GraphRAG for podcast transcripts

[Kaveh Hadjari (kavhad)]

One of the demos of GraphRAG published on youtube (https://www.youtube.com/watch?v=NV3vyWG2nxg) uses transcripts from "Behind the Tech" podcast episodes with Kevin Scott.

• Was the transcript text in the same format present on the microsoft web page (https://www.microsoft.com/en-us/behind-the-tech)?
• Did you use a custom prompt for any of the generation task, e.g. entity relationship extraction, etc?

[Alonso Guevara (AlonsoGuevara)]

Hi Kaveh Hadjari (@kavhad)
Per your questions:
1- Yes, we downloaded them from the Behind the tech webpage and converted into .txt. files
2- Yes, our prompts were generated by the auto templating feature, you can learn more about it here: https://microsoft.github.io/graphrag/posts/prompt_tuning/auto_prompt_tuning/

[Alonso Guevara (AlonsoGuevara)]

Oh, couple of details I overlooked!

1. Yes, everything as I said but... We did a preprocess step to include also the episode information as part of the document.
2. Podcast processing was actually one of the motivations for Prompt Auto Templating, so we manually modified the prompts for this. Now that the Auto Templating is live, I would instead suggest to use that feature.

[Nikhil Maddirala (nikhilmaddirala)]

Hi Alonso Guevara (@AlonsoGuevara) — I'm trying to test a similar use case (podcast transcripts) and I have a couple of questions. Consider this excerpt from page 5 of the Document "Behind the Tech with Kevin Scott | Episode 63: Mike Schroepfer" and suppose this excerpt is split into 3 chunks (TextUnits):

MIKE SCHROEPFER: Yeah. Me too. And I – like we can talk, because you know, like again, it was – just a fortunate time, because like, the internet, you know, the World Wide Web. We already had Gopher, and a bunch of other things, and Telnet, and what was it, Usenet/NNTP, when I got there, but then, you know, the web obviously came out while I was at Stanford, and so that was just, like amazing, but on the hardware side, I agree with you.

I mean, you know, when I joined Facebook, now Meta, in 2008, you know, those first couple of years – everyone thinks of Meta as a software company – and I was like, those first couple of years, like our biggest problem was the site was just growing so quickly that we had all these emergency fire drills, like we’re literally going to run out of capacity and everything’s going to crash.

And so a combination of like software optimization and like, how do we uncork the like hardware growth rate thing? And as you well know, better than I, as well as I do, it’s like, you can’t just, like order a new datacenter and have it next week. You’re like, "Oh crap, we gotta …"

And like, I remember in these conversations, it’s like, "Okay, we’re going to build a datacenter," like we’d have a 2- to 4-year, like lead time, between the go and – like, it’s operational, like, how much capacity do you need? And I was like, "I don’t know," like it’s so hard to predict that far in advance.

So like, we literally have to order steel now, and like buy some land. So like, you can’t tell me this in nine months. You need to tell me now, how much we’re building. And so like that was just a – like, and then we built, you know, lots of datacenters, and then we made another move into consumer hardware.

And so I joke, I made two transitions, one from software to hardware. And then when we got into consumer hardware it was like, "Oh, we’ve been building datacenters. I know this hardware thing. This is easy." And it’s like, "Oh, no. No, no, no." It’s a totally different game when – like, if the consumer returns the thing, you’ve now lost a ton of money because, like, you just ate everything.

And so making sure that they like it and there’s customer support and your price – like, there’s a whole different ballgame in consumer hardware. So it’s like software, enterprise hardware, consumer hardware, in terms of like difficulty, I would say. And so that has been really fun and has actually informed a lot of like, in my climate-tech investing, we’re investing mostly in hardware, and a lot of it is enterprise, but some of it’s consumer. You know, we’re selling products that your average person is going to buy. And so anyway, so we can talk a lot about that. It is definitely a shift and a lot of fun.

My questions:

1. How do we know that each TextUnit is associated to the speaker "MIKE SCHROEPFER"? The speaker name is only in the first chunk. This would be relevant for queries such as "What were the topics discussed by the speaker MIKE SCHROEPFER?"
2. How do we know that each unit is from the Document titled "Episode 63: Mike Schroepfer"? There seems to be no way to add metadata to the Text. This would be relevant for queries such as "What were the topics discussed in Episode 63?"

Thanks in advance! :)

EDIT: I notice the only way to add metadata is to use a csv format with the text in column and metadata in other columns. Is this right? If yes, how is this metadata (e.g. episode title) used for indexing and/or for querying?

[bmaltais]

Alonso Guevara (@AlonsoGuevara) I tried running the auto prompt on a local ollama based system and got the following error when performing the Generaring entity extraction prompt:

KeyError: 'Could not automatically map gemma2 to a tokeniser. Please use tiktoken.get_encoding to explicitly get the tokeniser you expect.'

I am specifying the Google gemma2 model for the task...

Same error for llama3

[Alonso Guevara (AlonsoGuevara)]

Hi bmaltais
I'll fix this, it is a bug that assumes cl100k_base as the encoding model. I'll prepare the PR