Does anyone think the prompts are vague?

[Zhanwei Zhang (IT-Bill)]

Prompt quality is one of the most important factors in GraphRAG. However, it seems that many developers do not give enough consideration to the quality of the prompts. The local_search_system_prompt and global_search_system_prompt have remained unchanged since the release of GraphRAG.

There are several inaccuracies in the prompts:

1. Improper delimiter: The --- delimiter is not recommended by OpenAI. Instead, it is suggested to use """. OpenAI Prompt Engineering.

2. Unclear/Inconsistent references:

   • In the local search prompt, terms like "Data Table" (referring to the data in the tables) and "response length and format" (referring to the target response length and format) lack clarity and consistency.
   • In the global search prompt, references like "analysts" (referring to analyst reports) and "a dataset" are similarly vague and need more precise definitions.

The lower the tier of the LLM, the higher the demand for prompt accuracy in more difficult tasks; conversely, the higher the tier, the less strict the requirement for prompt accuracy in such tasks. Local search and global search are difficult tasks in GraphRAG, so I think the prompts need to be updated and improved.