fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI#1637
fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI#1637winklemad wants to merge 1 commit into
Conversation
… OpenAI The MCP server never passed a cross_encoder to Graphiti(), so it fell back to OpenAIRerankerClient and required OPENAI_API_KEY even on fully non-OpenAI setups. Add a CrossEncoderFactory that picks a reranker from the LLM provider, then the embedder provider, and falls back to the local BGE reranker, and wire it into both Graphiti() construction sites. Fixes getzep#1393
|
I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: example@example.com or I have read the CLA Document and I hereby sign the CLA behalf of my company, e-mail: example@example.com Signature is valid for 6 months. This bot will be retriggered when the Contributor License Agreement comment has been provided. Posted by the CLA Assistant Lite bot. |
|
I have read the CLA Document and I hereby sign the CLA |
|
I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: winklemad@outlook.com |
1 similar comment
|
I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: winklemad@outlook.com |
|
@winklemad Tested this on a clean clone: the MCP cross-encoder used to always be OpenAI regardless of config, which crashes if you only have a Gemini or Anthropic key. With your patch the factory returns the right reranker per provider and falls back to BGE local. Ran the PR tests plus provider-combo tests, all pass. |
|
Thanks @Naseem77 — appreciate the clean-clone test, and good to have the FalkorDB reranker path independently confirmed. Status for maintainers, in case it helps: I've signed the CLA (winklemad@outlook.com) and Ruff is green — the remaining Required checks (CLA re-check, pyright, unit-tests) are all just waiting on first-time-contributor workflow approval rather than any actual failure. The branch is behind |
abouchard11
left a comment
There was a problem hiding this comment.
Tested this on my production stack, complementary to @Naseem77's clean-clone FalkorDB run: Neo4j 5.26 + Gemini LLM (gemini-2.5-flash) + gemini-embedding-001, config loaded through the server's actual GraphitiConfig loader. The factory selected GeminiRerankerClient and a live rank() call against the Gemini API returned correct ordering. test_cross_encoder_factory.py + test_factories.py: 26/26 pass locally.
Two findings on the fallback path, with a suggestion inline:
- On a base install (no
providersextra),sentence-transformersis missing, so when neither provider has a native reranker (e.g. Anthropic LLM + Voyage embedder) the BGE import raisesImportError.graphiti_mcp_server.pycatches it, logs a warning, and passescross_encoder=None— and graphiti-core then defaults toOpenAIRerankerClient(), which is exactly the #1393 crash this PR fixes, now hidden behind a misleading warning. Reproduced by blockingsentence_transformersin the venv. The suggestion below at least makes the failure actionable; whether reranker-init failure should be fatal is a maintainer call. BGERerankerClient()instantiatesCrossEncoder('BAAI/bge-reranker-v2-m3')in its constructor — a ~2.3 GB synchronous download at server startup the first time the fallback runs. Folded a heads-up into the suggested log line; probably worth a README note for Docker users too.
This supersedes my #1669 (Gemini-only take on the same idea), so I'm closing that in favor of this one. Also happy to pick up the explicit reranker: config block you floated in the description as a follow-up PR once this lands.
| logger.info('No provider reranker available, using local BGERerankerClient') | ||
| from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient | ||
|
|
||
| return BGERerankerClient() |
There was a problem hiding this comment.
Makes the failure actionable when sentence-transformers isn't installed, and flags the first-run download:
| logger.info('No provider reranker available, using local BGERerankerClient') | |
| from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient | |
| return BGERerankerClient() | |
| logger.info( | |
| 'No provider reranker available, using local BGERerankerClient ' | |
| '(downloads BAAI/bge-reranker-v2-m3, ~2.3 GB, on first run)' | |
| ) | |
| try: | |
| from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient | |
| except ImportError as e: | |
| raise ValueError( | |
| 'No provider reranker is available for this configuration, and the local ' | |
| 'BGE fallback requires the optional sentence-transformers dependency. ' | |
| "Install the MCP server's 'providers' extra (uv sync --extra providers) " | |
| 'or configure an OpenAI or Gemini key so a provider reranker can be used.' | |
| ) from e | |
| return BGERerankerClient() |
What & why
The MCP server never passed a
cross_encodertoGraphiti(), so it fell back toOpenAIRerankerClient()and requiredOPENAI_API_KEYon startup, even when the LLM and embedder were both configured for other providers (Anthropic, Gemini, Groq). That made the server unusable without an OpenAI key (#1393).Change
CrossEncoderFactoryinservices/factories.py. It picks a reranker from the LLM provider, then the embedder provider, and falls back to the localBGERerankerClient. graphiti-core only ships OpenAI/Gemini/BGE rerankers, so for an Anthropic or Groq LLM the factory reuses the embedder provider's key. The reporter's Anthropic-LLM + Gemini-embedder setup, for example, now reranks via Gemini instead of crashing on a missing OpenAI key.cross_encoder=into both the FalkorDB and Neo4jGraphiti()calls.Testing
tests/test_cross_encoder_factory.pycovers the OpenAI case and the Anthropic-LLM + Gemini-embedder case. The full factory suite passes (26 tests), andruff check/ruff formatare clean.Happy to adjust the provider-to-reranker mapping, or add an explicit
rerankerconfig block if you'd rather have that than inference.Fixes #1393