Skip to content

fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI#1637

Open
winklemad wants to merge 1 commit into
getzep:mainfrom
winklemad:fix/mcp-configurable-cross-encoder
Open

fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI#1637
winklemad wants to merge 1 commit into
getzep:mainfrom
winklemad:fix/mcp-configurable-cross-encoder

Conversation

@winklemad

Copy link
Copy Markdown

What & why

The MCP server never passed a cross_encoder to Graphiti(), so it fell back to OpenAIRerankerClient() and required OPENAI_API_KEY on startup, even when the LLM and embedder were both configured for other providers (Anthropic, Gemini, Groq). That made the server unusable without an OpenAI key (#1393).

Change

  • Add CrossEncoderFactory in services/factories.py. It picks a reranker from the LLM provider, then the embedder provider, and falls back to the local BGERerankerClient. graphiti-core only ships OpenAI/Gemini/BGE rerankers, so for an Anthropic or Groq LLM the factory reuses the embedder provider's key. The reporter's Anthropic-LLM + Gemini-embedder setup, for example, now reranks via Gemini instead of crashing on a missing OpenAI key.
  • Pass cross_encoder= into both the FalkorDB and Neo4j Graphiti() calls.

Testing

tests/test_cross_encoder_factory.py covers the OpenAI case and the Anthropic-LLM + Gemini-embedder case. The full factory suite passes (26 tests), and ruff check / ruff format are clean.

Happy to adjust the provider-to-reranker mapping, or add an explicit reranker config block if you'd rather have that than inference.

Fixes #1393

… OpenAI

The MCP server never passed a cross_encoder to Graphiti(), so it fell back to OpenAIRerankerClient and required OPENAI_API_KEY even on fully non-OpenAI setups.

Add a CrossEncoderFactory that picks a reranker from the LLM provider, then the embedder provider, and falls back to the local BGE reranker, and wire it into both Graphiti() construction sites.

Fixes getzep#1393
@zep-cla-assistant

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. For privacy information, see our Privacy Notice. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: example@example.com

or

I have read the CLA Document and I hereby sign the CLA behalf of my company, e-mail: example@example.com

Signature is valid for 6 months.


This bot will be retriggered when the Contributor License Agreement comment has been provided. Posted by the CLA Assistant Lite bot.

@winklemad

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@winklemad

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: winklemad@outlook.com

1 similar comment
@winklemad

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: winklemad@outlook.com

@Naseem77

Copy link
Copy Markdown
Contributor

@winklemad Tested this on a clean clone: the MCP cross-encoder used to always be OpenAI regardless of config, which crashes if you only have a Gemini or Anthropic key. With your patch the factory returns the right reranker per provider and falls back to BGE local. Ran the PR tests plus provider-combo tests, all pass.

@winklemad

Copy link
Copy Markdown
Author

Thanks @Naseem77 — appreciate the clean-clone test, and good to have the FalkorDB reranker path independently confirmed.

Status for maintainers, in case it helps: I've signed the CLA (winklemad@outlook.com) and Ruff is green — the remaining Required checks (CLA re-check, pyright, unit-tests) are all just waiting on first-time-contributor workflow approval rather than any actual failure. The branch is behind main but merges cleanly; happy to rebase if that's easier. Whenever someone can approve the CI run or take a look, I'll address any feedback quickly.

@abouchard11 abouchard11 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this on my production stack, complementary to @Naseem77's clean-clone FalkorDB run: Neo4j 5.26 + Gemini LLM (gemini-2.5-flash) + gemini-embedding-001, config loaded through the server's actual GraphitiConfig loader. The factory selected GeminiRerankerClient and a live rank() call against the Gemini API returned correct ordering. test_cross_encoder_factory.py + test_factories.py: 26/26 pass locally.

Two findings on the fallback path, with a suggestion inline:

  1. On a base install (no providers extra), sentence-transformers is missing, so when neither provider has a native reranker (e.g. Anthropic LLM + Voyage embedder) the BGE import raises ImportError. graphiti_mcp_server.py catches it, logs a warning, and passes cross_encoder=None — and graphiti-core then defaults to OpenAIRerankerClient(), which is exactly the #1393 crash this PR fixes, now hidden behind a misleading warning. Reproduced by blocking sentence_transformers in the venv. The suggestion below at least makes the failure actionable; whether reranker-init failure should be fatal is a maintainer call.
  2. BGERerankerClient() instantiates CrossEncoder('BAAI/bge-reranker-v2-m3') in its constructor — a ~2.3 GB synchronous download at server startup the first time the fallback runs. Folded a heads-up into the suggested log line; probably worth a README note for Docker users too.

This supersedes my #1669 (Gemini-only take on the same idea), so I'm closing that in favor of this one. Also happy to pick up the explicit reranker: config block you floated in the description as a follow-up PR once this lands.

Comment on lines +432 to +435
logger.info('No provider reranker available, using local BGERerankerClient')
from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient

return BGERerankerClient()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes the failure actionable when sentence-transformers isn't installed, and flags the first-run download:

Suggested change
logger.info('No provider reranker available, using local BGERerankerClient')
from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient
return BGERerankerClient()
logger.info(
'No provider reranker available, using local BGERerankerClient '
'(downloads BAAI/bge-reranker-v2-m3, ~2.3 GB, on first run)'
)
try:
from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient
except ImportError as e:
raise ValueError(
'No provider reranker is available for this configuration, and the local '
'BGE fallback requires the optional sentence-transformers dependency. '
"Install the MCP server's 'providers' extra (uv sync --extra providers) "
'or configure an OpenAI or Gemini key so a provider reranker can be used.'
) from e
return BGERerankerClient()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] MCP Server: cross_encoder/reranker not configurable, hardcoded to OpenAI — requires OPENAI_API_KEY even with non-OpenAI providers

3 participants