Problem: Each direct findAndRerank request constructs a new Nvidia provider and Quarkus REST client inside a synchronized factory before the vector and lexical reads begin. Burst traffic therefore creates many independent clients and connection pools, defeats effective pooling/backpressure, and does not deterministically close those clients.
Proposal: Cache one direct reranking provider/client per configured provider and model for the Data API process lifetime. Reuse it across tenants because credentials and tenant headers are supplied per call, close it at shutdown, and keep embedding-gateway wrappers request-scoped.
Acceptance criteria:
- Repeated and concurrent direct-provider lookups return the same instance.
- Different models do not share instances.
- Gateway-backed providers remain request-scoped.
- Cached clients close on application shutdown.
- Existing reranking behavior and credentials remain unchanged.
Problem: Each direct findAndRerank request constructs a new Nvidia provider and Quarkus REST client inside a synchronized factory before the vector and lexical reads begin. Burst traffic therefore creates many independent clients and connection pools, defeats effective pooling/backpressure, and does not deterministically close those clients.
Proposal: Cache one direct reranking provider/client per configured provider and model for the Data API process lifetime. Reuse it across tenants because credentials and tenant headers are supplied per call, close it at shutdown, and keep embedding-gateway wrappers request-scoped.
Acceptance criteria: