As Information Retrieval (IR) shifts toward neural and hybrid search, Sparse Vectors (lexical embeddings like SPLADE, ELSER, and BM25 variants) have become first-class citizens. However, we lack a standard way to identify, reference, and cache these vectors across different systems.
The Sparse URL project defines a deterministic, content-addressed URI scheme (sparse://) to provide a universal "ID card" for any sparse vector.
- Deterministic Identity: The same vector + the same model always produces the same URI.
- Compact & Fixed-Length: Solves the "URI too long" error by hashing large term-weight dictionaries into a 32-character digest.
- Model-Aware: Prevents "semantic collision" by binding vectors to their specific authority/model lexicon.
- Immutable: Content-addressing ensures that if the weights change, the URI changes.
/RFCs: The core specifications (Start with RFC 0001)./reference-impl: Standard libraries for Python, JavaScript, and Rust to ensure bit-perfect hashing./examples: Sample vectors and their corresponding URIs for testing.
A sparse vector with thousands of dimensions is compressed into a manageable string:
Input:
{
"authority": "huggingface.co",
"model": "splade-v2",
"terms": {"AI": 0.85, "lexical": 0.92, ...}
}
Resulting URI:
sparse://huggingface.co/v1/a3f8c2d19e4b7a6f8e3c1d0b9f2a7e4c
🤝 Contributing
We are currently in the Draft/Proposal phase. We specifically need feedback on:
1. Float-to-string canonicalization rules (ensuring 0.900000 hashes the same everywhere).
2. Authority registration for common models.
3. Integration patterns for Vector Databases (Milvus, Pinecone, Qdrant, etc.).
Please see CONTRIBUTING.md to get started.
📜 License
This project is licensed under the Apache License 2.0. See the LICENSE file for details.