add vector search benchmarks - #311
Open
vkarpov15 wants to merge 4 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds basic vector search benchmark scripts to compare insert and query performance between Astra tables and collections.
Changes:
- Introduces
benchmarks/writeVector.mjsto create/sync acontenttable or collection, then bulk-insert ~21k vector documents in batches and log throughput. - Introduces
benchmarks/queryVector.mjsto run 2,000 parallelized ANN-style vector queries against thecontentmodel and log query performance metrics.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| benchmarks/writeVector.mjs | Sets up the Content schema and collection/table, then bulk-inserts vectorized movie documents and prints insert performance stats. |
| benchmarks/queryVector.mjs | Uses the existing Content schema to perform repeated vector similarity queries and prints query performance stats. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Collaborator
Author
|
I checked and confirmed that writing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
Basic benchmarks for vector search:
21k documents just have a
textandvectorproperty. Paradoxically, it seems like collections are actually faster than tables in this case. A big reason why collections are faster though is because collections project out the vector field by default, it seems like nearly 90% of the overhead of tables vs collections is eliminated by projecting out the vector field on the read path.Tables
{"inserted":21347,"seconds":376.129264,"docsPerSecond":56.754425,"secondsPerBatch":0.352395}{"queries":2000,"parallelism":10,"seconds":55.601865,"queriesPerSecond":35.970016,"secondsPerBatch":0.278009}{"queries":2000,"parallelism":10,"seconds":32.994885,"queriesPerSecond":60.615457,"secondsPerBatch":0.164974}Collections
{"inserted":21347,"seconds":222.619178,"docsPerSecond":95.890211,"secondsPerBatch":0.208572}{"queries":2000,"parallelism":10,"seconds":28.87107,"queriesPerSecond":69.273497,"secondsPerBatch":0.144355}Which issue(s) this PR fixes:
Fixes #
Checklist