Repository navigation
threads: prefetch pipeline and parallel ingest on Windows - #155
Merged
Merged
Conversation
A small portability header, src/odbc_threads.h, gives odbc_reader.c and odbc_bind.c one mutex, condition variable and joinable thread on both platforms: pthreads on POSIX, SRWLOCK, CONDITION_VARIABLE and _beginthreadex on Windows. The _WIN32 compile-outs around the prefetch pipeline and the parallel ingest pool go, as does the ingest_connections clamp to 1 on Windows; adbc.odbc.prefetch and adbc.odbc.ingest_connections now behave the same on Linux, macOS and Windows. Docs updated to match. Bundled crate sources synced. Linux: C tests 5316 checks / ctest 10 of 10; prefetch and parallel-ingest suites 45 of 45 against PostgreSQL 18 through psqlodbc.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap item: both features were pthreads and compiled out on
_WIN32, so Windows reads ran unpipelined andadbc.odbc.ingest_connectionswas clamped to 1.Change.
src/odbc_threads.his a small portability header: one mutex, one condition variable and one joinable thread, pthreads on POSIX and SRWLOCK / CONDITION_VARIABLE /_beginthreadexon Windows (a heap cell carries the pthread-shaped entry point across the__stdcalltrampoline).odbc_reader.candodbc_bind.cuse it in place of pthreads, the_WIN32guards around the prefetch pipeline and the parallel ingest pool are gone, and so is the Windows clamp inodbc_driver.c. The options now behave the same on all three platforms; the docs that said otherwise (README, install-windows, prefetch, options, go, rust, python, performance, FAQ, roadmap) are updated. Crate sources synced.Verified on Linux (behaviour must not change there): C tests 5316 checks, 0 failures; ctest 10/10;
tests/test_prefetch.py+tests/test_parallel_ingest.py45/45 against PostgreSQL 18 through psqlodbc, including the mid-stream error and the abort-while-prefetching cases.Windows: this PR's CI builds the library on windows-latest x64 and Win32 and runs the C unit tests; the prefetch and parallel-ingest suites only run on Linux in CI, so the Windows run of those two suites is a manual step on the Windows boot before 0.1.4 is tagged.