Repository navigation
Make lz4 optional on Python 3.14+ - #1139
Conversation
Import lz4 lazily via compression.py so the package works without it. available_compression only lists lz4 when installed, so compress=True falls through to zstd/gzip. Add a clickhouse-connect[lz4] extra. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
joe-clickhouse
left a comment
There was a problem hiding this comment.
Looks good, thanks @29antonioac. I added clearer install instructions when lz4 is missing, updated the docs and changelog, simplified the extra, and added CI coverage without lz4. These changes make the optional dependency easier to use and keep that path tested. I also updated the PR description to accurately represent the final state of the PR.
|
Thank you so much @joe-clickhouse! Really appreciate the support ❤️ It'll come handy for 3.14+ to avoid installing extra deps in Lambda 😊 |
There was a problem hiding this comment.
🔵 Needs a closer look
The 1.x bare-install compatibility change requiring an extra for Python 3.14+ clients requesting lz4 has no established explicit maintainer approval, as required by repository guidance.
No open issues 🎉
What changed in this PR
Makes lz4 optional on Python 3.14+, allowing default installations to use standard-library zstd without building lz4 from source. Default dependencies below Python 3.14 remain unchanged.
Changes:
- Add the
lz4extra and guard compression imports with installation guidance when lz4 is requested but unavailable. - Update documentation and docstrings to explain the zstd insert fallback.
- Add missing-lz4 tests and a Python 3.14 CI job, skipping lz4-specific tests when unavailable.
No findings remain. Tests, linting, and type checking were not run during this review.
| File | Description |
|---|---|
tests/unit_tests/test_streaming_source.py |
Skip lz4 streaming tests when unavailable. |
tests/unit_tests/test_driver/test_httputil.py |
Guard the lz4 response test. |
tests/unit_tests/test_driver/test_compression.py |
Test missing-lz4 imports and errors. |
tests/integration_tests/test_streaming.py |
Skip unavailable lz4 parameter cases. |
setup.py |
Make lz4 optional on Python 3.14+ and add an extra. |
docs/index.mdx |
Document installation and wheel implications. |
docs/additional-options.mdx |
Explain compression fallback and optional lz4 support. |
clickhouse_connect/driver/streaming.py |
Use the guarded lz4 decompressor helper. |
clickhouse_connect/driver/httputil.py |
Remove mandatory lz4 imports from response handling. |
clickhouse_connect/driver/compression.py |
Detect lz4 availability and guard codec creation. |
clickhouse_connect/driver/_backend/httpcommon.py |
Provide missing-lz4 guidance and guarded decompression. |
clickhouse_connect/driver/_backend/chdb_backend.py |
Guard lz4 decompression in chDB. |
clickhouse_connect/driver/__init__.py |
Update sync and async compression docstrings. |
CHANGELOG.md |
Record the dependency and fallback changes. |
.github/workflows/on_push.yml |
Add Python 3.14 coverage without lz4. |
Review effort: Balanced
Approved as a deliberate 1.x change: lz4 becomes an extra on 3.14+ only, nothing changes below 3.14. |
Summary
On Python 3.14+, lz4 is optional. Install
clickhouse-connect[lz4]to use it. Default installs below 3.14 are unchanged.python-lz4lacks wheels for Python 3.14+ free-threaded builds, CPython 3.15, and musllinux. Its latest release was November 2025. These platforms must build it from source, which requires a C compiler. Python 3.14 includes zstd, so default compression no longer needs a third-party package.Without lz4:
compress=Truestill gets zstd responses because the server prefers zstd.compress="lz4"raises ProgrammingError with instructions to install the extra.With lz4 installed, compression behaves as before.
Changes