You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
add a local SenseVoiceASR provider backed by FunASR, with lazy model loading, model caching, serialized inference, 20-minute chunking, native timestamp parsing, and a separately configured fsmn-vad model
expose model and device selection through the CLI, TOML configuration, environment variables, and desktop settings
normalize encoded audio bytes to 16 kHz mono WAV before inference
keep funasr, torch, and torchaudio in the optional sensevoice extra and hide SenseVoice from the desktop model list when that complete runtime is unavailable
add focused provider, dependency-gating, and CLI configuration tests
The default iic/SenseVoiceSmall checkpoint supports Chinese, Cantonese, English, Japanese, and Korean. This PR does not add the OpenAI-compatible FunASR server, Paraformer, speaker diarization, or a universal performance claim across FunASR checkpoints.
Desktop release builds do not currently bundle the optional PyTorch/FunASR runtime. With this PR, those builds no longer advertise an unusable SenseVoice option; source/Python installations can enable it with pip install 'videocaptioner[sensevoice]'.
Validation
TDD regression: dependency-incomplete desktop model-list test failed before the runtime gate, then passed after the fix
Follow-up for head 658b2ee: the three new workflow runs are still action_required with zero jobs, so no test has failed; they need maintainer approval before GitHub can start them:
This head also fixes a desktop packaging gap found during review: SenseVoice is now hidden unless the complete optional runtime (funasr, torch, and torchaudio) is available. Local verification is green: 92 focused tests, Ruff and targeted Pyright, sdist/wheel build, optional-runtime imports, and real CPU transcription of the Chinese and English fixtures.
Could a maintainer approve the current runs when convenient?
Pushed head aeaa7de5e6cafab6655b0210920b64c6027a6f7f after rechecking this PR in an environment that installs the optional SenseVoice runtime.
What changed:
Fixed the missing-optional-dependency test so it patches the same import path used by production code: importlib.import_module("funasr"). This keeps the test meaningful even when videocaptioner[sensevoice] is installed.
Ran the current project Ruff formatter on the PR-touched CLI/config/entity files so the current locked toolchain passes ruff format --check.
Fresh validation on ind-gpu8 with uv sync --group dev --extra sensevoice:
uv run pytest tests/test_asr/test_sensevoice_asr.py tests/test_asr/test_platform_utils.py tests/test_cli/test_config.py tests/test_cli/test_parser.py -q
# 78 passed
uv run ruff check <PR-touched ASR/CLI/test files>
uv run ruff format --check <PR-touched ASR/CLI/test files>
uv run pyright videocaptioner/core/asr/sensevoice.py videocaptioner/cli/validators.py
python -m py_compile <PR-touched Python files>
git diff --check
uv build --sdist --wheel
All of the above passed. GitHub reports the PR as mergeable; the remaining workflow runs still need maintainer approval before they can execute on the fork.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SenseVoiceASRprovider backed by FunASR, with lazy model loading, model caching, serialized inference, 20-minute chunking, native timestamp parsing, and a separately configuredfsmn-vadmodelfunasr,torch, andtorchaudioin the optionalsensevoiceextra and hide SenseVoice from the desktop model list when that complete runtime is unavailableThe default
iic/SenseVoiceSmallcheckpoint supports Chinese, Cantonese, English, Japanese, and Korean. This PR does not add the OpenAI-compatible FunASR server, Paraformer, speaker diarization, or a universal performance claim across FunASR checkpoints.Desktop release builds do not currently bundle the optional PyTorch/FunASR runtime. With this PR, those builds no longer advertise an unusable SenseVoice option; source/Python installations can enable it with
pip install 'videocaptioner[sensevoice]'.Validation
uv run --frozen pytest tests/test_cli tests/test_dubbing tests/test_asr/test_sensevoice_asr.py tests/test_asr/test_platform_utils.py -q(92 passed)uv run --frozen ruff checkandruff format --checkon all changed filesuv run --frozen pyright videocaptioner/core/utils/platform_utils.py(0 errors)uv sync --frozen --extra sensevoicefollowed by successful imports of FunASR 1.3.14, Torch 2.13.0, and torchaudio 2.11.010timestamped segments each, non-empty expected text)uv build(sdist and wheel built successfully)git diff --checkFull-repository
pyrightstill reports four pre-existing return-type errors in translation/TTS modules that are outside this PR's diff.Closes #1137