feat: add MiniMax cloud TTS engine - #971
Conversation
Register a cloud speech engine in the TTS engine registry with the current speech models, global/CN endpoint selection, request controls (voice/audio settings, language boost, output format) and audio formats (mp3/wav/flac/pcm). Adds preset-voice listing/validation and unit tests.
📝 WalkthroughWalkthroughAdds MiniMax as a cloud TTS engine with regional API access, preset and cloned voices, request validation, audio decoding, backend factory integration, and unit tests for configuration, synthesis, cloning, and error handling. ChangesMiniMax TTS support
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant GenerationRequest
participant MiniMaxTTSBackend
participant MiniMaxT2A
GenerationRequest->>MiniMaxTTSBackend: generate text and voice settings
MiniMaxTTSBackend->>MiniMaxT2A: POST T2A payload with API key
MiniMaxT2A-->>MiniMaxTTSBackend: JSON response with hex audio
MiniMaxTTSBackend->>MiniMaxTTSBackend: decode audio to float32 samples
MiniMaxTTSBackend-->>GenerationRequest: audio samples and sample rate
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/backends/minimax_backend.py`:
- Around line 89-100: Extend LANGUAGE_BOOST_MAP with MiniMax-supported boost
names for the explicitly supported language codes ar, he, da, el, fi, hi, ms,
nl, no, pl, sv, and tr, ensuring GenerationRequest.language values are mapped to
their corresponding language names instead of falling back to "auto".
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 7275aeea-b206-40b8-84c6-5be22c8774b0
📒 Files selected for processing (6)
backend/backends/__init__.pybackend/backends/minimax_backend.pybackend/models.pybackend/routes/profiles.pybackend/services/profiles.pybackend/tests/test_minimax_backend.py
| LANGUAGE_BOOST_MAP = { | ||
| "en": "English", | ||
| "zh": "Chinese", | ||
| "ja": "Japanese", | ||
| "ko": "Korean", | ||
| "de": "German", | ||
| "fr": "French", | ||
| "ru": "Russian", | ||
| "pt": "Portuguese", | ||
| "es": "Spanish", | ||
| "it": "Italian", | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate files =="
fd -a 'minimax_backend\.py|models\.py$' . | sed 's#^\./##'
echo "== inspect minimax backend relevant section =="
if [ -f backend/backends/minimax_backend.py ]; then
wc -l backend/backends/minimax_backend.py
sed -n '1,180p' backend/backends/minimax_backend.py | cat -n
fi
echo "== search request language type/constants =="
rg -n "GenerationRequest|language|LANGUAGE_BOOST_MAP|boost" backend -S || trueRepository: jamiepine/voicebox
Length of output: 24857
🌐 Web query:
MiniMax HTTP Speech T2A supported languages boost API language_code
💡 Result:
MiniMax text-to-audio (T2A) models support 40 widely used global languages [1][2][3]. The language_boost parameter in the API is used to enhance recognition or synthesis accuracy for these languages, allowing you to specify a language name or set it to "Automatic" [4]. For certain languages with specific dialectal needs, such as Cantonese, the parameter may require a combined string like "Chinese,Yue" [5][6]. The 40 supported languages include: Chinese (including Cantonese), English, Spanish, French, Russian, German, Portuguese, Arabic, Italian, Japanese, Korean, Indonesian, Vietnamese, Turkish, Dutch, Ukrainian, Thai, Polish, Romanian, Greek, Czech, Finnish, Hindi, Bulgarian, Danish, Hebrew, Malay, Persian, Slovak, Swedish, Croatian, Filipino, Hungarian, Norwegian, Slovenian, Catalan, Nynorsk, Tamil, and Afrikaans [1][2][3]. Note that certain languages, such as Filipino, Tamil, and Persian, are specifically supported by the newer speech-2.6 series models and later [7][8]. When using the API, ensure you select the appropriate model (e.g., speech-2.8-hd, speech-2.8-turbo, or speech-2.6-*) to access the full range of language capabilities [1][8].
Citations:
- 1: https://platform.minimax.io/docs/api-reference/api-overview
- 2: https://platform.minimax.io/docs/guides/speech-t2a-async
- 3: https://platform.minimax.io/docs/guides/speech-t2a-websocket
- 4: https://replicate.com/minimax/speech-2.6-turbo/readme
- 5: https://platform.minimax.io/docs/api-reference/speech-t2a-http
- 6: https://reference-server.pipecat.ai/en/stable/_modules/pipecat/services/minimax/tts.html
- 7: https://pipecat-docs.readthedocs.io/en/latest/api/pipecat.services.minimax.tts.html
- 8: https://docs.pipecat.ai/api-reference/server/services/tts/minimax
🌐 Web query:
MiniMax T2A language_boost exact value Arabic Danish Greek Finnish Hebrew Hind
💡 Result:
In the MiniMax Text-to-Audio (T2A) API, the language_boost parameter is used to specify a language or dialect to enhance recognition and synthesis accuracy, preventing pronunciation errors [1][2][3]. Arabic, Danish, Greek, Finnish, Hebrew, and Hindi are all supported, valid values for this parameter [1][3][4]. When using this parameter, you must provide the language name exactly as defined in the API's supported list [1][5]. The supported values for the requested languages are: - Arabic [1][4] - Danish [1][4] - Greek [1][4] - Finnish [1][4] - Hebrew [1][4] - Hindi [1][4] Other common valid values include 'auto' (for automatic detection), 'English', 'Chinese', 'French', 'German', and many others [1][3][5]. The parameter defaults to null if not specified [1][6]. Note that while the model supports these languages, the speech-01 and speech-02 series models have specific limitations regarding Persian, Filipino, and Tamil, which are not currently supported for enhancement via this parameter [1].
Citations:
- 1: https://platform.minimax.io/docs/api-reference/speech-t2a-http
- 2: https://platform.minimax.io/docs/faq/about-apis
- 3: https://novita.ai/docs/api-reference/model-apis-minimax-speech-02-turbo
- 4: https://platform.minimax.io/docs/api-reference/api-overview
- 5: https://fal.ai/models/fal-ai/minimax/speech-02-turbo/api
- 6: https://platform.minimax.io/docs/guides/mcp-guide
🌐 Web query:
site:platform.minimax.io/api-reference/speech-t2a-http T2A language_boost Arabic Hebrew Finnish
💡 Result:
Yes — the MiniMax T2A HTTP docs list Arabic, Hebrew, and Finnish as available language_boost options. The docs also note that language_boost can be set to "auto" if the language is unknown. [1]
Map the remaining explicitly supported request languages.
GenerationRequest.language allows ar, he, da, el, fi, hi, ms, nl, no, pl, sv, and tr, but LANGUAGE_BOOST_MAP sends "auto" for all of them. Add MiniMax-supported boost names for these codes so GenerationRequest.language is preserved.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/backends/minimax_backend.py` around lines 89 - 100, Extend
LANGUAGE_BOOST_MAP with MiniMax-supported boost names for the explicitly
supported language codes ar, he, da, el, fi, hi, ms, nl, no, pl, sv, and tr,
ensuring GenerationRequest.language values are mapped to their corresponding
language names instead of falling back to "auto".
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/backends/minimax_backend.py`:
- Around line 244-259: The voice prompt cache flow around
_create_cloned_voice_sync must not persist a newly created voice_id before its
first successful T2A activation. Move cache_voice_prompt until after the
activation request succeeds, or invalidate the cached entry when activation
fails, while preserving reuse of valid cached prompts. Add coverage for failed
activation and recovery from an inactive cached voice.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 148f0d2a-9f2c-43d1-8cc4-07d692a28cf9
📒 Files selected for processing (3)
backend/backends/minimax_backend.pybackend/services/profiles.pybackend/tests/test_minimax_backend.py
🚧 Files skipped from review as they are similar to previous changes (1)
- backend/services/profiles.py
| audio_hash = get_cache_key(audio_path, reference_text) | ||
| cache_key = f"minimax_{audio_hash}" if use_cache else None | ||
| if cache_key: | ||
| cached = get_cached_voice_prompt(cache_key) | ||
| if isinstance(cached, dict) and cached.get("voice_id"): | ||
| return cached, True | ||
|
|
||
| requested_voice_id = f"voicebox_{audio_hash[:24]}" | ||
| voice_id = await asyncio.to_thread(self._create_cloned_voice_sync, audio_path, requested_voice_id) | ||
| voice_prompt = { | ||
| "voice_type": "cloned", | ||
| "voice_id": voice_id, | ||
| } | ||
|
|
||
| if cache_key: | ||
| cache_voice_prompt(cache_key, voice_prompt) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🌐 Web query:
According to the current MiniMax Voice Clone documentation, what action keeps a newly cloned voice available after the first 168 hours, and how should an application handle a clone that was never activated?
💡 Result:
According to the official MiniMax documentation, newly cloned voices are initially in an inactive, temporary state and will be automatically deleted if not used within 168 hours (7 days) [1][2][3]. To keep a cloned voice permanently, an application must use the voice_id to synthesize audio at least once via any Text-to-Audio (T2A) speech synthesis API within that 168-hour window [1][4][2]. It is important to note that preview or audition actions within the cloning API itself do not count as activation and will not persist the voice [1][2][3]. If a clone remains unused (never activated) for longer than 168 hours, it will expire and be automatically deleted by the system [1][2][3]. Consequently, applications should handle such clones by ensuring a real synthesis request is performed immediately after creation to confirm the voice is saved for long-term use [5][3]. Additionally, the documentation notes that cloned voices are not visible or queryable via the Get Voice API until they have been successfully used at least once for speech synthesis [6].
Citations:
- 1: https://platform.minimax.io/docs/api-reference/api-overview
- 2: https://platform.minimax.io/docs/api-reference/voice-design-intro
- 3: https://minimax-cac98058.mintlify.dev/docs/faq/about-apis
- 4: https://novita.ai/docs/api-reference/model-apis-minimax-voice-cloning
- 5: https://docs.nano-gpt.com/api-reference/endpoint/voice-cloning
- 6: https://platform.minimax.io/docs/api-reference/voice-management-get
🏁 Script executed:
#!/bin/bash
printf '%s\n' '--- target implementation ---'
sed -n '210,275p' backend/backends/minimax_backend.py
printf '%s\n' '--- cache helpers and voice-prompt call sites ---'
rg -n -C 4 'get_cached_voice_prompt|cache_voice_prompt|create_voice_prompt|_create_cloned_voice_sync' backend
printf '%s\n' '--- relevant tests ---'
rg -n -C 3 'minimax|voice_prompt|voice_id|clone' tests backend | head -300Repository: jamiepine/voicebox
Length of output: 50376
🏁 Script executed:
#!/bin/bash
printf '%s\n' '--- MiniMax generation path ---'
ast-grep outline backend/backends/minimax_backend.py
sed -n '270,430p' backend/backends/minimax_backend.py
printf '%s\n' '--- profile creation and generation ordering ---'
sed -n '521,645p' backend/services/profiles.py
sed -n '45,105p' backend/services/generation.py
printf '%s\n' '--- cache implementation ---'
sed -n '1,115p' backend/utils/cache.py
printf '%s\n' '--- MiniMax tests ---'
sed -n '220,340p' backend/tests/test_minimax_backend.pyRepository: jamiepine/voicebox
Length of output: 21133
🏁 Script executed:
#!/bin/bash
python3 - <<'PY'
import ast
from pathlib import Path
files = {
"backend/backends/minimax_backend.py": Path("backend/backends/minimax_backend.py"),
"backend/services/profiles.py": Path("backend/services/profiles.py"),
"backend/services/generation.py": Path("backend/services/generation.py"),
}
trees = {name: ast.parse(path.read_text()) for name, path in files.items()}
def find_function(tree, name):
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name == name:
return node
raise AssertionError(name)
create = find_function(trees["backend/backends/minimax_backend.py"], "create_voice_prompt")
generate = find_function(trees["backend/backends/minimax_backend.py"], "generate")
def calls_in_order(node):
result = []
for child in ast.walk(node):
if isinstance(child, ast.Call):
fn = child.func
if isinstance(fn, ast.Name):
result.append((child.lineno, fn.id))
elif isinstance(fn, ast.Attribute):
result.append((child.lineno, fn.attr))
return sorted(result)
print("create_voice_prompt calls:", calls_in_order(create))
print("generate calls:", calls_in_order(generate))
profile_tree = trees["backend/services/profiles.py"]
generation_tree = trees["backend/services/generation.py"]
profile = find_function(profile_tree, "create_voice_prompt_for_profile")
generation = find_function(generation_tree, "execute_generation")
print("profile prompt creation call lines:",
[n.lineno for n in ast.walk(profile)
if isinstance(n, ast.Call)
and isinstance(n.func, ast.Attribute)
and n.func.attr == "create_voice_prompt"])
print("generation ordering:",
[(n.lineno, n.func.attr) for n in ast.walk(generation)
if isinstance(n, ast.Call)
and isinstance(n.func, ast.Attribute)
and n.func.attr in {"create_voice_prompt_for_profile", "generate_chunked"}])
PYRepository: jamiepine/voicebox
Length of output: 687
🏁 Script executed:
#!/bin/bash
python3 - <<'PY'
import ast
from pathlib import Path
path = Path("backend/services/generation.py")
tree = ast.parse(path.read_text())
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
print(f"{node.name}: line {node.lineno}")
PYRepository: jamiepine/voicebox
Length of output: 312
Do not cache an unactivated cloned voice.
MiniMax keeps a cloned voice only after a successful T2A request uses its voice_id within 168 hours. This code caches the ID before that request. If the first T2A request fails, later requests can reuse the inactive ID.
Cache the voice only after successful activation, or invalidate it when activation fails. Add tests for failed activation and stale-cache recovery.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/backends/minimax_backend.py` around lines 244 - 259, The voice prompt
cache flow around _create_cloned_voice_sync must not persist a newly created
voice_id before its first successful T2A activation. Move cache_voice_prompt
until after the activation request succeeds, or invalidate the cached entry when
activation fails, while preserving reuse of valid cached prompts. Add coverage
for failed activation and recovery from an inactive cached voice.
1 participant
Reason: Add MiniMax cloud speech generation to the TTS engine registry with the current speech models, global/CN endpoint selection, request controls, and audio formats.
What this adds
A new cloud TTS engine that plugs into the existing
TTSBackendprotocol, solong-text chunking, trimming and preset-voice profile handling all work
unchanged. Nothing is downloaded — each request is a single HTTP call.
backend/backends/minimax_backend.py— newMiniMaxTTSBackend:MINIMAX_TTS_MODELS):speech-2.8-hd(default),speech-2.8-turbo,speech-2.6-hd,speech-2.6-turbo,speech-02-hd,speech-02-turbo,speech-01-hd,speech-01-turbo. Unknown model idsfall back to the default.
MINIMAX_API_REGION:global_en(
https://api.minimax.io/v1/t2a_v2, default) andcn_zh(
https://api.minimaxi.com/v1/t2a_v2). Unknown/empty values fall back tothe global host.
voice_setting(voice id, speed,vol, pitch),
audio_setting(sample rate, format, channel),language_boost(derived from the request language,autofallback) andoutput_format.MINIMAX_AUDIO_FORMATS):mp3,wav,flac,pcm(default).
pcmis decoded directly to float32; container formats aredecoded through soundfile.
MINIMAX_API_KEYor~/.env.local, and API errors (base_resp.status_code) surface as clearruntime errors.
backend/backends/__init__.py— registersminimaxinTTS_ENGINES, wiresthe factory, and reports a clear API-key error from
ensure_model_cached_or_raiseinstead of a download message.backend/models.py— allowsengine="minimax"onGenerationRequest.backend/services/profiles.py/backend/routes/profiles.py— expose thepreset voice ids for validation and the
/profiles/presets/{engine}listing.backend/tests/test_minimax_backend.py— unit tests for region/endpointselection, payload construction, response parsing and audio decoding (no
network calls).
Checks
python -m ruff checkon the new/changed backend files — clean.python -m pytest backend/tests/test_minimax_backend.py— 19 passed.Configuration is required at runtime: set
MINIMAX_API_KEY(and optionallyMINIMAX_API_REGION).Summary by CodeRabbit
New Features
Bug Fixes