feat(algo): let any model run — litellm-backed context window + base-name capability tables

[shmulc8]

Goal

Modernize model handling so any model runs without a per-model code edit, and stop the "add a model to the list" PR treadmill — while keeping the explicit, tested context-window values the project relies on.

Changes

1. get_max_tokens never fails; litellm is the safety net

Resolution order:

MAX_TOKENS override (exact or base model name)
 -> litellm's built-in, self-updating registry (exact or base model name)
 -> config.custom_model_max_tokens
 -> DEFAULT_MODEL_MAX_TOKENS (conservative) + warning

An unknown model used to raise; now it runs. The table stays the authoritative, deterministic source for the models it lists (checked first); litellm + default only cover models that used to error out — so a brand-new model works with no edit. Direct MAX_TOKENS[...] readers route through get_max_tokens() (cap=False for the uncapped-window callers).

2. Capability tables keyed by base model name

base_model_name() strips provider prefix, Bedrock region/vendor infixes, and cloud-version/date suffixes, so every spelling collapses to one key. Capability sets drop from ~45–50 provider-prefixed rows to a handful of base names, matched via base_model_name(model) in <set> — adding a model is one line per capability instead of one per provider×region, closing the silent gap where a missing prefix (e.g. bedrock/eu.) sent an unsupported arg. Same normalizer keys the litellm lookup (bedrock/us.anthropic.claude-sonnet-4-6-v1:0 → claude-sonnet-4-6).

3. Corrected stale context windows (verified vs OpenAI/Anthropic docs + litellm)

• gpt-5/5.1 200K→272K; gpt-5.2/5.2-codex/5.3-codex 400K→272K (272K is the enforced input cap — 400K over-budgeted and risked 400s); gpt-5.1-chat-latest 200K→128K (200K exceeded the model).
• o1/o1-2024-12-17/o3-mini 204800→200000 (204800 exceeded the real limit).
• claude-sonnet-4-6/claude-opus-4-6 (all spellings) 200K→1M.

4. Dropped retired / unused models

Whole Claude 3.x line (incl. 3.7), Claude 2/instant, retired Opus 4 / Sonnet 4 20250514 snapshots, dead gpt-3.5 snapshots. litellm + never-fail still serve any stragglers. claude-3-7-sonnet also removed from CLAUDE_EXTENDED_THINKING_MODELS.

5. litellm bump 1.84.0 → 1.91.0

Why keep the table instead of fully delegating to litellm?

Verified against litellm's live registry + official docs:

• litellm has data bugs — reports gpt-5.4-mini/nano at 1.05M (real: 400K; it copied the flagship's number). Kept as explicit overrides.
• litellm's map is fetched from the network at import, falling back to a pinned bundle offline — full delegation would make the default model's budget non-deterministic (what test_get_max_tokens.py pins).
• litellm's capability data is wrong — it lists Opus 4.7/4.8, Sonnet 5, Fable 5 as supporting temperature, but Anthropic's docs state those reject non-default temperature/top_p/top_k with a 400 on every request. So the capability lists are correct and stay.

litellm is the safety net for unknowns, not the source of truth for knowns — the pattern Aider uses.

Tests

• New test_base_model_name.py; test_litellm_claude_extended_thinking.py updated for set storage; test_get_max_tokens.py updated so the unknown-model case asserts the default instead of raising, and sonnet-4-6/opus-4-6 assert the corrected 1M.

🤖 Generated with Claude Code

[qodo-free-for-open-source-projects]

PR Summary by Qodo

refactor(algo): key model capability tables by base model name

✨ Enhancement 🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

AI Description

• Introduces base_model_name() normalizer that strips provider prefixes, Bedrock region/vendor
 infixes, and cloud-version/date suffixes so all provider spellings of a model collapse to one
 canonical key.
• Converts all capability lists (USER_MESSAGE_ONLY_MODELS, NO_SUPPORT_TEMPERATURE_MODELS,
 CLAUDE_EXTENDED_THINKING_MODELS, SUPPORT_REASONING_EFFORT_MODELS, STREAMING_REQUIRED_MODELS)
 from lists with ~50 provider-prefixed entries to compact base-name sets; all membership checks now
 use base_model_name(model) in .
• Adds a litellm fallback in get_max_tokens() for models absent from MAX_TOKENS, and introduces
 a cap=False parameter for callers that need the uncapped context window; routes all direct
 MAX_TOKENS[...] readers through get_max_tokens().
• Bumps litellm 1.84.0 → 1.91.0 so the fallback recognizes more models.
• Adds test_base_model_name.py covering prefix stripping, suffix normalization, dotted-version
 preservation, and capability-set membership; updates existing extended-thinking tests to reflect
 set-based storage.

Diagram

graph TD
    A["base_model_name()"] --> B["Capability Sets\n__init__.py"]
    A --> C["litellm_ai_handler.py"]
    B --> C
    D["get_max_tokens()\nutils.py"] --> E["litellm registry\nfallback"]
    D --> F["MAX_TOKENS table"]
    C --> D
    G["Tool callers\npr_code_suggestions\npr_help_docs\npr_help_message\npr_similar_issue"] --> D
    H["token_handler.py"] --> D

    subgraph Legend
        direction LR
        _fn["Function"] ~~~ _data[("Data Store")] ~~~ _ext{{"External"}}
    end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Also normalize MAX_TOKENS keys to base names

• ➕ Eliminates the remaining ~300 provider-prefixed entries in MAX_TOKENS
• ➕ Makes the entire token-limit system consistent with capability sets
• ➖ Touches every direct MAX_TOKENS reader simultaneously — higher blast radius
• ➖ Better as a follow-up once get_max_tokens() is the sole lookup path (as the PR notes)

2. Try base_model_name(model) in litellm fallback

• ➕ Increases fallback hit rate for provider-prefixed model strings litellm keys by base name
• ➖ Minor scope addition; litellm's key format is not always predictable

Recommendation: The PR's approach is well-chosen: normalizing at lookup time (rather than at storage time in MAX_TOKENS) is the right incremental step, as the PR description itself notes. The main alternative worth considering is normalizing MAX_TOKENS keys too, but the PR deliberately defers this to avoid touching every direct reader at once — a sound risk-management decision. One minor concern: the litellm fallback in _litellm_max_tokens queries by the original (provider-prefixed) model name, which may miss entries if litellm keys by base name; this could be addressed by also trying base_model_name(model) as a second lookup.

Files changed (11) +137 / -125

Enhancement (1) +20 / -6

utils.pyAdd litellm fallback and cap=False parameter to get_max_tokens() +20/-6

Add litellm fallback and cap=False parameter to get_max_tokens()

• Adds '_litellm_max_tokens()' helper that queries litellm's built-in context-window registry. Extends 'get_max_tokens()' with a 'cap' parameter (default 'True') so callers can request the uncapped window, and adds litellm as a third fallback tier before raising an error for unknown models.

pr_agent/algo/utils.py

Refactor (7) +67 / -115

__init__.pyAdd base_model_name() normalizer and convert capability lists to base-name sets +44/-80

Add base_model_name() normalizer and convert capability lists to base-name sets

• Introduces the 'base_model_name()' function and '_PROVIDER_INFIX' constant to strip provider prefixes, Bedrock region/vendor infixes, and cloud-version/date suffixes from model identifiers. Converts all five capability collections ('USER_MESSAGE_ONLY_MODELS', 'NO_SUPPORT_TEMPERATURE_MODELS', 'SUPPORT_REASONING_EFFORT_MODELS', 'CLAUDE_EXTENDED_THINKING_MODELS', 'STREAMING_REQUIRED_MODELS') from provider-prefixed lists to compact base-name sets, reducing 'NO_SUPPORT_TEMPERATURE_MODELS' from ~45 entries to ~21 and 'CLAUDE_EXTENDED_THINKING_MODELS' from ~50 to 6.

pr_agent/algo/init.py

litellm_ai_handler.pyApply base_model_name() to all capability membership checks in chat_completion +11/-11

Apply base_model_name() to all capability membership checks in chat_completion

• Imports 'base_model_name' and wraps every capability set lookup ('user_message_only_models', 'no_support_temperature_models', 'support_reasoning_models', 'claude_extended_thinking_models', 'streaming_required_models') with 'base_model_name(model) in <set>'. Also normalizes the 'claude_extended_thinking_models_override' config entries to base names so prefixed overrides continue to work.

pr_agent/algo/ai_handlers/litellm_ai_handler.py

token_handler.pyRoute MAX_TOKENS lookup in _calc_claude_tokens through get_max_tokens() +3/-3

Route MAX_TOKENS lookup in _calc_claude_tokens through get_max_tokens()

• Replaces the direct 'MAX_TOKENS[model]' access in '_calc_claude_tokens' with a call to 'get_max_tokens(model)' so the litellm fallback applies here too.

pr_agent/algo/token_handler.py

pr_code_suggestions.pyReplace MAX_TOKENS direct access with get_max_tokens(model, cap=False) +2/-6

Replace MAX_TOKENS direct access with get_max_tokens(model, cap=False)

• Removes the 'MAX_TOKENS' import and replaces the conditional 'MAX_TOKENS[model]' / 'get_max_tokens(model)' pattern with a single 'get_max_tokens(model, cap=False)' call to obtain the uncapped context window.

pr_agent/tools/pr_code_suggestions.py

pr_help_docs.pyReplace MAX_TOKENS direct access with get_max_tokens(model, cap=False) +2/-6

Replace MAX_TOKENS direct access with get_max_tokens(model, cap=False)

• Removes the 'MAX_TOKENS' import and simplifies the token-limit lookup to 'get_max_tokens(model, cap=False)', eliminating the duplicated conditional pattern.

pr_agent/tools/pr_help_docs.py

pr_help_message.pyReplace MAX_TOKENS direct access with get_max_tokens(model, cap=False) +2/-5

Replace MAX_TOKENS direct access with get_max_tokens(model, cap=False)

• Removes the 'MAX_TOKENS' import and replaces the conditional lookup with 'get_max_tokens(model, cap=False)' for the uncapped documentation-fitting use case.

pr_agent/tools/pr_help_message.py

pr_similar_issue.pyReplace direct MAX_TOKENS[MODEL] accesses with get_max_tokens(MODEL) +3/-4

Replace direct MAX_TOKENS[MODEL] accesses with get_max_tokens(MODEL)

• Removes the 'MAX_TOKENS' import and replaces three direct 'MAX_TOKENS[MODEL]' dictionary lookups with 'get_max_tokens(MODEL)' so the litellm fallback applies to the similar-issue token budget checks.

pr_agent/tools/pr_similar_issue.py

Tests (2) +49 / -3

test_base_model_name.pyNew unit tests for base_model_name() normalizer and capability-set membership +45/-0

New unit tests for base_model_name() normalizer and capability-set membership

• Adds 'TestBaseModelName' covering provider-prefix stripping (anthropic/, bedrock/ with region+vendor+version, vertex_ai/), date/cloud-version suffix normalization, dotted-version preservation, and end-to-end capability-set membership checks.

tests/unittest/test_base_model_name.py

test_litellm_claude_extended_thinking.pyUpdate extended-thinking tests to expect set-based storage +4/-3

Update extended-thinking tests to expect set-based storage

• Updates two assertions to compare against '{"custom-claude-model"}' and '{"custom-claude-model", "another-model"}' (sets) instead of lists, and updates the comment to reflect that entries are now normalized to base model names.

tests/unittest/test_litellm_claude_extended_thinking.py

Other (1) +1 / -1

requirements.txtBump litellm 1.84.0 → 1.91.0 +1/-1

Bump litellm 1.84.0 → 1.91.0

• Updates the litellm dependency to 1.91.0 so the new '_litellm_max_tokens' fallback recognizes a broader set of models.

requirements.txt

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (4) 📘 Rule violations (4) 📜 Skill insights (0)

1. MAX_TOKENS uses single quotes 📘 Rule violation ⚙ Maintainability ⭐ New

Description

Modified/added entries in MAX_TOKENS use single-quoted strings, which conflicts with the repo
Python formatting standard (double quotes). This can cause Ruff/format checks to fail and introduce
style drift across the codebase.

Code

pr_agent/algo/init.py[R53-66]

+    'gpt-5-nano': 272000,  # 272K, but may be limited by config.max_model_tokens
+    'gpt-5-mini': 272000,  # 272K, but may be limited by config.max_model_tokens
+    'gpt-5': 272000,
+    'gpt-5-2025-08-07': 272000,
+    'gpt-5.1': 272000,
+    'gpt-5.1-2025-11-13': 272000,
+    'gpt-5.1-chat-latest': 128000,
+    'gpt-5.1-codex': 272000,
+    'gpt-5.1-codex-mini': 272000,
+    'gpt-5.2': 272000,  # 272K, but may be limited by config.max_model_tokens
+    'gpt-5.2-2025-12-11': 272000,  # 272K, but may be limited by config.max_model_tokens
    'gpt-5.2-chat-latest': 128000,  # 128K, but may be limited by config.max_model_tokens
-    'gpt-5.2-codex': 400000,  # 400K, but may be limited by config.max_model_tokens
-    'gpt-5.3-codex': 400000,  # 400K, but may be limited by config.max_model_tokens
+    'gpt-5.2-codex': 272000,  # 272K, but may be limited by config.max_model_tokens
+    'gpt-5.3-codex': 272000,  # 272K, but may be limited by config.max_model_tokens

Evidence

PR Compliance ID 15 requires Python changes to follow repository lint/format standards including
preferring double quotes. The modified MAX_TOKENS entries shown use single quotes on the changed
lines.

AGENTS.md: Python Code Must Conform to Repository Lint/Format Standards (Ruff + isort Conventions, 120 Char Lines, Double Quotes)
pr_agent/algo/init.py[53-66]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Modified lines in `pr_agent/algo/__init__.py` introduce/retain single-quoted string literals in `MAX_TOKENS`, conflicting with the repository Python formatting standard (Ruff formatter prefers double quotes).

## Issue Context
These are modified/added lines in this PR, so they are in-scope for style compliance and may fail CI lint/format checks.

## Fix Focus Areas
- pr_agent/algo/__init__.py[53-66]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. base_model_name import unsorted 📘 Rule violation ⚙ Maintainability

Description

The modified from pr_agent.algo import (...) block is not isort-sorted (e.g., base_model_name is
appended after the constants). This can violate Ruff/isort checks and causes avoidable churn in
future automated formatting runs.

Code

pr_agent/algo/ai_handlers/litellm_ai_handler.py[R14-17]

                         NO_SUPPORT_TEMPERATURE_MODELS,
                         STREAMING_REQUIRED_MODELS,
                         SUPPORT_REASONING_EFFORT_MODELS,
-                           USER_MESSAGE_ONLY_MODELS)
+                           USER_MESSAGE_ONLY_MODELS, base_model_name)

Evidence

PR Compliance ID 14 requires Ruff/isort-conformant import ordering. The updated import block adds
base_model_name but leaves the imported names unsorted within the parenthesized list.

AGENTS.md: Python code must conform to repository lint/style constraints (Ruff: line length 120, import ordering, string style): AGENTS.md: Python code must conform to repository lint/style constraints (Ruff: line length 120, import ordering, string style): AGENTS.md: Python code must conform to repository lint/style constraints (Ruff: line length 120, import ordering, string style): AGENTS.md: Python code must conform to repository lint/style constraints (Ruff: line length 120, import ordering, string style)
pr_agent/algo/ai_handlers/litellm_ai_handler.py[13-17]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The `from pr_agent.algo import (...)` names are not ordered in an isort-compliant way after adding `base_model_name`.
## Issue Context
The repo enables Ruff isort checks (`I001`), so unsorted import members can fail linting.
## Fix Focus Areas
- pr_agent/algo/ai_handlers/litellm_ai_handler.py[13-17]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Bedrock v2 name mangled 🐞 Bug ≡ Correctness

Description

base_model_name() strips any trailing -v: suffix, which incorrectly removes the -v2:1 portion
from Bedrock’s anthropic.claude-v2:1 identifier and collapses it to claude. This can cause
base-name matching to miss for plausible regional spellings (e.g.
bedrock/us.anthropic.claude-v2:1), leading capability gating and token-window selection to fall
back to the wrong model key.

Code

pr_agent/algo/init.py[R16-22]

+    m = model.split("/", 1)[-1]           # drop provider prefix (anthropic/, bedrock/, vertex_ai/, ...)
+    m = m.replace("@", "-")               # vertex date separator -> '-' (matches anthropic spelling)
+    m = re.sub(r"-v\d+:\d+$", "", m)      # bedrock cloud-version suffix (-v1:0)
+    parts = m.split(".")
+    while len(parts) > 1 and parts[0] in _PROVIDER_INFIX:   # bedrock region + vendor infixes
+        parts.pop(0)
+    return ".".join(parts)                # dotted model versions (gpt-5.2) are preserved

Evidence

The regex re.sub(r"-v\d+:\d+$", "", m) will strip -v2:1 from anthropic.claude-v2:1, and the
subsequent dot-splitting/popping removes anthropic, yielding the incorrect base name claude. The
repo’s token table includes bedrock/anthropic.claude-v2:1, demonstrating -v2:1 is part of the
model identifier; and existing tests show Bedrock regional spellings (bedrock/us.*,
bedrock/eu.*, etc.) are expected inputs, making a regional ...claude-v2:1 spelling plausible.

pr_agent/algo/init.py[9-22]
pr_agent/algo/init.py[147-176]
tests/unittest/test_litellm_chat_completion_core.py[76-89]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`base_model_name()` currently removes any suffix matching `-v\d+:\d+$`. This pattern matches Bedrock Claude v2 identifiers like `anthropic.claude-v2:1`, which are *model identity* (not deploy decoration), causing normalization to incorrectly return `claude`.
### Issue Context
This normalization is used for capability gating and for best-effort model registry lookups. Incorrect normalization can make lookups miss and fall back to defaults.
### Fix Focus Areas
- pr_agent/algo/__init__.py[16-22]
- tests/unittest/test_base_model_name.py[1-45]
### Suggested change
- Restrict the Bedrock deploy-suffix removal to the known deploy pattern (e.g. `-v<digits>:0`), or otherwise ensure `claude-v2:1` is preserved.
- Add a unit test asserting:
- `base_model_name("bedrock/anthropic.claude-v2:1") == "claude-v2:1"`
- `base_model_name("bedrock/us.anthropic.claude-v2:1") == "claude-v2:1"`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View more (1)

4. Stacked prefix not stripped 🐞 Bug ≡ Correctness

Description

base_model_name() strips only the first provider prefix segment, so stacked names like
azure/openai/ normalize to openai/ instead of the intended base model name. As a result,
LiteLLMAIHandler capability gates that now rely on base_model_name(model) in  can fail to apply
(e.g., skipping required streaming or incorrectly adding unsupported params) when stacked prefixes
occur.

Code

pr_agent/algo/init.py[R16-22]

+    m = model.split("/", 1)[-1]           # drop provider prefix (anthropic/, bedrock/, vertex_ai/, ...)
+    m = m.replace("@", "-")               # vertex date separator -> '-' (matches anthropic spelling)
+    m = re.sub(r"-v\d+:\d+$", "", m)      # bedrock cloud-version suffix (-v1:0)
+    parts = m.split(".")
+    while len(parts) > 1 and parts[0] in _PROVIDER_INFIX:   # bedrock region + vendor infixes
+        parts.pop(0)
+    return ".".join(parts)                # dotted model versions (gpt-5.2) are preserved

Evidence

base_model_name() only removes the first slash-delimited prefix, but LiteLLMAIHandler can create
stacked prefixes by prepending azure/ to a model that may already be prefixed (and even documents
azure/openai/...). Since capability gates now use base_model_name(model) for set membership, an
incompletely normalized value (e.g., openai/qwq-plus) will not match base-name sets (e.g.,
qwq-plus).

pr_agent/algo/init.py[9-22]
pr_agent/algo/ai_handlers/litellm_ai_handler.py[440-477]
pr_agent/algo/ai_handlers/litellm_ai_handler.py[510-559]
pr_agent/algo/ai_handlers/litellm_ai_handler.py[640-646]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`base_model_name()` currently removes only one leading `<provider>/` segment via `model.split("/", 1)[-1]`. However, the codebase can produce/accept stacked prefixes (e.g., `azure/openai/<model>`), which then normalize incorrectly and cause capability-set membership checks to miss.
### Issue Context
`LiteLLMAIHandler.chat_completion()` can prepend `azure/` to an existing model string, and the file explicitly documents stacked-prefix cases. Capability checks for user-message-only, temperature support, reasoning_effort, extended thinking, and streaming now depend on `base_model_name()`, so incomplete normalization breaks these checks.
### Fix Focus Areas
- pr_agent/algo/__init__.py[9-22]
- tests/unittest/test_base_model_name.py[9-45]
### What to change
1. Update `base_model_name()` to repeatedly strip *known provider prefixes* (e.g., `azure`, `openai`, `anthropic`, `bedrock`, `vertex_ai`, `huggingface`, etc.) while the leftmost segment matches a known provider.
- Use a loop like `prefix, _, rest = m.partition('/')` and only drop `prefix` when it’s in a `KNOWN_PROVIDERS` set.
- This preserves non-provider slashes like HuggingFace org/model (`meta-llama/Llama-2...`) while still handling `huggingface/meta-llama/Llama-2...`.
2. Add a unit test that asserts stacked prefixes normalize correctly, e.g.:
- `base_model_name("azure/openai/qwq-plus") == "qwq-plus"`
- optionally add a regression check that `base_model_name("huggingface/meta-llama/Llama-2-7b-chat-hf") == "meta-llama/Llama-2-7b-chat-hf"` (or whatever behavior you intend for that case).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

5. qwq-plus max tokens mismatch 🐞 Bug ≡ Correctness ⭐ New

Description

STREAMING_REQUIRED_MODELS is keyed by the base model name qwq-plus, but MAX_TOKENS only
defines a provider-prefixed key openai/qwq-plus. If the configured model is qwq-plus,
get_max_tokens() will miss the explicit override and may fall back to litellm/default,
under-budgeting the context window and causing unnecessary prompt clipping.

Code

pr_agent/algo/init.py[R246-248]

+STREAMING_REQUIRED_MODELS = {
+    "qwq-plus",
+}

Evidence

MAX_TOKENS contains the provider-prefixed key openai/qwq-plus but no base-name key, while
STREAMING_REQUIRED_MODELS uses the base name qwq-plus and base_model_name explicitly
normalizes openai/qwq-plus to qwq-plus. Since get_max_tokens only checks exact match and then
base_model_name(model), a configuration of model="qwq-plus" will not hit the explicit override.

pr_agent/algo/init.py[81-99]
pr_agent/algo/init.py[245-248]
pr_agent/algo/utils.py[1029-1032]
tests/unittest/test_base_model_name.py[34-38]
pr_agent/settings/configuration.toml[5-10]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`qwq-plus` is now treated as a base-name capability key, but the token override table only contains `openai/qwq-plus`. When users configure `model = "qwq-plus"` (consistent with other unprefixed OpenAI model names), `get_max_tokens()` will not find the table override.

## Issue Context
- `base_model_name("openai/qwq-plus") == "qwq-plus"`
- `get_max_tokens()` checks `MAX_TOKENS.get(model)` and then `MAX_TOKENS.get(base_model_name(model))`

## Fix Focus Areas
- pr_agent/algo/__init__.py[81-100]
- pr_agent/algo/__init__.py[245-248]

## Expected fix
- Add `"qwq-plus": 131072` to `MAX_TOKENS` (or otherwise ensure base-name and provider-prefixed spellings map to the same explicit override) so `model="qwq-plus"` and `model="openai/qwq-plus"` resolve to the same context window deterministically.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Hardcoded DEFAULT_MODEL_MAX_TOKENS fallback 📘 Rule violation ⚙ Maintainability

Description

get_max_tokens() introduces a hardcoded default context window (`DEFAULT_MODEL_MAX_TOKENS =
32000) that changes runtime behavior when a model is unknown to both MAX_TOKENS and litellm`.
This should be driven via Dynaconf defaults (and optionally repo overrides) rather than a fixed
constant in code.

Code

pr_agent/algo/utils.py[R992-1039]

+# Context window used when a model is unknown to both the MAX_TOKENS overrides and litellm.
+# Conservative but non-zero so any model still runs (token clipping stays sane) rather than failing.
+DEFAULT_MODEL_MAX_TOKENS = 32000
+
+
+def _litellm_max_tokens(model):
+    """Context window from litellm's built-in, self-updating registry; None if litellm doesn't know it.
+
+    Tries the exact model string first, then the base model name (see base_model_name) since litellm
+    keys its registry by canonical names and often doesn't recognize provider-prefixed / region /
+    cloud-version spellings (e.g. 'bedrock/us.anthropic.claude-sonnet-4-6-v1:0' -> 'claude-sonnet-4-6').
"""
-    Get the maximum number of tokens allowed for a model.
-    logic:
-    (1) If the model is in './pr_agent/algo/__init__.py', use the value from there.
-    (2) else, the user needs to define explicitly 'config.custom_model_max_tokens'
+    import litellm
+    for name in (model, base_model_name(model)):
+        try:
+            info = litellm.get_model_info(name)
+            ctx = info.get("max_input_tokens") or info.get("max_tokens")
+            if ctx:
+                return ctx
+        except Exception:
+            continue
+    return None
-    For both cases, we further limit the number of tokens to 'config.max_model_tokens' if it is set.
+
+def get_max_tokens(model, cap=True):
+    """
+    Get the maximum number of input tokens allowed for a model. Resolution order:
+    (1) MAX_TOKENS in './pr_agent/algo/__init__.py' — a small override layer for values litellm
+        gets wrong or lacks (matched by exact then base model name).
+    (2) litellm's built-in, self-updating model registry (exact then base model name).
+    (3) 'config.custom_model_max_tokens' if set (>0) — user-declared budget for private/unknown models.
+    (4) DEFAULT_MODEL_MAX_TOKENS with a warning — never raises, so any model runs.
+
+    When cap=True, further limit the number of tokens to 'config.max_model_tokens' if it is set.
This aims to improve the algorithmic quality, as the AI model degrades in performance when the input is too long.
+    Pass cap=False when the uncapped context window is wanted (e.g. fitting full documentation into the prompt).
"""
settings = get_settings()
-    if model in MAX_TOKENS:
-        max_tokens_model = MAX_TOKENS[model]
-    elif settings.config.custom_model_max_tokens > 0:
-        max_tokens_model = settings.config.custom_model_max_tokens
-    else:
-        get_logger().error(f"Model {model} is not defined in MAX_TOKENS in ./pr_agent/algo/__init__.py and no custom_model_max_tokens is set")
-        raise Exception(f"Ensure {model} is defined in MAX_TOKENS in ./pr_agent/algo/__init__.py or set a positive value for it in config.custom_model_max_tokens")
+    max_tokens_model = (MAX_TOKENS.get(model) or MAX_TOKENS.get(base_model_name(model))
+                        or _litellm_max_tokens(model))
+    if not max_tokens_model:
+        if settings.config.custom_model_max_tokens > 0:
+            max_tokens_model = settings.config.custom_model_max_tokens
+        else:
+            max_tokens_model = DEFAULT_MODEL_MAX_TOKENS
+            get_logger().warning(
+                f"Model {model} is unknown to MAX_TOKENS and litellm; using default {DEFAULT_MODEL_MAX_TOKENS}. "
+                f"Set config.custom_model_max_tokens to specify its context window.")

Evidence

The PR adds a new hardcoded default (DEFAULT_MODEL_MAX_TOKENS = 32000) and uses it as the fallback
when both MAX_TOKENS and litellm do not recognize the model. This introduces new behavior
controlled by a literal rather than a Dynaconf setting, violating the configuration-driven
requirement.

AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values): AGENTS.md: Configuration Must Be Driven by Dynaconf Settings Files (Not Hardcoded Values)
pr_agent/algo/utils.py[992-1039]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`DEFAULT_MODEL_MAX_TOKENS = 32000` is a behavior-affecting fallback hardcoded in `pr_agent/algo/utils.py`. Per repo compliance, new runtime behavior defaults should be configurable via Dynaconf (defaults in `pr_agent/settings/configuration.toml` with optional `.pr_agent.toml` overrides).
## Issue Context
This value is used when a model is unknown to both `MAX_TOKENS` and `litellm`, so it can materially affect token clipping and prompt sizing across the app.
## Fix Focus Areas
- pr_agent/algo/utils.py[992-1043]
- pr_agent/settings/configuration.toml[34-44]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. _litellm_max_tokens() masks exceptions 📘 Rule violation ⛨ Security

Description

_litellm_max_tokens() catches Exception and returns None, which can silently hide unexpected
failures (e.g., internal litellm errors) and make debugging/token-limit behavior inconsistent.
This violates the requirement to avoid broad exception masking at configuration/logging boundaries.

Code

pr_agent/algo/utils.py[R992-999]

+def _litellm_max_tokens(model):
+    """Context window from litellm's built-in registry; None if litellm doesn't know the model."""
+    try:
+        import litellm
+        info = litellm.get_model_info(model)
+        return info.get("max_input_tokens") or info.get("max_tokens")
+    except Exception:
+        return None

Evidence

PR Compliance ID 21 requires avoiding broad exception masking. The new _litellm_max_tokens()
implementation catches Exception unconditionally and returns None, suppressing all errors from
litellm.get_model_info() (including unexpected ones).

pr_agent/algo/utils.py[992-999]
Best Practice: Learned patterns

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`_litellm_max_tokens()` uses a broad `except Exception:` and returns `None`, which can mask unexpected errors and reduce observability.
## Issue Context
The code is used as a fallback for determining model context windows; swallowing unexpected exceptions can hide real failures and lead to confusing downstream behavior.
## Fix Focus Areas
- pr_agent/algo/utils.py[992-999]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

8. Repeated token lookup in loop 🐞 Bug ➹ Performance

Description

pr_similar_issue now calls get_max_tokens(MODEL) inside per-issue and per-comment loops, adding
repeated settings/function overhead where the value is constant for the run. This can slow down
large index ingestions unnecessarily.

Code

pr_agent/tools/pr_similar_issue.py[R639-641]

+                                self.token_handler.count_tokens(comment_body) < get_max_tokens(MODEL):
                          comment_record = Record(
                              id=issue_key + ".comment_" + str(j + 1),

Evidence

The ingestion loop calls get_max_tokens(MODEL) for each issue and comment. get_max_tokens()
begins by calling get_settings() and then performs resolution logic, which is unnecessary to
repeat when MODEL is constant.

pr_agent/tools/pr_similar_issue.py[411-446]
pr_agent/algo/utils.py[1016-1033]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`get_max_tokens(MODEL)` is invoked repeatedly inside loops processing many issues/comments, even though `MODEL` is constant.
### Issue Context
`get_max_tokens()` performs a settings lookup and additional logic; repeating it thousands of times is avoidable.
### Fix Focus Areas
- pr_agent/tools/pr_similar_issue.py[411-446]
### Suggested change
- Compute `max_tokens = get_max_tokens(MODEL)` once per function (or per batch) and reuse it in the loop conditions.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

_{ⓘ  1 issues published inline · 2 in summary}

• Author self-review: I have reviewed the code review findings, and addressed the relevant ones.

[qodo-free-for-open-source-projects]

2. Stacked prefix not stripped 🐞 Bug ≡ Correctness

base_model_name() strips only the first provider prefix segment, so stacked names like
azure/openai/<model> normalize to openai/<model> instead of the intended base model name. As a
result, LiteLLMAIHandler capability gates that now rely on base_model_name(model) in <set> can
fail to apply (e.g., skipping required streaming or incorrectly adding unsupported params) when
stacked prefixes occur.

Agent Prompt

### Issue description
`base_model_name()` currently removes only one leading `<provider>/` segment via `model.split("/", 1)[-1]`. However, the codebase can produce/accept stacked prefixes (e.g., `azure/openai/<model>`), which then normalize incorrectly and cause capability-set membership checks to miss.

### Issue Context
`LiteLLMAIHandler.chat_completion()` can prepend `azure/` to an existing model string, and the file explicitly documents stacked-prefix cases. Capability checks for user-message-only, temperature support, reasoning_effort, extended thinking, and streaming now depend on `base_model_name()`, so incomplete normalization breaks these checks.

### Fix Focus Areas
- pr_agent/algo/__init__.py[9-22]
- tests/unittest/test_base_model_name.py[9-45]

### What to change
1. Update `base_model_name()` to repeatedly strip *known provider prefixes* (e.g., `azure`, `openai`, `anthropic`, `bedrock`, `vertex_ai`, `huggingface`, etc.) while the leftmost segment matches a known provider.
   - Use a loop like `prefix, _, rest = m.partition('/')` and only drop `prefix` when it’s in a `KNOWN_PROVIDERS` set.
   - This preserves non-provider slashes like HuggingFace org/model (`meta-llama/Llama-2...`) while still handling `huggingface/meta-llama/Llama-2...`.
2. Add a unit test that asserts stacked prefixes normalize correctly, e.g.:
   - `base_model_name("azure/openai/qwq-plus") == "qwq-plus"`
   - optionally add a regression check that `base_model_name("huggingface/meta-llama/Llama-2-7b-chat-hf") == "meta-llama/Llama-2-7b-chat-hf"` (or whatever behavior you intend for that case).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[qodo-free-for-open-source-projects]

Code review by qodo was updated up to the latest commit 684d529

[qodo-free-for-open-source-projects]

1. base_model_name import unsorted 📘 Rule violation ⚙ Maintainability

The modified from pr_agent.algo import (...) block is not isort-sorted (e.g., base_model_name is
appended after the constants). This can violate Ruff/isort checks and causes avoidable churn in
future automated formatting runs.

Agent Prompt

## Issue description
The `from pr_agent.algo import (...)` names are not ordered in an isort-compliant way after adding `base_model_name`.

## Issue Context
The repo enables Ruff isort checks (`I001`), so unsorted import members can fail linting.

## Fix Focus Areas
- pr_agent/algo/ai_handlers/litellm_ai_handler.py[13-17]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[qodo-free-for-open-source-projects]

2. Bedrock v2 name mangled 🐞 Bug ≡ Correctness

base_model_name() strips any trailing -v<digits>:<digits> suffix, which incorrectly removes the
-v2:1 portion from Bedrock’s anthropic.claude-v2:1 identifier and collapses it to claude. This
can cause base-name matching to miss for plausible regional spellings (e.g.
bedrock/us.anthropic.claude-v2:1), leading capability gating and token-window selection to fall
back to the wrong model key.

Agent Prompt

### Issue description
`base_model_name()` currently removes any suffix matching `-v\d+:\d+$`. This pattern matches Bedrock Claude v2 identifiers like `anthropic.claude-v2:1`, which are *model identity* (not deploy decoration), causing normalization to incorrectly return `claude`.

### Issue Context
This normalization is used for capability gating and for best-effort model registry lookups. Incorrect normalization can make lookups miss and fall back to defaults.

### Fix Focus Areas
- pr_agent/algo/__init__.py[16-22]
- tests/unittest/test_base_model_name.py[1-45]

### Suggested change
- Restrict the Bedrock deploy-suffix removal to the known deploy pattern (e.g. `-v<digits>:0`), or otherwise ensure `claude-v2:1` is preserved.
- Add a unit test asserting:
  - `base_model_name("bedrock/anthropic.claude-v2:1") == "claude-v2:1"`
  - `base_model_name("bedrock/us.anthropic.claude-v2:1") == "claude-v2:1"`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[qodo-free-for-open-source-projects]

Code review by qodo was updated up to the latest commit d73a9d1

[qodo-free-for-open-source-projects]

1. max_tokens uses single quotes 📘 Rule violation ⚙ Maintainability

Modified/added entries in MAX_TOKENS use single-quoted strings, which conflicts with the repo
Python formatting standard (double quotes). This can cause Ruff/format checks to fail and introduce
style drift across the codebase.

Agent Prompt

## Issue description
Modified lines in `pr_agent/algo/__init__.py` introduce/retain single-quoted string literals in `MAX_TOKENS`, conflicting with the repository Python formatting standard (Ruff formatter prefers double quotes).

## Issue Context
These are modified/added lines in this PR, so they are in-scope for style compliance and may fail CI lint/format checks.

## Fix Focus Areas
- pr_agent/algo/__init__.py[53-66]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[qodo-free-for-open-source-projects]

Code review by qodo was updated up to the latest commit 3943da6