Skip to content

Failed to initialize Weave tracing #6320

Description

@suhaibmujahid

Regressed by #6288

2026-07-16T00:20:36.544910Z		2026-07-16 00:20:36,543 - uvicorn.error - INFO - Started server process [1]
2026-07-16T00:20:36.544935Z		2026-07-16 00:20:36,543 - uvicorn.error - INFO - Waiting for application startup.
2026-07-16T00:20:36.581801Z		2026-07-16 00:20:36,580 - mcp.server.streamable_http_manager - INFO - StreamableHTTP session manager started
2026-07-16T00:20:36.581815Z		2026-07-16 00:20:36,580 - bugzilla-broker - INFO - bugzilla broker ready on 0.0.0.0:8765 (read-only)
2026-07-16T00:20:36.581820Z		2026-07-16 00:20:36,580 - uvicorn.error - INFO - Application startup complete.
2026-07-16T00:20:36.582137Z		2026-07-16 00:20:36,581 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8765 (Press CTRL+C to quit)
2026-07-16T00:20:40.168588Z	INFO	Default STARTUP TCP probe succeeded after 3 attempts for container "broker" on port 8765.
2026-07-16T00:20:44.597398Z		2026-07-16 00:20:44,595 - hackbot_runtime.anthropic_wif - INFO - Anthropic auth: GCP Workload Identity Federation (identity token file /tmp/anthropic-wif-x6gjrqbd/identity-token, refresh every 1800s)
2026-07-16T00:20:44.597412Z		2026-07-16 00:20:44,595 - hackbot_runtime - INFO - Configured Anthropic WIF authentication
2026-07-16T00:20:44.627082Z		2026-07-16 00:20:44,625 - hackbot_runtime.wandb_wif - INFO - W&B auth: GCP Workload Identity Federation (identity token file /tmp/wandb-wif-ytr9updh/identity-token, refresh every 1800s)
2026-07-16T00:20:44.627096Z		2026-07-16 00:20:44,625 - hackbot_runtime - INFO - Configured W&B WIF authentication
2026-07-16T00:20:49.173572Z		�[36m�[1mweave�[0m: Please login to Weights & Biases (https://wandb.ai) to continue...
2026-07-16T00:20:49.173587Z		2026-07-16 00:20:49,171 - weave.trace.weave_init - INFO - Please login to Weights & Biases (https://wandb.ai) to continue...
2026-07-16T00:20:49.173757Z		�[36m�[1mweave�[0m: Logging into Weights & Biases
2026-07-16T00:20:49.173763Z		2026-07-16 00:20:49,171 - weave.compat.wandb.wandb_thin.login - INFO - Logging into Weights & Biases
2026-07-16T00:20:49.173767Z		�[36m�[1mweave�[0m: You can find your API key in your browser here: https://wandb.ai/authorize?ref=weave
2026-07-16T00:20:49.173771Z		2026-07-16 00:20:49,171 - weave.compat.wandb.wandb_thin.login - INFO - You can find your API key in your browser here: https://wandb.ai/authorize?ref=weave
2026-07-16T00:20:49.173953Z		�[36m�[1mweave�[0m: Paste an API key from your profile and hit enter:
2026-07-16T00:20:49.173958Z		2026-07-16 00:20:49,171 - weave.compat.wandb.wandb_thin.login - INFO - Paste an API key from your profile and hit enter:
2026-07-16T00:20:49.174412Z		/usr/local/lib/python3.12/getpass.py:91: GetPassWarning: Can not control echo on the terminal.
2026-07-16T00:20:49.174423Z		  passwd = fallback_getpass(prompt, stream)
2026-07-16T00:20:49.174433Z		Warning: Password input may be echoed.
2026-07-16T00:20:49.174576Z		Enter your Weights & Biases API key: 
2026-07-16T00:20:49.224679Z		2026-07-16 00:20:49,222 - httpx - INFO - HTTP Request: GET https://trace.wandb.ai/server_info "HTTP/1.1 200 OK"
2026-07-16T00:20:49.334526Z		2026-07-16 00:20:49,332 - httpx - INFO - HTTP Request: POST https://api.wandb.ai/graphql "HTTP/1.1 200 OK"
2026-07-16T00:20:49.363493Z		2026-07-16 00:20:49,361 - httpx - INFO - HTTP Request: POST https://api.wandb.ai/graphql "HTTP/1.1 401 Unauthorized"
2026-07-16T00:20:49.364058Z		�[36m�[1mweave�[0m: Unable to access `moz-bugbug/hackbot-dev`.
2026-07-16T00:20:49.364064Z		2026-07-16 00:20:49,361 - weave.wandb_interface.project_creator - ERROR - Unable to access `moz-bugbug/hackbot-dev`.
2026-07-16T00:20:49.364116Z		�[36m�[1mweave�[0m: {'message': 'user is not logged in', 'path': ['upsertModel'], 'extensions': {'code': 'PERMISSION_ERROR'}}
2026-07-16T00:20:49.364122Z		2026-07-16 00:20:49,362 - weave.wandb_interface.project_creator - ERROR - {'message': 'user is not logged in', 'path': ['upsertModel'], 'extensions': {'code': 'PERMISSION_ERROR'}}
2026-07-16T00:20:49.367524Z		2026-07-16 00:20:49,362 - hackbot_runtime - ERROR - Failed to initialize Weave tracing; continuing without it
2026-07-16T00:20:49.367618Z	ERROR	Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/hackbot_runtime/tracing.py", line 60, in _init_weave
    weave.init(project)
  File "/opt/venv/lib/python3.12/site-packages/weave/trace/api.py", line 176, in init
    return weave_init.init_weave(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/weave/trace/weave_init.py", line 285, in init_weave
    client = weave_client.WeaveClient(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/weave/trace/weave_client.py", line 480, in __init__
    resp = self.server.ensure_project_exists(entity, project)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/weave/trace_server_bindings/remote_http_trace_server.py", line 106, in ensure_project_exists
    project_creator.ensure_project_exists(entity, project)
  File "/opt/venv/lib/python3.12/site-packages/weave/wandb_interface/project_creator.py", line 52, in ensure_project_exists
    return _ensure_project_exists(entity_name, project_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/weave/wandb_interface/project_creator.py", line 162, in _ensure_project_exists
    _raise_project_access_error(entity_name, project_name, exception)
  File "/opt/venv/lib/python3.12/site-packages/weave/wandb_interface/project_creator.py", line 108, in _raise_project_access_error
    raise UnableToCreateProjectError(
weave.wandb_interface.project_creator.UnableToCreateProjectError: You do not have permission to access or create project `moz-bugbug/hackbot-dev`. Your API key may belong to a service account or user that is not a member of the `moz-bugbug` team, or that lacks write access. Verify the key's team/entity scope.

Metadata

Metadata

Assignees

Labels

Type

Fields

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions