Skip to content

feat: add password-based authentication support - #250

Merged
echarles merged 1 commit into
datalayer:mainfrom
EnyMan:feat/password-auth
Jul 30, 2026
Merged

feat: add password-based authentication support#250
echarles merged 1 commit into
datalayer:mainfrom
EnyMan:feat/password-auth

Conversation

@EnyMan

@EnyMan EnyMan commented May 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds password-based authentication as an alternative to token auth, enabling the MCP server to work with Jupyter deployments that use password login and XSRF protection.

Related issues: Partially addresses #183 (XSRF-protected Jupyter deployments without Bearer tokens), related to #61 (WebSocket authentication failures)

Changes

  • New JupyterPasswordAuth class (auth.py) handles the /login flow: obtains XSRF cookie, POSTs credentials, verifies the session, and keeps a requests.Session alive so subsequent callers see live cookie values rather than a login-time snapshot.
  • CLI options --runtime-password, --document-password, --jupyter-password (with corresponding env vars RUNTIME_PASSWORD, DOCUMENT_PASSWORD, JUPYTER_PASSWORD) follow the same priority/fallback pattern as existing token options. Password takes precedence over token when both are set for the same server.
  • Auth cookies are injected into JupyterServerClient, KernelClient, and the collaboration WebSocket connection. The fresh X-XSRFToken header is read per-request from the live cookie jar, so cookie rotation is handled correctly.
  • Two separate auth instances: _runtime_password_auth for kernel operations and _document_password_auth for collaboration. When runtime_url == document_url the runtime auth is reused; when URLs differ and --document-password is set, a second login is performed. A warning is logged when the document server is genuinely different and only --runtime-password is set.
  • The collaboration WebSocket now uses jupyter-nbmodel-client's native additional_headers (NbModelClient) and headers (get_notebook_websocket_url) parameters to inject the Cookie/XSRF auth — see "Removed monkey-patch" below.
  • Automatic re-login on session expiry: Jupyter session cookies expire (30 days by default), so long-running sessions could go stale. NotebookConnection now catches 401/403 on the collaboration-session request, re-authenticates (ServerContext.relogin_document()JupyterPasswordAuth.relogin()), re-injects the fresh cookies, and retries once.
  • Refactored duplicated _init_mcp_server_mode logic in server_context.py into a single method with proper resource cleanup on partial failure. ServerContext.reset() now closes auth sessions to release connection pools.
  • The CLI decorator was split into _remote_connection_options (the subset that connect can forward over /api/connect) and _connection_options (adds local-only password options used by start). This also fixed a pre-existing TypeError in connect_command on main.
  • Comprehensive test suite (test_auth.py) covering the login flow (success, timeout, connection error at each stage, 5xx, 401/403, missing-xsrf, unexpected-status, response-body propagation), header generation, session injection, config normalization, dual runtime/document auth scenarios, session expiry / re-login, and e2e tests against a real password-protected Jupyter server.

How it works

  1. On startup, if a password is configured, ServerContext creates a JupyterPasswordAuth instance and calls .login() to authenticate.
  2. The resulting session cookies are injected into JupyterServerClient's HTTP session, and the auth's own requests.Session is kept alive so cookies remain readable for the document side.
  3. For KernelClient and collaboration API requests, auth headers (Cookie + X-XSRFToken) are passed via the headers kwarg and computed per-request from the live cookie jar.
  4. The runtime_auth_headers and document_auth_headers properties on ServerContext always read the latest cookies from the active session (not the stale login-time snapshot).
  5. If a collaboration request returns 401/403 (expired cookie), the connection re-authenticates and retries once with the fresh cookies.

Removed monkey-patch

Earlier revisions of this PR monkey-patched the module-level connect imported in jupyter_nbmodel_client.client to inject the Cookie header during the WebSocket handshake, because NbModelClient exposed no way to pass extra headers. That patch was not concurrency-safe.

This is now resolved upstream: jupyter-nbmodel-client#61 added additional_headers= to NbModelClient and headers= to get_notebook_websocket_url (released in jupyter-nbmodel-client 0.14.8). The monkey-patch and the re-implemented URL helper have been removed; the minimum dependency is bumped to >=0.14.8.

Known limitation

The automatic re-login currently covers the collaboration / document path (NotebookConnection). Kernel-management HTTP requests that go through jupyter_server_client internals are not yet wrapped in the same retry; this can be a follow-up.

Test plan

  • uv run --extra test pytest tests/test_auth.py passes (unit + e2e, including session-expiry/re-login tests against a real password-protected Jupyter server)
  • Test with a password-protected Jupyter server: jupyter lab --IdentityProvider.token='' --ServerApp.password='argon2:...'
  • Verify token-based auth still works unchanged (backward compatible)
  • Verify that --jupyter-password falls back correctly when individual passwords are not set
  • Verify that jupyter-mcp-server connect --help shows no --*-password options (passwords are local-only; not forwardable over /api/connect)

@echarles

Copy link
Copy Markdown
Member

NbModelClient (from jupyter-nbmodel-client) does not expose an additional_headers parameter for its WebSocket connection. As a workaround, NotebookConnection.aenter temporarily monkey-patches the connect function to inject the Cookie header during WebSocket establishment. This is thread-safe (restore happens in a finally block) but ideally should be replaced by an upstream additional_headers parameter in jupyter-nbmodel-client.

should this be fixed in upstream jupyter-nbmodel-client? not sure how this would help this PR?

@EnyMan

EnyMan commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

@echarles It would allow us to get rid of this not great piece of code:

def _connect_with_auth(uri, **kwargs):
   kwargs.setdefault("additional_headers", {})
   kwargs["additional_headers"]["Cookie"] = cookie_header
   return original_connect(uri, **kwargs)

nbmodel_client_module.connect = _connect_with_auth

in jupyter_mcp_server/notebook_manager.py:113

But if we are fine with it, we can keep it as is. But it's somewhat fragile if the nbmodel_client_module changes in how it works, this will break.

@echarles

Copy link
Copy Markdown
Member

Related issues: Partially addresses #183 (XSRF-protected Jupyter deployments without Bearer tokens), related to #61 (WebSocket authentication failures)

As we are talking about security, very clear explanations would be useful. Is it possible to create a security section / page in the docs, and explain the various ways and implications the jupyter mcp server can be configured, included those changes.

@abbbe

abbbe commented May 20, 2026

Copy link
Copy Markdown
Contributor

I wonder if the tests run against the real instance of a jupyter server or the test code only test a standalone instance of the newly added class mean to implement password based auth?

@EnyMan

EnyMan commented May 20, 2026

Copy link
Copy Markdown
Contributor Author

@abbbe i tested this implementation with claude code agains real jupyter lab server multiple times

@abbbe

abbbe commented May 21, 2026

Copy link
Copy Markdown
Contributor

I am not speaking on behalf of the project here, mere a party interested in the feature, but “works for me” is not quite the same as proper CI. The project test suite already has all the machinery to spawn Jupiter Lab server etc, it is not too complicated to integrate your tests.

@EnyMan

EnyMan commented May 21, 2026

Copy link
Copy Markdown
Contributor Author

@echarles Good point — I've added documentation for the password auth feature

Security docs (docs/docs/reference/security/index.mdx):

  • New "Password Authentication" section covering the login flow, XSRF handling, simplified and advanced configuration (with JSON and CLI examples), a comparison table of password vs token auth, priority behavior, and how to set up a Jupyter server password
  • Updated the "Current Limitations" section to reflect that password-protected XSRF deployments are now supported, while noting that SSO/OAuth/IAM scenarios from Support XSRF-protected Jupyter deployments without Bearer tokens #183 remain open
  • Kept the token-based workarounds as "Alternatives for Token-less Environments"

Configuration docs (docs/docs/reference/configuration/index.mdx):

  • Added JUPYTER_PASSWORD, DOCUMENT_PASSWORD, RUNTIME_PASSWORD to the environment variable tables
  • Added --jupyter-password, --runtime-password, --document-password to the CLI options reference
  • Added a "Password-Protected Jupyter Server" configuration example
  • Updated the priority section to document password-over-token precedence

Let me know if you'd like any changes to the structure or content.

@echarles

Copy link
Copy Markdown
Member

@EnyMan Awesome!

Maybe @abbbe can go through a first pass on these changes as he has been key to introduce more security recently, I can then review as follow-up?

@abbbe

abbbe commented May 21, 2026

Copy link
Copy Markdown
Contributor

@EnyMan this is a very cool feature, actually wanted to implement it myself but never could wrap my head around it.

I am happy to try to review it, but would you be open to do the following to begin with:

  • align your fork with the latest main (now it is 9 commits behind),
  • Implement proper CI (run tests against real Jupyter instance as done by token auth, I can point you to specific old commits you can use for inspiration, it is not that complicated)
  • confirm you have done a basic automated analysis of your PR (here is one done by Claude https://claude.ai/code/session_01EYGdzWYfnApH8AjX28nFLF). I do not imply AI has a final say here and aware it is subject to hallucinations, but I believe it is a valuable tool to catch low hanging fruits.

Comment thread docs/docs/reference/configuration/index.mdx Outdated
@abbbe

abbbe commented May 21, 2026

Copy link
Copy Markdown
Contributor

@echarles It would allow us to get rid of this not great piece of code:

def _connect_with_auth(uri, **kwargs):
   kwargs.setdefault("additional_headers", {})
   kwargs["additional_headers"]["Cookie"] = cookie_header
   return original_connect(uri, **kwargs)

nbmodel_client_module.connect = _connect_with_auth

in jupyter_mcp_server/notebook_manager.py:113

But if we are fine with it, we can keep it as is. But it's somewhat fragile if the nbmodel_client_module changes in how it works, this will break.

I think this (extending nbmodel client) is the best approach, all this monkey patching business is rather sketchy.

@EnyMan

EnyMan commented May 21, 2026

Copy link
Copy Markdown
Contributor Author

Better test are on the way

@EnyMan
EnyMan force-pushed the feat/password-auth branch from ef07222 to 53121d4 Compare May 21, 2026 22:17
@EnyMan

EnyMan commented May 22, 2026

Copy link
Copy Markdown
Contributor Author

@abbbe Thanks for looking at this I did several rounds against grumpy opus reviewer.

@EnyMan
EnyMan requested a review from abbbe May 22, 2026 13:31
@echarles

Copy link
Copy Markdown
Member

@abbbe Any further comment before I review this?

@abbbe

abbbe commented May 29, 2026 via email

Copy link
Copy Markdown
Contributor

Comment thread docs/docs/reference/configuration/index.mdx Outdated
Comment thread docs/docs/reference/security/index.mdx Outdated
Comment thread jupyter_mcp_server/CLI.py Outdated
Comment thread jupyter_mcp_server/notebook_manager.py Outdated
Comment thread tests/test_auth.py Outdated
Comment thread tests/test_auth.py
Comment thread tests/conftest.py Outdated
@abbbe

abbbe commented May 29, 2026

Copy link
Copy Markdown
Contributor

There is something else. The current implementation logs in once, but does not retry if session goes stale (cookie expires) throughout the session. I understand by default jupyter cookies expire in 30 days and normal desktop users are unlikely to be affected, but long running sessions might fail. I did not investigate this in details, because things might change depending on how the race condition will be fixed. Something to look into...

@abbbe

abbbe commented May 29, 2026

Copy link
Copy Markdown
Contributor

Also, I think setting token to blank value is not useful but create risks. Not useful, because password-based authentication gives access to API even without IdentityProvider.token="".

Dangerous, because if things go wrong with the password based authentication (for whatever reason), user ends up with zero authentication.

@EnyMan

EnyMan commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review i should be able to look at it sometime this week.

@EnyMan

EnyMan commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

@abbbe Good catch on the Jupyter cookie expiration i haven't thought about it. As for the implementation of it i plan to look for 401/403 an re-auth. By any chance do you know if the collaboration WebSocket surfaces this cleanly? I am also going to wait for the nbmodel_client_module change so i can do that and the re-auth in the same pass.

@abbbe

abbbe commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

By any chance do you know if the collaboration WebSocket surfaces this cleanly?

My guess is web socket is not affected, in a sense that once it is open with a valid cookie, the web socket itself just stays open even if the cookie used to open it has expired.

@EnyMan

EnyMan commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

@abbbe Removed the monkey-patch and I added the re-login flow to the collaboration/document path. Do you want it added also to the Kernel management path jupyter_server_client?

@EnyMan
EnyMan requested a review from abbbe June 5, 2026 19:33
@EnyMan

EnyMan commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

While testing this branch I hit a bug: use_notebook(mode="create") failed under password auth with 403: '_xsrf' argument missing from POST. Fixed it and added an E2E test for create mode.

@echarles

Copy link
Copy Markdown
Member

@EnyMan Is this ready for review?

@EnyMan

EnyMan commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

@EnyMan Is this ready for review?

Yes, it is. Let me rebase to master really quickly, ok it might not be that quick since the CLI changes

@EnyMan
EnyMan force-pushed the feat/password-auth branch from df44208 to ff38f60 Compare July 23, 2026 09:28
@EnyMan

EnyMan commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Fixing the test plus some new additions that were since added to main.

@echarles

Copy link
Copy Markdown
Member

thx, are the items listed in the first note still opened?

 Verify token-based auth still works unchanged (backward compatible)
 Verify that --jupyter-password falls back correctly when individual passwords are not set
 Verify that jupyter-mcp-server connect --help shows no --*-password options (passwords are local-only; not forwardable over /api/connect)

@EnyMan

EnyMan commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

thx, are the items listed in the first note still opened?

 Verify token-based auth still works unchanged (backward compatible)
 Verify that --jupyter-password falls back correctly when individual passwords are not set
 Verify that jupyter-mcp-server connect --help shows no --*-password options (passwords are local-only; not forwardable over /api/connect)

The last one is done. For the first two, I can either test manually or try to write tests for them. The manual test might need to wait until tomorrow.

@EnyMan

EnyMan commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

OK verified the two last two points

@EnyMan
EnyMan force-pushed the feat/password-auth branch from 1f5b316 to 68b9ce8 Compare July 24, 2026 09:31
@echarles

Copy link
Copy Markdown
Member

Just merged #306 and release V 1.1.0 which has internal breaking changes, no changes for the users. The logic is to use the code-sandboxes package instead of the jupyter-kernel-client, this has introduced conflict on your PR, sorry for that!

@EnyMan

EnyMan commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Rebased and opened up datalayer/code-sandboxes#13; it's the same kind of issue as with the nb-client.
Also, the test fails due to an unbound MCP dependency; they released v2.

@echarles

Copy link
Copy Markdown
Member

Rebased and opened up datalayer/code-sandboxes#13; it's the same kind of issue as with the nb-client.

Great, merged and code sandboxes 0.17.0 release with that

Also, the test fails due to an unbound MCP dependency; they released

#328 is merged and pins mcp<2 - new release on its way.

Add support for authenticating against password-protected Jupyter
servers, in addition to token/anonymous auth.

- auth.py: log in to /login, capture the XSRF token and session cookie,
  and expose auth headers/cookies for HTTP, WebSocket, and collaboration
  API requests; re-authenticate on 401/403 when the session cookie
  expires and use a fresh cookie for the WebSocket handshake after
  relogin.
- Inject the X-XSRFToken header on notebook-create and other POST
  requests when using password (cookie) auth.
- Thread auth headers into the kernel client (create_kernel and the
  execute_code cross-kernel connection) and the collaboration WebSocket
  (NbModelClient additional_headers); drop the token when password auth
  supplies cookie/XSRF headers so it can't override them.
- CLI/config: add --runtime-password / --document-password /
  --jupyter-password options (with matching env vars) that fall back to
  the shared --jupyter-password and take precedence over the tokens.
- Docs: document password authentication in the security and
  configuration references.
- Tests: unit tests (test_auth.py) plus e2e tests against a real
  password-protected Jupyter server; the e2e read-back polls the
  collaboration YDoc to tolerate its eventual consistency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@EnyMan
EnyMan force-pushed the feat/password-auth branch from cbcd2dc to 4f26f2b Compare July 30, 2026 09:27
@EnyMan

EnyMan commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Rebased and pushed. So unless something breaks again, ready to review @echarles

@echarles echarles left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thx @EnyMan

@echarles
echarles merged commit d7e12a3 into datalayer:main Jul 30, 2026
20 of 21 checks passed
AmirF194 added a commit to AmirF194/jupyter-mcp-server that referenced this pull request Aug 6, 2026
ServerContext only ever fetches the `_xsrf` cookie when a code sandbox
password is configured (PR datalayer#250). When neither a password nor a token
is set, code_sandbox_auth_headers stays permanently {}, so every
state-changing request (kernel management, collaboration API) fails
with "'_xsrf' argument missing from POST" on deployments where auth is
handled externally: --IdentityProvider.token='', JupyterHub
single-user servers, or managed environments (SageMaker Studio, Colab
Enterprise).

Add JupyterAnonymousAuth, a JupyterPasswordAuth subclass that performs
only the anonymous GET /login step to pick up the _xsrf cookie,
skipping the password POST and verification. Wire it into
_init_mcp_server_mode's existing no-password branch when
code_sandbox_token is also unset. Every reader of
_code_sandbox_password_auth (code_sandbox_auth_headers,
relogin_code_sandbox, document auth reuse) already dispatches through
get_headers()/relogin()/close(), so no other call site needed to
change.

Scoped to the code sandbox / shared-URL case reported in the issue; a
document server configured at a different URL with no credentials of
its own is unchanged (still silently unauthenticated, same as before).

Fixes datalayer#183
echarles pushed a commit that referenced this pull request Aug 6, 2026
…et (#333)

* fix(auth): fetch anonymous XSRF cookie when no password or token is set

ServerContext only ever fetches the `_xsrf` cookie when a code sandbox
password is configured (PR #250). When neither a password nor a token
is set, code_sandbox_auth_headers stays permanently {}, so every
state-changing request (kernel management, collaboration API) fails
with "'_xsrf' argument missing from POST" on deployments where auth is
handled externally: --IdentityProvider.token='', JupyterHub
single-user servers, or managed environments (SageMaker Studio, Colab
Enterprise).

Add JupyterAnonymousAuth, a JupyterPasswordAuth subclass that performs
only the anonymous GET /login step to pick up the _xsrf cookie,
skipping the password POST and verification. Wire it into
_init_mcp_server_mode's existing no-password branch when
code_sandbox_token is also unset. Every reader of
_code_sandbox_password_auth (code_sandbox_auth_headers,
relogin_code_sandbox, document auth reuse) already dispatches through
get_headers()/relogin()/close(), so no other call site needed to
change.

Scoped to the code sandbox / shared-URL case reported in the issue; a
document server configured at a different URL with no credentials of
its own is unchanged (still silently unauthenticated, same as before).

Fixes #183

* fix(auth): don't let an unreachable server abort MCP_SERVER init

The anonymous XSRF fetch added in the previous commit ran eagerly inside
_init_mcp_server_mode, so any config with no token and no password (the
process-global default many unrelated tests and code paths construct)
now performs a real network GET at ServerContext init time. A server
that isn't reachable yet turned that into a RuntimeError that aborted
startup, where before this feature existed the same config did zero
network I/O.

Extract the login attempt into _try_anonymous_auth, which returns the
auth object either way (inject_into_session/get_headers already no-op
when _authenticated is False), and degrade to no cookie on a connection
error instead of raising. Explicit password auth is unaffected: a user
who configured a password still gets a hard failure if login fails.

Caught this via CI on the open PR (6 previously-passing tests failed
across the full matrix); reproduced locally and confirmed the same 6
pass again after this change, with no new ruff findings.

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>

---------

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants