Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
a291451
bump
echarles Jul 27, 2026
53c901d
user avatar
echarles Jul 30, 2026
c9f4436
runtimes
echarles Aug 1, 2026
45e95ce
bump
echarles Aug 1, 2026
5166e6f
tests
echarles Aug 2, 2026
f46d6a9
user badge
echarles Aug 2, 2026
50cfcb8
user badge
echarles Aug 2, 2026
fcabe6f
signin
echarles Aug 2, 2026
b746894
lint
echarles Aug 3, 2026
e2ede05
input text
echarles Aug 4, 2026
60cad24
theme
echarles Aug 5, 2026
96015ba
external token
echarles Aug 8, 2026
ee0b2cb
state defaults
echarles Aug 8, 2026
37a41be
dev
echarles Aug 9, 2026
6ffca32
fix: build
echarles Aug 9, 2026
e56dd8e
animation
echarles Aug 10, 2026
017cd65
animated text
echarles Aug 11, 2026
f2a2634
config
echarles Aug 12, 2026
f45d640
datalayer run
echarles Aug 12, 2026
c27b4a5
bump
echarles Aug 12, 2026
3eec93c
status
echarles Aug 13, 2026
116bfea
animated text
echarles Aug 13, 2026
3337828
spaces
echarles Aug 13, 2026
aef2397
bump
echarles Aug 13, 2026
89939d2
readme
echarles Aug 14, 2026
90ec86d
readme
echarles Aug 14, 2026
6c88f9a
animated text
echarles Aug 14, 2026
9d4ddb1
datalayer box
echarles Aug 15, 2026
b76498f
credit
echarles Aug 15, 2026
b51fada
collaboratoin
echarles Aug 16, 2026
da5cc3d
Automatic application of license header
github-actions[bot] Aug 16, 2026
f454043
principal
echarles Aug 16, 2026
6e551a5
apperance
echarles Aug 16, 2026
66ae4f6
Automatic application of license header
github-actions[bot] Aug 16, 2026
9e993b3
bump
echarles Aug 16, 2026
01e1584
bump
echarles Aug 17, 2026
5d4fd03
principal
echarles Aug 18, 2026
45b0fa2
principal
echarles Aug 18, 2026
ef0a07e
apperance
echarles Aug 18, 2026
a85b92f
bump
echarles Aug 18, 2026
62069ac
label
echarles Aug 18, 2026
ca45d80
use user
echarles Aug 19, 2026
7b034ff
sandbox
echarles Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
File renamed without changes.
61 changes: 19 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%20Sponsor&message=%E2%9D%A4&logo=GitHub&style=flat&color=1ABC9C)](https://github.com/sponsors/datalayer)

# ☰ Datalayer Core
# ☰ Datalayer Core

<p align="center">
<strong>Python and Typescript libraries for Datalayer</strong>
Expand All @@ -18,7 +18,7 @@

## Overview

Datalayer Core is the foundational package that powers the [Datalayer AI Platform](https://datalayer.app/). It provides a TypesScript and Python packages as a Command Line Interface (CLI) for AI engineers, data scientists, and researchers to seamlessly integrate scalable compute runtimes into their workflows.
Datalayer Core is the foundational package that powers the [Datalayer AI Platform](https://datalayer.ai). It provides a TypesScript and Python packages as a Command Line Interface (CLI) for AI engineers, data scientists, and researchers to seamlessly integrate scalable compute runtimes into their workflows.

This package serves as the base foundation used by many other Datalayer packages, containing core application classes, configuration, and unified APIs for authentication, runtime management, and code execution in cloud-based environments.

Expand Down Expand Up @@ -66,8 +66,6 @@ npm install

## Quick Start with Python

### 1. Authentication

Set your Datalayer token as an environment variable:

```bash
Expand All @@ -89,31 +87,19 @@ if client.authenticate():
print("Successfully authenticated!")
```

### 2. Runtime, Snapshots, and Evals

Runtime execution, snapshot workflows, and evals CLI are now documented and maintained in:
## Architecture

- [agent-runtimes README](https://github.com/datalayer/agent-runtimes/blob/main/README.md)
- [agent-runtimes CLI docs](https://agent-runtimes.datalayer.tech/cli)
Datalayer Core serves as the foundation for the entire Datalayer ecosystem:

Use `agent-runtimes` for runtime workloads (`RuntimeClient`, `@datalayer`, snapshots, and evals), and use `datalayer-core` for account/platform operations (authentication, secrets, API keys, usage, profile).
- **Base Classes**: Core application classes inherited by other Datalayer packages
- **Configuration Management**: Centralized configuration system for all Datalayer components
- **Authentication Layer**: Unified authentication across all Datalayer services
- **Runtime Abstraction**: Common interface for different types of compute runtimes
- **Resource Management**: Automatic cleanup and lifecycle management

## Examples

Examples have moved to Agent Runtimes:

- [Agent Runtimes examples README](https://github.com/datalayer/agent-runtimes/blob/main/examples/README.md)
- [Agent Runtimes examples directory](https://github.com/datalayer/agent-runtimes/tree/main/examples)

## Platform Integration

Datalayer adds AI capabilities and scalable compute runtimes to your development workflows. The platform is designed to seamlessly integrate into your existing processes and supercharge your computations with the processing power you need.

Key platform features accessible through this Client and CLI:

- **Remote Runtimes**: Execute code on powerful remote machines with CPU, RAM, and GPU resources
- **Multiple Interfaces**: Access and consume runtimes through Python Client, CLI, or other integrated tools
- **Scalable Compute**: Dynamically scale your computational resources based on workload requirements
- [OTEL example README](./examples/otel/README.md)

## Documentation

Expand Down Expand Up @@ -193,16 +179,6 @@ This Client is designed to be simple and extensible. We welcome contributions! P

For issues and enhancement requests, please use the [GitHub issue tracker](https://github.com/datalayer/core/issues).

## Architecture

Datalayer Core serves as the foundation for the entire Datalayer ecosystem:

- **Base Classes**: Core application classes inherited by other Datalayer packages
- **Configuration Management**: Centralized configuration system for all Datalayer components
- **Authentication Layer**: Unified authentication across all Datalayer services
- **Runtime Abstraction**: Common interface for different types of compute runtimes
- **Resource Management**: Automatic cleanup and lifecycle management

## Use Cases

- **AI/ML Development**: Scale your machine learning workflows with cloud compute using Client or CLI
Expand All @@ -219,15 +195,16 @@ This project is licensed under the [BSD 3-Clause License](https://github.com/dat

- **Documentation**: [Datalayer Platform Documentation](https://datalayer.ai/docs/)
- **Issues**: [GitHub Issues](https://github.com/datalayer/core/issues)
- **Community**: [Datalayer Platform](https://datalayer.app/)
- **Community**: [Datalayer Platform](https://datalayer.ai)

---

<p align="center">
<img src="https://assets.datalayer.tech/datalayer-25.svg" alt="Datalayer Logo" width="200"></img>
</p>
<div align="center">

<p align="center">
<strong>🚀 AI Agents for Data Analysis</strong><br></br>
<a href="https://datalayer.app/">Get started with Datalayer today!</a>
</p>
**If this project is helpful to you, please give us a ⭐️**

Made with ❤️ by [Datalayer](https://datalayer.ai)

<img src="https://assets.datalayer.tech/datalayer-25.svg" alt="Datalayer Logo" width="200"></img>

</div>
9 changes: 5 additions & 4 deletions datalayer_core/authn/authn.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,14 @@ def __init__(self, iam_url: str, storage: Optional[TokenStorage] = None):
"""
self.iam_url = iam_url

# Extract datalayer_url from iam_url (remove /api/iam/v1 suffix if present)
datalayer_url = iam_url.replace("/api/iam/v1", "")
# The service URL the credentials are stored under: the IAM one,
# without the path of its API.
service_url = iam_url.replace("/api/iam/v1", "")

# CRITICAL: Pass datalayer_url as service_name to KeyringStorage for backwards compatibility
# CRITICAL: Pass the service URL as service_name to KeyringStorage for backwards compatibility
self.storage: TokenStorage
if storage is None:
keyring_storage = KeyringStorage(service_name=datalayer_url)
keyring_storage = KeyringStorage(service_name=service_url)
if keyring_storage.is_available():
self.storage = keyring_storage
else:
Expand Down
8 changes: 4 additions & 4 deletions datalayer_core/authn/server/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,23 @@
from typing import Optional

from datalayer_core.authn.server.http_server import get_token
from datalayer_core.utils.urls import DEFAULT_DATALAYER_URL
from datalayer_core.utils.urls import DEFAULT_DATALAYER_IAM_URL

logging.basicConfig(level=logging.INFO)

logger = logging.getLogger(__name__)


DATALAYER_URL = DEFAULT_DATALAYER_URL
IAM_URL = DEFAULT_DATALAYER_IAM_URL


if __name__ == "__main__":
from sys import argv

if len(argv) == 2:
ans = get_token(DATALAYER_URL, port=int(argv[1]))
ans = get_token(IAM_URL, port=int(argv[1]))
else:
ans = get_token(DATALAYER_URL)
ans = get_token(IAM_URL)

handle: Optional[str] = None
token: Optional[str] = None
Expand Down
27 changes: 13 additions & 14 deletions datalayer_core/authn/server/http_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ def do_GET(self) -> None:
elif path in {"/", "/datalayer/login/cli"}:
config_json = json.dumps(
{
"datalayerUrl": self.server.datalayer_url, # type: ignore
"iamUrl": self.server.iam_url, # type: ignore
"iamUrl": self.server.iam_url, # type: ignore
"whiteLabel": False,
}
Comment on lines 153 to 158
Expand Down Expand Up @@ -229,8 +229,8 @@ class AuthHTTPServer(HTTPServer):
The server address and port.
RequestHandlerClass : Callable
The request handler class.
datalayer_url : str
The runtime URL.
iam_url : str
The URL of the IAM service, which the login goes through.
bind_and_activate : bool, default True
Whether to bind and activate the server.
"""
Expand All @@ -239,7 +239,7 @@ def __init__(
self,
server_address: tuple[Union[str, bytes, bytearray], int],
RequestHandlerClass: t.Callable[[t.Any, t.Any, t.Self], BaseRequestHandler],
datalayer_url: str,
iam_url: str,
bind_and_activate: bool = True,
) -> None:
"""
Expand All @@ -251,14 +251,13 @@ def __init__(
The server address and port.
RequestHandlerClass : Callable
The request handler class.
datalayer_url : str
The runtime URL.
iam_url : str
The URL of the IAM service, which the login goes through.
bind_and_activate : bool, default True
Whether to bind and activate the server.
"""
# Use DatalayerURLs for proper URL configuration
self._urls = DatalayerURLs.from_environment(datalayer_url=datalayer_url)
self.datalayer_url = self._urls.datalayer_url
self._urls = DatalayerURLs.from_environment(iam_url=iam_url)
self.iam_url = self._urls.iam_url
self.user_handle = None
self.token = None
Expand Down Expand Up @@ -299,15 +298,15 @@ def finish_request(self, request: t.Any, client_address: str) -> None:


def get_token(
datalayer_url: str, port: Optional[int] = None, logger: logging.Logger = logger
iam_url: str, port: Optional[int] = None, logger: logging.Logger = logger
) -> Optional[tuple[str, str]]:
"""
Get the user handle and token.

Parameters
----------
datalayer_url : str
The runtime URL.
iam_url : str
The URL of the IAM service, which the login goes through.
port : int or None, default None
The port to use for the authentication server.
logger : logging.Logger, default logger
Expand All @@ -329,8 +328,8 @@ def get_token(
)
sys.argv = [
"",
"--DatalayerExtensionApp.datalayer_url",
datalayer_url,
"--DatalayerExtensionApp.iam_url",
iam_url,
"--ServerApp.disable_check_xsrf",
"True",
]
Expand All @@ -339,7 +338,7 @@ def get_token(
# return None if httpd.token is None else (httpd.user_handle, httpd.token)
return None
else:
httpd = AuthHTTPServer(server_address, LoginRequestHandler, datalayer_url)
httpd = AuthHTTPServer(server_address, LoginRequestHandler, iam_url)
logger.info(
f"Waiting for user logging, open http://localhost:{port}. Press CTRL+C to abort.\n"
)
Expand Down
2 changes: 1 addition & 1 deletion datalayer_core/authn/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ def __init__(self, service_name: str = DEFAULT_DATALAYER_IAM_URL):
"""Initialize keyring storage.

Args:
service_name: Service name for keyring entries (MUST be datalayer_url for backwards compatibility)
service_name: Service name for keyring entries (MUST be the IAM URL for backwards compatibility)
"""
self.service_name = service_name
self._keyring: Any = None
Expand Down
Loading
Loading