Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions API_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

### Added

- **Managed app deployments — config-file mounts (`files:`)** (part of the App Deployments epic) — the app `compose` grammar now supports injecting read-only config files into a container (rendered into a ConfigMap, or a Secret when `sensitive`, and mounted via `subPath`), so file-driven apps (e.g. strfry's `strfry.conf`) are deployable. Each service's `files:` entry declares a `path` plus either inline templated `content` (with `${…}` filled from `config`/`secrets`) or `content_from` a `config` field (including a new `type: file` for customer-supplied whole files). Validated by the shared parser (and therefore the admin catalog API): absolute non-traversal paths, a single content source, ≤256 KiB inline content, `content_from` must reference a declared field, and a file may not overlap a read-write data volume. Distinct from `volumes:` (persistent read-write PVCs).
- **Managed app deployments — compose validation** (part of the App Deployments epic) — the admin app-catalog endpoints (`POST`/`PATCH /api/admin/v1/apps`) now fully parse and validate the app's `compose` document (via a new shared `lnvps_compose` parser, the same code the operator will use to render Kubernetes objects) instead of only checking it's non-empty. Invalid or unsafe compose is rejected at catalog-edit time: unknown `depends_on`, `expose: ingress` on a non-HTTP port, absolute/`..`-traversal volume mount paths, and malformed `backup` blocks all return a clear `400`.
- **Managed app deployments — customer catalog & deployment views** (part of the App Deployments epic) — new read-only customer endpoints: `GET /api/v1/apps` and `GET /api/v1/apps/{id}` browse the catalog of offered apps (each exposes its docker-compose-style `compose` so clients can render the deploy form), and `GET /api/v1/app-deployments` and `GET /api/v1/app-deployments/{id}` list/show your own deployments (name, hostname, status, desired state, billing subscription). Ordering and lifecycle control land in a later increment.
- **Managed app deployments — admin catalog & clusters** (part of the App Deployments epic) — new admin endpoints to manage a catalog of predefined "apps" (e.g. Nostr relay, Blossom) deployed on shared Kubernetes infrastructure, and the clusters they run on. `GET/POST /api/admin/v1/apps` and `GET/PATCH/DELETE /api/admin/v1/apps/{id}` manage catalog apps (each defined by a docker-compose-style `compose` YAML plus subscription pricing); `GET/POST /api/admin/v1/app_clusters` and `GET/PATCH/DELETE /api/admin/v1/app_clusters/{id}` manage clusters (each bound to a `region` — which provides the billing company — and an ingress domain). Gated behind a new `app` admin permission (granted to `super_admin` by default). No customer-facing surface yet — deployment ordering and the operator reconcile land in later increments.
Expand Down
267 changes: 252 additions & 15 deletions lnvps_compose/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,37 @@ pub struct Service {
/// Advisory startup ordering hints (k8s has no hard ordering; apps retry).
#[serde(default)]
pub depends_on: Vec<String>,
/// Config files injected read-only into the container (ConfigMap/Secret),
/// separate from `volumes` (which are read-write PVCs for app data).
#[serde(default)]
pub files: Vec<File>,
/// Optional backup method for this service's data.
#[serde(default)]
pub backup: Option<Backup>,
}

/// A config file injected into a container (rendered into a ConfigMap, or a
/// Secret when `sensitive`) and mounted **read-only** at `path` via `subPath`
/// so it drops in as a single file without shadowing the directory.
///
/// Exactly one content source is used: an inline templated `content` (with
/// `${…}` filled from `config`/`secrets`), or `content_from` a `config` field
/// (e.g. `type: file`) whose value the customer supplies verbatim.
#[derive(Debug, Clone, Deserialize)]
pub struct File {
/// Absolute in-container mount path.
pub path: String,
/// Inline templated file content.
#[serde(default)]
pub content: Option<String>,
/// Name of a `config` field whose value is used as the file content.
#[serde(default)]
pub content_from: Option<String>,
/// Render into a Secret instead of a ConfigMap (holds secret material).
#[serde(default)]
pub sensitive: bool,
}

/// How a port is exposed.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Deserialize)]
#[serde(rename_all = "lowercase")]
Expand Down Expand Up @@ -153,6 +179,9 @@ pub enum FieldType {
String,
Int,
Bool,
/// Multiline free-form content, typically referenced by a file's
/// `content_from` to let the customer supply a whole config file.
File,
}

/// A service's backup method.
Expand Down Expand Up @@ -206,6 +235,52 @@ impl Compose {
bail!("service '{sname}': depends_on unknown service '{dep}'");
}
}
// Config files: valid path, single content source, size-bounded,
// and not overlapping a data volume.
for f in &svc.files {
check_abs_no_traversal(sname, "file", &f.path)?;
match (&f.content, &f.content_from) {
(Some(_), Some(_)) => {
bail!(
"service '{sname}': file '{}' has both content and content_from",
f.path
)
}
(None, None) => {
bail!(
"service '{sname}': file '{}' needs content or content_from",
f.path
)
}
(Some(c), None) => {
if c.len() > MAX_FILE_BYTES {
bail!(
"service '{sname}': file '{}' content exceeds {MAX_FILE_BYTES} bytes",
f.path
);
}
}
(None, Some(field)) => {
if !self.config.iter().any(|cf| &cf.name == field) {
bail!(
"service '{sname}': file '{}' content_from references unknown config field '{field}'",
f.path
);
}
}
}
// A config file must not land inside a read-write data volume.
for v in &svc.volumes {
if f.path == v.path || path_is_within(&f.path, &v.path) {
bail!(
"service '{sname}': file '{}' overlaps data volume mount '{}'",
f.path,
v.path
);
}
}
}

// A backup entry is exactly one of command | volume.
if let Some(b) = &svc.backup {
match (&b.command, &b.volume) {
Expand All @@ -227,15 +302,24 @@ impl Compose {
Ok(())
}

/// Every distinct env var name referenced as `${…}` across all services.
/// Every distinct env var name referenced as `${…}` across all services —
/// in env values and in inline file `content` templates.
pub fn referenced_vars(&self) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let push = |val: &str, out: &mut Vec<String>| {
for name in extract_refs(val) {
if !out.contains(&name) {
out.push(name);
}
}
};
for svc in self.services.values() {
for val in svc.env.values() {
for name in extract_refs(val) {
if !out.contains(&name) {
out.push(name);
}
push(val, &mut out);
}
for f in &svc.files {
if let Some(content) = &f.content {
push(content, &mut out);
}
}
}
Expand All @@ -262,9 +346,72 @@ impl Compose {
}
Ok(out)
}

/// Resolve every service's config files to their final (path, content,
/// sensitive) form: inline `content` has `${…}` substituted; `content_from`
/// takes the customer-supplied value from `vars`. Errors on unknown refs.
pub fn resolve_files(
&self,
vars: &HashMap<String, String>,
) -> Result<HashMap<String, Vec<ResolvedFile>>> {
let mut out = HashMap::new();
for (sname, svc) in &self.services {
let mut files = Vec::new();
for f in &svc.files {
let content = match (&f.content, &f.content_from) {
(Some(c), _) => substitute(c, vars)?,
(_, Some(field)) => vars
.get(field)
.cloned()
.ok_or_else(|| anyhow!("file '{}': unresolved config '{field}'", f.path))?,
(None, None) => bail!("file '{}': no content source", f.path),
};
files.push(ResolvedFile {
path: f.path.clone(),
content,
sensitive: f.sensitive,
});
}
out.insert(sname.clone(), files);
}
Ok(out)
}
}

/// A config file with its final rendered content, ready to become a ConfigMap
/// (or Secret when `sensitive`) mounted read-only at `path`.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ResolvedFile {
pub path: String,
pub content: String,
pub sensitive: bool,
}

/// Maximum inline file / ConfigMap content size we accept (well under the k8s
/// ~1 MiB ConfigMap limit).
const MAX_FILE_BYTES: usize = 256 * 1024;

/// Whether `path` sits inside directory `dir` (both absolute).
fn path_is_within(path: &str, dir: &str) -> bool {
let dir = dir.trim_end_matches('/');
path.starts_with(&format!("{dir}/"))
}

/// Validate an in-container path: absolute, not root, no `..` traversal.
fn check_abs_no_traversal(service: &str, label: &str, path: &str) -> Result<()> {
if !path.starts_with('/') {
bail!("service '{service}': {label} path '{path}' must be absolute");
}
if path == "/" {
bail!("service '{service}': {label} path cannot be '/'");
}
if path.split('/').any(|seg| seg == "..") {
bail!("service '{service}': {label} path '{path}' must not contain '..'");
}
Ok(())
}

/// Validate a volume mount path: absolute, no `..` traversal, not root.
/// Validate a volume mount: name is a slug, path is absolute/non-traversal.
fn validate_mount_path(service: &str, name: &str, path: &str) -> Result<()> {
if name.is_empty()
|| !name
Expand All @@ -273,15 +420,7 @@ fn validate_mount_path(service: &str, name: &str, path: &str) -> Result<()> {
{
bail!("service '{service}': volume name '{name}' must be a lowercase slug");
}
if !path.starts_with('/') {
bail!("service '{service}': volume '{name}' path must be absolute");
}
if path == "/" {
bail!("service '{service}': volume '{name}' cannot mount at '/'");
}
if path.split('/').any(|seg| seg == "..") {
bail!("service '{service}': volume '{name}' path must not contain '..'");
}
check_abs_no_traversal(service, "volume", path)?;
Ok(())
}

Expand Down Expand Up @@ -485,4 +624,102 @@ config:
let vars = HashMap::new();
assert!(substitute("${oops", &vars).is_err());
}

const STRFRY: &str = r#"
services:
strfry:
image: ghcr.io/hoytech/strfry:latest
ports:
- { name: ws, container: 7777, protocol: http, expose: ingress }
files:
- path: /etc/strfry.conf
content: |
relay { info { name = "${relay_name}"; } }
- path: /etc/custom.conf
content_from: custom_conf
- path: /etc/secret.key
content: "${API_KEY}"
sensitive: true
volumes:
- { name: db, path: /app/db, size: 5Gi }

secrets:
- { name: API_KEY, generate: token }

config:
- { name: relay_name, label: "Relay name", type: string, default: "My Relay" }
- { name: custom_conf, label: "Custom config", type: file }
"#;

#[test]
fn parses_and_resolves_files() {
let c = Compose::parse(STRFRY).unwrap();
let files = &c.services["strfry"].files;
assert_eq!(files.len(), 3);
assert!(files[2].sensitive);

// referenced_vars picks up ${…} in file content too.
let mut refs = c.referenced_vars();
refs.sort();
assert_eq!(refs, vec!["API_KEY", "relay_name"]);

let mut vars = HashMap::new();
vars.insert("relay_name".to_string(), "Zap Relay".to_string());
vars.insert("API_KEY".to_string(), "deadbeef".to_string());
vars.insert("custom_conf".to_string(), "my custom file body".to_string());

let resolved = c.resolve_files(&vars).unwrap();
let sf = &resolved["strfry"];
assert!(sf.iter().any(|f| f.path == "/etc/strfry.conf"
&& f.content.contains("name = \"Zap Relay\"")));
// content_from injects the customer-supplied value verbatim.
assert!(
sf.iter()
.any(|f| f.path == "/etc/custom.conf" && f.content == "my custom file body")
);
// sensitive file flagged for a Secret.
assert!(
sf.iter()
.any(|f| f.path == "/etc/secret.key" && f.sensitive)
);
}

#[test]
fn rejects_bad_files() {
// both content and content_from
assert!(
Compose::parse(
"services:\n a:\n image: x\n files:\n - { path: /e.conf, content: 'x', content_from: y }\n"
)
.is_err()
);
// neither content nor content_from
assert!(
Compose::parse(
"services:\n a:\n image: x\n files:\n - { path: /e.conf }\n"
)
.is_err()
);
// content_from unknown config field
assert!(
Compose::parse(
"services:\n a:\n image: x\n files:\n - { path: /e.conf, content_from: nope }\n"
)
.is_err()
);
// traversal path
assert!(
Compose::parse(
"services:\n a:\n image: x\n files:\n - { path: /etc/../x, content: 'y' }\n"
)
.is_err()
);
// file overlaps a data volume
assert!(
Compose::parse(
"services:\n a:\n image: x\n files:\n - { path: /app/db/f.conf, content: 'y' }\n volumes:\n - { name: db, path: /app/db, size: 1Gi }\n"
)
.is_err()
);
}
}
11 changes: 10 additions & 1 deletion work/app-deployments.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ images (higher isolation risk — design the boundary in now).
- [ ] Create deployment (validate config vs compose env schema → subscription + line item
(type App) + payment invoice, mirroring VM order); delete/stop/start; renew via subscription.

### Increment 4a — Shared compose parser (DONE, PR pending)
### Increment 4a — Shared compose parser (DONE, PR #218 merged)
- [x] New `lnvps_compose` crate (serde + serde_yaml, no heavy deps): typed model (`Compose`
/`Service`/`Port`/`Volume`/`SecretDecl`/`ConfigField`/`Backup`), `parse` + `validate`
(ingress=http only, mount-path traversal guard, depends_on refs, backup command|volume),
Expand All @@ -86,6 +86,15 @@ images (higher isolation risk — design the boundary in now).
on catalog create/update (same parser the operator will use).
- [x] Unit tests (9) in the crate + admin validator test updated.

### Increment 4a+ — Config files (`files:`) (DONE, PR pending)
- [x] `files:` per service: read-only injected config, `content` (templated) or `content_from`
(a `config` field, incl. new `type: file`), optional `sensitive` (→ Secret vs ConfigMap).
Operator mounts via `subPath` at `path`.
- [x] Validation: absolute/non-traversal path, single content source, ≤256 KiB inline,
`content_from` must be a declared config field, file must not overlap a data volume.
- [x] `resolve_files(vars) -> {service: [ResolvedFile{path,content,sensitive}]}`;
`referenced_vars` also scans file content. Distinct from `volumes:` (PVC, read-write).

### Increment 4b — Operator reconcile — TODO
- [ ] `lnvps_operator/src/app_deployments.rs`: use `lnvps_compose` to map compose + deployment
config → Namespace + Deployment/StatefulSet + Service + Ingress + PVC + Secret (generated) +
Expand Down
Loading