Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 123 additions & 2 deletions docs/HERMES.md
Original file line number Diff line number Diff line change
Expand Up @@ -658,10 +658,131 @@ AETHER_AUTOMATION=1 AETHER_AUTOMATION_SOCKET=aethersdr-hl2 \
- Launch the app as the **foreground process of a backgrounded shell**;
launching it with `&` inside a foreground command gets it killed with the
shell's process group.
- First WDSP channel open costs **~19 s** generating FFTW wisdom; every later
open — any receiver, any sample rate — is **40–175 ms**. The planning cost is
- An earlier first WDSP channel open took **~19 s** generating FFTW wisdom;
warm opens in that observation were **40–175 ms**. These are historical
observations, not a current cold-open estimate or a timeout budget; the later
bench observations below were substantially slower. The planning cost is
not a bug and cannot be optimised away, but it is now paid **off the GUI
thread** and reported in the connect animation; see §22.

- **Warm-open timings assume the wisdom cache survives. Redirecting `HOME`
can move that cache too, depending on the other environment variables.**
An isolated profile can therefore turn a warm open into a cold one.

For this macOS recipe, the config variables and the WDSP cache have separate
resolution rules. `HOME` affects the cache only when neither a non-empty
`AETHER_WDSP_WISDOM_DIR` nor `XDG_CACHE_HOME` overrides it:

| variable | what it moves |
|---|---|
| `CFFIXED_USER_HOME` | Qt's config and log locations — `QStandardPaths::writableLocation(GenericConfigLocation)`, so the settings store and `LogManager`'s rotated log |
| `XDG_CONFIG_HOME` | the same locations on platforms that consult it |
| **`HOME`** | locations resolved from `QDir::homePath()`; also the WDSP wisdom cache on macOS/Linux when `AETHER_WDSP_WISDOM_DIR` and `XDG_CACHE_HOME` do not override it |

After the explicit `AETHER_WDSP_WISDOM_DIR` override, macOS/Linux resolve
`WdspChannel::wisdomPath()` through `$XDG_CACHE_HOME`, **else** `$HOME/.cache`,
with `/aethersdr/wdsp-fftw-wisdom` appended. Windows uses `LOCALAPPDATA`
instead of those two variables. An empty resolved directory falls back to
the system temporary directory. Wisdom is imported before the channels are
built and exported after.

**Without the explicit override, an inherited `XDG_CACHE_HOME` means that
redirecting `HOME` does not move the wisdom cache.** If that inherited path
still names the operator's cache, the "isolated" run reads it — and, because
the export is unconditional in an app process, **writes to it too**. That is the dangerous outcome, not the slow
one: the run is fast, looks correct, and quietly rewrites a file outside the
profile it was supposed to be confined to. `XDG_CACHE_HOME` is commonly
exported on Linux and rarely on macOS, which is exactly the kind of difference
that makes a harness behave one way on a developer's machine and another in
CI.

Without either cache override, redirecting `HOME` to a fresh directory on
macOS/Linux gives the first isolated run a cold cache even on a machine that
has connected a hundred times. If an inherited `XDG_CACHE_HOME` still points
at the operator's cache, redirecting `HOME` alone does not isolate it.

**VERIFY THE PATH THE PROCESS GOT, NOT THE ONE YOU ASKED FOR.** This is the
general form and it is worth more than the specific trap: a harness that
exports a variable and prints that it exported it has confirmed its own
intent, not the outcome. Read the environment of the running process —
`ps eww <pid>` on macOS, `/proc/<pid>/environ` on Linux — or log
`wisdomPath()` from inside the app and compare it against what you meant. A
request that is accepted, validated and reported as fine, then discarded
further down, produces exactly the same output as one that worked.

**How often you pay that depends on your profile's lifetime, so be deliberate
about it.** The recipe above exports a stable `$T=/tmp/aether-hl2-test` and
`mkdir -p`s it, so a cache resolved under that profile survives between runs
and only the first launch is slow — until something clears `/tmp`. A harness
that discards both the profile and its cache, using `mktemp -d`, a cleanup
trap, or a fresh container, has **no warm run at all**: every launch is a
first open, and that is the case that reads as a hang. Which shape you have is not
visible from the symptom, so decide it rather than discover it.

The earlier ~19 s observation does not predict these runs, and the recorded
evidence does not establish why they differ. Do not infer a receiver-count
multiplier or a portable cold-open time from these separate observations.
WDSP builds every FFT with `FFTW_PATIENT`, 35 plans per channel.
Two independent measurements, both on
this bench, changing only whether that cache was reachable:

| what was measured | cache present | cache absent |
|---|---|---|
| connect, via the bridge | **4.1 s** | **still running at 150 s** (abandoned, not a completion time) |
| first `WdspChannel` open, direct | **86 ms** (warm re-open) | **98.3 s** on an idle machine; **188.1 s** at one-minute load 38–40 |

The second row is the one to quote, because it is a completion time rather
than the moment an observer gave up. Raw result and runner are in the bench
notebook repo, not this one: `hl2-lab`, at
`streams/hl2-telemetry/runs/d57_bench_quiet_result.txt` and
`d57_bench_quiet.py`. Quiet throughout — 21 load samples at 5 s intervals, all
between 3.3 and 4.1. Note what it says about the first
row: **150 s was not a failure**, it was a working open that had not finished
yet. That abandoned observation does not justify a 150 s failure bound;
the landed #5415 watchdog allows 600 s for this phase.

**The fix is one variable**, and the code already provides it for its own
tests — see the `AETHER_WDSP_WISDOM_DIR` branch at the top of
`WdspChannel::wisdomPath()`:

```bash
# ABSOLUTE, and resolved BEFORE HOME is redirected.
export AETHER_WDSP_WISDOM_DIR=/var/tmp/aethersdr-harness-wisdom
```

It is checked **first and unconditionally**, ahead of `XDG_CACHE_HOME`,
`LOCALAPPDATA` and `HOME`, so it is the only one of these that binds whatever
else the environment carries. That — not the convenience — is why it is the
fix: redirecting `HOME` is a guess about which branch of the resolution order
a machine will take.

**Do not write `$HOME/...` here.** `HOME` is the variable this very recipe
redirects, so a `$HOME`-relative path lands *inside* the temporary tree and is
deleted with it — the fix silently undoes itself, and the symptom is
indistinguishable from not having applied it. On a single command line with
`HOME=$T` in front, `$HOME` happens to expand from the caller's environment
and it appears to work; in a script that sets `HOME` first, it does not. Use
an absolute path.

Point it at a directory that OUTLIVES the run. The harness then pays the
planning cost once instead of on every launch, and keeps the isolation the
rest of the recipe is for.

Two traps worth naming, both of which cost a day here. There are **two**
FFTW wisdom files — `AudioEngine::wisdomFilePath()` under
`~/.config/AetherSDR/` for NR2, and `WdspChannel::wisdomPath()` under
`~/.cache/aethersdr/` for the channels — and the startup log line
`Audio NR2 wisdom summary:` followed by `status=missing` refers to the
**first**, which is routinely absent and says nothing about the second.

**The DSP-setup diagnostic from #5415 has landed.** `Hl2Backend` logs
`HL2 DSP setup: opening` at phase start and `HL2 DSP setup: chains open after`
when setup returns. The watchdog warns after 10 s, repeats every 30 s while
waiting, and reports a connection error at 600 s. A timeout invalidates the
attempt; it cannot interrupt an in-progress WDSP open, whose eventual
completion releases the stale chains. These logs and the bounded wait answer
the silence reported in #5413. The initial `ok`/`deferred` reply still means
the connect was accepted, not that the radio is connected.
- The `tools/hl2/` Python spike defaults to broadcasting
`255.255.255.255`, which fails on macOS with `OSError 65` when multiple
interfaces are up. Use `--bcast <subnet>.255`. The in-app Qt sweep is fine.
Expand Down
Loading