Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ jobs:
cache-to: type=gha,mode=max
- run: npm ci
- run: node build/assemble.mjs
- run: npm run build:scripting
- run: node --test test/scripting/api.test.mjs test/scripting/quota.test.mjs
- run: |
python3 -m venv .venv
.venv/bin/pip install fonttools==4.59.0 brotli==1.2.0
Expand All @@ -36,6 +38,7 @@ jobs:
npm ci
npx playwright install --with-deps chromium webkit
npm run test:browser
BROWSER_CHANNEL=chromium node test/scripting/api-browser.mjs
node test/pages-cache.mjs
- run: node build/release.mjs
- uses: actions/upload-artifact@v4
Expand All @@ -52,6 +55,16 @@ jobs:
uses: actions/upload-pages-artifact@v3
with:
path: _site
ci:
name: CI
if: always()
needs: wasm
runs-on: ubuntu-24.04
steps:
- name: Require successful build and tests
env:
BUILD_RESULT: ${{ needs.wasm.result }}
run: test "$BUILD_RESULT" = success
pages:
if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
needs: wasm
Expand Down
48 changes: 48 additions & 0 deletions .github/workflows/scripting.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Native scripting proof
on:
workflow_dispatch:
pull_request:
paths:
- 'native/**'
- 'build/**'
- 'test/scripting/**'
- 'src/**'
- 'package*.json'
- '.github/workflows/scripting.yml'
permissions:
contents: read
jobs:
scripting:
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
context: .
file: build/Dockerfile
target: scripting-export
outputs: type=local,dest=artifacts/scripting
cache-from: type=gha,scope=scripting
cache-to: type=gha,scope=scripting,mode=max
- run: npm ci
- run: node build/assemble-scripting.mjs
- run: node --test test/scripting/api.test.mjs test/scripting/quota.test.mjs
- run: node --test test/scripting/proof.test.mjs
- run: docker build -f build/Dockerfile.scripting-native -t fontforge-wasm:scripting-native .
- run: node test/scripting/parity.mjs
- run: |
python3 -m venv .venv
.venv/bin/pip install fonttools==4.59.0 brotli==1.2.0
.venv/bin/python test/scripting/verify-parity.py
- run: npx playwright install --with-deps chromium webkit
- run: node test/scripting/browser.mjs
env:
BROWSER_CHANNEL: chromium
- run: node test/scripting/api-browser.mjs
env:
BROWSER_CHANNEL: chromium
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ conversion and browser font loading. No external application is needed.
`build/sources.json` pins archives and their SHA-256 digests. `build/patch.py`
contains small checked platform changes; `native/patch-formats.py` preserves
Unicode information in Type 11 exports. The C adapter exposes only a narrow
conversion ABI, not a general scripting interface. Keep browser filesystem access
conversion ABI. A separate scripting target exposes the upstream interpreter;
see docs/scripting-api.md and run both scripting test suites for changes to it. Keep browser filesystem access
inside Emscripten's in-memory filesystem and run each job in its own worker.
The WASM memory ceiling is a linear-memory limit, not a total process limit.

Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,3 +197,11 @@ not remove the license obligations of applications distributing a combined work.
The live demo is hosted on GitHub Pages. Successful builds of `main` deploy it
automatically after the native and browser tests pass. Run `node build/pages.mjs`
after building to assemble the same static site locally.

### Native scripting (unreleased)

The development branch exposes a typed `execute(script, options)` API for Node and
browsers, with isolated workers, streamed logs, cancellation and runtime filesystem
budgets. See the [API contract and example](docs/scripting-api.md). Build its
separate assets with `npm run build:scripting`; `convert()` loads only the existing
conversion engine. This API has not yet been published in a release.
9 changes: 9 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,11 @@ and only missing server-supported outputs use fallback.

## Milestone 2 — FontForge native scripting and CLI semantics

The first [native scripting proof](docs/native-scripting-proof.md) is implemented
as an opt-in development target: real upstream scripts, disposable workers,
exit/log capture, native parity and browser tests. It is not yet a public SDK API
or a released scripting feature; the work below remains the milestone scope.

Enable FontForge's own scripting interpreter in a separate build target. Start
with `fontforge -lang=ff -script` semantics and then `-c`; do not label a custom
command parser as CLI compatibility. Python support is a later milestone.
Expand Down Expand Up @@ -138,3 +143,7 @@ measure memory/download/startup costs, publish browser support and compatibility
matrices, review third-party source distribution, and define semver guarantees
for the SDK, worker protocol and native ABI. Keep non-stable APIs explicitly
experimental until these gates pass.

The unreleased [execute API](docs/scripting-api.md) now adds typed Node/browser
execution, live diagnostics, cumulative filesystem budgets and explicit capability
errors. Public playground and broader editing-operation parity remain next.
16 changes: 16 additions & 0 deletions build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -81,5 +81,21 @@ RUN mkdir -p /output/dist /output/sources \
COPY build/licenses.py /product/build/licenses.py
RUN python3 /product/build/licenses.py \
&& tar czf /output/sources/emscripten-runtime.tar.gz -C /emsdk/upstream/emscripten system src LICENSE
# Opt-in proof build: the normal conversion artifact remains unchanged.
FROM builder AS scripting-builder
COPY build/patch-scripting.py /product/build/patch-scripting.py
RUN python3 /product/build/patch-scripting.py
# FreeType also uses setjmp; it must agree with the interpreter's exception ABI.
RUN emcmake cmake -S /work/freetype -B /work/freetype/out \
-DCMAKE_C_FLAGS="-O2 -I/opt/wasm/include -fwasm-exceptions -sSUPPORT_LONGJMP=wasm" \
&& cmake --build /work/freetype/out --target install -j4
RUN emcmake cmake -S /work/fontforge -B /work/fontforge/out \
-DENABLE_NATIVE_SCRIPTING=ON -DCMAKE_C_FLAGS="-O2 -I/opt/wasm/include -fwasm-exceptions -sSUPPORT_LONGJMP=wasm" \
&& cmake --build /work/fontforge/out --target fontforge-script -j4
RUN mkdir -p /script-output \
&& cp /work/fontforge/out/wasm-wrapper/fontforge-script.* /script-output/
FROM scratch AS scripting-export
COPY --from=scripting-builder /script-output/ /

FROM scratch AS export
COPY --from=builder /output/ /
17 changes: 17 additions & 0 deletions build/Dockerfile.scripting-native
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Native reference uses the same hash-verified FontForge source as the WASM build.
FROM emscripten/emsdk:5.0.7@sha256:4e332f7343b6f66320bf72f7ecc01a3d9f3866721a13b0e5c7b96505d6ab148a
RUN apt-get update && apt-get install -y --no-install-recommends \
ninja-build gettext libglib2.0-dev libfreetype-dev libxml2-dev libltdl-dev \
libwoff-dev libbrotli-dev && rm -rf /var/lib/apt/lists/*
COPY build/fetch.py build/sources.json /product/build/
RUN mkdir -p /work /sources && python3 /product/build/fetch.py fontforge
RUN cmake -S /work/fontforge -B /work/fontforge/out -G Ninja \
-DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
-DENABLE_GUI=OFF -DENABLE_NATIVE_SCRIPTING=ON \
-DENABLE_PYTHON_SCRIPTING=OFF -DENABLE_PYTHON_EXTENSION=OFF \
-DENABLE_LIBSPIRO=OFF -DENABLE_LIBGIF=OFF -DENABLE_LIBJPEG=OFF \
-DENABLE_LIBPNG=OFF -DENABLE_LIBREADLINE=OFF -DENABLE_LIBTIFF=OFF \
-DENABLE_WOFF2=ON -DENABLE_HARFBUZZ=OFF -DENABLE_DOCS=OFF \
&& cmake --build /work/fontforge/out --target fontforgeexe -j4
RUN cp /work/fontforge/out/bin/fontforge /usr/local/bin/fontforge-reference
ENTRYPOINT ["/usr/local/bin/fontforge-reference"]
6 changes: 6 additions & 0 deletions build/assemble-scripting.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// SPDX-License-Identifier: GPL-3.0-or-later
import { cp, mkdir } from 'node:fs/promises';
import { build } from 'esbuild';
await mkdir('dist', { recursive: true });
for (const name of ['fontforge-script.mjs', 'fontforge-script.wasm']) await cp(`artifacts/scripting/${name}`, `dist/${name}`);
await build({ entryPoints: ['src/script-browser-worker.js'], outfile: 'dist/script-browser-worker.mjs', bundle: true, platform: 'browser', format: 'iife', define: { 'import.meta.url': 'self.location.href' }, external: ['node:*'] });
4 changes: 4 additions & 0 deletions build/check-package.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,7 @@ for (const file of ['dist/browser-worker.mjs', 'dist/fontforge-core.mjs', 'dist/
}
const wasm = await readFile('dist/fontforge-core.wasm');
if (wasm.subarray(0, 4).toString('hex') !== '0061736d') throw new Error('Invalid WASM artifact.');

for (const name of ['fontforge-script.mjs', 'fontforge-script.wasm', 'script-browser-worker.mjs']) {
if (!(await stat(`dist/${name}`)).size) throw new Error('Run npm run build:scripting before packaging.');
}
15 changes: 15 additions & 0 deletions build/patch-scripting.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SPDX-License-Identifier: GPL-3.0-or-later
from pathlib import Path
p = Path('/work/fontforge/fontforge/scripting.c')
s = p.read_text()
needle = 'if ( found!=NULL ) {'
assert s.count(needle) == 1, 'Upstream dispatch changed; review capability guard'
guard = """if ( found!=NULL ) {
if (!strcmp(name, "AutoTrace") || !strcmp(name, "Autotrace") ||
!strcmp(name, "AskUser")) {
EM_ASM({ Module['onCapabilityError']?.(); });
ScriptError(&sub, "Command requires unavailable external programs or interactive UI");
goto docall_skipfunc;
}
"""
p.write_text('#include <emscripten.h>\n' + s.replace(needle, guard))
4 changes: 3 additions & 1 deletion demo-service-worker.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ const VERSION = 'fontforge-wasm-demo-0.2.0-alpha.1-formats';
const ASSETS = [
'examples/browser/', 'examples/browser/demo.js',
'examples/fonts/Roboto-Regular.ttf', 'examples/fonts/Roboto-Regular.otf', 'examples/fonts/LICENSE.txt',
'src/formats.js', 'src/containers.js', 'src/index.js', 'src/validate.js', 'src/browser-worker.js', 'src/runtime.js',
'src/formats.js', 'src/containers.js', 'src/index.js',
'src/execute.js',
'src/script-validate.js', 'src/validate.js', 'src/browser-worker.js', 'src/runtime.js',
'dist/browser-worker.mjs', 'dist/fontforge-core.wasm',
].map(path => new URL(path, self.registration.scope).href);
self.addEventListener('install', event => {
Expand Down
123 changes: 123 additions & 0 deletions docs/native-scripting-proof.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# Native scripting proof

Historical feasibility baseline. The follow-up [public API](scripting-api.md)
adds typed execution and runtime filesystem budgets; the original harness below
remains an internal parity test.

This implements the first feasibility gate in [Epic #1](https://github.com/warting/fontforge-wasm/issues/1).
It is an opt-in development build, not a released `execute()` API or a public
script playground. The existing conversion distribution and OFC integration
are unchanged.

## What runs

`native/script.c` initializes headless FontForge and calls upstream
`ProcessNativeScript`. It accepts only `-lang=ff -script file [args...]` and
`-lang=ff -c script [args...]`. It does not implement a replacement parser or
pretend to support the full CLI, stdin, Python or desktop GUI.

The proof runs this native script against an SFD source:

```text
Open($1);
Print($fontname);
Print($2);
Generate($2);
```

Arguments are passed as argv entries, without a shell. Tests cover Unicode and
shell-like characters, syntax errors, missing files and `Quit(7)`.

## Lifecycle decision

The upstream interpreter owns process-global state and terminates with `exit`.
Normal completion exits 0; non-interactive script errors exit 1. Do not patch
these into returns or reuse a module after exit. One job owns one fresh worker,
module and MEMFS. The supervisor terminates the worker on completion, error,
timeout or cancellation, then releases its browser blob URL.

The generated module uses `noInitialRun`, exported `callMain`, `onExit`, and
`EXIT_RUNTIME=1`. Its filesystem remains readable after a normal interpreter
exit, allowing the harness to copy explicitly requested output files. An
unexpected WASM trap is an execution failure, not a successful script exit.

The conversion build's function-pointer emulation combined with JS-based
setjmp/longjmp produced invalid output from wasm-opt when scripting was linked.
The separate script target uses WASM exceptions (`-fwasm-exceptions` and
`SUPPORT_LONGJMP=wasm`). FreeType must be rebuilt with the same longjmp ABI.
Conversion retains its existing flags and dependency artifacts.

## Reproduce

Prerequisites: Docker, Node >=22, npm dependencies, Python with
`fonttools==4.59.0` and `brotli==1.2.0`, and Playwright Chrome/Chromium + WebKit.
Run from the repository root:

```sh
npm ci
docker build -f build/Dockerfile --target scripting-export --output type=local,dest=artifacts/scripting .
node --test test/scripting/proof.test.mjs
docker build -f build/Dockerfile.scripting-native -t fontforge-wasm:scripting-native .
node test/scripting/parity.mjs
python test/scripting/verify-parity.py
node test/scripting/browser.mjs
```

The browser test defaults to installed Google Chrome. Set `BROWSER_CHANNEL=chromium`
for Playwright's bundled Chromium. `SCRIPT_BROWSER=Chrome` or `WebKit` selects
one engine. Use `PLAYWRIGHT_BROWSERS_PATH` when browsers are installed elsewhere.

The native reference independently downloads and hash-verifies the same pinned
FontForge source in `build/sources.json`; it uses native system dependencies
rather than the WASM dependency builds. Its container has networking disabled
at execution and uses `LANG=C.UTF-8`, matching the proof's UTF-8 arguments.

Results are written below ignored `test/results/scripting/`. Native and WASM
stdout/exit codes are compared. FontTools independently compares cmap, glyph
order, names, metrics, OpenType layout tables and every decomposed glyph outline.
Build timestamps are not required to match.

## Browser verification

The browser harness bundles the same runtime into a disposable worker. It tests
SFD-to-OTF, script failure, concurrent jobs, cancellation of an infinite loop,
timeout and successful execution after cancellation. It loads the worker code,
WASM and input explicitly before running the network-independent tests.

Chrome runs with Playwright offline mode enabled. On the tested macOS WebKit
runner, `setOffline(true)` prevents even a standalone `postMessage("ok")` blob
worker from starting. WebKit therefore runs with HTTP/HTTPS requests blocked,
and the test asserts that execution makes zero such requests. This proves no
network dependency during execution; it does not verify Safari service-worker
caching, offline reload or every Safari release.

## Measured build

With pinned Emscripten 5.0.7, the script WASM is 7,312,950 bytes and its generated
module is 80,624 bytes (uncompressed). The existing conversion WASM is 4,976,470
bytes. Keep scripting as a separate, opt-in download. These are local build
measurements, not a versioned release promise.

## Boundaries still to implement

The internal harness limits supplied files, copied outputs, logs and execution
time. WASM memory is capped at 512 MiB. It is **not** a production sandbox/API:

- Writable filesystem quotas must apply while scripts run, not just when copying
output. The current harness only limits input to 16 MiB, returned output to
32 MiB, supplied/selected file counts to 64, and logs to 64 KiB.
- Audit subprocess/network-dependent native commands and provide structured
capability errors. Existing platform stubs do not establish comprehensive
command compatibility.
- Define public argv/$0, working-directory, filename, partial-output, directory
output and error contracts. Tests currently select ordinary output files only.
- Add typed browser/Node `execute()`, complete resource validation, streaming
diagnostics, bounded assets and browser crash recovery.
- Expand native parity to selection, subsetting, metrics, transforms, contour
edits, source preservation and multi-file outputs.
- Build the public playground, cache/update integration and offline reload tests
only after those contracts are ready.

Do not publish this harness as a general-purpose script execution service.
Do not enable it on OFC's server/API endpoints. No product deployment is needed
for this feasibility gate.
Loading
Loading