RoSeMAN

Roobonomics Seensors Measure Analytics and Archive Node -- the CPS indexer and analytics backend for sensors.social. RoSeMAN indexes Connectivity Protocol data from the Robonomics parachain, verifies signed payloads, stores canonical messages in MongoDB and serves them via REST API.

Deprecation notice: Datalog/RWS indexing, the measurements/cities compatibility projection and the Sensor/Story APIs are deprecated and retained only for existing deployments. New deployments should use CPS indexing and the V3 Connectivity API. See Legacy functionality deprecation.

Features

• Robonomics blockchain indexer (Polkadot) — reads finalized blocks and processes CPS payload events.
• Legacy compatibility — Datalog/RWS handlers and their storage/API remain available only for existing deployments.
• CPS ingestion — snapshots configured numeric NodeIds, handles realtime cps.PayloadSet, resolves legacy binary CIDs through IPFS or reads compressed batches directly from CPS storage, selects Zstandard/XZ/LZ4 from cps.meta, then verifies Ed25519 signatures before storing public Urban/Insight measurements.
• Reverse geocoding — derives country/region/city from sensor coordinates.
• REST API — sensor data (V1/V2), public Connectivity Protocol messages (V3), story list and indexer status. See docs/api_endpoints.md.
• Prometheus metrics at /metrics.
• Flexible composition — every functional module (API, INDEXER, MEASUREMENT, GEOCODING) is toggled by environment flags, which makes it possible to run the REST API, indexer and IPFS processor as separate instances.

Stack

• NestJS 11 + TypeScript (ESM)
• MongoDB via Mongoose
• @polkadot/api + robonomics-api-augment
• Buf Protobuf, Ed25519, Zstandard, XZ/LZMA2 and LZ4 frame for CPS payloads
• ESLint + Prettier

A detailed description of the indexer, data formats and DB schemas is in docs/indexer.md.

Project structure

src/
├── api/                  REST controllers: sensor (V1/V2), connectivity (V3), story, status
├── robonomics/           chain connection, block indexer, CPS snapshot and handlers
├── measurement/          legacy/CPS processors, IPFS fetcher and protocol codecs
├── geocoding/            sensor reverse geocoding
├── metrics/              Prometheus metrics
├── database/
│   ├── schemas/          Mongoose schemas
│   └── repositories/     DB access (Repository pattern)
├── config/               configs (app, robonomics, ipfs, geocoding, cps)
├── common/               constants and utilities
└── app.module.ts         dynamic module composition driven by env flags

Configuration

Environment files:

• .env — shared settings + REST API (see .env.example)
• .env.polkadot — Polkadot CPS indexer (see .env.polkadot.example); its handler allowlist contains only cps-payload-set

Key flags for splitting processes:

Variable	Purpose
API_ENABLED	REST API + Prometheus
INDEXER_ENABLED	Robonomics block scanner
MEASUREMENT_ENABLED	CPS queue processing plus deprecated compatibility jobs
GEOCODING_ENABLED	Deprecated reverse geocoding of the cities projection
CPS_ENABLED	CPS snapshot, realtime handler and CPS processor
CPS_CANONICAL_STORAGE_ENABLED	Canonical CPS occurrence writes (default off)
CPS_RAW_PAYLOAD_STORAGE_ENABLED	Exact CPS payload archive writes (default off)
CPS_NODE_IDS	Snapshot NodeIds and optional realtime allowlist
ENABLED_HANDLERS	Allowlist of indexer handlers (comma-separated)
DISABLED_HANDLERS	Denylist of handlers (applied on top of the allowlist)

The full list of variables and defaults is in the *.example files and in docs/indexer.md. CPS_ENABLED only activates work inside enabled indexer/measurement modules. An empty CPS_NODE_IDS disables snapshot discovery but allows realtime events from any numeric NodeId.

Running

Locally (dev)

npm install
npm run start:dev                  # API + all modules from .env
npm run start:dev:polkadot         # Polkadot indexer (.env.polkadot)

Production

npm run build
npm run start:prod                 # API
npm run start:polkadot             # Polkadot indexer

Docker

The docker-compose.yml ships MongoDB, REST API and the Polkadot indexer. The image is built directly from the current project sources by the multi-stage Dockerfile. Compose passes the common .env to every application and overlays .env.polkadot for the indexer.

cp .env.example .env
cp .env.polkadot.example .env.polkadot
docker compose up -d --build

No separate npm run build or external Docker image is required. REST API will be available at http://localhost:3000/api, metrics — at /metrics. The host ports, image name, MongoDB credentials and Compose project name can be changed in .env.

npm scripts

Command	Description
build	Build via nest build
start / start:dev	Run (with watch in dev)
start:prod	Run the built dist/main
sync-indexes	Sync MongoDB indexes with the schemas (see docs/database.md)
backfill-connectivity	Backfill canonical CPS payloads/records for completed anchors
format	Prettier over src/ and test/
lint	ESLint with autofix
test	Jest unit tests, sequentially (--runInBand)
test:e2e	Jest with the config from test/

Documentation

The full documentation lives in docs/:

• Architecture — modules, run modes, data flow
• Deployment — npm/Docker, multi-instance setup
• Indexer — BlockIndexer, handlers, IPFS, geocoding, DB schemas, configuration
• REST API — HTTP layer design, validation, error handling
• Endpoint reference — REST endpoint reference
• Database — Repository pattern, repositories overview
• Metrics — Prometheus /metrics endpoint
• Legacy deprecation — deprecated components, runtime signals and migration targets