RoSeMAN

Roobonomics Seensors Measure Analytics and Archive Node -- the indexer and analytics backend for sensors.social. RoSeMAN indexes legacy sensor datalogs and the CPS protocol from the Robonomics parachain, verifies IPFS payloads, stores measurements in MongoDB and serves them via REST API.

Features

• Robonomics blockchain indexer (Polkadot/Kusama) — reads finalized blocks and processes datalog.NewRecord events and RWS extrinsics.
• IPFS loader — asynchronous processing of datalog records with CIDs: fetches JSON via a list of gateways with fallback, parses it and stores sensor measurements.
• CPS ingestion — snapshots configured numeric NodeIds, handles realtime cps.PayloadSet, reads binary CIDs, decodes raw/XZ/zlib protobuf batches and verifies Ed25519 signatures before storing public Urban/Insight measurements.
• Reverse geocoding — derives country/region/city from sensor coordinates.
• REST API — sensor data (V1/V2), story list, indexer status. See docs/api_endpoints.md.
• Prometheus metrics at /metrics.
• Flexible composition — every functional module (API, INDEXER, MEASUREMENT, GEOCODING) is toggled by environment flags, which makes it possible to run the REST API, indexer and IPFS processor as separate instances.

Stack

• NestJS 11 + TypeScript (ESM)
• MongoDB via Mongoose
• @polkadot/api + robonomics-api-augment
• Buf Protobuf, Ed25519 and XZ/LZMA2 for CPS payloads
• ESLint + Prettier

A detailed description of the indexer, data formats and DB schemas is in docs/indexer.md.

Project structure

src/
├── api/                  REST controllers: sensor (V1/V2), story, status
├── robonomics/           chain connection, block indexer, CPS snapshot and handlers
├── measurement/          legacy/CPS processors, IPFS fetcher and protocol codecs
├── geocoding/            sensor reverse geocoding
├── metrics/              Prometheus metrics
├── database/
│   ├── schemas/          Mongoose schemas
│   └── repositories/     DB access (Repository pattern)
├── config/               configs (app, robonomics, ipfs, geocoding, cps)
├── common/               constants and utilities
└── app.module.ts         dynamic module composition driven by env flags

Configuration

Environment files:

• .env — shared settings + REST API (see .env.example)
• .env.polkadot — Polkadot indexer (see .env.polkadot.example)
• .env.kusama — Kusama indexer (see .env.kusama.example)

Key flags for splitting processes:

Variable	Purpose
API_ENABLED	REST API + Prometheus
INDEXER_ENABLED	Robonomics block scanner
MEASUREMENT_ENABLED	Legacy datalog and CPS queue processing
GEOCODING_ENABLED	Reverse geocoding
CPS_ENABLED	CPS snapshot, realtime handler and CPS processor
CPS_NODE_IDS	Snapshot NodeIds and optional realtime allowlist
ENABLED_HANDLERS	Allowlist of indexer handlers (comma-separated)
DISABLED_HANDLERS	Denylist of handlers (applied on top of the allowlist)

The full list of variables and defaults is in the *.example files and in docs/indexer.md. CPS_ENABLED only activates work inside enabled indexer/measurement modules. An empty CPS_NODE_IDS disables snapshot discovery but allows realtime events from any numeric NodeId.

Running

Locally (dev)

npm install
npm run start:dev                  # API + all modules from .env
npm run start:dev:polkadot         # Polkadot indexer (.env.polkadot)
npm run start:dev:kusama           # Kusama indexer  (.env.kusama)

Production

npm run build
npm run start:prod                 # API
npm run start:polkadot             # Polkadot indexer
npm run start:kusama               # Kusama indexer

Docker

The docker-compose.yml ships MongoDB and three application instances: REST API, Polkadot indexer, Kusama indexer. Configuration is supplied to the containers via bind-mounts of the corresponding .env files.

cp .env.example .env
cp .env.polkadot.example .env.polkadot
cp .env.kusama.example .env.kusama
docker compose up -d

REST API will be available at http://localhost:3000/api, metrics — at /metrics.

npm scripts

Command	Description
build	Build via nest build
start / start:dev	Run (with watch in dev)
start:prod	Run the built dist/main
sync-indexes	Sync MongoDB indexes with the schemas (see docs/database.md)
format	Prettier over src/ and test/
lint	ESLint with autofix
test	Jest unit tests, sequentially (--runInBand)
test:e2e	Jest with the config from test/

Documentation

The full documentation lives in docs/:

• Architecture — modules, run modes, data flow
• Deployment — npm/Docker, multi-instance setup
• Indexer — BlockIndexer, handlers, IPFS, geocoding, DB schemas, configuration
• REST API — HTTP layer design, validation, error handling
• Endpoint reference — REST endpoint reference
• Database — Repository pattern, repositories overview
• Metrics — Prometheus /metrics endpoint