Vendor-neutral runtime interface: runtime-loadable sensor plugins, generic transports, and replay/live parity

[drwnz]

Summary

This is the umbrella/tracking issue for an eight-PR stacked series that introduces a vendor-neutral, runtime-loadable plugin architecture beneath Nebula's existing vendor driver collection. A sensor vendor ships one .so and one descriptor JSON; the host discovers it, loads it through a versioned C ABI, constructs the transports it declares, routes packets to it, and consumes typed outputs — without modifying a single existing vendor file.

• PR 1 — test/runtime-common-return-mode — pin generic ReturnMode parsing contract
• PR 2 — feat/vendor-neutral-common-contracts — ROS-free common data model
• PR 3 — feat/vendor-neutral-decoder-contracts — plugin ABI & decoder runtime contract
• PR 4 — feat/vendor-neutral-sample-plugin — reference plugin implementation
• PR 5 — feat/vendor-neutral-packet-sources — generic UDP/TCP/CAN/HTTP/PCAP transports
• PR 6 — feat/vendor-neutral-runtime-core — plugin registry & packet routing
• PR 7 — feat/vendor-neutral-session-runners — live & replay session orchestration
• PR 8 — docs/vendor-neutral-runtime-interface — architecture & ABI documentation

Each PR is based on the one above it (PR 1 is based on main). Review order = stack order; merge order = stack order.

Motivation / problem statement

Nebula today is a compile-time, vendor-coupled driver collection:

1. Compile-time vendor binding. Every supported sensor is a statically linked vendor package. Adding a sensor means adding packages to this repo, recompiling everything, and shipping a new release. Third parties can't add a sensor without forking.
2. ROS leaks into the core. The legacy decoder base (nebula_driver_base.hpp) includes sensor_msgs/msg/point_cloud2.hpp, so even the "core" decoder layer can't build or run without a ROS workspace.
3. Per-vendor transport code. Each vendor's hw_interface re-implements socket setup/teardown around the shared connection classes; there is no common "packet in → decoder" pipeline.
4. No replay parity. PCAP replay, where it exists, takes a different code path than live ingestion, so a replay test doesn't prove the live pipeline.

Current architecture

graph TD
    ROS["nebula_ros<br/>(per-vendor node wrappers, params)"]
    H["nebula_hesai_*<br/>decoders / hw"]
    V["nebula_velodyne_*<br/>decoders / hw"]
    R["nebula_robosense_*<br/>decoders / hw"]
    DEC["nebula_core_decoders: NebulaDriverBase<br/>#include &lt;sensor_msgs/msg/point_cloud2.hpp&gt; ⚠️"]
    HW["nebula_core_hw_interfaces: UdpSocket, CAN, ..."]
    COM["nebula_core_common: configs, point types"]

    ROS --> H
    ROS --> V
    ROS --> R
    H --> DEC
    V --> DEC
    R --> DEC
    DEC --> HW
    HW --> COM

Loading

One decoder + hw_interface package per vendor, all statically linked, all selected at compile time.

Proposed architecture

graph TD
    APP["host application<br/>(ROS node, CLI tool, test, ...)"]
    REG["SensorRegistry (PR 6)<br/>descriptor discovery · dlopen · ABI check"]
    SO["libvendor_plugin.so<br/>+ descriptor JSON (runtime!)"]
    PLUGIN["SensorPlugin<br/>declares packet & transport requirements"]
    LIVE["LiveTransportGraph (PR 7)<br/>builds sources from requirements"]
    REPLAY["ReplaySessionRunner (PR 7)<br/>opens PCAP"]
    ROUTER["PacketRouter (PR 6)<br/>SAME router for live & replay"]
    RT["SensorDecoderRuntime::process_packet()"]
    OUT["SensorDecodedOutput (bounded variant)<br/>output / error / progress callbacks"]

    APP --> REG
    SO -. loaded at runtime .-> REG
    REG --> PLUGIN
    PLUGIN --> LIVE
    PLUGIN --> REPLAY
    LIVE --> ROUTER
    REPLAY --> ROUTER
    ROUTER --> RT
    RT --> OUT

    subgraph layers ["supporting layers"]
        L5["nebula_core_hw_interfaces (PR 5): UDP / TCP / CAN / HTTP / PCAP PacketSources"]
        L3["nebula_core_decoders (PR 3): SensorPlugin / SensorDecoderRuntime ABI"]
        L2["nebula_core_common (PR 2): SensorPacket, views, configs — ROS-FREE"]
    end

Loading

The inversion: vendors no longer write transport or node code. The plugin declares what it needs ("UDP data on port X", "CAN status on ID Y"); the host constructs the transports, routes packets to the declared channels, and consumes typed outputs.

---

Implementation plan — PR-by-PR detail

PR 1 — test/runtime-common-return-mode (+56 lines)

What: A single regression test pinning the behavior of return_mode_from_string() in nebula_core_common: generic names (SingleFirst, SingleStrongest, SingleLast, Dual) parse; vendor-specific combos (LastStrongest, FirstStrongest) return UNKNOWN at the common layer.

Why it exists: It's the anchor of the stack. The vendor-neutral layer relies on the common/vendor split of ReturnMode semantics — this test turns an implicit convention into an enforced contract before new code starts depending on it. Zero runtime changes; it also seeds the test target pattern reused by later PRs.

PR 2 — feat/vendor-neutral-common-contracts (~+563)

What: The ROS-free data model in nebula_core_common:

• SensorPacket — owning packet: transport kind, channel, timestamp, optional source/destination endpoints, optional CAN metadata, payload vector.
• SensorPacketView — non-owning hot-path view; construction from temporaries is deleted (both && and const&&) so it cannot dangle.
• SensorEndpoint/SensorCanMetadata — fixed-size (std::array<char,46> IPv6-max / 16-byte IFNAMSIZ) so the packet hot path never heap-allocates; over-length input throws std::length_error.
• SensorDecodedOutput — a bounded std::variant (pointcloud, radar detections/objects, status, diagnostics, telemetry).
• SensorConfiguration : public LidarConfigurationBase — extends the existing config chain rather than forking a parallel one.
• Standalone CMake package export so the package builds with or without ament.

Why: Everything above this layer (plugins, routing, sessions) needs a vocabulary that is allocation-disciplined and ROS-free. Verified: no rclcpp/sensor_msgs/std_msgs anywhere transitively. The "extend, don't fork" config decision means existing vendor configuration code keeps working unmodified.

PR 3 — feat/vendor-neutral-decoder-contracts (~+406)

What: The plugin ABI in nebula_core_decoders:

• SensorPlugin — pure-virtual: metadata(), packet_requirements(config), live_transport_requirements(config), create_decoder_runtime().
• SensorDecoderRuntime — pure-virtual decode host: configure, process_packet(const SensorPacketView&), flush, output/error/progress callback setters, plus an experimental set_sink() RT hook (virtual-dispatch, allocation-free path; honestly flagged as not yet wired).
• sensor_plugin_export.hpp — the three extern "C" symbols every plugin exports: create_nebula_sensor_plugin, destroy_nebula_sensor_plugin, nebula_plugin_abi_version (returns kNebulaPluginAbiVersion = 1).
• PacketChannelRequirement / LiveTransportRequirement — how a plugin declares its traffic (transport kind, channel, port/CAN-ID, optional payload signature with offset/bytes/mask).
• Legacy NebulaDriverBase + its sensor_msgs dependency preserved behind explicit TODO(drwnz): remove scaffolding, gated on the ament environment.

Why: This is the actual vendor-neutral boundary. extern "C" symbols are mangling-proof so plugins can be built with different compilers/standard libraries than the host; the explicit ABI version lets the registry reject incompatible binaries instead of crashing. Keeping the legacy ROS base intact (marked temporary) means existing vendor decoders compile untouched while migration happens gradually — the stack's core "no vendor files changed" guarantee.

PR 4 — feat/vendor-neutral-sample-plugin (~+510)

What: A complete reference implementation: SampleSensorPlugin + SampleSensorDecoderRuntime in nebula_sample_decoders, the descriptor nebula_sample_plugin.json, a draft-07 JSON schema, and the adapter that copies from SensorPacketView into the pre-existing SampleDecoder.

Why: A contract without a reference implementation invites divergent interpretations. The sample is the executable specification — it's what PR 6/7 tests load via real dlopen, and it's the template a vendor copies (verified to override every pure-virtual with exact signatures, export the three symbols correctly, and pair new/delete across the factory/destroy boundary). Deliberately not a production sensor, so contract questions stay separate from vendor questions.

PR 5 — feat/vendor-neutral-packet-sources (~+1510)

What: Generic transport ingress in nebula_core_hw_interfaces: a PacketSource interface and five implementations —

• UDP (surfaces the sender endpoint into SensorPacket.source),
• TCP (documented as stream-chunked; framing left to plugins),
• CAN (correct EFF flag/mask extended-ID handling),
• HTTP polling source (configurable timeout/interval) + HttpControlEndpoint for request/response control channels,
• PCAP replay with full IP-fragment reassembly (interval-merge based, handles out-of-order fragments).

Shared utilities give every source the same lifecycle (shared_ptr<atomic<bool>> run-flag captured into the worker thread; join-on-stop; callback exceptions caught and logged, never thread-fatal). Also fixes two latent bugs in the existing UdpSocket (drop-counter wrap off-by-one, sender-port filtering).

Why: This replaces the per-vendor hw_interface pattern. Transport handling is written once, hardened once (RAII fds, race-free stop, exception policy), and reused by every plugin — a vendor never opens a socket again. The PCAP source produces packets with the same metadata shape (source/destination endpoints) as the live sources, which is the precondition for replay parity in PR 7.

PR 6 — feat/vendor-neutral-runtime-core (~+1609)

What: The nebula_core_runtime package:

• SensorRegistry — discovers descriptor JSONs (NEBULA_PLUGINS_PATH, AMENT_PREFIX_PATH, COLCON_PREFIX_PATH), validates them (must declare ≥1 known model; unknown models rejected; duplicate packages keep-first-with-warning; cross-package duplicate model ownership warned), then dlopens the library, resolves factory/destroy, checks nebula_plugin_abi_version (mismatch → rejected; missing → warn-and-accept for legacy .sos). The returned shared_ptr<SensorPlugin>'s deleter captures the library handle, so dlclose can never precede plugin destruction — the classic plugin-system use-after-free is structurally impossible.
• PacketRouter — sorted flat vectors matched via std::lower_bound on port/CAN-ID plus optional payload signatures; stamps the channel on the view; documented as externally-serialized (no internal locking — an honest contract).
• Tests including two purpose-built test .sos exercising both ABI rejection paths with real dlopen.

Why: Registry and router are split from session orchestration deliberately — discovery/validation rules and routing rules are the security- and correctness-critical surface, and this keeps them reviewable in isolation. The flat-vector router is the hot path: O(log n) match, no allocation, no locks.

PR 7 — feat/vendor-neutral-session-runners (~+1227)

What: The orchestration hosts that make the layers usable:

• ReplaySessionRunner — config → registry lookup → plugin load → runtime + router construction → PCAP source; teardown-before-rebuild so a throwing configure() leaves the runner cleanly unconfigured, never half-built.
• LiveTransportGraph — translates each LiveTransportRequirement into the matching PR 5 source (UDP→bind host_ip, TCP→connect sensor_ip, CAN→interface from extra_params, HTTP→control endpoint); three-mutex design (lifecycle / state / processing) with snapshot-under-lock packet dispatch.
• Both runners expose get_router_metrics() and wrap user callbacks in identical swallow-and-log policy.
• Tests: a genuine end-to-end replay (writes a real 10-packet PCAP, runs it through the real source thread, asserts a decoded PointCloud) plus a multi-transport test plugin driving all four transport-mapping branches.

Why: This is where the central architectural promise lands: replay and live traffic flow through the same PacketRouter::route() → process_packet() dispatch. A PCAP regression test therefore exercises the same routing logic production uses — previously impossible. Destructors stop source threads before member destruction so a callback can never land on a dead router.

PR 8 — docs/vendor-neutral-runtime-interface (~+658)

What: docs/vendor_neutral_runtime_interface.md — architecture, package map, contracts, plugin ABI + descriptor format, routing semantics, live/replay behavior, threading and RT notes, sample-plugin guidance — linked from README, design, integration, API reference, index, and site nav.

Why: A plugin ABI only works if external authors can implement it from the docs alone. The doc was verified claim-by-claim against the code and tracked the code edits in lockstep (e.g., when truncation-asserts became std::length_error throws in PR 2, the doc was updated the same day). It explicitly states vendor adapters are future work and the experimental sink is not yet a functional RT path — it documents what exists, not what's aspired to.

---

Design rationale

Decision	Justification
Runtime plugins (dlopen) over compile-time linking	Sensors can be added/updated without touching or recompiling Nebula; vendors can ship proprietary plugins; host and driver release cycles decouple.
extern "C" + explicit ABI version	C symbols survive compiler/STL differences; the version gate converts "mysterious crash" into "rejected with a logged error". The deleter-captures-library pattern makes unload-order bugs unrepresentable.
ROS-free core	The decode pipeline becomes usable in non-ROS contexts (tooling, tests, RT processes) and testable without a ROS workspace; ROS conversion moves to the edges where it belongs. The legacy sensor_msgs base is retained behind marked-temporary scaffolding so nothing breaks meanwhile.
Declared requirements / inversion of control	The plugin states what traffic it needs; the host owns how sockets are created and supervised. Transport hardening (RAII, stop semantics, exception policy) is implemented once instead of per vendor.
Replay ≡ live routing	PCAP tests become real pipeline tests. This is the single biggest testability win — the e2e replay test in PR 7 proves the same path live traffic takes.
Fixed-size hot-path types + non-owning views	No heap allocation per packet; deleted temporaries-to-view conversions make the dominant lifetime bug a compile error. The RT sink hook reserves an allocation-free output path without overpromising (explicitly experimental).
Stacked, strictly-layered PRs	Each layer (data model → ABI → reference impl → transport → registry/routing → orchestration → docs) is independently reviewable; in review, every issue was attributable to exactly one layer.
Sample plugin instead of a first vendor port	Contract validation is decoupled from vendor complexity; the sample doubles as the test fixture and the vendor template.

Out of scope / future work

• No production vendor is migrated by this stack. Hesai / Robosense / Velodyne / Continental packages are untouched; vendor adapter PRs land separately, one vendor at a time, against this substrate.
• Wiring the experimental SensorOutputSink RT path into the sample runtime and LiveTransportGraph.
• TCP message framing (the TCP source is documented as stream-chunked; framing is currently the plugin's responsibility).
• Removal of the temporary legacy sensor_msgs compatibility scaffolding in nebula_core_decoders once vendor decoders migrate.
• A started-session end-to-end test for LiveTransportGraph (live socket → route → PointCloud); configure-time coverage exists for all four transports.
• A plain-CMake (non-ament) CI job, so the non-ROS build path is continuously validated.

Review / validation

Each PR carries its own review procedure and build validation (see the per-PR descriptions); the stack builds with:

colcon build --packages-up-to nebula_core_runtime \
  --event-handlers console_direct+ \
  --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo

Documentation (PR 8) is validated with pre-commit over the changed files.