Team UTA-RVL - Event-Guided Semantic Segmentation with CoSEC ; EBMV @ ECCV2026

Official code for Team UTA-RVL's 3rd place submission to the EBMV @ ECCV 2026 Event-Guided Semantic Segmentation with CoSEC.

Requirements

• Python 3.11 or newer
• uv
• All training, experiments, and inference were tested on Ubuntu and Debian with NVIDIA RTX3090 and A40.

Create the environment:

uv sync --extra ml

Contents

configs/                  two training stages and final inference
scripts/                  training, reproduction, and zip validation
src/eccv_challenge/       minimal data, model, training, and I/O code
trained_weights/          checksum manifest and downloaded final checkpoint

The pinned public initialization is downloaded automatically from Hugging Face:

• NVIDIA SegFormer-B5 fine-tuned on Cityscapes
• model revision: 2c6f153e4c23c229e2fa2b188eb250607e030cd8

The model's Hugging Face license and upstream Cityscapes terms apply to that checkpoint.

Release Artifacts

Download the release artifacts and extract the trained_weights/ directory into this repository. The result must include:

trained_weights/segformer_b5_rgbd_zero_epoch_0004.pt

Verify the download before inference:

sha256sum -c trained_weights/SHA256SUMS

Data Layout

Place the official challenge data at data/segmentation_challenge, or set DATA_ROOT when invoking a script:

data/segmentation_challenge/
├── train/<Sequence>/
│   ├── img_co_left/<frame>.png
│   ├── segment_co/<frame>.png
│   ├── events_co_left.h5
│   └── timestamps.txt
└── test/<Sequence>/
    ├── img_co_left/<frame>.png
    ├── events_co_left.h5
    └── timestamps.txt

Events are present in the official layout but are not loaded or used by this method.

Reproduce The Submission

uv run ./scripts/reproduce_submission.sh

The output is outputs/reproduction/submission.zip. The submitted archive's SHA-256 is:

ba1abddab24b5297f4e5c3b09c48cb1b377c7b9acd7a65348ba71d20cf6ed12e

The exact zip hash can vary across PNG/zlib library versions even when masks are identical. The validator checks all frame names, shapes, dtypes, and class IDs against the supplied test tree.

Optional paths and device:

DATA_ROOT=/path/to/data \
WEIGHTS=/path/to/segformer_b5_rgbd_zero_epoch_0004.pt \
OUT_DIR=/path/to/output \
uv run ./scripts/reproduce_submission.sh

Train From The Public Cityscapes SegFormer-B5 Model

Two stages are involved:

1. Fine-tune RGB SegFormer-B5 for eight epochs on the released Fold-A training sequences with full-resolution frames, weighted cross entropy, and random horizontal flips. Stage 2 uses the Stage 1 epoch-6 checkpoint.
2. Widen the input stem to six channels by copying the RGB kernels and zero initializing the additional kernels, then fine-tune for four epochs on all 14 released labeled sequences. The released model is epoch 4.

Run both stages with:

uv run ./scripts/train_from_pretrained.sh

Use the supplied checkpoint for exact submitted predictions as training can be stochastic.

See trained_weights/README.md for checkpoint provenance.