Workzone Perception

A multi-modal perception pipeline for detecting roadwork zones using synchronized RGB camera and LiDAR point cloud data. The system benchmarks six deep learning architectures across two tasks:

• Semantic Segmentation — pixel-level detection of barriers and road beacons (DeepLabv3+, Swin-UperNet, SegFormer)
• 3D Object Detection — 3D bounding box prediction from camera-only, LiDAR-only, and camera-LiDAR fusion inputs (SMOKE, PointPillars, MVXNet)

---

Hardware

Task	GPU	Notes
3D Object Detection	2× NVIDIA RTX 4090 (24 GB each)	Distributed training via dist_train.sh
Semantic Segmentation	1× NVIDIA RTX 4080 (16 GB)	Single-GPU training

---

Sensor Suite

Sensor	Model	Spec
Camera	Basler	1920 × 1200 px, colour
LiDAR	Ouster OS1-128	128 beams, float32 [x, y, z, intensity] binary
GPS/IMU	Certus GNSS/INS	Centimetre-level GPS, 200 Hz IMU

---

Dataset

RZDG-Real — 1357 frames, KITTI format

• Classes: Barrier, Road_beacon
• Detection split: 1085 train / 272 val (80/20, seed 42)
• Segmentation split: 687 train / 172 val (80/20, seed 42)

RZDG-Sim — CARLA simulator data

• Detection: ~6872 train / 1719 val
• Segmentation: ~6420 train / 802 val

Dataset: https://github.com/chrisyan/RZDG

---

Quick Start

# 1. Clone this repository
git clone https://github.com/ShyamShenoi/ROAD_WORKZONE_PERCEPTION.git
cd ROAD_WORKZONE_PERCEPTION

# 2. Set up 3D detection environment (creates conda env rzdg_det)
bash object_detection/setup/install.sh

# 3. Set up segmentation environment (creates conda env rzdg_seg)
bash semantic_segmentation/setup/install.sh

# 4. Prepare detection dataset (point to your RZDG-Real root)
conda activate rzdg_det
python object_detection/data/prepare_dataset.py \
    --input_dir /path/to/RZDG-Real \
    --output_dir object_detection/data/kitti

# 5. Train detection models
bash object_detection/training/train.sh smoke
bash object_detection/training/train.sh pointpillars
bash object_detection/training/train.sh mvxnet

# 6. Train segmentation models
conda activate rzdg_seg
bash semantic_segmentation/training/train.sh deeplabv3plus
bash semantic_segmentation/training/train.sh swin
bash semantic_segmentation/training/train.sh segformer

---

Project Structure

workzone-perception/
├── README.md
├── .gitignore
│
├── object_detection/
│   ├── setup/
│   │   ├── install.sh                ← One-command environment setup
│   │   └── verify.sh                 ← Check install is correct
│   ├── data/
│   │   ├── prepare_dataset.py        ← Convert + split + generate .pkl infos
│   │   ├── verify_dataset.py         ← Sanity-check every file
│   │   └── visualize_sample.py       ← Project LiDAR→image, draw BEV
│   ├── configs/
│   │   ├── smoke/smoke_rzdg_real.py
│   │   ├── pointpillars/pointpillars_rzdg_real.py
│   │   └── mvxnet/mvxnet_rzdg_real.py
│   ├── training/
│   │   ├── train.sh                  ← ./train.sh <model>
│   │   └── monitor_training.py       ← Plot live loss curves
│   └── evaluation/
│       ├── evaluate.sh               ← ./evaluate.sh <model>
│       ├── compute_ap.py             ← AP3D + APBEV computation
│       ├── generate_table.py         ← Reproduce detection results
│       └── measure_inference_time.py ← GPU timing with CUDA synchronize
│
└── semantic_segmentation/
    ├── setup/
    │   └── install.sh
    ├── data/
    │   ├── prepare_dataset.py        ← LabelMe JSON → PNG masks
    │   ├── verify_dataset.py
    │   └── visualize_sample.py       ← Overlay masks on images
    ├── configs/
    │   ├── deeplabv3plus/deeplabv3plus_r101_rzdg_real.py
    │   ├── swin/swin_upernet_rzdg_real.py
    │   └── segformer/segformer_mit-b5_rzdg_real.py
    ├── training/
    │   ├── train.sh
    │   └── simple_train.py           ← Standalone PyTorch training (no mmseg)
    └── evaluation/
        ├── evaluate.sh
        ├── compute_metrics.py        ← mIoU + confusion matrix from scratch
        └── generate_table.py         ← Reproduce segmentation results

---

Switching Between RZDG-Real and RZDG-Sim

Every config and data script has clearly marked switch comments:

# ── RZDG-Real  ────────────────────────────────────────────────────────────────
data_root = '../data/kitti/'          # 1357 real frames
# ── RZDG-Sim (CARLA) — uncomment these lines instead ─────────────────────────
# data_root = '../data/kitti_sim/'    # ~8000 CARLA frames
# img_scale = (1280, 720)             # CARLA default resolution
# ─────────────────────────────────────────────────────────────────────────────

Key differences for RZDG-Sim:

• Image resolution: 1280×720 (vs 1920×1200 Real)
• Segmentation masks from CARLA semantic camera — no manual annotation needed
• Calibration matrices differ — regenerate with your CARLA camera parameters