Date: 2026-03-09
Model: Qwen/Qwen3-VL-8B-Instruct
Dataset: 200 videos (MicroCV subset)
Overall Type Accuracy: 29.0%
The model shows severe class imbalance in predictions:
- t-bone collisions: 90% accuracy ✅ (only collision type it understands)
- head-on collisions: 2.6% accuracy ❌ (predicts as t-bone 56%, rear-end 39%)
- sideswipe collisions: 2.7% accuracy ❌ (predicts as rear-end 65%, t-bone 32%)
- rear-end collisions: 26% accuracy ❌ (predicts as t-bone 73%)
- single-vehicle: 0% accuracy ❌ (predicts as t-bone 57%, rear-end 43%)
Key Finding: The model has learned a strong bias toward t-bone and rear-end predictions, likely because these are easier to identify from relative vehicle angles. Head-on and sideswipe collisions require understanding relative motion along the same axis, which the model fails to grasp.
What the model gets right:
- Vehicles approaching at 90-degree angles
- Intersection collisions where paths cross perpendicularly
- Clear visual cue: one vehicle moving horizontally, another vertically
Visual Pattern (from grid images): ![T-Bone Example: Town03_t-bone_clear_29.mp4]
- Frame 0-5: Blue car enters roundabout from right, white car travels horizontally
- Frame 6-9: Clear perpendicular approach visible
- Frame 10: Impact at 90-degree angle at roundabout center
- Model correctly identifies: "blue car entering from the right collides with white car moving horizontally, forming a clear t-bone intersection"
Example correct reasoning:
"At F7 8.4s, a red car turning left collides with a black car moving straight, forming a clear t-bone intersection."
Why it works: The perpendicular angle is visually distinctive and matches the model's spatial understanding. From a top-down view, the crossing paths are unambiguous.
Predicted as:
- t-bone: 22 videos (56.4%)
- rear-end: 15 videos (38.5%)
- head-on: 1 video (2.6%) ✓
What the model sees instead:
The model sees one vehicle "behind" another in the frame and assumes same-direction travel.
Visual Pattern (from grid images - Town05_head-on_clear_37.mp4): ![Head-On Example: Predicted as rear-end]
- Frame 0-3: White bus traveling down (toward camera), red car traveling up (away from camera) on same road
- Frame 4-8: Vehicles appear vertically aligned in the frame as they approach
- Frame 9-11: Contact occurs front-to-front, but model sees "red car behind white bus"
- Model incorrectly reasons: "At F10 14.8s, a red car is directly behind a white bus and appears to make contact with its rear"
Why this fails: The top-down camera angle makes vehicles traveling in opposite directions appear "stacked" or aligned. The model interprets this spatial arrangement as "one behind the other" rather than "approaching from opposite ends of the same road."
Critical issue: The model doesn't track direction of motion across frames. It only looks at single-frame spatial position.
The model interprets vehicles approaching from opposite directions as crossing perpendicular paths.
Example errors:
Town05_head-on_night_18.mp4 → Predicted: t-bone
Reasoning: "At F10 17.5s, a vehicle moving vertically collides with a vehicle
moving horizontally, consistent with a t-bone collision."
Why this fails: The model sees two vehicles meeting at any angle and assumes perpendicular motion. In reality, head-on collisions on straight roads may have slight curvature or camera angles that make the vehicles appear to be at an angle.
Predicted as:
- rear-end: 24 videos (64.9%)
- t-bone: 12 videos (32.4%)
- sideswipe: 1 video (2.7%) ✓
What the model sees instead:
The model sees two vehicles in the same lane and assumes one is rear-ending the other.
Visual Pattern (from grid images - Town06_sideswipe_rain_01.mp4): ![Sideswipe Example: Predicted as rear-end]
- Frame 0-3: White van (top lane) and yellow taxi (bottom lane) both traveling forward, parallel
- Frame 4-8: Vehicles remain side-by-side, yellow taxi slightly ahead
- Frame 9-11: Glancing side-to-side contact as white van drifts into yellow taxi's lane
- Model incorrectly reasons: "In frame F9 at 14.8s, a white van is directly behind a yellow taxi and appears to be making contact with its rear"
Why this fails: The top-down camera angle makes side-by-side vehicles appear front-to-back depending on which vehicle is slightly ahead. The model doesn't recognize:
- Parallel motion (both moving same direction)
- Adjacent lane positioning (side-by-side, not front-to-back)
- Lateral contact (door-to-door, not bumper-to-bumper)
Critical issue: Sideswipe collisions involve subtle lateral drift while maintaining forward motion. The model sees "one vehicle ahead of another" and assumes rear-end, missing the side-by-side context.
The model sees an angle between vehicles and assumes perpendicular collision.
Visual Pattern (from grid images - Town04_sideswipe_wet_22.mp4 - WORST SPATIAL ERROR): ![Worst Spatial Error: Sideswipe predicted as t-bone]
- Frame 0-1: Red bus (right lanes) and white car (left lanes) both traveling in same direction
- Frame 2-5: Red bus crosses into white car's path at a shallow angle (15-20 degrees, not 90)
- Frame 6-11: Side-glancing collision as bus sidesweeps car
- Model incorrectly reasons: "At F3 5.6s, a red bus moving right-to-left collides with a white car moving top-to-bottom, forming a clear t-bone intersection"
Why this fails: The model sees any angle between vehicles as "perpendicular." It doesn't distinguish between:
- 90-degree angle (t-bone): vehicles on perpendicular roads
- 15-30 degree angle (sideswipe): vehicles on same road, one drifting laterally
Spatial error explanation: By picking frame 3 (early) and assuming t-bone at intersection center, the model predicts a collision location far from the actual side-swipe contact point (frame 6+, different location on road).
Predicted as:
- t-bone: 56 videos (72.7%)
- rear-end: 20 videos (26.0%) ✓
What the model sees instead:
The model over-predicts t-bone collisions for rear-ends, suggesting it sees any angle between vehicles as a perpendicular intersection.
Visual Pattern (from grid images - Town04_rear-end_wet_19.mp4): ![Rear-End Example: Predicted as t-bone]
- Frame 0-3: Red car and blue car both traveling forward (same direction) at a slight angle due to road curve
- Frame 4-5: Red car slows/stops ahead, blue car approaches from behind
- Frame 6-11: Blue car strikes red car from behind (classic rear-end)
- Model incorrectly reasons: "At F6 11.2s, a red car moving horizontally collides with a blue car moving vertically, forming a clear t-bone intersection"
Why this fails: The top-down camera angle + road curvature makes vehicles appear to be at an angle. The model sees this angle and assumes "perpendicular paths" rather than "same road, same direction, curved alignment."
Critical issue: Rear-end collisions require understanding:
- Same direction travel (both moving forward on same road)
- Speed differential (rear vehicle faster than front vehicle)
- Front-to-back contact (rear bumper of front vehicle ↔ front bumper of rear vehicle)
The model only looks at spatial position and angle, missing the motion context.
Pattern: Many rear-ends occur on curved roads or at intersections where vehicles are turning. The curved path creates an apparent "angle" between vehicles, which the model misinterprets as perpendicular t-bone motion.
Predicted as:
- t-bone: 4 videos (57.1%)
- rear-end: 3 videos (42.9%)
What the model sees instead:
The model hallucinates a second vehicle even when only one is present.
Example error:
Town05_head-on_night_39.mp4 → Predicted: single (correct classification but wrong type)
Reasoning: "A red car is seen hitting a wall at F10 18.6s, with visible deformation
and smoke indicating impact."
Why this fails: The model is trained to expect two vehicles in collision. When it sees only one, it either:
- Misinterprets the environment (wall, barrier) as another vehicle
- Sees the vehicle at an angle and assumes t-bone/rear-end with an "invisible" vehicle
Visual Pattern: Single-vehicle accidents involve hitting static objects (walls, barriers, trees). The model needs to recognize lack of a second moving vehicle.
Mean Temporal Error: 5.71 seconds
Median Temporal Error: 4.80 seconds
Range: 0.05s to 16.6s
Bias: Model predicts too late in 88.5% of cases (177/200 videos).
| Video | GT Type | Pred Type | Error | GT Time | Pred Time |
|---|---|---|---|---|---|
| Town05_head-on_night_43.mp4 | head-on | rear-end | 16.6s | 7.8s | 24.4s |
| Town03_head-on_night_32.mp4 | head-on | t-bone | 16.4s | 3.5s | 19.8s |
| Town03_t-bone_night_00.mp4 | t-bone | t-bone | 14.5s | 6.4s | 20.9s |
| Town05_head-on_night_00.mp4 | head-on | t-bone | 14.1s | 5.9s | 20.0s |
Pattern: The model consistently selects frame 10 (or late frames) as the collision point, even when the collision happens much earlier. This suggests:
- Bias toward final frames: The prompt may be training the model to look for "maximum visual damage" rather than "first contact"
- Night/wet conditions: Many worst errors occur in night/rain conditions where early contact may be visually subtle
- Frame selection issue: With only 12 frames across 10-25 second videos, early collisions may not have enough pre-collision context
Recommendation: The model should be trained to identify "moment of first contact" rather than "most obvious damage frame."
Mean Spatial L2 Error: 0.2102
Range: 0.0064 to 0.4799
Spatial errors measure how far the predicted (x, y) collision location is from the ground truth. Large errors suggest:
- Model predicts collision at wrong location in frame
- Model identifies wrong vehicles as involved
- Model picks a late frame where vehicles have moved significantly
| Video | GT Type | Pred Type | Spatial Error |
|---|---|---|---|
| Town04_sideswipe_wet_22.mp4 | sideswipe | t-bone | 0.4799 |
| Town03_t-bone_rain_47.mp4 | t-bone | rear-end | 0.4671 |
| Town05_rear-end_wet_236.mp4 | rear-end | rear-end | 0.4660 |
| Town04_rear-end_clear_08.mp4 | rear-end | rear-end | 0.4615 |
Pattern: Large spatial errors occur when:
- Sideswipes are misclassified (model picks wrong location)
- Late temporal predictions (vehicles have moved by frame 10)
- Multi-vehicle scenes (model identifies wrong pair)
✅ Clear visual signature: Vehicles crossing at 90 degrees
✅ Distinctive motion pattern: Horizontal vs vertical movement
✅ Frame-by-frame visibility: Perpendicular approach is visible in multiple frames
✅ Prompt alignment: "t-bone" is well-defined in the prompt as "90-degree impact"
❌ Ambiguous camera angle: Two vehicles approaching head-on may appear aligned/stacked
❌ Motion confusion: Model doesn't understand "opposite directions on same road"
❌ Frame alignment: Early frames show both vehicles distant; late frames show them "stacked"
❌ Prompt weakness: "head-on" definition may not emphasize "opposite direction travel"
❌ Subtle contact: Glancing side-by-side collision is less visually dramatic
❌ Parallel motion: Model doesn't recognize "same direction, adjacent lanes"
❌ Camera angle: Side-by-side vehicles may appear front-to-back from certain angles
❌ Prompt weakness: "sideswipe" may not be well-explained in the prompt
❌ Angle confusion: Model sees any angle as "perpendicular"
❌ Speed differential: Model doesn't track relative velocity (one vehicle catching up)
❌ T-bone bias: Model defaults to t-bone for most collision scenarios
Current problem: Model focuses on "vehicle A is at position X, vehicle B is at position Y" rather than "vehicle A is moving in direction D1, vehicle B is moving in direction D2."
Fix: Add explicit motion analysis to the prompt:
Step 1: For each vehicle, determine its direction of travel:
- Track vehicle position across frames 0-5 to establish trajectory
- Classify direction as: forward/backward/left/right relative to camera view
- Identify if vehicles are in same lane, parallel lanes, or perpendicular paths
Step 2: Classify collision type based on relative motion:
- Head-on: Vehicles traveling opposite directions on same road (both forward, meeting frontally)
- Rear-end: Vehicles traveling same direction, one catching up from behind
- Sideswipe: Vehicles traveling parallel (same general direction), making contact on sides
- T-bone: Vehicles traveling perpendicular directions (one horizontal, one vertical)
- Single: Only one vehicle moving, hits static object
Add explicit head-on cues:
Head-on collision indicators:
- Both vehicles approaching each other on the same road/lane
- Front-to-front impact (not front-to-side or front-to-rear)
- Vehicles were traveling in opposite directions before impact
- Key visual: Both vehicles visible moving toward each other in early frames
- NOT the same as rear-end (which is same direction)
- NOT the same as t-bone (which is perpendicular paths)
Example reasoning template:
"In frames 0-5, vehicle A is traveling forward (toward camera) while vehicle B is traveling backward (away from camera) on the same road. At frame 8, they make front-to-front contact in the center of the lane → Head-on collision."
Add explicit sideswipe cues:
Sideswipe collision indicators:
- Vehicles traveling in same general direction (parallel motion)
- Vehicles in adjacent lanes (side-by-side, not front-to-back)
- Contact occurs on the side of vehicles (door-to-door), not front/rear
- Glancing impact (not a direct perpendicular or head-on crash)
- Key visual: Both vehicles visible moving forward, one drifts laterally into the other
Example reasoning template:
"In frames 0-8, vehicle A (left lane) and vehicle B (right lane) are both traveling forward. At frame 9, vehicle A drifts right and makes side-to-side contact with vehicle B → Sideswipe collision."
Add explicit rear-end cues:
Rear-end collision indicators:
- Both vehicles traveling same direction on same road
- One vehicle (rear) is catching up to another vehicle (front)
- Front vehicle may be slowing, stopped, or moving slower
- Impact is rear bumper of front vehicle ↔ front bumper of rear vehicle
- Key visual: In early frames, rear vehicle is behind and closing distance
Example reasoning template:
"In frames 0-7, vehicle A (red car) is traveling behind vehicle B (white truck), both moving in the same direction. Vehicle B begins slowing at frame 5. At frame 8, vehicle A makes contact with vehicle B's rear bumper → Rear-end collision."
Current problem: Model picks frame 10 (or late frames) even when collision happens earlier.
Fix:
Timestamp Selection Rule:
- Select the FIRST frame where the two vehicles make physical contact
- Do NOT select a later frame showing maximum damage/deformation
- If contact is visible in frame 6, select frame 6 (not frame 10)
- Physical contact = bumpers/bodies touching, not just "positioned to collide"
Current problem: Model defaults to t-bone for ambiguous cases.
Fix: Add explicit counter-examples:
Common Mistakes to Avoid:
- NOT t-bone: If both vehicles are traveling same direction → likely rear-end or sideswipe
- NOT rear-end: If vehicles are traveling opposite directions → likely head-on
- NOT head-on: If vehicles approach at right angle → likely t-bone
Add explicit single-vehicle cues:
Single-vehicle collision indicators:
- Only ONE moving vehicle visible in the scene
- Vehicle impacts a static object (wall, barrier, pole, tree, building)
- No second vehicle is involved in the collision
- Key visual: One vehicle moving, then sudden deceleration/deformation upon hitting object
Current problem: With 12 frames across 10-25 second videos, model may not have enough context.
Fix:
Frame Analysis Strategy:
- Frames 0-3: Establish initial positions and directions of all vehicles
- Frames 4-7: Track motion and identify when vehicles begin to interact
- Frames 8-11: Identify moment of first contact and impact type
- Use frame differences to infer motion direction (not just single-frame position)
Head-on:
Collision Type: HEAD-ON
- Definition: Two vehicles traveling in OPPOSITE directions on the same road collide front-to-front
- Key indicators:
* Both vehicles approach each other (one moving forward, one moving toward it)
* Impact is front bumper to front bumper
* Vehicles were in same lane or head-on lanes before impact
- Motion pattern: Opposite directions → closing distance → front-to-front contact
- NOT head-on if: Vehicles traveling same direction (that's rear-end)
- NOT head-on if: Vehicles crossing perpendicular paths (that's t-bone)
Sideswipe:
Collision Type: SIDESWIPE
- Definition: Two vehicles traveling in the SAME general direction make side-to-side contact
- Key indicators:
* Both vehicles moving in same direction (parallel motion)
* Vehicles in adjacent lanes (side-by-side positioning)
* Contact occurs on side of vehicles (door-to-door, not front/rear)
* Glancing impact (not a direct head-on or perpendicular crash)
- Motion pattern: Parallel travel → lateral drift → side-to-side contact
- NOT sideswipe if: One vehicle behind the other (that's rear-end)
- NOT sideswipe if: Vehicles approach at right angle (that's t-bone)
Rear-end:
Collision Type: REAR-END
- Definition: Two vehicles traveling in the SAME direction, with rear vehicle striking front vehicle from behind
- Key indicators:
* Both vehicles moving in same direction on same road
* Rear vehicle catches up to front vehicle
* Impact is rear bumper of front vehicle ↔ front bumper of rear vehicle
- Motion pattern: Same direction travel → rear vehicle closing distance → front-to-rear contact
- NOT rear-end if: Vehicles moving opposite directions (that's head-on)
- NOT rear-end if: Vehicles approach at angle/perpendicular (that's t-bone)
- NOT rear-end if: Vehicles side-by-side (that's sideswipe)
Step 1: ANALYZE MOTION DIRECTION
For each vehicle visible in frames 0-5:
1. Track position across frames to determine direction of travel
2. Classify as: forward/backward/left/right relative to camera
3. Identify relative motion:
- Same direction (parallel) → potential rear-end or sideswipe
- Opposite directions (head-on trajectory) → potential head-on
- Perpendicular directions (crossing paths) → potential t-bone
- Single vehicle only → potential single-vehicle
Step 2: IDENTIFY FIRST CONTACT
Scan frames 0-11 to find the FIRST frame where vehicles make physical contact.
Do NOT select a later frame showing maximum damage.
Step 3: CLASSIFY COLLISION TYPE
Based on motion pattern + contact type, classify as head-on/rear-end/sideswipe/t-bone/single.
After examining actual grid images from the error browser, several patterns emerge:
Top-Down Perspective Problems:
-
Vehicles aligned on same road appear "stacked"
- Head-on collisions look like rear-ends because both vehicles appear vertically aligned
- Model can't distinguish "approaching from opposite ends" vs "one behind the other"
-
Curved roads create apparent angles
- Vehicles on same curved road appear at 30-45 degree angles
- Model interprets these as "perpendicular" and predicts t-bone
- Reality: both traveling same direction, just following road curvature
-
Side-by-side vehicles appear front-to-back
- Sideswipe scenarios show vehicles in adjacent lanes
- Whichever vehicle is slightly ahead appears to be "in front"
- Model assumes front-back relationship = rear-end, missing the parallel lanes context
Late Frame Bias:
- Most grid images show the model selecting Frame 10 or Frame 11 (final frames)
- By these late frames, vehicles have often moved significantly post-impact
- Ground truth collisions often occur at Frame 5-7
- Late selection causes both temporal error AND spatial error (vehicles moved from impact point)
Early Frame Context Loss:
- Frames 0-3 often show critical approach context (direction, lanes, parallel motion)
- Model reasoning rarely references early frames
- Focus on final frames means model only sees "result" not "process"
T-Bone Success (90% accuracy):
- Frame sequence clearly shows: separate roads → convergence → 90-degree impact
- Motion paths are unambiguous even in single frames
- Perpendicular angle is distinctive regardless of camera position
Head-On Failure (2.6% accuracy):
- Frame sequence shows: both vehicles on same road, closing distance
- In late frames, vehicles appear overlapped/stacked, obscuring who was going which direction
- Model needs early frames to see "one vehicle top-left, one vehicle bottom-right, both moving toward center"
Sideswipe Failure (2.7% accuracy):
- Frame sequence shows: vehicles side-by-side → slight drift → glancing contact
- Subtle lateral motion is hard to detect without frame-by-frame comparison
- Contact point is door-to-door, not bumper-to-bumper, but model can't see this detail
Rear-End Failure (26% accuracy):
- Frame sequence shows: both vehicles on same road → front vehicle slows → rear vehicle closes gap
- Model sees the "angle" from camera perspective, not the "same direction" motion
- Speed differential invisible in single-frame analysis
| Issue | Impact | Fix Priority |
|---|---|---|
| Head-on → t-bone misclassification | 56% of head-ons wrong | 🔴 CRITICAL |
| Sideswipe → rear-end misclassification | 65% of sideswipes wrong | 🔴 CRITICAL |
| Rear-end → t-bone misclassification | 73% of rear-ends wrong | 🔴 CRITICAL |
| Temporal bias (too late) | 88% predict late | 🟡 HIGH |
| Single-vehicle hallucination | 100% wrong | 🟢 MEDIUM |
Next Steps:
- ✅ Update prompt v9 with motion direction analysis
- ✅ Add explicit counter-examples for each collision type
- ✅ Emphasize "first contact" over "maximum damage" for temporal selection
- 🔄 Run MicroCV with prompt v9 to validate improvements
- 🔄 Target metric: >70% type accuracy (currently 29%)