Real-Time Detection of Rare Traffic Situations Using RGB-LiDAR Fusion and a Rule-Based Safety Agent in CARLA

Rare and safety-critical traffic situations remain challenging for autonomous driving (AD) because they are poorly represented in common training data and may include objects outside standard detector classes. This paper presents a real-time RGB-LiDAR fusion framework for detecting and reacting to rare traffic situations in CARLA. The approach combines YOLOv8-based RGB perception, bird’s-eye-view (BEV) LiDAR clustering, decision-level fusion, an interpretable rule-based safety agent with hysteresis, and an automatic emergency braking (AEB) override. Fused observations are classified as semantic-geometric detections, semantic-only detections, or geometric-only obstacle candidates, where unmatched LiDAR clusters are treated conservatively as candidate-level physical evidence. The framework was evaluated on three CARLA maps and 3CSim-inspired corner-case scenarios, comprising 19253 frames. On a manually annotated subset of 1200 frames, the full pipeline achieved 93.7% precision, 94.7% recall, and a 94.2% F1-score. The CPU implementation processed one frame in 34.7 ms on average, remaining within the 50 ms budget of a 20 Hz simulation tick.