---
title: "DAITFO — RL Traffic Signal Control Benchmark"
description: "A custom Gymnasium environment wrapping SUMO that benchmarks a PPO agent against fixed-time and queue-based heuristic controllers across 27 controlled runs. The heuristic won every metric — and that honest negative result is the actual finding, not a failure."
canonical: "https://shivanshutiwari.in/projects/daitfo"
lastUpdated: "2026-09-02"
---

# DAITFO — RL Traffic Signal Control Benchmark

A custom Gymnasium environment wrapping SUMO that benchmarks a PPO agent against fixed-time and queue-based heuristic controllers across 27 controlled runs. The heuristic won every metric — and that honest negative result is the actual finding, not a failure.

**Stack:** Python, Kotlin, Stable-Baselines3, Gymnasium, SUMO, Jetpack Compose, Docker, GitHub Actions  
**Date:** 2026  

## The Problem

Fixed-timing traffic signals can't adapt to real-time demand, which wastes fuel and delays emergency vehicles at intersections. The obvious modern answer is reinforcement learning. I wanted to know if that answer actually holds up against a much simpler queue-based heuristic — or if it's just assumed to be better because it's newer.

## What I Built

I built a custom Gymnasium environment wrapping SUMO (a standard traffic microsimulator) via its TraCI interface, exposing queue lengths and waiting times per lane as the observation space. I trained a PPO agent (Stable-Baselines3) on this environment and benchmarked it head-to-head against a fixed-time controller and a queue-based heuristic — across 3 controllers × 3 demand levels × 3 seeds, 27 controlled runs in total.

I also built a companion Android app (Kotlin/Compose) with role-based monitoring views for citizens, emergency responders, and traffic inspectors.

## Architecture

```
SUMO Simulator
 ├── Fixed-Time Controller
 ├── Heuristic Controller (queue-based, actuated)
 └── PPO Agent (RL)
        │
        ▼
  MetricsCollector → wait time, queue length, throughput
        │
        ▼
  benchmark_results.csv
```

## The Result

The heuristic won. On every metric, at every demand level.

On medium demand: the heuristic averaged 83s wait time. PPO averaged 136s. Fixed-time was 357s.

That's not a failed project to me — it's the actual finding. I initially had a bug where the heuristic had zero yellow-transition overhead, which made the early comparison look favorable to RL. Once I fixed that and re-ran the full benchmark, the result flipped completely. I kept the negative result instead of quietly reworking the framing.

The more interesting nuance I found: tightening the RL agent's minimum phase duration got one seed out of five to match the heuristic (84.6s) — but two of five seeds catastrophically failed instead, with vehicles getting permanently stuck (max wait times over 199,000 seconds in a simulation that should run 3,600). Shorter minimum phases buy peak performance and cost training reliability. That tradeoff is the real technical takeaway I walked away with.

## Key Decisions & Tradeoffs

- I built a custom Gymnasium env instead of using sumo-rl's built-in — I wanted full control over yellow-phase handling, at the cost of maintaining the TraCI integration by hand.
- I used a binary action space (keep/switch) so the comparison to the heuristic would be apples-to-apples. That meant I couldn't express phase-specific durations as a result.
- I trained only on medium demand — simpler experiment design, but low-demand generalization suffered (40s vs. the heuristic's 10.5s).
- I descoped the infrastructure I'd originally planned (Kafka, Neo4j, Redis, an LLM layer) to focus on one defensible, well-measured result instead of a sprawling half-built system.

## Highlights

- Heuristic beat PPO on every metric at every demand level — the honest negative result is the finding
- Custom Gymnasium environment wrapping SUMO via TraCI with full yellow-phase control
- 27 controlled runs: 3 controllers × 3 demand levels × 3 seeds
- Discovered fundamental reliability tradeoff: shorter minimum phases buy peak performance but cost training stability
- Companion Android app (Kotlin/Compose) with role-based monitoring for citizens, responders, and inspectors

---
*Source: https://shivanshutiwari.in/projects/daitfo*