Metro’s new bus prediction system is pretty accurate, but leave yourself some extra time

Metro recently swapped out its seven-year-old bus predictions system for a new one called BusETA. Last week, fourteen Greater Greater Washington contributors and staff audited BusETA for accuracy. Overall, the system performed well enough, but buses sometimes came earlier than predicted, and “ghost buses” are still real.

Real-time departure information has totally changed bus travel, as any rider with a smart phone can now triangulate the real location of the bus. That means the ability to minimize wait times or choose a travel mode with more confidence; we’re no longer reliant on a printed schedule that can be shredded by congestion, incidents, events, breakdowns, or weather.

WMATA’s predictions come via a combination of real-time data from GPS transponders on the buses and computer models that predict bus arrivals using historic data about traffic patterns. Once they’re made, WMATA publishes predictions on its website. The marketplace for independent prediction apps like Transit Tracker and Citymapper is also pretty crowded, as WMATA publishes tools that any developer can use.

A recent change in who provides WMATA’s prediction technology means NextBus, the widely used but proprietary system, is out, and BusETA is in.

Our contributors recorded how BusETA did one morning

For the audit, which we did on Thursday, April 14th, each participant went to a specific bus stop and called up the prediction for the stop by the stop’s unique ID number. While BusETA will also give predictions by bus line, we audited only the bus stop interface. Each participant took a screenshot of the prediction and then recorded the actual arrival times of the predicted bus(es). Our participants audited a total of 27 buses covering 24 lines at 17 locations from 6 to 9 am.

Here’s how close each bus came to arriving when BusETA said it would:

A positive error value means the bus was early. The variance of the error clearly increases as the prediction time increases (the further the bus is away, the worse BusETA is at predicting the arrival time).

However, the latest bus was an F4 bus that Gray Kimbrough watched from across the street: It was nine minutes late as it passed through Silver Spring, heading away from downtown. It is notable that this bus was audited at a location near the end of its run. It could be that the further into the bus’ run the stop is, the less accurate the prediction is because there have been more opportunities for the bus to encounter delays.

Buses were early as often as they were late. One contributor missed his bus because the prediction told him he had five minutes, and he actually only had three.

Anecdotally, it seems like BusETA might under-predict bus arrivals more often than NextBus did (i.e. the bus shows up ‘early’).

If so, this is a major problem, because when you miss a bus by only one or two minutes, you have to wait the entire headway of the bus line for the next one, which is the worst-case delay scenario. Based on the results of our audit, I’d recommend factoring in a three minute margin of error when using BusETA.

There were still ghost buses (either buses that were not predicted, or predicted buses that didn’t show). Jonathan Neeley got an H8 prediction, but when he refreshed it two minutes before it was supposed to arrive, the bus had disappeared (the actual bus did arrive a moment after, however). Steven Yates read that the 74 was 11 minutes away…just as it pulled up.

Overall, though, BusETA worked more often than it didn’t. Brendan Casey said that for his commute, BusETA is far more accurate than Transit or Citymapper.

The BusETA technology is different, and likely better

NextBus had a lot of accuracy problems. WMATA’s switch to BusETA means it has joined the open source OneBusAway project, which is also used in Atlanta and New York City. That means that all the old apps that used the NextBus standard don’t work for DC any more.

The switch to an open source standard via BusETA should promote innovation and help interested parties understand how and why various prediction apps are working. Anybody can contribute back and improve the OneBusAway project since the code is freely available as an open-source project.

This potentially makes the software quite powerful: If someone wants to write a feature in, they can pull the freely-available code, edit it, and publish it back for approval. This freedom allows the application to be more feature-rich than it might otherwise be, and be developed faster than a typical commercial application.

In the long run, an open-source standard will hopefully mean more and better apps for DC.

Thank you to Gray Kimbrough, Chris Slatt, Abby Lynch, Jim Titus, Sebastian Galeano, Steven Yates, Brendan Casey, Bryan Barnett-Woods, Ronit Dancis, Andrea Adleman, Angela Martinez, Jonathan Neeley and Sarah Guidi for participating in this flash audit.