Metro’s Vital Signs a small step towards transparency

Since Richard Sarles took the helm, WMATA has started releasing more statistics through the monthly Vital Signs Report and Metro Scorecard. These are models for measuring a transit agency’s performance, but WMATA could go further and provide more raw performance data to the public, more frequently.

At the August meeting of the WMATA Riders’ Advisory Council, Chief Performance Officer Andrea Burnside gave an overview of the Vital Signs Report’s first year.

Vital Signs is part of a much larger program of “performance-based management” at the agency, though it’s the only part made publicly available. Employees at all levels of the agency, from front-line managers to the General Manager, get more detailed reports.

The reports provide a monthly overview of key performance indicators (KPIs) for Metrorail, Metrobus, MetroAccess, and the Metro Transit Police Department. The report contains not only the KPIs themselves, but also explains, for each KPI, why the information is relevant, why the performance has changed since the last reporting period, and shows the steps being taken to improve performance.

This is all valuable information, but should not be the only glimpse the public gets into how the transit system is doing.

An example of the current framework’s shortcomings is that the public lacks access to statistics on the performance of the most heavily-used escalators, such as those at the seven transfer stations in the system. Burnside’s response to the RAC was that the Vital Signs Report was only intended to convey system-wide statistics.

There’s a great escalator status page showing current outages, but not their past performance. Having the raw data available would make it possible to analyze things like escalator malfunctions at transfer stations, and generate more specific performance indicators than those Metro unveils in the report.

The benefits of releasing raw performance data are substantial. An enterprising software developer could, for example, build a customized dashboard which would allow riders to view statistics specific to stations and routes they use. This would empower riders by allowing them to better understand the agency’s performance in a personally meaningful way.

Unfortunately, WMATA’s strategy for reporting performance indicators (pictured above) entails releasing only the least detailed of reports to the public, while the General Manager receives an intermediate level of detail, and heads of individual departments receive the most detailed data.

This makes it impossible for members of the public to independently analyze the data, whether to create their own more localized metrics, develop charts and maps to better visualize the data, or to merge the data with outside information, such as weather patterns or traffic conditions.

Performance data also should to be released more often. WMATA intends to cut the Vital Signs Report’s production from monthly to quarterly, saying that many of the actions being taken to improve performance are difficult to measure on a month-to-month basis. Expecting to see results one month after a change is implemented, Burnside said, is unreasonable. However, this change also means that there will be times that the latest publicly-available data will be at least three months old.

While monthly reporting is a good start, WMATA should begin to take steps towards making vital information available on a real-time or near-real-time basis. Many indicators, like bus and rail on-time performance, and the availability of elevators and escalators, are based on electronic records that the agency already uses for operational purposes. So it’s only a matter of developing the necessary analysis tools for riders to be able to, for example, examine the on-time performance for each day’s service at the end of the operating day.

WMATA released service reports daily until February 15, 2011. These detailed Metrobus and Metrorail ridership and summarized Metrorail service disruptions. Metro discontinued these, maintaining that they were too labor-intensive to produce. Metro should make daily service reports available once again, and they should be enhanced to provide additional valuable data.

San Francisco Muni’s exemplary Service Delivery Daily Reports provide a considerable amount of information about the previous day’s service: the availability of each type of vehicle in the agency’s fleet on that day, operator absenteeism, and precise information on accidents and any incident causing more than 20 minutes’ delay to a line. They detail the start and end times of the incident, the location and nature of the incident, actions taken to restore service, and the identities of transit vehicles, staff members, and police units involved in the incident.

As fantastic as Muni’s daily reports are, they can’t be put to their most effective use because they are released in rigid PDF format. If one wants to conduct an automated analysis of the data, as our own Michael Perkins did with Metrorail disruption reports, one cannot simply load them into a spreadsheet or database without time-consuming copying and pasting of each individual number.

Releasing Metro’s monthly Vital Signs Report to the public is a good first step, but it should release information to the public on a daily basis which is already gathered electronically, to encourage different types of analysis. Summaries like the Vital Signs Report, which take time to prepare because of the depth of analysis involved, are best released on a monthly or quarterly basis. But raw performance data, including service disruption reports, should be posted considerably more frequently.