If FMCSA is right about inspections, there's a problem

The Federal Motor Carrier Safety Administration yesterday issued a long press release proclaiming that the commercial motor vehicle (CMV) roadside and traffic enforcement (TD) interventions the agency oversees had saved an estimated 472 lives in fiscal 2012. Considering the relatively small number of lives FMCSA itself says will be saved by some of its high-profile rulemakings – 19 lives a year for hours of service and 26 lives for the electronic logging device rule, for example – this is pretty big news.

Except it's really not.

FMCSA's announcement is based on data from the agency’s Roadside Intervention Effectiveness Model (RIEM) – you know, one of those indecipherable statistical projections based on a whole bunch of assumptions and extrapolations that are virtually impossible to verify in the real world. I mean, think about it: How can you really be sure an action saved a life or prevented a crash without constructing some kind of super-unethical experiment?

There are lots of stones to throw, and you could start by raising the same sorts of objections regarding methodology and data quality that virtually the whole trucking industry has pinned on the Compliance, Safety, Accountability program's Safety Measurement System. Yes, that same SMS that Congress last year said was unfit for public consumption and ordered a major review of.

But let's simplify things by accepting that FMCSA's model is generally accurate. In that case, we should be more troubled than pleased by the findings.

One of the model's assumptions is that TE inspections are about four times as effective as roadside inspections (RIs) in reducing crashes, injuries and deaths. TE inspections are those conducted over the road following moving violations such as speeding or reckless driving while the more frequent roadside inspections (RIs) are typically conducted at fixed locations, such as weigh stations. It's not clear why TE inspections are so much more effective, but that's what the model says.

But here's the problem. The number of TE inspections peaked in fiscal 2006 and had fallen by more than 43% by fiscal 2012. Under FMCSA's own analysis those interventions could have saved 641 lives instead of 472 had TE inspections remained at their fiscal 2006 level.

And the news gets worse because TE inspections have continued to drop and are about 60% below the peak. Using the same math FMCSA uses, applying the RIEM to fiscal 2015 roadside interventions actually saved 461 lives. But based on TE inspections at the fiscal 2006 level, 685 lives would have been saved.

But there's some good news: The real issue might l be incomplete data. It's possible that highway safety has improved despite - or, in a sense, because of - fewer TE inspections.

Huh? Read on.

A foreseeable outcome

Given that TE inspections result from speeding and other moving violations, a drop in inspections might actually be a good thing. It could mean that truckers are slowing down for a variety of reasons, including safety concerns and a desire for better fuel economy.

While slower speeds might account for some of the reduction in inspections, a more plausible cause for most of the decrease lies with a change to federal law in 2005.

To understand this argument, you need to know that practically all inspections – whether TE and RIs – are conducted by state and local agencies. Most of FMCSA's budget goes to reimburse these partners for their efforts through something called the Motor Carrier Safety Assistance Program, or MCSAP.

Another important fact: Only a tiny fraction of law enforcement officers are qualified to conduct thorough inspections of CMVs. Given that, Congress decided in 2005 that it would be a good idea to encourage states to leverage the vast majority of law enforcement officers to police commercial driver's behavior on the road.

To do this, Congress deleted what had previously been a requirement that in order for a law enforcement agency to get reimbursement for a traffic stop, it had to conduct an inspection. Considering that state budgets are always under stress, if an agency can get money without having to conduct an inspection, it is less likely to conduct the inspection. It's not rocket science.

From a safety standpoint, what Congress did in 2005 might very well be a good thing. But we don't know because FMCSA doesn't compile data from traffic stops that don't lead to inspections in any usable form. Basically, state agencies submit reimbursement vouchers to the FMCSA office in each state, and that information isn't reported to headquarters or to any central database.

The bottom line is that either we are seeing crashes, injuries or deaths that could have been avoided or FMCSA’s RIEM is flawed due to a lack of data.

Definite impacts

Whether the reduced TE inspections indicate degraded or improved safety is uncertain, but we do know one thing: It has starved data for the Unsafe Driving Behavior Analysis Safety Improvement Category (BASIC) in the Safety Measurement System. TE inspections are the only source of data for that BASIC. TE inspections are down more than 35% since FMCSA launched CSA.

The lack of data also could undermine FMCSA's proposed carrier safety fitness determination rule, which relies heavily on data from the BASICs – including Unsafe Driving – to rate carriers directly without an audit.

In any event, FMCSA's big announcement is problematic. If the RIEM is accurate, then we have a problem. If it isn't accurate, FMCSA can't claim credit for anything.