Inspection Efficiency Evaluation

Data to address the safety (inspection efficiency) goals and objectives described above were collected through various methods: (1) interviews and site visits with various KTC and KVE personnel; (2) a 2-week field study at the Laurel County inspection site in June 2007; (3) various federal and state safety data sources; and (4) past federal studies that relate to CMV crashes and safety.  Listed below are the main data sources used.

• Interviews with KVE inspectors and KTC specialists
• USDOT numbers for all trucks going through the ISSES portal at the Laurel County station during a 2-week field study (during normal daytime hours).
• NORPASS (electronic screening/preclearance) bypass decisions per truck for one week during field study
• Electronic copies of inspections performed during 2-week field study
• Electronic copies of Kentucky statewide inspections spanning over 2.5 years
• SAFER (Safety and Fitness Electronic Record) carrier and inspection tables obtained from the Volpe Center at the time of the field study
• Kentucky Clearinghouse
• Large Truck Crash Causation Study (LTCCS)
• 2003 National Truck Fleet Safety Survey
• Large Truck Crash Facts – 2005.

The goal of roadside enforcement is to avoid as many crashes as possible by putting unsafe vehicles OOS before the OOS conditions present on the vehicle contribute to a crash.  A means to this end is to improve the inspection selection process in such a way that the greatest benefit can result from a fixed number of inspections.  This makes the most efficient use of limited time, human resources, and facilities.  The overall approach of this evaluation was to first assess the effectiveness of the current inspection selection methods at selecting high-risk trucks.

In addition, alternative methods for selecting vehicles for inspection were evaluated based on potential availability of information from the above data sources.  Several forms of available evidence and inspection selection methods were combined in various ways to develop hypothetical scenarios for the safety analysis:

• Selecting vehicles randomly for inspection, to provide a starting point from which to assess the contribution of the inspectors’ knowledge and experience.
• The current vehicle selection process used in Kentucky, which relies primarily on inspector judgment.
• Using electronic screening² to eliminate all low- and medium-risk carriers from selection consideration, so that inspectors can focus on high-risk trucks or those with insufficient safety information in federal databases.  This approach uses the carrier’s ISS score, a rating system promoted by USDOT.
• Using the carrier’s vehicle and driver OOS rates, which are the metrics preferred by Kentucky in roadside enforcement. 
• Using information on OOS violations with a high relative crash risk
• Using thermal/IR brake images from the ISSES.

Finally, the evaluation measured the success of these new inspection selection methods by simulating what would happen if inspectors used these kinds of information to select high-risk trucks for inspection.  The measures used to estimate success were the estimated number of crashes, injuries, and fatalities avoided.

Kentucky’s current approach to inspection selection, which at some sites involves the use of the Kentucky Clearinghouse and historic out-of-service (OOS) rates, is described in the Technical Report (USDOT 2008).  Also included in that report is a detailed account of the field observational study data collection, characteristics of truck traffic at the Laurel County station, a discussion of inspection efficiency (defined as the degree to which inspectors choose high-risk trucks for inspection), a discussion of current and potential future alternative approaches to increasing safety by improving the efficiency of selecting commercial vehicles for inspection, and an analysis of the usefulness of a carrier’s credentialing status relative to their safety information in identifying high-risk trucks.  This Summary Report focuses on only the results and implications of the inspection efficiency evaluation for commercial vehicle safety.

Table 2 presents a summary of large trucks involved in crashes in 2005³ both nationally and within Kentucky.

The most important benefit expected from the deployment of the ISSES and other CVISN technologies, especially electronic screening and safety information exchange, is a reduction in CMV‑related crashes through improved enforcement of the Federal Motor Carrier Safety Regulations (FMCSRs).  The principal hypothesis to be tested is that the ISSES and CVISN technologies will help enforcement staff focus inspection resources on high‑risk carriers.  This will result in more OOS orders for the same number of inspections—thereby removing from service additional trucks and drivers that would have caused crashes because of vehicle defects and driver violations of safety regulations. 

Table 3 lists some key safety statistics obtained from the published literature.  Most of these data are used in the crash avoidance analysis; others are provided for reference.  According to FMCSA, 8.5 million large trucks in 2005 traveled approximately 233 billion miles in the U.S.  Also in 2005, the last year for which complete statistics are available, 441,000 trucks were involved in crashes, resulting in approximately 114,000 injuries and 5,212 deaths.  In order to determine the impact of removing OOS violators from the roadway on the number of crashes, it is necessary to estimate certain probabilities associated with crash causation.  One important component to the statistical crash reduction model is being able to estimate the relative risk of driver and vehicle OOS violations in truck crashes.  Specifically, we would like to know the probability that an OOS condition exists on a truck given a crash has occurred involving that truck.  Before the FMCSA-sponsored LTCCS, there were not reliable estimates of this probability for either vehicle or driver OOS violations as there had not been sufficient data to support calculation of reliable estimates.  By focusing on the pre-crash condition of the truck, the LTCCS provides the right type of data for this analysis.  The LTCCS data was used to calculate various probabilities that were used as inputs to the crash avoidance model (USDOT 2006a).

Summary of Safety Modeling Approach

Ultimately, safety benefits will be realized only to the extent that targeted inspections and improved compliance translate into reductions in numbers of crashes.  The premise of targeted inspections is that, for the same number of inspections performed, additional drivers and vehicles operating with OOS conditions will be removed from the roadway.  Furthermore, all of the conditions leading to the OOS order will be fixed and “stay fixed” for a period of time after the inspection.  Therefore, crashes that would have occurred during this period are prevented because the OOS conditions that would have caused the crashes were eliminated.  The safety benefit of ISSES and CVISN technologies is determined by comparing the number of crashes avoided under a baseline scenario (i.e., with pre-ISSES or CVISN roadside enforcement strategies and technology) with the number of crashes avoided under a number of deployment scenarios involving the ISSES and CVISN.  It is assumed under each scenario that the corresponding number of injuries and fatalities avoided are proportional to the number of crashes avoided.

A statistical model of crash avoidance was developed, based on research on the Safe-Miles model developed for FMCSA at the Volpe Center to estimate the benefits of MCSAP, the Motor Carrier Safety Assistance Program (VNTSC 1999).  Although the model used in the present Kentucky safety benefits analysis is different from the one used in Safe-Miles, certain model parameters such as the number of “safe miles” a truck travels following an OOS order, were used in this Kentucky analysis.  The approach to safety benefits estimation in the Kentucky evaluation was adapted from the approach documented in Chapter 5 of the CVISN Model Deployment Initiative (MDI) Evaluation (USDOT 2002).

Table 3. Relevant national safety and safety enforcement statistics on large trucks.

Statistic Description	Value	Source 1
Number of large trucks	8.5 million	Large Truck Crash Facts 2005 (USDOT 2007b)
Large truck annual vehicle miles traveled (VMT)	233 billion	Large Truck Crash Facts 2005 (USDOT 2007b)
Large trucks involved in crashes (2005) Injuries from large truck crashes (2005) Fatalities from large truck crashes (2005)	441,000 114,000 5,212	Large Truck Crash Facts 2005 (USDOT 2007b)
Large trucks involved in property damage-only crashes Large trucks involved in injury-only crashes Large trucks involved in fatal crashes	354,000 82,000 4,932	Large Truck Crash Facts 2005 (USDOT 2007b)
Large truck crash rate (truck crashes/100 million VMT) = 441,000 truck crashes/233 billion VMT	189.3	Derived
Commercial vehicle (non-bus) vehicle inspections performed (2005) Commercial vehicle (non-bus) driver inspections (2005) Total CV (non-bus) inspections (driver or vehicle) (2005) Kentucky annual commercial vehicle (non-bus) vehicle inspections performed (2005) Kentucky annual commercial vehicle (non-bus) driver inspections performed (2005) Kentucky annual commercial vehicle (non-bus) (driver or vehicle) inspections performed (2005)	1,949,375 2,669,679 2,708,856 44,142 86,028 86,077	Annual Summary of Roadside Inspections - NAFTA Safety Stats (A&I website) Kentucky Historical Inspection Data
Percent of vehicles placed OOS (2005) Percent of drivers placed OOS (2005) Kentucky percent of vehicles placed OOS (2005-Sept 2007) Kentucky percent of drivers placed OOS (2005 - Sept 2007) Kentucky percent of vehicles or drivers placed OOS (2005 - Sept 2007)	24.0% 7.0% 9.5% 4.7% 13.6%	Annual Summary of Roadside Inspections - NAFTA Safety Stats (A&I website) Kentucky Inspection Data (2005 - Sept 2007)
Percent of VMT with vehicle OOS conditions (2003) Percent of VMT with driver OOS conditions (2003) Percent of inspections that found at least one OOS vehicle violation given a OOS driver violation was found Percent of VMT with brake-related OOS conditions	28% 5% 49% 14%	2003 National Truck Fleet Safety Survey (TFSS) (USDOT 2006b) 1996 National Survey (Star 1997)
Percent of large CMV crashes with vehicle OOS condition present	32.4%	Derived from LTCCS
Percent of large CMV crashes with driver OOS condition present	17.2%	Derived from LTCCS

To summarize, the statistical model used terms such as the following:

• The probability that a truck has an OOS violation given that it was inspected
• The probability of a crash given that a vehicle has an OOS violation
• The probability that a vehicle has a particular OOS violation or group of violations (e.g., vehicle or driver OOS condition) given that it is in a crash (based in part on LTCCS crash factors data)
• The probability of a crash
• The probability that a vehicle has an OOS condition
• The national crash rate for large trucks
• The number of safe miles (SM) traveled as a result of “fixing” an OOS condition.

National data on rates of injury and fatality per truck-involved crash were used to derive the numbers of injuries and fatalities that could be avoided, given a certain number of crashes avoided.

Deployment Scenarios

Truck traffic at most inspection sites is very heavy, and inspectors cannot inspect every CMV that passes by.  Thus, there needs to be a sound methodology for narrowing down the pool of trucks from which inspectors have to choose.  Seven overall scenarios are presented in this section, a few of which have been divided into sub-scenarios.  The seven deployment scenarios present different methods for selecting vehicles for inspection with the goal being to select trucks that yield the most OOS orders.  Using the crash avoidance model, these scenarios illustrate the estimated safety benefits of the ISSES and other CVISN technologies.  Table 4 provides a high-level summary of the seven scenarios presented in this section.  A more thorough description of each roadside enforcement (RE) scenario follows the table.

Table 4. High-level overview of roadside enforcement scenarios.

Scenario Number	Screening Criteria Used in Scenario
	Random Only	Inspector Experience and Judgment	Electronic Screening with Snapshots	KY OOS Rate Algorithm	Vehicle and Driver OOS Rates Using Threshold	Brake and Driver OOS Rates	Infrared Images and Driver OOS Rate
RE-0	X
RE-1		X
RE-2		X	X
RE-3		X	X	X
RE-4		X	X		X
RE-5		X	X			X
RE-6		X	X				X

RE-0:  Random Selection.  Enforcement officers (inspectors) select CMVs for inspection in a random manner without using personal experience, judgment, or any ISSES or CVISN technologies.  This is not one of the roadside enforcement strategies being considered, nor is it a realistic strategy to employ.  However, the calculation of safety benefits under this scenario is useful for determining the contribution of the inspectors’ knowledge and experience during the vehicle selection process.

RE-1:  Baseline—Pre-ISSES/CVISN.  Inspectors select CMVs for inspection using personal experience and judgment, but without the aid of ISSES or most CVISN technologies.  Electronic screening is assumed to be used at its current level as of June 2007.  This baseline scenario is analyzed twice.  First, safety benefits are calculated based on Kentucky vehicle and driver OOS rates, which are significantly lower than the national average.  Then, the analysis is performed assuming that Kentucky’s vehicle and driver OOS rates were on par with national estimates—referred to as RE-1a.

RE-2:  Mainline Electronic Screening based on ISS Score.  State deploys electronic screening with safety snapshots at all major inspection sites.  Motor carriers that are classified as low- and medium-risk based on ISS scores (comprising approximately 60 percent of trucks on the road) enroll in the electronic screening program, are equipped with transponders, and are allowed to bypass inspection sites.  Inspectors use current practices to select vehicles for inspections from the remaining 40 percent of trucks in the high-risk and insufficient data categories.

RE-3: Electronic Screening based on Kentucky OOS Rate Inspection Selection Algorithm.  State utilizes Kentucky OOS rate inspection selection algorithm at all inspection sites that utilize electronic screening.  Every vehicle that enters the inspection station is identified accurately by the ISSES’ ALPR and USDOT readers.  Safety information for each carrier is obtained from the Kentucky Clearinghouse.  Based on the safety information, the algorithm identifies trucks for inspection as described in Section 6.2.  Inspectors select vehicles for inspection from this pool of identified trucks, while non-identified trucks continue to the mainline.  Trucks with transponders are subject to the same algorithm already built into NORPASS. 

RE-4: Electronic Screening based on high vehicle and/or driver OOS rates.  State utilizes the ISSES and/or electronic screening at all major inspection sites.  This scenario is similar to RE-3 in that each truck is screened via the ISSES based on the vehicle and driver OOS rate of the carrier.  However, RE-4 differs in that a threshold OOS rate is established for both vehicles and drivers such that all trucks with OOS rates exceeding the corresponding thresholds are brought into the inspection station for inspection, while all others are allowed to bypass inspection sites.  The threshold rates are chosen such that only trucks with the highest OOS rates are candidates for inspection.  The threshold values can vary depending on both the truck traffic and the rate at which inspections can be performed at the site.  As part of RE-4, three specific threshold values are considered.

RE-5: Electronic screening based on high driver OOS or brake violation rates.  State utilizes the ISSES and/or electronic screening at all major inspection sites.  Each truck is screened via the ISSES based on its OOS or violation rate for violations that have a high relative risk for crash.  In this scenario, vehicles are screened based on their brake violation and overall driver OOS rates as they appear in SAFER.  A distinction is made here between violation and OOS rates.  SAFER contains a violation rate for brakes but not a brake OOS rate.  Thus, violation rates are used as a safety index for brake issues, while the driver OOS rate is used to screen for driver issues.  Both brakes and driver OOS violations have been found to have a high relative risk for crashes.  This scenario differs from RE-4 in that vehicles are screened on their brake violation rate as opposed to their overall vehicle violation rate in an attempt to catch those vehicles that have a violation that has a higher relative risk for crash.  Similar to RE-4, all trucks with violation rates exceeding the threshold are candidates for inspection, while all others are allowed to bypass inspection sites.  Moreover, the threshold rates are chosen such that only trucks with the highest rates are selected for inspection and the thresholds can vary depending on the amount of inspection personnel available at a given station.  As part of RE-5, three specific threshold values are considered.

RE-6: Electronic screening based on infrared screening and high driver OOS violation rate.  State utilizes the ISSES at all major inspection sites.  Each truck is screened via two criteria: the thermal (IR) imaging system on the ISSES and the driver OOS rate of the carrier.  In this scenario, vehicles are screened based on the presence of a brake violation through the IR image produced by the ISSES and the driver OOS rate as it appears in SAFER.  This scenario is similar to RE-5 in that both brake and driver OOS violations are used as screening criteria.  RE-6 differs from RE-5 in that vehicles are screened for brake violations via IR imaging as opposed to brake violation rates obtained from SAFER.  All trucks with a potential brake violation as detected from the IR image or trucks with driver OOS rates exceeding various thresholds are candidates for inspection, while all others are allowed to bypass inspection sites.  Inspection efficiency data from an earlier FMCSA report on the IRISystem (USDOT 2000) were used in this scenario.⁴

In summary, RE-0 is the most basic selection process of selecting vehicles randomly and is presented mainly to assess the contribution of the inspectors’ knowledge and experience during the vehicle selection process, which is represented in the baseline scenario RE-1.  The remaining five scenarios all make use of progressively more involved selection criteria.  Electronic screening is employed in RE-2 to eliminate all low- and medium-risk carriers from selection consideration.  Although this scenario helps improve inspection selection efficiency by allowing inspectors to focus only on high-risk vehicles or those with insufficient data, there are still too many vehicles remaining in these categories for roadside enforcement officials to inspect them all.  As a result, scenarios RE-3 through RE-6 provide various methods to further narrow down the number of vehicles that inspectors have to choose from.  RE-3 is based on the Kentucky OOS rate inspection selection algorithm, which selects vehicles for inspection at different rates depending on their OOS rates.  RE-4 and RE-5 take a slightly different approach in selecting only those vehicles with the highest probability of having particular kinds of OOS violations as measured by some safety index.  RE-6 examines the benefits when IR imaging is used to screen for brake violations.

Broader Definition of “Electronic Screening”

In the CMV law enforcement community, the term “electronic screening” signifies a transponder-based mainline preclearance system, such as NORPASS, HELP/PrePass, Oregon Green Light, or equivalent.  Such systems provide roadside enforcement personnel the ability to detect and identify and (optionally) weigh CMVs at mainline speeds.  For purposes of this report, Scenarios RE-3 through RE-6 expand the definition of “electronic screening” to include other means of achieving a similar goal, namely to use computers and telecommunication technology to identify and prescreen vehicles in real time.  In Scenarios RE-3 through RE-6, ISSES or an equivalent system is used for identifying trucks moving slowly through a weigh station.  The basic function is the same as transponder-based preclearance, the only difference being the truck’s speed at the point of decision (red light, pull-in, green-light, bypass).  In these four scenarios, it is assumed that some trucks carry transponder tags and some do not.  Furthermore, it is assumed that all trucks approaching the station are subject to electronic or computer-based, real-time prescreening—at high or low speeds—as an aid to the inspector’s decision process.  These four scenarios also diverge from the usual definition of “electronic screening” in that, for purposes of modeling and analysis, they introduce screening decision criteria that are different from the criteria believed to be used in the prevailing mainline e-screening programs or partnerships (NORPASS, PrePass, and Oregon Green Light).

Summary of Safety Benefits/Inspection Efficiency Results

Table 5 summarizes the major results of this safety benefits analysis.  According to the model, current roadside enforcement strategies (RE-1) are responsible for avoiding 126 truck-related crashes, which represents about 4.4 percent of the 2,853 crashes in Kentucky that occur annually, based on 2005 crash statistics.  Furthermore, it is estimated that current roadside enforcement activities are responsible for preventing 33 injuries and 2 deaths. 

The safety benefits realized increases with each scenario RE-2 through RE-6.  The maximum benefit is achieved with RE-6, where 755 crashes are avoided if the top 5 percent of vehicles in terms of driver OOS violations are inspected in conjunction with IR screening.  This implies that about 26 percent of Kentucky’s 2,853 annual truck-related crashes could be avoided under RE-6.  In reality, this figure is an overestimate, because national crash rates were used in the safety benefit calculations, because reliable crash rates for Kentucky were not available.

To put the crash avoidance numbers into context, consider that the number of large trucks involved in crashes in Kentucky (2,853) is low relative to the 441,000 large trucks involved in crashes nationally, representing only 0.6 percent of national crashes.  Also, the percent of Kentucky crashes relative to the number of inspections performed in Kentucky is about 3.3 percent.  Comparatively, the national rate of crashes relative to the number of inspections is about 16 percent.  Therefore, relative to the number of inspections, Kentucky’s crash rate is smaller than the national crash rate.  The exact reason for this is unknown, but possible explanations include a lower volume of traffic in Kentucky, less congested highways, or a smaller number of large cities.

Table 5. Estimated safety benefits of the ISSES and CVISN under selected deployment scenarios and assumptions.

Scenario	Description		Numbers of Safety Events Avoided1			Additional2 Safety Events Avoided (ISSES/CVISN Benefit)
			Crashes	Injuries	Fatalities	Crashes	Injuries	Fatalities
RE-0	Random Selection		183	47	2
RE-1	Baseline - Pre ISSES/CVISN Using Kentucky OOS Rates		126	33	2
RE-1a	Pre ISSES/CVISN Using National OOS Rates		214	55	3
RE-2	Mainline Electronic Screening Based on ISS Score		189	49	2	63	16	0
RE-3	Electronic Screening based on Kentucky OOS Rate Inspection Selection Algorithm		227	59	3	101	26	1
RE-4	Electronic Screening based on high vehicle and/or driver OOS rates3	5% 10% 25%	306 217 134	79 56 35	4 3 2	180 91 8	46 23 2	2 1 0
RE-5	Electronic screening based on high driver or brake violation rates3	5% 10% 25%	476 353 221	123 91 57	6 4 3	350 227 95	90 58 24	4 2 1
RE-6	Electronic screening based on infrared screening and high driver OOS violation rate3	5% 10% 25%	755 644 544	196 167 141	9 8 7	629 518 418	163 134 108	7 6 5

Recalculating the safety benefits achieved when the national number of vehicle and driver inspections in 2005 is used instead of Kentucky inspection figures in Equation (5) finds that implementing RE-6 avoids about 6.5 percent of all national crashes.  This figure makes more sense in the context of the number of total crashes.

It is not possible to know the exact percentage of crashes caused by driver or brake OOS violations.  However, as discussed earlier, there is a 12.2 percent increase in relative crash risk for driver OOS violations, a 4.4 percent increase in crash risk for vehicle violations, and a 7.7 percent increase in crash risk for brake OOS violations.  Since a vehicle could have more than one type of violation, the three crash risk figures cannot be added to obtain the total increase in crash risk.  However, these figures suggest that if there were no driver or brake OOS violations present in the population, no more than about 20 percent of crashes could be avoided.  This is the maximum possible benefit if all OOS violations were removed from trucks traveling on the road.  This fact helps to put the Kentucky results into context and to provide an upper bound on the crash avoidance numbers for Kentucky.

---

² The term “electronic screening” is defined, for purposes of this study, as using any computer-based, real-time information source to aid in selecting trucks for inspection, whether the truck carries a transponder or not, and whether the screening occurs at mainline or ramp/sorter-lane speeds.  Further details are provided below.

³Although more current crash statistics are available, the safety benefits analysis is performed using a baseline year of 2005 because that was the last year for which complete data were available from all of the relevant sources.

⁴ The IRISystem technology was purchased by IIS (the vendor for the ISSES technology under evaluation) in 2003.  IIS continues to manufacture IRISystem vans, and the IRISystem designer participates in all of IIS’s thermal imaging applications.

Previous | Table of Contents | Next

ITS Home | US DOT Home | Privacy | Feedback

United States Department of Transportation - ITS