4.0 Summary of Evaluation Goals, Objectives, and Hypotheses
This independent evaluation conducted on behalf of the USDOT is intended to document the performance and benefits of the ISSES from a national point of view and provide practical information on commercial vehicle safety and efficiency that will be useful to other states considering the deployment of similar equipment. Safety-related results from this independent evaluation are also being incorporated into the national evaluation of the Commercial Vehicle Information Systems and Networks (CVISN) Deployment Program under a separate task order with the USDOT (2006d,e; 2007a).
The goals and objectives of the Kentucky evaluation, summarized below, are described in the Evaluation Strategy (USDOT 2005a), which was prepared based on research, phone contacts, and site visits in July and August 2005. Related measures, data sources, methods, and anticipated deliverables for the independent evaluation were described in an Evaluation Plan (USDOT 2006c).
Two goal areas, with respective objectives and hypotheses, guided the evaluation. Although each objective is numerically related to a particular goal area, most of the objectives were interconnected, and were instrumental in achieving more than one goal.
Goal 1 To estimate whether the ISSES will make highways measurably safer and more secure.
Objective 1.1 Measure subsystem and integrated system performance characteristics.
Hypotheses: The radiation monitor accurately alerts inspectors to potential radiation hazards, and produces minimal false alarms.
The thermal inspection device enables inspectors to see potential heat-related defective or malfunctioning equipment that might not be readily visible otherwise.
The laser scanner accurately logs the passage of trucks through the ISSES apparatus, and signals other subsystems.
The ISSES performs with a minimum of unscheduled downtime.
Objective 1.2 Use data from the field test to determine the distributions of kinds of vehicles traversing the weigh station under normal conditions. This provides a baseline for reference in assessing the highway safety benefits of the ISSES.
Hypothesis: The distribution of commercial vehicles passing the London site, relative to the respective motor carriers' SafeStat score ranges, is similar to that of the national population of commercial vehicles.
Goal 2 To determine how the ISSES makes the inspection process more efficient and effective, in turn contributing to improved highway safety.
Objective 2.1 Determine the degree of user acceptance and the perceived usefulness and usability of the ISSES as deployed, and quantify deployment and operating costs related to the ISSES.
Hypotheses: Inspectors and state transportation managers believe that ISSES enables roadside inspectors to perform their job functions better.
Inspectors believe that ISSES should be deployed more widely.
In deploying similar systems, officials at other sites believe their system enables them to perform their job functions better.
Inspectors found their training and user documentation for ISSES to be helpful to them in their normal course of duties.
In deploying ISSES, Kentucky incurred one-time start-up and recurring costs that were clearly defined and measurable.
Objective 2.2 Measure the ability of the ISSES to improve inspection selection efficiency, and in turn to yield reductions in crashes and breaches of highway security.
Hypothesis: The ISSES can help inspectors focus their efforts on higher-risk trucks.
Objective 2.3 Explore options for integrating the data available from the ISSES with existing safety, enforcement, and administrative data sources, and prepare models or plausible scenarios for Kentucky or other states to apply.
Hypothesis: Data from ISSES can yield important information for commercial vehicle enforcement and administration when combined with data from other state and federal sources.