FAST-TRAC Evaluation:
Evaluation Summary Report

June 2000

PDF Version 11.9MB

The University of Michigan
ITS Research Laboratory
200 Engineering Programs Building
2609 Draper Drive
Ann Arbor, Michigan 48109-2101
Program Offices: (734) 764-4333
FAX: (734) 763-1674

Rebecca Unterriker
Steven Underwood
Andrea Frank
Yael Waldman

Preface

This Evaluation Summary Report provides an overview of some of the major evaluation studies planned and conducted for the FAST-TRAC program. Evaluations were completed by a consortium of researchers under the management of the University of Michigan. As the plans for deployment of the FAST-TRAC system components evolved over time, the evaluation plans were adjusted accordingly. This report summarizes the major evaluation studies as they were actually conducted. The original evaluation plans are presented in the “FAST-TRAC Evaluation Plan.” The evaluation components were carried out under a series of five contractual phases. Summary reports of individual studies describe the evaluation goals, methodology employed, and results when available. Where studies have been completed, a reference to the full report(s) is given.

Contents

• Preface
  Contents
  Tables and Figures
  Introduction
  • Overview and Early History
  • Program Evolution and Recent History
  • Evaluation Objectives
  • Evaluation Overview
    • ALI-Scout Route Guidance Evaluation
    • Traffic Control System Evaluation
    • Global Evaluation
    • Integrated System Evaluation
• Route Guidance Evaluations
  • Driver Perceptions and Behaviors
    • Objectives
    • Method
    • Pre-testing and Pilot Testing
    • Natural Use Study (NUS)
    • Choice Modeling Study
  • Human Factors
    • Assumptions
    • Test Subjects
    • Instrumented Car
    • On-Road Pilot Test
    • Destination Entry Laboratory Study
    • On-Road Test
    • Supplemental On-Road Testing
    • Human Factors Results
  • Technical Performance
    • Evaluation Questions
    • Data Collection Methods
    • Results
  • Traffic Modeling
    • Evaluation Questions
    • Mesoscopic Traffic Simulation
    • Measures of Effectiveness
    • Data Collection
    • Results
• Traffic Control System Evaluation
  • Accident Analysis
    • Objectives
    • Methodology
    • Results
  • Intersection Delay
    • Scope
    • Methodology
    • Validation of Video Reduction Method
    • Data Collection Activities
    • Computer Modeling
    • Coordination with Other Activities
    • Results
  • Corridor Delay
    • Measure of Travel Time
    • Data Collection Activities
    • Coordination with Other Activities
    • Expected Impact of Advanced Traffic Management System
    • Results
  • System Delay and Capacity
    • Evaluation Questions
    • Method
    • Results
  • Special Event Analysis
    • Purpose
    • Methodology
    • Results
  • Incident Response Analysis
    • Purpose
    • Evaluation Questions
    • Methodology
    • Results
• Global Evaluation
  • Cost Evaluation
    • Methodology
    • Results
  • Global Analysis
    • Method
    • Coordination with Other Evaluation Studies
    • Stakeholder Analysis
    • Traffic and Higher Order Impacts
    • Individual Impacts
    • Driver Operation and Interface Design
    • Technical
    • Instruments
    • Results
• Integrated System Evaluation
  • Stakeholder Analysis
    • Method for Quantifying Stakeholder Values
    • Stakeholder Group Representatives
    • Design
    • Data collection
    • Instruments
    • Facilities
    • Results
  • Institutional Issues
    • Issue Identification Methods
    • Interviews
    • Questionnaires
    • Case Study
    • Results
  • Traffic Information Management Systems Evaluation
  • Oakland County Traveler Study
    • Survey Method
  • Procedure
  • Participant Selection Results
  • Web Survey Set-up/functional specification
  • System Integration Case Study
  • System Integration Model

Tables and Figures

Table 1:    Percentage of Subjects Assigning Some Level of Importance to Various Factors Related to ALI-Scout-Like Systems
Table 2:    A Summary of the Percentage of People Who Indicated Which Vehicle Options They Would Buy if They Had $2,500 to Spend on Options for a New Car
Table 3:    Dollars Willing to Pay For ALI-Scout Option on New Car
Table 4:    Average Responses to How Frequently Each Factor was at Least Partially a Reason for not Following a Recommended Turn
Table 5:    Percentages of People within Each Age Group and Gender who selected the Various Options for a new Car
Table 6:    Average Dollar Amount Respondents were Willing to pay for ALI-Scout by Age Group and Gender
Table 7:    Average Ratings of Importance for Factors in the Operation of ALI-Scout-Like Systems by Age Group and Gender
Table 8:    Average Ratings for Frequency with Which Each Reason was involved in Deciding not to Follow TetraStar Turn Recommendation by Age Group and Sex
Table 9:    Average Ratings of How Frequently TetraStar Was Used for Various Trip Purposes by Age Group and Sex
Table 10: Average Ratings for Several Sources of Route Guidance Information by Age Group and Sex
Table 11:  Percent of Subjects Indicating Which ATIS Gave Them the More Positive Impression by In-Vehicle Guidance Feature
Table 12:  Percent of Subjects Indicating Which ATIS They Would Prefer to Have in Three Purchase/Rental Situations
Table 13:  Classifications for In-Vehicle Navigation Assistance Systems and the Example Systems Tested in the Troika Driving Study
Table 14:  Trip Time Savings (Percentage and Seconds) for ALI-Scout and TetraStar Over Written Instructions as a Function of Driver Familiarity and Traffic Conditions
Table 15:  Average Rating for Various Sources of Route Guidance as a Function of Navigation System Respondent used in Study
Table 16: Incident Scenarios
Table 17: Average Trip Time (Min) in the No-Incident Case
Table 18: Average Trip Time (Min) in the Incident Case
Table 19: Total Accidents on Selected Corridors Before and After SCATS
Table 20: Change in Turn-Related Accidents Before and After SCATS
Table 21:  Difference In Total Travel Time From The No Control Change Scenarios
Table 22: Difference In Congestion Duration From The No Control Change Scenarios (Both-Lane Closure Incident)
Table 23: Difference In Total Travel Time From The No Control Change Scenarios (10-Minute Both-Lane Incident)
Table 24: Difference In Congestion Duration From The No Control Change Scenarios (10-Minute Both-Lane Closure Incident)
Table 25: Stakeholder Groups to the FAST-TRAC Global Evaluation Process
Table 26: Ratio of Observed to Expected Cluster Membership, by Stakeholder
Table 27: Coding and Frequency of Responses
Figure 1: ALI-Scout Route-Guidance System Interface
Figure 2: High Level Flow Chart of Original Concept
Figure 3: Geographic Extent of southeast Michigan ITS Deployment
Figure 4: Geographic Extent of FAST-TRAC Deployment Effort
Figure 5: Critical Aspects of SCATS (ATMS) and ALI-SCOUT (ATIS)
Figure 6: Original Concept of Logical System Integration
Figure 7: ALI-Scout Interface
Figure 8: ALI-Scout Keyboard
Figure 9: ALI-Scout Keyboard
Figure 10: Traffic Modeling Evaluation Framework
Figure 11: Scenarios for Market and Route Guidance Computation
Figure 12: Delay Along I-75
Figure 13: Moving average speed on NB John R.Road-May 17, 1993 (Maple Road to Long Lake Road-morning period
Figure 14: How have the new traffic signals affected traffic congestion (B6) and safety (B7)?
Figure 15: Would you support or oppose Road Commission efforts to equip more intersections with high-tech traffic signals?
Figure 16: The Road Commission for Oakland County could distribute real-time traffic information in different ways. Which of the following methods appeal to you?
Figure 17: Contents of the Systems Integration Model
Figure 18: Stakeholders in the systems Integration Process

Introduction

FAST-TRAC is an Intelligent Transportation System (ITS) that integrates advanced traffic control with a variety of advanced traffic information systems through centralized collection, processing, and dissemination of traffic data. The Road Commission for Oakland County (RCOC) manages the program. Although the project originated within the City of Troy in 1992, it has expanded to encompass a number of the areas comprising Oakland County including Troy, Rochester Hills, Auburn Hills, Pontiac, Milford, and South Lyon.

From the beginning of FAST-TRAC, the mission was to implement an "integrated traffic management and traffic information system leading to improved mobility and safety on the roads and freeways of Oakland County." The original tests and plans called for the phased deployment of three specific systems: the ALI-Scout route guidance system (ALI-Scout), the Sydney Coordinated Adaptive Traffic System (SCATS), and the Autoscope™-2003 (Autoscope) video vehicle detection system.

The concept of integration was expanded to include communication with the Michigan Department of Transportation's (MDOT) traffic operations center, ERINet users, and the SMART suburban transit system. The new Transportation Information Management System (TIMS) located at Road Commission for Oakland County’s traffic operations center in Pontiac is the hub of this continuously evolving communications network.

An important element of the FAST-TRAC program was a test of the ALI-Scout route guidance system. ALI-Scout, an Advanced Traveler Information System (ATIS) technology, is the U.S. adaptation of the Euro-Scout System developed in Germany by the parent company of Siemens Automotive. The three major components included: a network of roadside infrared beacons, vehicles equipped with on-board computer systems, and a central computer that contains route guidance and traveler information. Vehicles could exchange traffic information with the central computer via an infrared communication link between the vehicles and roadside beacons.

The second-generation ALI-Scout system provided dynamic turn-by-turn guidance to drivers who had units installed in their vehicle. ALI-Scout vehicles communicated with infrared beacons at the roadside, sending travel times to the traffic control center, and receiving sequential routing instructions from the center. The travel times sent to the center told the system where congestion might have been occurring. This traffic information was used by the system to compute the fastest routes based on an historical travel time database. The routes were then communicated back to the vehicles as they passed by the beacons.

The ALI-Scout system was dismantled after the initial phase of the test was completed. This difficult adjustment accompanied a gradual and adaptive change in focus for the FAST-TRAC program, from an initial, limited test of SCATS and ALI-Scout integration, to a more comprehensive and general focus on systems integration through the TIMS.

Another important element of the FAST-TRAC test was the Sydney Adaptive Traffic System (SCATS) developed by the Road and Traffic Authority of New South Wales in Australia. The system is considered to be adaptive because it adjusts traffic signal timing in response to changes in traffic patterns to enable more vehicles to get through the system with fewer stops and less delay. Installation of the SCATS traffic control system continues to expand at intersections on the urban street network of Oakland County. Autoscope™ video image processors, that serve as replacements for loop detectors and provide intersection traffic counts to SCATS, augment the traffic controllers. The SCATS system was widely deployed and tested in FAST-TRAC, and continues to be expanded.

This chapter provides an overview of the approaches used to evaluate the complete FAST-TRAC system including the individual ALI-Scout, SCATS, and Autoscope™ subsystems, as well as the complete integration of ATIS and ATMS (Advanced Traffic Management Systems) in Oakland County. Specific details including the evaluation goals and objectives, the methods for addressing the objectives and research designs are addressed in the appropriate chapters of this report.

The evaluation studies include: (1) user perceptions and behaviors analysis, (2) human factors evaluation, (3) technical performance evaluation, (4) traffic modeling, (5) intersection delay analysis, (6) corridor delay analysis, (7) system delay and capacity analy­sis, (8) special events analysis, (9) incident response analysis, (10) accident analysis, (11) cost evaluation, (13) global analysis, (14) stakeholder analysis, (15) institutional issues evaluation and (16) TIMS evaluation.

Overview and Early History

Oakland County is located in the center of southeastern Michigan, about 15 miles north of the City of Detroit. It is part of the greater Detroit metropolitan area. The county is well connected to Detroit and other neighboring centers by an extensive network of interstate highways and state and local roads. The historically rural county has 37 communities and 24 townships. The communities in the southeast portion of the county have experienced an exceptionally significant influx of industry and population over the last two decades. The population growth in the county over from 1980 to 1995 was 12%. Today, the county which covers 910 square miles of land area, is Michigan’s most populated county with roughly 1.1 million residents as of 1997. More significant in terms of traffic generation may be that in the same time period two thirds of all new office development in Michigan took place in Oakland County.

Traffic operations in Oakland county are partly managed by the Road Commission of Oakland County (RCOC). RCOC (established 1913) has jurisdiction over nearly 50% of the county’s road infrastructure (ca. 2500 miles), 139 bridges and approximately 1000 traffic signals. In addition, RCOC has a maintenance contract for 300 miles of state highway within the county. Funding comes primarily from the state fuel tax and license plate fees (not property taxes), although some municipalities have opted to contribute additional funds in the form of road millages.

In 1987, the 61 local government units of Oakland County reported concern with traffic congestion. Increased traffic and demands on the road infrastructure threatened the quality of life in the communities. Under the guidance of Oakland County’s Road Commission a strategic planning process was initiated in order to develop approaches to address the traffic situation. Cost estimates for the road widening and improvements necessary to handle the increased demand exceeded the financial capabilities of the local governments. Innovative traffic management approaches were explored as an alternative to road construction. The adoption of a traffic management approach was seen as a logical extension of efforts to increase road and traffic safety, a top priority for Oakland County’s Road Commission since 1977. As a consequence, the Road Commission’s focus evolved from traditional road construction and maintenance toward a traffic management oriented approach to respond to increased travel demands and increase the focus on traffic safety.

In 1988, RCOC presented a conceptual plan using an advanced traffic management system as part of a comprehensive road improvement program. Approximately $2 million of the $100 million plan was to be used for a computerized traffic signal system. Partly a political decision to gain support of constituents, this computerized traffic signal system was to be used in the highly populated southeast of the county (City of Troy). At that time, only the budget for the computerized traffic signal system was approved. While planning to implement the system, it quickly became apparent that the $ 2 million would not be sufficient to deploy all elements of a computerized traffic system. Nevertheless this funding represented the seed for the FAST-TRAC program.

In 1992, the FAST-TRAC program was invoked as a major ITS Operational Test in Southeastern Michigan, integrating an ATMS and an ATIS (route guidance). The City of Troy served as a testbed for a small-scale, traffic control system composed of 28 intersections under automated, adaptive signal control (SCATS) and 17 ALI-Scout beacon-equipped cars as the ATIS component. Over the years, the project has evolved and grown considerably in size and geographic coverage. ITS equipment and installation was increased incrementally. For example, in June 1996, a total of 5 regional computers controlled approximately 270 intersections over southern Oakland County. By the completion of the project in August 1998, the number of controlled intersections had been expanded to 350, including approximately 20 closed circuit television cameras to perform automated traffic surveillance and monitoring. Several inter-agency and intra-agency links for data sharing and exchange were established. One link exists with the Michigan Department of Transportation’s (MDOT) urban expressway instrumentation project in Detroit. Other links facilitate retrieval and input of pertinent traffic events from police, emergency services and traffic departments. Furthermore, integration of operations with the Suburban Mobility Authority for Regional Transportation (SMART) public transportation service was achieved. Figures 3 and 4 give an overview of the deployed systems and geographic areas of deployment in Oakland county and Southeast Michigan.

Integration of the many systems involved in FAST-TRAC was a concerted effort in which the Road Commission for Oakland County (RCOC), a local public agency, collaborated with consultants, systems vendors from the private industry and other public authorities. FAST-TRAC's integration of various ITS systems was accomplished through strong leadership from the implementing agency and a willingness of the involved parties and vendors to cooperate and work toward a common goal. The major FAST-TRAC partners included Siemens Automotive, Odetics ITS (previously Rockwell Systems, Inc. (ISS), AWA Traffic Systems - America, Inc. (AWATSA)^[1], the Suburban Mobility Authority Regional Transportation (SMART), the Michigan Department of Transportation (MDOT), and the University of Michigan.

FAST-TRAC represents a considerable financial investment in the future of traffic operations in Oakland county. Oakland County officials were successful in securing federal grants under ISTEA (Intermodal Surface Transportation Efficiency Act) legislation, which included language to promote and further ITS technology testing and deployment. Since the program’s inception in the early 1990’s, different FAST-TRAC components, such as field tests, systems design and integration have been jointly funded by RCOC, U.S. DOT and the MDOT.

Program Evolution and Recent History

Oakland County’s FAST-TRAC project represents a prime example of learning by doing. At the onset of the project very little experience existed regarding deployment and integration of ITS technologies for traffic management. The National ITS Architecture was undergoing initial development and could not be used to guide integration planning. The courage to experiment and the flexibility of officials and project partners in adjusting to technical advancements and shifts in political agendas proved to be a major asset of the project. As a result of the mutual learning process and changes in political climates, both the overall FAST-TRAC project goals and the objectives for the system integration changed over time.

The initial stage of FAST-TRAC was driven by local traffic management needs and political earmarking strategies. Faced with declining funds and the related inability to address growing traffic capacity needs through construction, Oakland County’s Road Commission leadership aggressively pursued ideas to improve traffic operations and management. A primary objective at that time was to extend the traffic signal system in the county to increase safety and optimize traffic flow on the existing roads rather than fund additional road construction. It soon became apparent that a high-tech system, offering real-time, area-wide adaptive traffic signal control would probably fulfill local traffic management needs but would be too expensive for local pockets. Lobbying and earmarking for a federally funded ITS operational test seemed a feasible and appropriate approach to secure the necessary funding, given that Congressman Bob Carr could show significant investment by local governments and industry.

Ideas for FAST-TRAC and system integration were developed as early as 1988 as key individuals met at a transportation planning conference where new technologies were introduced, amongst them video-image processing as an alternative to inductive loops. Haugen and Associates, a consulting firm helped to establish a fruitful contact between Siemens, ISS and county officials, that ultimately led to the realization of FAST-TRAC. By the end of 1990, officials and county politicians managed to get hold of start up funding of U$ 3,700,000 from the county and industry to launch a small test with 28 adaptive traffic signals and a beacon based route guidance system. This quick start phase built a base on which a larger proposal for an ITS operational test was then submitted at the federal level.

In this proposal, FAST-TRAC was conceived as a field test that integrates an advanced traffic management system with an advanced traveler information system in the City of Troy. Core components of the systems integration effort were the Sydney Coordinated Adaptive Traffic System (SCATS), ALI-Scout route guidance system and AUTOSCOPE vehicle detection. FAST-TRAC goals were to:

• improve travel times
• reduce accidents
• improve air quality
• test integration of an ATMS (SCATS) with an ATIS (ALI-Scout)

From December 1990 to summer of 1992, planning, procurement and implementation of a “quick phase” installation of 28 SCATS intersections was undertaken. In the meantime federal funding was approved to place an additional 70 intersections in Troy on the network of the adaptive signal system, to establish a traffic operation center and to install more ALI-Scout beacons and equip more vehicles with the Ali-Scout route guidance system.   Figure 5 depicts the critical parts of the two major systems, SCATS and ALI-Scout, whose integration was the original FAST-TRAC concept.

During this first stage, planning had an informal character and was based on the vision and personal bonds of a few determined politicians and managers. The interaction between the project partners was in small meetings and agreements were forged with handshakes and conversation. Everybody was excited and cooperative in a spirit of invention and exploration.

However, the FAST-TRAC project gaining ITS operational test status started to stretch local experience and human resources, especially since in an unusual circumstance it was decided that RCOC was to administer the federal grant locally - a responsibility typically assumed by the State transportation agency. The exception to prior practice in grant administration was a result of extensive administrative restructuring at the State level. RCOC was thrown into unknown territory. Federal procurement regulations and requirements for ITS field tests place a heavy administrative burden on the agency. Organization and administration of the program needed to be adjusted. This was done by establishing an internal management team which oversaw administrative aspects of the program and introducing procedural improvements.

With the deployment of ITS technology progressing quickly further ideas and opportunities in other jurisdictions emerged, such as collaboration with MDOT’s ITS deployment project in Southeast Michigan. Over time it became apparent that a systematic approach to systems integration would be needed.

Figure 5 Critical Aspects of SCATS and ALI-SCOUT

The project planning process moved into a semi-structured mode with the formal establishment of a FAST-TRAC Systems Integration Subcommittee in November 1992. RCOC invited representatives from Siemens, AWATSA, ISS, FHWA, MDOT, Rockwell and The University of Michigan to:

• discuss and identify system integration issues,
• determine a strategic path, and
• establish strong relationships amongst the project partners.

The committee met regularly and worked on conceptual issues regarding system integration (e.g. functional integration, data integration, jurisdiction and subsystem integration).

Furthermore, RCOC had put out a formal request for proposal to hire a systems integrator to aid in the formidable task of carrying out the envisioned system integration. Responses to this RFP came not surprisingly from the Aerospace and military industries. Both aerospace and military suppliers could provide the engineering background and experience of systems integration, however, none of them had worked with local governments before and vice versa. Local governments were not used to the problem-solving approaches in aerospace systems integration projects. Aerospace and defense industry companies were not familiar with local government needs and operations practices. A mutual and at times costly learning process was started as the integrators worked hard to develop a larger systems integration concept. Through this process information flows and systems needs were mapped and hierarchies and links outlined. Several attempts were stalled and the proposal had to be revised.

As the project progressed, RCOC broadened the conceptual and geographic scope of FAST-TRAC. As a consequence, FAST-TRAC moved away from the dualism of integrating a traffic management system and a traveler information system toward the integration of multiple intelligent transportation systems. Shortcomings of the early concept were identified. These include the inherent inefficiency of traffic management on a partial road network without information about the traffic status of adjacent arterials or highways. Likewise, the ALI-Scout system provided route guidance only to a small subset of vehicles. A set of functional systems components were added to improve upon the effectiveness of future operations. The new concept identified public transportation, emergency road services and commercial fleet operations as important elements that need to be taken into account in systems integration. In contrast to early systems design attempts in Phase I, the system architecture was now conceived not as a hardwired system with core components, but as a loosely coupled network of interacting transportation subsystems at different jurisdictional levels and agency responsibilities. Partners to participate were the Michigan Department of Transportation (MDOT) with its freeway operation, SMART’s computer aided dispatch, and various law enforcement agencies whose services were to be linked and integrated into Oakland County’s TIMS.

The intent of deploying these different technologies was to facilitate the mitigation and avoidance of traffic congestion through optimization of network trips and maximization of network capacities. These objectives were to be achieved by collecting traffic flow information, managing traffic flows through signal timing, and disseminating congestion information to travelers to encourage re-routing and trip avoidance. According to the Systems Requirement Specifications (Report Draft, Aug. 10, 95; Rockwell), goals and objectives of the FAST-TRAC project were to provide:

• cost-effective traffic operations,
• measurable transportation,
• environmental improvements.

Evaluation Objectives

The FAST-TRAC project evolved out of an initial concept of combining SCATS, ALI-Scout, and Autoscope™ as described in the original 1990 proposal by the Road Commission for Oakland County to the Federal Highway Administration. In October 1991, the U.S. Congress appropriated $10 million for the FAST-TRAC operational field test. Federal moneys were combined with $2 million from RCOC and contributions from private partners including Siemens Automotive. FAST-TRAC was designated as an operational field test to evaluate applications of new technologies and systems concepts, facilitating the transition from research and design to operational use. A key feature of FAST-TRAC has been the integration of ATIS and ATMS in a single geographic location.

This Evaluation Summary Report began in a series of Evaluation Subcommittee meetings during which the project partners and the evaluation team defined the evaluation objectives and the approaches that would be most appropriate for evaluating the FAST-TRAC systems. The FAST-TRAC Evaluation Subcommittee meetings produced a consolidated list of objectives that guided the development of the work plans for the individual studies summarized in this document. These work plans were revised to include many of the changes that came into effect during the life of the project.

The original evaluation objectives were derived from the system de­sign objectives for each of the individual subsystems: ALI-Scout, SCATS, and Autoscope™. However, the set of objectives was expanded beyond these basic subsystems to encompass a more general notion of systems integration. While some of the original objectives relate to a specific sub­system including ALI-Scout, SCATS, and Autoscope™, other objectives pertain to the system as a whole -- the integrated FAST-TRAC system. The following is the consensus for the original list of the primary system and subsystem objectives to have been addressed in the evaluation.

The objectives for ALI-Scout (dynamic route guidance) were:

• To help drivers find their way in unfamiliar areas, avoid navigational errors, and to avoid getting caught in unex­pected traffic congestion;
• To provide an interface that is safe and easy to use;
• To provide a perceptible benefit to the driver who will purchase the in-vehicle components of the system;
• To guide vehicles more efficiently and thereby alleviate traffic congestion;
• To perform reliably under various weather and usage conditions;
• To cost less than what potential customers are willing to pay (i.e., all production and distribution costs).

The objectives for SCATS (coordinated adaptive traffic signal control) were:

• To increase the level of service on the urban street network (i.e., reduction in travel times, delays, and stops);
• To provide well-timed signal progressions on the streets in response to traffic demand at any time of day;
• To have positive second-order impacts (i.e., reduce traffic ac­cidents and fatalities, fuel consumption, vehicle emissions, and vehicle operating costs);
• To perform reliably under various weather and usage conditions;
• To be easy to operate, assisting the staff at the traffic control center in performing their diagnostic, operation, and recon­figuration tasks;
• To cost less than or equal to what the public revenues are likely to support (including training, installation, maintenance, operations, etc. costs).

The objectives for Autoscope™ (video im­age detection system) were:

• To measure standard variables of vehicle presence, volume, and occupancy (i.e., as a replacement for loop detectors);
• To add flexibility to the deployment of detectors in the area in­cluding adjustment, addition, and removal of detectors un­der all weather conditions;
• To reduce traffic disruptions during maintenance and repair of the detec­tors;
• To maintain operation during road construction and other events;
• To improve performance in the long term by providing a means for upgrading the software;
• To cost less than or equal to what the public revenues are likely to support (including training, installation, maintenance, operations, etc. costs).

The primary integrated system objectives were:

• To disseminate centralized traffic information to a range of potential users;
• To provide integrated service at an acceptable cost.

Objectives related to the integration of ALI-Scout and SCATS were dropped in response to the early evaluation results for ALI-Scout test.

These system and subsystem objectives provided the focal point for the FAST-TRAC evaluation.

Evaluation Overview

The evaluation summaries are organized into four groups. The first set of summaries address the ALI-Scout route guidance system and include the natural use study, troika driver study, choice modeling, human factors, technical performance, and traffic modeling evaluations.

The second set of summaries address the SCATS coordinated adaptive traffic control system and include evaluations of intersection delay, corridor delay, system delay and capacity, response to special events, response to incidents, response to accidents, and video image processing.

The third set of summaries address the process of systems integration as well as the integrated system itself. Evaluation studies include the systems integration case study, systems integration model, the traveler study, institutional issues, and a stakeholder analysis.

The final set of summaries present a global evaluation of the FAST-TRAC project which includes a cost evaluation and a global analysis.

ALI-Scout Route Guidance Evaluation

This section describes a set of studies designed to evaluate many facets of the ALI-Scout route guidance system and route guidance in general. The ‘User Perceptions and Behaviors’ studies were a group of evaluations that focused primarily on understanding how ALI-Scout users perceived and valued in-vehicle route guidance during the ALI-Scout test. Other studies in this section include ‘Human Factors’ which examined ease-of-use and safety factors related to route guidance systems, ‘Technical Performance’ which addressed the performance of the ALI-Scout hardware, software, and databases tested, and ‘Traffic Modeling’ which analyzed the impact of various levels of market penetration of a route guidance system in the Southeast Michigan area.

User Perceptions and Behaviors - Perhaps the most important questions about the ALI-Scout route guidance system were: “What do the drivers think of the system? What aspects do they like or dislike? Do they perceive a net benefit? What would they be willing to pay for this service?” This set of studies examined how the test drivers used the system, and how use of the system affected them. In particular, the studies addressed differences in drivers’ perceptions, behaviors, and attitudes that may be attributable to: (1) ALI-Scout (versus drivers who did not use ALI-Scout), (2) types of guidance, and (3) driver characteristics.

Natural Use Evaluation - In the “natural use” evaluation, study subjects used ALI-Scout for approximately one year. The concept the evaluation was to allow a large number of drivers to become familiar with the features and benefits of the system as they would use ALI-Scout in their everyday driving tasks. At intervals throughout the use period, the evaluation team questioned participants about various aspects of the system. In general, the purpose was to obtain opinions from many types of drivers under static and dy­namic guidance conditions. Data was collected through inter­views, questionnaires, focus groups, driver logs, and vehicle tracking. Conjoint analysis was used to assess the drivers' preferences for the various features of ALI-Scout. It determined the drivers’ relative valuation of the features including display format, delay update times, network coverage, entering multi-stop destinations, and so on. The analysis also provided information on the subjects' marginal willingness to pay for these features.

Troika Driver Evaluation – The name “troika” implies that three participants in the experiment were joined together, and simultaneously faced the same task. This evaluation study plotted drivers of ALI-Scout equipped vehicles against drivers of vehicles without ALI-Scout (with a traditional road map), and drivers using a map-based autonomous route guidance system. The drivers were instructed to drive to the same destination through the same beacon-equipped network. The objectives were to (1) determine where the drivers went, (2) note differences in their routes, and (3) assess the differences in trip times between the equipped vehicles and non-equipped vehicles. Measures of effectiveness included travel time, travel distance, queue times, number of turns, number of stops, driver stress, and driver attitudes. Routing differences were compared and evaluated.

Choice Modeling – Using quantitative data, the choice modeling study evaluated user preferences regarding three classes of traveler services: 1) traffic reports, 2) route advice, and 3) (advanced) emergency roadside assistance. To assess consumer preferences regarding service bundles and products, a questionnaire was presented to a set of subjects who were asked to respond as to whether they would buy a service/product or not. Both price and service types were explored.

Human Factors Evaluation - The human factors evaluation addressed issues related to the ease of use and safety tests of the ALI-Scout in-vehicle unit from the driver’s standpoint. The evaluation activities began with an on-road pilot test to collect preliminary information on the use of ALI-Scout followed by two experiments. The first experiment took place in a controlled laboratory setting and evaluated how demanding it was for drivers to input their destinations. The second experiment took place on the road and addressed the drivers’ ability to recover from “being lost” (in other words, deliberately or mistakenly deviating from the given driving instructions).

Technical Performance Evaluation - The technical performance evaluation addressed the performance of hardware, software, and data­bases by looking at four subsystems: the central com­puter, the vehicle-roadway communication, the in-vehicle equipment including dead-reckoning, and the probe transmissions from the vehicle. It also addressed whether the system could grow and continue to be compatible with new and im­proved systems over time.

Traffic Modeling Evaluation - Traffic simulation was used to model the impact of in­creasing market penetration of ALI-Scout. The initial evaluation was used for planning the field implementation of the evalua­tion. The evaluation was used to assess the number of ALI-Scout equipped vehicles that would have been required to generate dynamic traffic data around the Chrysler Technology Center (CTC). For the ALI-Scout subsystem, each of the equipped vehicles also served as a traffic probe. This probe capability was only of value when enough vehicles were in the network. The question that this study pursued was “How many probe vehicles are enough to test the dynamic traffic update capability of ALI-Scout?”

Traffic Control System Evaluation

The FAST-TRAC evaluation also addressed the coordination of traffic signals that adapt to traffic demand in real-time. As traffic demand levels fluctuate throughout the day, the signals responded by coordinating their cycles in a manner that favored the flow of traffic along the arterial routes with the highest demand. The goal was for SCATS coordination of signals to increase the traffic throughput of the system as well as smooth traffic flow. Desired indirect impacts included reduced accidents, emissions, and energy use.

The evaluation of the traffic control system addressed changes in traffic flows at selected intersections, along selected corridors, within the overall system, and in response to special events, and incidents. Possible changes in accident patterns were also monitored.

Intersection Delay Evaluation - One of the most frustrating aspects of automotive travel is needless delay at signalized intersections. Reducing delay through the adaptation of signals to traffic demand and the coordination of high volume intersections improves travel for the individual driver as well as increases the effectiveness of the transportation system as a whole.

The intersection delay evaluation measured the delay at various approaches to selected SCATS-equipped intersections in the morning, afternoon, and evening. The data was analyzed to identify any differences in the delay measures between the before- versus after-SCATS installation conditions.

Corridor Delay Evaluation - The primary objective of SCATS was to increase system throughput by coordinating signal cycles and adapting to changes in traffic demand. High-demand corridors have series of signals with common cycle lengths appropriately offset to enable platoons of vehicles to proceed through the corridor relatively unimpeded. If the volume at any of the intersections changes due to an incident, a special event in the area, or for other reasons, the coordination of the intersections will gradually adapt to the change in demand. It was expected that the coordination among the signals and the adaptation to traffic demand would result in reduction in delays along high-volume corridors. The corridor delay evaluation tested whether travel time along selected corridors changed after the installation of SCATS.

High-volume corridors were selected for monitoring based on SCATS implementation, historical traffic volumes, intersection delays, peak-hour levels of service, accident history, and road geometry criteria. Travel time and average moving speed along the corridors were collected in two ways: (1) timing vehicles at the beginning and end of the corridor, and (2) sending "floating" test vehicles through the corridors and logging their times. The stop bar at the intersection marked the beginning and ending points of the corridors. Video image processing methods were tested for collecting corridor travel time data.

System Delay and Capacity Evaluation - The evaluation hypothesis was that the net impact of the SCATS deployment should be an increase in network capacity enabling greater vehicle throughput and/or less delay. The system-level impact of SCATS could not be measured directly from the empirical analysis of sampled intersections and corridors. Rather, the data analysis had to be extended to the entire network for a full system-wide analysis. The plan was to evaluate the level of service (LOS) for each intersection approach in the network. The Highway Capacity Manual Software was used to determine the maximum flow at each level of service. Maximum flows were compared with existing peak period flows to determine the LOS and the unused capacity of each approach. This information helped in assessing the potential for re-routing traffic during an incident.

Special Event Evaluation – This evaluation hypothesized that SCATS should respond to unusual changes in demand that may arise from special events like sporting events that attract large volumes of traffic. The Pontiac Silverdome, home of the Detroit Lions, hosted a World Cup soccer match in the summer of 1994. This provided a unique opportunity to test the capabilities of SCATS to respond to traffic generated by a major event. Because the Silverdome is a major attraction and a source of regular traffic jams, the SCATS system was extended to the Silverdome area in anticipation of the World Cup games.

A "before and after" experimental design was used in the proximity of the Silverdome on two events with similar expected traffic volume characteristics. Field data was collected for intersections in the deployment area including percentage flow for each leg, queue length at each cycle, signal timing, traffic volume, and any construction activity that might modify traffic flow. Volumes and delays were measured and compared at events before and after the SCATS deployment. Multiple measures were made for repeated events before and after deployment.

Incident Response Evaluation - Traffic incidents, like lane-blocking accidents, are another source of delay that could be reduced through the implementation of traffic control systems such as SCATS. The incident response evaluation involved collecting data on scheduled "incidents" like road maintenance in the SCATS deployment area. This data was used to calibrate a microscopic model of the deployment area with capabilities of representing both SCATS and standard fixed-time signals. Simulation runs produced measures of queue lengths, delays, and travel times under both SCATS and no-SCATS conditions.

Accident Evaluation - Improved traffic safety was a fundamental goal of the FAST-TRAC program. An expected benefit of SCATS was the reduction of certain types of accidents, for example rear-end accidents, that can be caused by discontinuities in the flow of traffic.

The accident evaluation employed a before-and-after with control experimental design. This required the monitoring of selected SCATS-controlled corridors and intersections and comparison against similar matched non-controlled corridors and intersections in the deployment area. Matching with non-controlled areas was necessary in order to factor out changes due to seasonal effects, traffic demand fluctuations, and other confounding factors.

Data were collected from the Transportation Accident Master from the Traffic and Safety Unit of the Michigan Department of Transportation (MDOT). This file was taken from the Michigan Accident Master, compiled by the Michigan State Police (MSP) from the Official Michigan Traffic Accident Report (form UD-10).

Global Evaluation

Evaluation plans included a summary analysis of the project as a whole.

Cost Evaluation (including ALI-Scout Cost Analysis) - The expected benefits of SCATS and ALI-Scout came at a cost. The cost evaluation assessed the cost of various components of SCATS and the system as a whole, including the integration of Autoscope™. Cost items included purchase price, installation cost, maintenance cost, training cost, operating cost, replacement cost, and overhead and administrative costs. The cost analysis addressed the cost per intersection, cost per corridor, and total system cost.

The costs of ALI-Scout were evaluated to determine the total cost at expected levels of production. Cost items include expected purchase price, subscription fees, installation cost, maintenance cost, training cost, operating cost, replacement cost, and over­head and administrative costs. The costs borne by the vehicle owner and the system operator were also assessed. Forecasted purchase prices and sub­scription fee assessments were based on the assumption of a competitive market and marginal profit rates. The cost assessment involved evaluating the system component costs and ex­pected manufacturing costs.

Global Analysis - The global analysis component of the evaluation study was quite different in nature from each of the other study elements. Whereas the other study elements analyzed a particular outcome of the FAST-TRAC system, the global analysis sought to integrate cost and benefit information from each of the other studies. The intent was for this analysis to provide a “road map” to enable users of the evaluation to assimilate and compare the disparate information coming from the rest of the project. The proposed method was a Multi-attribute Utility Technology (MAUT) style of analysis designed to elicit preferences on system attributes and to delineate desirable alternatives based on preferences of relevant stakeholder groups. Data supporting this analytical approach was the product of coordination with other study components.

Integrated System Evaluation

In addition to studying the SCATS and ALI-Scout subsystems individually, the evaluation plan called for evaluating the integration of ATIS and ATMS.

Stakeholder Evaluation - In addition to talking with individuals who were directly involved in the FAST-TRAC project, the evaluators interviewed community leaders on their responses to FAST-TRAC and the other similar ITS deployments. The evaluation called for identifying a range of stakeholder groups including emergency vehicle firms, de­livery services, environmental groups, consumer advocacy groups, etc. From these groups, the evaluation team identified individuals to represent the perspectives of the group for interviews.

The stakeholder analysis provided an opportunity to assess community values regarding the implementation of FAST-TRAC. Quantitative value assessment procedures provided measures of the relative value for safety, travel time improvements, potential neighborhood disruption, energy impacts, environmental impacts, and economic productivity from the various stakeholder perspectives. These quantitative values were used to assess the value associated with the various system conditions (i.e., SCATS only, ALI-Scout only, combined system, no system) in the global evaluation.

Institutional Issues - The institutional issues evaluation involved recording events throughout the FAST-TRAC tests, and the responses to difficult circumstances. Types of issues include legal, jurisdictional, organizational, and financial among others. The objective was to discuss non-technical issues with those who were involved in the events and to report their experiences. A series of individual and group interviews were conducted to document the background, history, goals, issues encountered, cause and effect relationships, and lessons learned. The information from the interviews was analyzed qualitatively. In later stages of the evaluation, institutional issues were monitored regularly on a monthly basis through individual interviews.

TIMS Evaluation - The final phase of the FAST-TRAC project is referred to as "TIMS Traffic Information Management System.” The goal of TIMS is to integrate traffic-related information in pursuit of improving road operations and traveler services information within the project area. Upon integration, the TIMS' various subsystems will become interactively linked, and therefore allow TIMS to act as a central system for data collection and dissemination. TIMS will enable RCOC to monitor traffic conditions throughout Oakland County and to provide travelers, businesses and agencies with real-time traffic information. The TIMS' evaluation was divided into two studies: a Traveler Study and the Systems Integration study.

Traveler Study – This study pursued several objectives: 1) to investigate how travelers perceive the FAST-TRAC system; 2) to solicit suggestions for future improvements; and 3) to examine how individual travelers with access to the traffic information provided by TIMS (Website) respond to it, i.e., how they use it and what they think about it, especially what benefit they believe it provides to them.

Systems Integration Study - This study consisted of two parts: the Case Study and the Systems Integration Model. The Case Study evaluated the systems integration process, obstacles and lessons learned in the effort to integrate multiple intelligent transportation systems locally and across jurisdictions. The Systems Integration Model generalized this process so that other agencies can use the process as an example for future systems integration.

Route Guidance Evaluations

Driver Perceptions and Behaviors

Objectives

The purposes and objectives of the User Perceptions and Behaviors evaluation component were:

1.      To understand how users of the ALI-Scout and TetraStar in-vehicle navigation systems perceived and valued the systems;

2.      To understand how drivers used the systems and how this use influenced important behavioral, perceptual, and affective outcomes;

3.      To understand how the behaviors and perceptions of users of the systems differed from each other;

4.      To understand the differential effects of static and dynamic route guidance;

5.      To understand how drivers’ use and perception of the systems differed based upon age, gender, and experience;

6.      To explore the safety impacts of driving in vehicles equipped with the in-vehicle navigation systems.

7.      To learn whether users of the two systems would purchase a route guidance device, and if so, how much they would be willing to pay.

The details of how these objectives were achieved and the criteria that were used for assessment of the objectives are described in detail in the following sections.

Method

The user perception and behavior element of the FAST-TRAC evaluation was broken down into four main components. These were:

• Pre-testing and Pilot Testing
• Natural Use Study
• Troika Driving Study
• Choice Modeling Study

The evaluation task components are described in detail in the following sections.

Pre-testing and Pilot Testing

The purpose of this component was to pretest and pilot test data collection methods and instruments planned for use in subsequent evaluation tasks. This testing included assessment and modification of survey instruments, vehicle tracking technologies, the ALI-Scout system, subject training materials, and other data collection strategies.

Subjects for the pretest were selected on the basis of convenience for the subjects and experimenters. It was found that a relatively limited number of subjects were sufficient for this task (about 40-60 subjects). While this initial testing phase was broken out separately from the other tasks, some additional pre-testing, pilot testing, and subject recruitment clearly was necessary as the Natural Use and Troika Driving Studies proceeded.

The first 60 vehicles to be equipped with ALI-Scout formed the basis of the pilot study. This fleet consisted predominately of the vehicles of the project’s industrial partners, as well as of the Road Commission for Oakland County. Thus, most of the subjects for the pilot study were lessees or users of fleet vehicles. Efforts were made to include the widest range of demographics in the pilot test subjects.

The pilot study was treated as a full-fledged part of the Natural Use Study (NUS), meaning that the necessary instruments, procedures, and protocols (such as surveys and subject training) were implemented as planned for the NUS. The pilot study offered a final opportunity for research staff to review and refine research plans and materials. In addition, the pilot study provided preliminary data that could be used as an early assessment of the project.

Natural Use Study (NUS)

This study served to provide measures of how drivers perceived and valued the ALI-Scout and TetraStar in-vehicle navigation systems. This study was referred to as “natural use” because the participants in this study drove and experienced the system within the context of their normal, everyday driving patterns. For example, subjects who commuted through the instrumented network on a daily basis experienced the system through their daily commute.

Reasonable effort was made to keep the evaluation tasks as invisible as possible to drivers. While it was not possible to keep the evaluation effort completely hidden from the subject drivers, much of their contact with the evaluation occurred after their experience with the system had concluded. In short, each study participant used the system as part of his or her naturally occurring travel needs.

The NUS consisted of three parts, each with slightly different methods, subject groups, or data collection instruments. Each part was conducted as a distinct study. These parts were:

• The ALI-Scout Natural Use Study-Personal Vehicles (NU-Personal)
• The ALI-Scout Natural Use Study-Leased Vehicles (NU-Leased)
• The TetraStar Natural Use Study-Leased Vehicles (NU-TetraStar)

Data analysis consisted primarily of analysis of variance (ANOVA) procedures.

The ALI-Scout Natural Use Study-Personal Vehicles (NU-Personal)

The general plan for this study was to 1) recruit participants from the population of drivers in the Oakland County study area who drove a vehicle of the correct make and model (for easy installation of ALI-Scout), 2) install ALI-Scout in their vehicle, 3) let them drive while the ALI-Scout system was providing static route-guidance, 4) survey them, then 5) let them continue driving while the system began to provide dynamic route guidance, and 6) survey them again.

Vehicles and Subjects - The procurement, availability, and allocation of vehicles were important considerations in the design of the NU-Personal study. Because the ALI-Scout system was installed somewhat differently in each vehicle model and year, there was a need to limit the number of different vehicle models in this study. Therefore, a list of valid vehicle makes, models, and years was developed. The list consisted of approximately 30 of the most commonly owned vehicles. Approximately 400 subjects were recruited from a variety of sources, potentially including automotive company lease fleets (as in the pilot study), utility service fleets, and through newspaper and television advertisements running in the Oakland County study area. Those volunteers who owned or leased a “valid” vehicle and drove in the Oakland County study area had an ALI-Scout unit installed in their vehicle. Every effort was made to include as wide of a variety of demographics as possible in study participants, but the vehicle and location constraints of the study were limiting factors.

Research Design and Methods - One main pseudo-independent variable was investigated in this study--type of route guidance. Study participants used the ALI-Scout system for up to one year. The study commenced while ALI-Scout was providing static route guidance. During the study duration, ALI-Scout began providing dynamic guidance instructions. Participants were surveyed once after at least one month of experience with static route guidance and again after at least one month of experience with dynamic route guidance. Type of guidance was considered a pseudo-independent variable because it co-varied with length of time participating in the study.

Static route guidance determines the fastest route by using speed limits and distances, whereas dynamic route guidance uses this information combined with information about recurrent traffic congestion.

There were numerous dependent measures in this study, all of which were collected through the questionnaire survey. Questions addressed several aspects of the ALI-Scout system and route guidance systems in general, including: driving and commuting behaviors, use of and familiarity with new technology, ALI-Scout operation and displays, use and opinions of the ALI-Scout system, crash and near-crash experience, importance of in-vehicle navigation technology, and willingness to pay.

Results – This section summarizes the results as reported in the following project deliverable: “Evaluating the Perceptions and Behaviors of ALI-Scout Users in a Naturalistic Setting” (Eby, Kostyniuk, Streff & Hopp, 1997).

The study took place between July 1995 and December 1996. Of the 369 subjects, 72.3 percent were male and 10.8 percent reported an income below $45,000. Subjects were generally well-educated and had a mean age of 41.6 years. Subjects completed two surveys. The survey was first administered after one month of participation, during static route guidance. It was then administered again in August 1996, four months after dynamic route guidance was introduced. Both surveys were mailed to subjects with a stamped, pre-addressed envelope.

About one third of the respondents did not live in the Oakland County study area. In general, study participants reported traveling out of town frequently - almost 90 percent had taken two or more out-of-town vacations in the last year.

Overall, users liked and found value in the ALI-Scout in-vehicle, route guidance system. Results show that the most desired system attributes were: quick updates to current road conditions, ease-of-use, and accuracy.

Respondents reported that ALI-Scout was both easy to use and accurate, but subjects were disappointed with its ability to update current road conditions. Updates were too slow. The few significant differences between the static and dynamic periods were too small to separate from other possible causes such as time or exposure to the system.

Subjects were asked which vehicle options they would buy if they had $2,500 to spend on options for a new car. With the ALI-Scout system priced at $500, 29.7 percent and 21.7 percent indicated they would buy it as an option in survey one and two, respectively. See the following table for details.

Subjects were further asked how much they would be willing to pay for the ALI-Scout as an option on a new car. The modal response in both surveys showed a willingness to pay somewhere between $200 and $399 for the ALI-Scout device, and a willingness to pay more during the static than dynamic phase.

Subjects also reported they would be willing to pay an average of $214 (survey one – static), and $141 (survey two – dynamic) to add the ALI-Scout system to their present car. They reported they would be willing to pay an average of $6 (survey one – static), and $14 (survey two – dynamic) per day for ALI-Scout as an option on a rental car.

The ALI-Scout Natural Use Study-Leased Vehicles (NU-Leased)

In order to have better control over the subject demographics, subject training, and collect more detailed use and perception data, it was necessary to lease a fleet of identical vehicles. These vehicles were installed with the ALI-Scout system and provided to volunteers to drive for one-month periods. The vehicles were passed from subject to subject over the course of one year. This technique allowed for recruitment of subjects from a much wider range of demographic variables since the type of vehicle owned by the subject was not a constraint.

Vehicles and Subjects - A fleet of 12 identical vehicles was leased from a local dealership and installed with the ALI-Scout device. Approximately 100 subjects participated. In order to ensure that the general population of the Oakland County study area was sampled as widely as possible, recruitment occurred at a Secretary of State office in the study area. From the pool of interested people, the researchers selected subjects quasi-randomly, with constraints that they travel regularly through the instrumented network, fit the dependent variables, and have good driving records.

Research Design and Methods - Two independent variables were investigated in this study. The first independent variable was three age groups chosen to represent potentially distinct users of in-vehicle navigation systems. These age groups were 19-29, 30-64, and 65-80 years of age. The second independent variable was gender. Thus, this study had six distinct groups participating (i.e., three age groups, each with two genders) in a between-subjects factorial design. This design allowed for independent assessment of the effects of age, gender, and the combinations of the variables.
Each group had approximately 17 subjects for a total of 102 subjects.

There were numerous dependent measures in this study. All subjects in this study completed the same questionnaire as participants in the NU-Personal study. This survey was completed during their fourth week of participation. In addition to these dependent measures, all subjects maintained a record of their daily driving by completing a driver log at the end of each day. Driver logs included information about each trip taken during the day (origin, destination, trip purpose, time, distance, use of ALI-Scout, and whether turn-by-turn guidance was received) and any unusual driving experiences or problems with ALI-Scout (e.g., near-crashes or a beacon not working). Finally, after participation, all destinations entered into ALI-Scout memory were recorded and tallied allowing further assessment of the extent of ALI-Scout use.

Results - This section summarizes the results as reported in the following project deliverable: “FAST-TRAC Natural Use Leased-Car Study” (Kostyniuk, Eby, Christoff, Hopp & Streff, 1997).

The average age of male study participants was 24.2 (sd=3.7) for the 19-to-29 year old age group, 46.1 (sd=9.5) for the 30-to-64 year old age group, and 70.9 (sd=4.0) for the 65-to-80 year old age group. The average age of female participants was 21.1 (sd=3.1) for the 19-to-29 year old age group, 42.6 (sd=8.1) for the 30-to-64 year old age group, and 71.8 (sd=4.5) for the 65-to-80 year old age group.

As shown by driver logs, subjects frequently received turn-by-turn instructions during the month of participation. Overall, the ALI-Scout system was received positively by the majority of drivers. Subjects were generally happy with the system’s attributes and performance and used the system frequently (about 75-80% of trips) for a variety of trip purposes.

When the entire ALI-Scout system was considered as a whole, subjects’ responses were generally positive. Subjects found the system to be easy-to-learn and easy-to-understand. Both the amount of information and advance warning were generally sufficient. The system was fairly accurate and produced little distraction while driving. In general, the system was liked to some degree. However, subjects reported that the ALI-Scout system did not produce a change in travel time, driving safety, or fuel consumption. ALI-Scout did, however, seem to produce a slight perceived reduction in traffic congestion. There were no differences between ages or genders on these measures. Also, subjects reported that ALI-Scout produced slight decreases in the frequency of several crash-related incidents, and no subject reported being in a crash.

The system’s features receiving the worst assessment were generally related to autonomous-mode guidance. Also, subjects thought that the beacon coverage area was too small. This was not surprising, since, as a test system, the area was limited in size.

Of those not following the provided guidance, subjects reported that the most common reason for not following a recommendation was that they “knew of a faster route.” This suggests a lack of confidence in a route guidance system’s ability to provide the fastest route.

The study showed few differences in ALI-Scout uses and perceptions between genders or between the two youngest age groups. There were, however, several differences between the oldest age group and the other two age groups. Older drivers used the system more frequently but had greater problems learning, understanding, and using the system than younger drivers. Members of the oldest age group had more difficulty learning the keyboard and reported more lack of proper function than those in the youngest age group.

While subjects reported that all four destination-entry methods (points of interest, map, address ranges, current location) were easy to use, but the list of points of interest was the easiest to use. The study also showed that subjects understood the information contained in the displays very well. Surprisingly, however, only about eight in every ten people correctly understood the information in the follow-main-road display. Subjects’ impressions were fairly positive for all displays except for three displays related to autonomous-mode guidance. This suggests that people had a difficult time using autonomous or “as-the-crow-flies” guidance information. This may have been because of the actual display method or it may simply be that this type of information is difficult for people to process and understand. However, subjects reported that they were close enough to final destinations when ALI-Scout switched over to autonomous mode about two-thirds of the time, and that they could find their destinations without difficulty most of the time.

As might be expected from the responses for the individual displays, subjects reported that the visual displays as a whole were easy to read under a variety of conditions, provided advance warning that was generally sufficient, and were fairly accurate with no differences between age group or genders. Subjects also indicated that the displays helped them find their way about one-half of the time, that their overall impressions were positive, and that the displays caused little distraction while driving again with no gender or age differences.

Subjects were also fairly positive in their assessment of the voice guidance feature of ALI-Scout. Respondents reported that the voice was quite easy to understand, the information and advance warning were sufficient, the voice was not distracting, the sound of the voice was generally liked, and the overall impressions were fairly positive. There were no differences between age groups or genders except for the opinions of the sound of the voice. Those in the middle age group liked the voice less than the subjects in the other two age groups.

As in the NU-Personal study, subjects were asked which vehicle options they would buy if they had $2,500 to spend on options for a new car.

Subjects were willing to pay only a limited amount of money to have the system as an option on a new car, to add it to their present car or, per day, to have it as an option on a rental car. Overall, subjects were willing to pay about $256 to have ALI-Scout on a new car and about $181 to have ALI-Scout added to their present car, with no significant differences between age groups and gender. Men were willing to pay only $3.70 for ALI-Scout as an option on a rental car, whereas women were willing to pay $7.40.

Subjects were asked to indicate the importance of factors in the operation of ALI-Scout-like systems.

The TetraStar Natural Use Study-Leased Vehicles (NU-TetraStar)

The NU-TetraStar study was conducted similarly to the NU-Leased study which used ALI-Scout. Once the NU-Leased study was completed, ALI-Scout units were removed and TetraStar units were installed. To allow for investigation of differential user behaviors and perceptions of the two in-vehicle navigation systems, subjects were drawn from the participants in the NU-Leased study and the same independent variables were investigated (age and gender). Subjects again participated for a one-month period and maintained daily driver logs. They completed a survey during their third week of participation. The driver logs were nearly identical to the ones used in the NU-Leased study. The survey questionnaire covered the same areas as the NU-Leased study, but had questions tailored to the specific features of the TetraStar system. An additional section in the survey investigated preferences between ALI-Scout and TetraStar. The study took place in monthly cycles over the course of six months. Ten subjects per condition participated (a total of 60 subjects).

Results - The following deliverable summarizes some of the results from the NU-Tetrastar study: “Driver Response to the TetraStar Navigation Assistance System by Age and Sex” (Kostyniuk, Eby, Hopp & Christoff, 1997).

The TetraStar system was similar to other commercially available products such as GuideStar or PathMaster. TetraStar provided static route guidance only; that is, it determined the fastest route between some origin and destination without taking in to account current traffic conditions. TetraStar determined the vehicle’s location through an on-board global positioning system (GPS), dead-reckoning calculations, and map matching. The TetraStar unit consisted of a four inch, color liquid crystal display (LCD) and several control buttons. TetraStar provided visual and voice, turn-by-turn navigation assistance to the driver. Visual instructions consisted of an electronic map, in which a highlighted route to the user-specified destination and the vehicle’s current location were shown, and driving-maneuver icons. The system also displayed the vehicle’s heading, the Euclidean distance and direction to the destination, and the current status of the GPS signals.

There were two independent variables in the study: gender (male and female) and age (19-to-29, 30-to-64, and 65-to-80 years of age). Sixty subjects were randomly selected from the 102 subjects who participated in an evaluation of the ALI-Scout ATIS (Kostyniuk et al., 1997). The study took place in seven monthly cycles from November 1996 through April 1997. During each cycle, five to ten subjects were given a project-leased, 1995 Mercury Sable equipped with the TetraStar system to use in their everyday driving for 28 days.

The average age of male study participants was 24.8 (sd=3.9) for the 19-to-29 year old age group, 44.3 (sd=8.3) for the 30-to-64 year old age group, and 70.1 (sd=3.8) for the 65-to-80 year old age group. The average age of female participants was 20.8 (sd=2.6) for the 19-to-29 year old age group, 43.3 (sd=8.3) for the 30-to-64 year old age group, and 72.4 (sd=4.8) for the 65-to-80 year old age group.

The survey was divided into five parts: TetraStar operation and displays, the TetraStar system, use of the TetraStar system, valuation, and comparison of TetraStar and ALI-Scout in-vehicle route guidance systems.

TetraStar operation and displays - Overall, subjects reported frequent use of TetraStar. They judged the destination entry system to be easy-to-learn and easy-to-use. There was a significant main effect of age group for ease of learning [F(2,46)=10.30; p<.0005]. Subjects thought that the destination selection system functioned properly most of the time. The most frequently used route calculation method was the shortest time route, with 91.8 percent of subjects giving it the highest ranking. Subjects judged all of the destination selection methods (street addresses, guidance history, intersections, points of interest, freeway entry/exit) of the system to be quite easy to use, except for the freeway entrance/exit ramp method.

The TetraStar system - Subjects reported that the display was fairly easy to read while driving and very easy to read while the vehicle was still. Subjects also reported that the visual displays helped him or her find the way most of the time.

Asked about the voice guidance feature, subjects reported that the voice instructions were very easy-to-hear, that the amount of information given was sufficient, and that this feature produced very little distraction while driving.

Overall, subjects reported following the given guidance instructions about two-thirds of the time with no differences between genders or among age groups. Those subjects that did not follow turn recommendations all of the time, were asked to indicate the frequency with which several factors were related to their decision no to follow the turn recommendations. There were no significant main effects except for a main effect of age group for the belief that the turn would take them into traffic congestion.

Subjects reported that the system as a whole was fairly easy-to-learn, gave a sufficient amount of information, and provided fairly accurate guidance. Subjects generally agreed that TetraStar helped them find their way, reduced their travel times, functioned properly, and produced only minimal distraction. Overwhelmingly, subjects indicated that they strongly liked the TetraStar system with no significant differences between age groups or sexes.

Use of the TetraStar system - Subjects judged the frequency with which they used TetraStar for commuting trips, work-related (non-commuting) trips, recreational trips, and other personal trips. As can be seen in the following table, TetraStar was used frequently for each type of trip.

Subjects rated the extent to which driving with TetraStar, as opposed to driving without TetraStar, changed their feelings while driving. Drivers reported that use of TetraStar produced a slight increase in feelings of confidence, attentiveness, safety, and relaxation.

Valuation - Subjects were asked to rate various types of route guidance. As can be seen in the following table, the route guidance information provided by TetraStar was the highest rated of all sources.

Similar results were obtained when subjects were asked to indicate which source of route guidance information they would like to use while driving in an unfamiliar area.

When asked to assume that TetraStar was available nationwide and to indicate the usefulness of TetraStar for various types of trips (out-of-town vacation, out-of-town business, commuting, local driving), subjects thought that TetraStar would be most useful for out-of-town vacation and business trips.

Subjects were asked how much they would be willing to pay for TetraStar as an option on a new car, to add TetraStar to their present car, and per day for having TetraStar on a rental car. Overall, subjects indicated that they were willing to pay an average of $503 (sd=$346) for TetraStar as an option on a new car, $357 (sd=$311) to add TetraStar to their present car, and $8.50 (sd=$14.60) per day to have TetraStar on a rental car.

Comparison of TetraStar and ALI-Scout Systems - Overwhelmingly, subjects preferred TetraStar over ALI-Scout reporting that TetraStar provided more accurate route guidance and was easier to use and program. The table below shows the percent of subjects indicating which guidance system gave them a more positive impression by in-vehicle guidance feature.

The following table shows the percent of subjects indicating which guidance system they would prefer to have in three purchase/rental situations.

There were few significant differences in the use and perceptions of the TetraStar system by sex and age group. However, men were more likely than women to want more navigation information, to be more distracted by the system and to want more advanced warning. Drivers in the oldest age group had greater problems in learning and using TetraStar.

Older Drivers - Reduction in attention resources, cognition, and perception makes navigating an automobile more difficult as people age. Since mobility is important to maintain quality of life, older drivers compensate for effects of aging by avoiding difficult, dangerous, and stressful situations and possibly by copiloting, that is, sharing piloting and navigation tasks with a passenger. The “Older Driver and Navigation Assistance Systems” study (Kostyniuk, Streff & Eby, 1997) examined navigation and copiloting of older people, with and without ITS in-vehicle navigation systems, and explored their need for special training for the navigation units. Group interviews of 18 drivers over age 64, who had substantial experience with two ITS in-vehicle navigation systems tested in the FAST-TRAC ITS deployment project were conducted.

Most participants in the study indicated that their own navigational skills did not change much as they aged, but that changes in the roads, traffic, and environment made navigating harder. Copiloting was found to be a common practice among the participants. Copilots served as an “extra set of eyes” and compensated for declines in reaction time and attention deficits. ITS in-vehicle navigational systems were thought to be reasonable copilots when driving alone, but a combination of both human and ITS in-vehicle copilots was preferred. Participants expressed a strong preference for “hands-on” training with the ITS in-vehicle navigation systems and some follow-up training after they had the unit for a few weeks. Also desired was a context-specific help feature that could provide instructions as the in-vehicle navigation system was being operated.The Troika Driving Study

This study provided the basis for assessing differences in perceptual and behavioral out­comes between drivers using ALI-Scout, TetraStar, and a control condition of written in­structions. While the Troika Study compared specific systems, it was also considered a test of conceptually different types of in-vehicle naviga­tion systems.

In recent years, many different navigation systems have been developed and these systems vary greatly in their appearance and features. However, all of these systems can be classified according to the scheme shown in Table 13. As shown in this table, certain systems determine the “best” route between some origin and destination without taking into account traffic conditions that may be encountered during the trip (static route guidance), whereas other sys­tems use information about potential traffic con­ditions at specific times to decide the “best” route (dynamic route guidance). In addition, some systems have information about the vehicle’s lo­cation (through, e.g., GPS or dead-reckoning) and can guide a person as he or she drives by giving instructions during the trip (as-you-drive), whereas other systems show the entire route or set of instructions to the driver in advance and no additional information is given during the trip (all-in-advance). While potential traffic condi­tions could be used in the calculation of a route and given to the driver all-in-advance, this infor­mation is not generally considered to be dynamic since it would not be responsive to the exact time the driver is driving and could not reroute during the trip. Therefore, the dynamic, all-in-advance box of Table 13 has been marked out. Listed in Table 13 are the specific systems to be tested in this study and how they fit into the general scheme of in-vehicle navigation systems.

The purpose of this study was to compare not only how people used the three systems and what they thought about them, but also to compare performance and what people thought about three distinct types of systems when they were used under identical conditions on the road. Identical conditions were achieved by having triplets, or troikas, of people drive similar vehicles at the same time to the same destinations. One person in the troika used ALI-Scout, one used TetraStar, and one used written instructions.

Vehicles and Subjects - A subset of the NU-Leased vehicle fleet was used for this study. One-half of the vehicles were installed with TetraStar and one-half installed with ALI-Scout. The written instructions condition was achieved by disabling the in-vehicle navigation devices. Approximately 360 subjects were recruited from the general population of the Oakland County study area and surrounding areas through advertisements in local and non-local newspapers. An experimental session took approximately 2.5 hours and the subjects were paid for their participation.

Research Design and Methods - There were four independent variables in the study: In-Vehicle Navigation System (ALI-Scout, TetraStar, Written Instructions), Driver Familiarity with Area (Familiar, Unfamiliar), Traffic Conditions (Peak, Non-peak), and Trip Number (First Trip, Second Trip). Driver familiarity was determined by self-report on a questionnaire completed at recruitment. Traffic conditions were varied by running the experiment during different times of day. Peak traffic conditions were defined as 7:00 am to 9:00 am and 4:00 pm to 6:00 pm. Non-peak traffic conditions were defined as 9:00 am to 11:30 am and 1:00 pm to 3:00 pm. So that any learning effects in the study could be assessed, each participant drove two trips. The first trip was from a specific origin to a specific destination (O-D) pair. The second trip was to drive the O-D pair in the opposite direction. The combination of these variables yielded 24 unique conditions in the study with the first three conditions run between subjects and the other 21 run within subjects. Power analysis suggested that the between-subjects variables required 30 subjects each, or 360 total subjects in the experiment.

The study was conducted by a team of four research assistants housed in a field office located in the study area. Subjects were run in sets of three (i.e., troikas). If three subjects were not available, the experimental session was not conducted. In order to increase the chances of getting three subjects at the correct time, for every three subjects scheduled, an alternate subject was also scheduled. All subjects were contacted the day before participation to remind them of the study time. If all four subjects arrived on time, the alternate was paid and rescheduled for another day as a primary subject. Upon arriving at the field office, the three subjects were placed in a conference room and assigned randomly to a navigation system condition. Subjects were given a brief lecture on the administrative and procedural aspects of the study. All subjects were told that their task was to drive between a given origin and destination “as quickly and safely as possible.” At the first destination, they were met by an experimenter who gave them a new O-D. This second O-D pair was to drive back to where they started. No subject was told about the second destination until the first destination was completed. In case of emergency or if a subject got lost, all vehicles were equipped with cellular phones.

After the administrative overview, the three subjects were separated and given training sessions that were specific to the system they were assigned. For the ALI-Scout and TetraStar subjects, function and presentation of navigation assistance information was conveyed through the use of a model car, a schematic road network, and a series of printed graphics. The subject in the written instructions condition simply waited in a reception area. Once the training was completed, subjects were placed in the appropriate vehicle, given the destination to which to drive and allowed to be­gin. To prevent subjects from following each other to the destinations, a several-minute gap between subjects was maintained.

A set of O-D pairs was used in the study, with a single pair used each day that was randomly selected from the set. Each O-D pair was matched on several variables including length, travel time, road classification, land use, type of traffic, direction of travel, proximity to beacons, and similarity of potential route sets. The complete trip, including heading, location, and speed was recorded during both trips using GPS vehicle tracking technology. At the completion of the second trip, the subject filled out a survey questionnaire.

There were numerous dependent variables in this study. The questionnaire investigated use of the system, level of satisfaction and distraction with the navigation assistance received, whether they felt lost, various impressions of the system, perceived safety, general routing preferences, opinions of navigation assistance in general, and willingness to pay. From the vehicle-tracking data, several dependent measures were extracted including problems finding the initial route, problems finding the destination, getting lost, the number of turns, the number of wrong turns, the time spent at zero velocity, trip distance, and trip duration.

Results - The following deliverable summarizes some of the results from the Troika Driving Study: “FAST-TRAC Troika Study” (Eby, Kostyniuk, Christoff, Hopp & Streff, 1997).

Of the 360 participants, 51.9 percent were male.

Study participants were asked about their route guidance preferences for vacation and out-of-town business trips. In both categories, the fastest route was the most preferred (80.7% for vacation trips and 86.6% for business trips.) For vacation trips, 40.2% of subjects indicated preference for the most scenic route, whereas only 7.9% indicated such preference for business trips. Preference for routes that avoid/utilize certain areas was indicated by 20.2% of respondents for vacation trips and 15.2% for business trips.

More than one-half of the study participants (57.2%) indicated that when they drove in urban areas they generally listened to traffic reports, and nearly all participants (98.1%) indicated that they were willing to divert from a driving route that they normally used in order to avoid traffic congestion. Roughly one-half of the participants reported that they were not confident in their ability to navigate in unfamiliar areas.

Overall, the study found that ALI-Scout users got lost more frequently, had more difficulty finding routes and destinations, made more wrong turns, and drove longer routes than did users of the other two systems.

When trips in which the driver got lost were removed and drivers had experience with the systems, the study found that users of both ALI-Scout and TetraStar had significantly shorter trip duration (5% overall) than users of written instructions. The following table shows trip time savings in trip two for users of the electronic navigation systems (ALI-Scout and TetraStar) over Written Instructions for all study variables. As can be seen in this table, the greatest trip time saving over Written Instructions occurred during the peak traffic conditions.

Analysis of trip duration by traffic conditions and of time spent at zero velocity showed no benefit of ALI-Scout’s dynamic route guidance over the static guidance of TetraStar.

Averaging across driver familiarity and traffic conditions, we find that a high level of confidence was reported by 66.4% of ALI-Scout users, while 86.7% of TetraStar and 91.4% of Written Instructions users reported high levels of confidence in the navigation accuracy of the navigation system.

There were significant differences in level of helpfulness by navigation system and driver familiarity. The main effect of navigation system resulted from the fact that users found the ALI-Scout and TetraStar systems to be less helpful in finding study destinations than the Written Instructions. Further, users found TetraStar to be more helpful than ALI-Scout. Averaging across familiarity and traffic conditions, users reported high levels of helpfulness 73.1% of the time for ALI-Scout, 81.5% of the time for TetraStar, and 86.3% of the time for Written Instructions.

In rating sources of route-guidance information for assistance in reaching destinations, navigation assistance utilizing verbal directions was rated least favorable, whereas electronic in-vehicle route guidance was rated the most favorable. There were no significant differences between users of the three navigation systems used in this study except for responses to electronic in-vehicle route guidance. For this source of information, those who had experience with electronic in-vehicle navigation assistance rated it more favorably than those who had Written Instructions.

Somewhat similar results were obtained in rating the likelihood of using various sources of route-guidance information in an unfamiliar area. Navigation assistance utilizing verbal directions was rated least likely to be used, whereas electronic in-vehicle route guidance was rated the most likely. There were significant differences between users of the three navigation systems used in this study in their responses to verbal directions from a passenger and electronic in-vehicle route guidance. In the case of verbal directions from a passenger, the significant difference resulted from the fact that those using Written Instructions during the study rated this type of information higher than those using the electronic navigation systems. The latter main effect was due to the fact that those who had experience with electronic in-vehicle navigation assistance rated it more favorably than those who had had Written Instructions.

ALI-Scout and TetraStar users were asked to indicate how much they would be willing to pay for the system as an option on a new car. The mean price ALI-Scout users were willing to pay was $342 while TetraStar users were willing to pay an average of $526.

Safety - Within all of the Natural Use studies, assessment of safety was an important concern. Several methods were employed to investigate and measure system safety. First, all users were queried on their perceptions of system safety and their safety-related experiences while driving the systems. Second, if and when any study vehicles were involved in a crash, the study team examined all available data (e.g., police reports and subject interviews) that might have indicated the extent to which the system contributed to the crash. Third, Natural Use subjects were asked to report perceived close calls on their daily driver logs.

Choice Modeling Study

Purpose - The purpose of the choice modeling study was to elicit quantitative data describing user preferences regarding three classes of traveler services: traffic reports, route advice, and (advanced) emergency roadside assistance. More specifically, the study sought to determine which of these traveler services consumers deemed important, how much consumers were willing to pay for the identified services, and how likely consumers were to actually purchase these services. The primary result of the study was an equation describing the probability that consumers would choose a market package composed of various implementations of the traffic reports, route advice, and emergency roadside assistance traveler services and price. Details regarding the implementation and results are found in Phase III Deliverable #16B: “Choice Modeling Final Report.”

Methodology - Choice modeling is a stated-preference technique that has been used to evaluate aggregate consumer preference or utility for products and services. The method is based on the assumption that a product or service can be described in terms of a “bundle” of attributes, which can take one, two or more values or levels. Each attribute, and its associated levels, is selected to characterize some aspect of the product or service under study (e.g., the attribute of purchase price at levels of $250, $750, $1250, and $1750). The attribute list should include features that are 1) commonly found to be, or thought to be, important to consumers and 2) measurable and appropriate for the purposes of the study (e.g., descriptive of products or services that exist or could soon exist in the marketplace). Each bundle of attribute levels can be considered as a potential alternative implementation of the product or service.

In the most basic form of choice modeling, data describing consumer preferences are obtained by presenting people with a series of carefully selected product/service bundles and asking them to respond to each, i.e., to state their preference by saying whether or not they would buy the bundle if it were available to them. In this case, only one bundle is considered at a time and the choice is between that implementation of the product or service and the status quo. Note that subjects responding to a choice model study are given the realistic task of evaluating a product or service in its entirety; subjects are not asked to evaluate attributes individually, as in the decision analytic approach.

If the researcher develops the series of bundles according to proper experimental design, and if study participants can be assumed to choose bundles rationally, i.e., will choose a bundle only if it provides more benefit than the status quo, then the degree to which each attribute-level influences aggregate respondent preference can be determined from the choice data. That is, aggregate coefficients for each attribute-level can be calculated from the choice data. Positive (negative) coefficients indicate positive (negative) utility. The greater (lesser) the magnitude of the coefficient, the greater (lesser) the impact the attribute-level should have on participants’ decision making, i.e., a large (small) coefficient for an attribute-level implies that the consumer is (is not) making significant use of the attribute-level to discriminate between product/service bundles.

The attribute-level coefficients form a “choice model” that describes the likelihood that consumers will purchase variously priced market packages composed of combinations of the attribute-levels. The choice model can be used either to reproduce the original judgments or to predict choice among new combinations of attribute-levels, i.e., to predict the probability that people will purchase product or service implementations that were not included in the study, but which can be described in terms of the attribute-levels tested. In this manner, design issues, such as which attributes and what levels to include in a product or service, may be addressed and market share may be estimated, prior to spending a great effort developing the product or service.

The model in the most basic case, where a subject answers “yes” or “no” when asked if they will purchase each product/service bundle as it is presented, is the binary logit^[2]:

                                               exp(lb'Xin)

       Probability(a|a,0) = ------------------------

                                         [1 + exp(lb'Xin)]

where:       a is the bundle being considered (one of I bundles) and “0” is the status quo, l is a scale constant, b' is a vector of utility coefficients, and Xin is a matrix of attributes for bundle i and individual n.

A primary advantage of the stated-preference choice model method, in contrast to a revealed preference approach, is that it enables evaluation of products or services that are not yet available in the marketplace. That is, the technique can be utilized to explore the effectiveness of, and evaluate traveler response to, (various implementations of) emerging products or services prior to implementation. Furthermore, decision makers can use the resulting choice model, combined with production and marketing cost data, to guide both the development and marketing of products or services.

Implementation - The basic approach in selecting attributes for this study was to consider the traveler service that could answer each of the most basic questions that a traveler might ask when considering a trip. For example,

Question:                                                                   Traveler Service:

What destination will best suit my needs?         Electronic Directory: Yellow Pages/Tour Book

What is the best way to reach the destination?   Route/Mode Advice

What are current traffic conditions?                     Traffic Information

How can I get help quickly while en route?          Emergency Assistance

What does all of this cost?                                       Price (not a Traveler Service)

Thus, in terms of this study, an attribute is a traveler service or price and the alternative bundles are market packages of traveler services. Five attributes, those deemed to be those most essential to definition of a traveler service package, were selected for this study: route advice, traffic information, emergency roadside assistance, purchase price, and monthly fee.

Subjects - Approximately 180 subjects were recruited to fill 6 experimental “cells” (30 per cell):

• males age 18-29
• females age 18-29
• males age 30-64
• females age 30-64
• males age 65 and over
• females age 65 and over

The age groups were selected for the study to be compatible with other portions of the evaluation being carried out at the University. The number of subjects per cell is empirically based and often used “rule of thumb” for studies of this type.

The pool of potential subjects was limited to drivers with at least a minimum of previous exposure to a route guidance system. This pool of subjects was selected to address a weakness in the choice model method: consumer response to a new category of product or service is difficult to gauge because consumers, by definition, have had no prior experience with the type of product or service and so might not readily understand its use and potential benefits. Thus, careful description of the product or service, and, if at all possible, some interaction with a prototype, is needed prior to asking subjects to respond to the choice questionnaire. This study primarily deals with route advice, traffic information, and emergency roadside assistance. The latter two categories of traveler service are extensions of types of services that most travelers are already familiar with. As a result, these services are relatively easily understood. However, route advice, in the form of route guidance systems, is a traveler service that is relatively new to travelers. Thus the benefits of a route advice service are potentially more difficult for travelers to envision. Subjects were recruited from two sources:

Subject Pool #1: The pool of subjects recruited as part of either the ALI-Scout Natural Use-Leased Vehicle Study or the ALI-Scout Natural Use-Personal Vehicle Study, both of which involved driving a vehicle equipped with a route guidance device. The exact means by which subjects were recruited in these studies can be found in the reports associated with the studies. These potential subjects already had experience with a route guidance system and only responded to the questionnaire.

Subject Pool #2: The residents of Senior Centers in Ann Arbor and the students in University of Michigan classes. Subjects were recruited from these groups to fill out the older and younger age groups, i.e., to take up where Subject Pool #1 left off. These subjects were exposed to route guidance technology prior to responding to the questionnaire by either having them ride as passengers in a University vehicle equipped with a route guidance system (and driven by a researcher) or by having them view a video describing route guidance systems. These subjects were recruited by contacting Senior Center staff and University faculty.

Data Collection - After testing, the choice questionnaires were mailed via first class U.S. mail to each of the potential subjects in Subject Pool #1. To boost the response rate, potential respondents were informed that those returning the survey would receive $5 in appreciation. In conformity with University regulations, the subject had to provide their name, address, and signature before the $5 could be disbursed. As this mailing did not produce sufficient numbers of respondents, several in-person efforts were made to recruit respondents from Subject Pool #2. The resulting choice data was analyzed via logistic regression to estimate coefficients for a model of consumer choice among the tested traveler services. The data and results are found in Phase III Deliverable #16B: “Choice Modeling Final Report.”

Results - The following deliverable summarizes some of the results from the Choice Modeling Study: “Market Assessment for Traveler Services: A Choice Modeling Study” (Reed, Underwood & Demski, 1997).

Four “choice models” were developed to describe the likelihood that participants would purchase variously priced market packages. Data from 282 subjects revealed the following:

The most appealing of the 4 models identified the following intuitively reasonable effects:

The probability of choosing traveler services is inversely and strongly related to price.

The effect of having a monthly service fee is large and negative, even for a fee of $5 per month.

Respondents state a strong desire for “Emergency Roadside Assistance” and “Dynamic Turn-by-turn Route Advice”, as well as for the traveler service combination of “Customized traffic reports on demand” and “Dynamic Turn-by-turn Route Advice”.

All else being equal, members of the 18 to 29 year old subject group are somewhat more likely to find Emergency Assistance appealing than members of the 30 to 64 year old subject group, while there is no difference between the 30 to 64 year old group and the 65 and over group in this regard.

All else being equal, female subjects are moderately more likely than males to choose a market package containing an Emergency Assistance feature.

All else being equal, females in the 65 and over age group and the 18 to 29 age group are moderately more likely and much more likely, respectively, to choose a market package than are either males or women in the 30 to 64 year old group.

Human Factors

If navigation systems are to be desired by the public, they must be safe and easy to use. In terms of safety, design guidelines (Green, Levison, Paelke, and Serafin, 1993) and certification procedures (Green, 1993a, b) are currently being devel­oped for the U.S. Department of Transportation, materials which could be the pre­cursors to regulations. However, more important is the public response to the driver interface. If products are not safe or easy to use, people will not buy them, or if they do, will not use them. There are many examples of driver interface products (electronic instruments in general, voice messages, etc.) whose initially predicted market potential was not realized because of usability problems. The purpose of this activity was to assess the ease of use and safety of the ALI-Scout route guidance system.

The following issues were addressed:

• Is ALI-Scout safe and easy to use for drivers of all ages and levels of experience?
• How easy and safe is it relative to other navigation systems and information sources?
• What kinds of problems do drivers of all ages encounter (and how can they be corrected)?
• When can simulated interfaces be used in place of real interfaces in safety and usability tests?

The research described here provides a careful, focused examination of how navigation systems might be used, both in the laboratory and in real, on-road settings. On-road experiments of the type conducted involve significant amounts of data—6 megabytes per subject in the research conducted here. Because of the detail involved, most of these experiments utilized a limited number of drivers who had been carefully selected on the dimensions that are most likely to affect performance: experience with the interface, driver age, and sex.

The approach was guided by three basic ideas. First, decisions about the safety of the driver interface are not absolute. Drivers can get route guidance information from maps and from naviga­tion systems other than ALI-Scout. Safety evalua­tions need to consider those alternatives. How­ever, due to funding constraints, the comparison of ALI-Scout with other interfaces was extremely limited.

Second, problems with the ALI-Scout interface were expected to be connected with two specific tasks - destination entry/retrieval and following a given route - both of which were examined in detail. These particular problems occur in systems other than ALI-Scout and offering solutions to them will have broad benefits.

Third, the instrumentation required to examine on-road use of ALI-Scout by drivers is quite complex. Pilot tests and small-scale studies were conducted first to iron out the problems. Also, software to summarize the vast amount of information collected was developed. Without such preparations, a full-scale on-road test would have been difficult.

The ALI-Scout Human Factors effort began with an on-road pilot test to collect preliminary data on the use of the ALI-Scout driver interface. Following were two main experiments, a labora­tory study of destination entry and an on-road test of route following.

Assumptions

• For the two main experiments, key press data was recorded manually from videotapes.
• The instrumented test vehicle would have sufficient power for the ALI-Scout unit and other equipment to be added.
• Update delays in SuperCard simulations of the ALI-Scout interface would not degrade the usability of the simulated interface since CPU speeds have increased to the point where response delays are a minor matter.
• The test vehicle was stored at a secure facility in the test area to avoid commuting in it on a daily basis during the test period.
• The level of training of subjects with the interface prior to testing was assumed to be consistent with that to be provided to intended customers. Training included showing subjects a training video, having them answer a few questions, and making sure they understood use of the device.

Test Subjects

Except for the pilot tests, all experiments involved an equal number of men and women in three age groups (under 30, 40-50, over 65). For the pilot test, subjects were recruited from Ann Arbor to facilitate coordination. For the on-road study, drivers were recruited mostly from Oakland County to minimize “dead” time driving to and from the test sections. Across the three sample groups, the extremes of the population are represented as well as the most likely group to purchase navigation equipment. Participants were paid a base rate of $10/hour for daytime driving and $15/hour for nighttime driving. In multiple session experiments, subjects were all paid the same: $15 after their first 1 hour session and $15 for their second 1.5 hour session plus a $20 bonus for completion. Subjects were recruited from previous lists of subjects and via newspaper ads. No subject participated in more than one experiment and all were unfamiliar with navigation systems.

Instrumented Car

The ALI-Scout driver interface was installed in a 1991 Honda Accord station wagon, the car used for all on-road tests. This instrumented car has sensors and recording hardware for lateral position (to the nearest 0.1 ft. at 10 Hz), speed (to the nearest 0.5 mph at 10 Hz), throttle position (to the nearest 0.5% at 30 Hz), steering wheel angle (to the nearest 0.5 degrees at 30 Hz), and brake on/off. Sensors for recording headway and other equipment required for safety evaluations were added.

There were also cameras to record the forward scene and the driver's head motions. Equipment to determine driver angle of gaze (to the nearest degree at 30 Hz) was added. Experimenters used a manual system (using off-head cameras) to record where drivers looked.

On-Road Pilot Test

This experiment determined what was required to carry out on-the-road studies of the ALI-Scout system. This test included installing the interface in the car, collecting some data with experimenters and real drivers, and analyzing it. The main concerns were getting the ALI-Scout interface installed and working reliably, the selection of routes, the amount of training subjects would require, the reliability of the data collection hardware, software, and sensors, identifying problems that are likely in analyzing the large data set, and the consistency of the safety ratings.

Method - An ALI-Scout unit was installed in the instrumented car and interfaced to the data recording equipment. In addition, a headway sensor, additional cameras and video equipment, and power-saving displays for the data logging equipment were installed. To verify proper operation of the equipment, the test route was driven by the experimenters, and then by non-project personnel.

Next, prior to the full-scale test, approximately four to six people unfamiliar with the ALI-Scout interface were given the preliminary training program. They drove the instrumented car to the site in Oakland County, had destinations keyed in, and followed the guidance. Navigation data (trip times, turns, etc.) and limited driver performance data (vehicle speed, lane position, etc.) was collected. The purpose of this effort was to determine if the training materials were adequate.

In the pilot experiment, drivers were driven to Oakland County. To provide a variety of road conditions, each subject drove to a series of four destinations. The routes selected provided a wide variety of road types (expressways, major city streets, residential streets), turn types, and speeds and exercised both guidance modes of the ALI-Scout (autonomous and guided). Routes were selected so that changes in traffic levels would be unlikely to alter the guidance provided by beacons. Each trip segment took about 15 minutes to drive. Segments were kept short, initially because of limits as to how long subjects could wear eye fixation recording headgear. At the end of the route, drivers were interviewed concerning the safety and ease-of-use of the driver interface.

Testing was iterative (not structure), with testing continuing with each sample until problems were no longer found.

Analysis - Since the purpose of this evaluation was to determine if the hardware and software functioned properly, if the instructions were understandable, and the protocol could be completed in the desired time, minimal analysis (but no formal reporting) of the driving data was performed. However, lane and speed variance were examined (to verify those measures were properly recorded) for a few drivers and selected route segments.

Destination Entry Laboratory Study

Drivers had to complete two tasks successfully to reach a destination: 1) select the destination and 2) follow the guidance messages. Only the second task has been considered at length in the literature though both tasks present problems to drivers.

Accordingly the focus of the laboratory work was on destination designation. The only study in the literature to examine alternative destination entry methods is Paelke (1992), though she did not examine the ALI-Scout interface. (See also Coleman, Loring, and Wiklund, 1991; and Dingus, Hulse, Krage, Szczublewski and Berry, 1991 for related information.).

Measures examined in this experiment were consistent with those in the literature--the number and nature of errors, the mean and range of times to enter destinations, and driver preferences for alternatives. Using these measures, the differences in user performance between the real device and a simulated interface (created using SuperCard) in retrieving and entering destination were examined. These results suggest where laboratory tests of rapidly prototyped simulations can be substituted for operational tests of real interfaces.

Method - The ALI-Scout interface was installed in a mock-up of a Chrysler G-body in a Human Factors Laboratory. Drivers were asked to key-in a series of destination requests into the interface. Inter-keystroke intervals and keys depressed were recorded, and each test session was videotaped. Two illumination conditions were simulated, dusk and night. Daytime conditions cannot be adequately simulated in the laboratory.

Also installed in the mock-up (but not concurrently with the real interface) was a SuperCard simulation. The same performance data was also collected from it. For input, a touch screen was used with supplemental auditory feedback (since feedback from switch movement was not available).

Each driver completed 15 destination-related operations for each task (entry and retrieval) for three test conditions (dusk-real interface, night-real interface, dusk-simulated interface). Pilot testing revealed that there were no differences in user performance for the simulated interface due to lighting conditions. Limited data on using the ALI-Scout manual and maps to look up destinations was also collected. The sequence of tasks and conditions was counterbalanced across drivers. Recovery from errors was given attention.

There were 12 drivers within each age group (36 total). Hand anthropometry was recorded to permit post hoc examination of the “fat finger” problem.

Analysis - The keying times were analyzed using ANOVA with the independent variables being age, sex, drivers nested in age and sex, real vs. simulated interface, dusk vs. night, the entry task, and other factors related to the address. Error differences were also examined, though there was not enough data to justify the same detailed analysis as was completed for keying times.

In addition, GOMS model predictions were compared with the actual keying times. GOMS (Card, Moran, and Newell, 1983) is a method for generating predictions of the time required for people to complete routine cognitive activities, such as entering a destination. Using GOMS predictions, many of the safety and usability engineering evaluations could be based on calculations, not experimentation.

On-Road Test

This was a comprehensive test of the safety and usability of the system, emphasizing normal driving situations. Critical factors included daytime vs. nighttime driving, congestion levels, and driver age. Control conditions (existing and optimal navigation methods) were included to provide baseline data.

Method - Prior to collecting data, the instrumented car was modified to collect more safety-related data (e.g., headway), to coordinate the data streams, and to improve the data quality (by adding a quad splitter, and a second lane tracker). Modifications of the lane tracker, so that it would function at night, were required. Because the current eye tracker restricted the driver's field of view and risk wais an issue, the original eye tracker in the vehicle was replaced. Eye fixations were recorded using a line of sight, off-head video recording system.

Four routes developed in the on-road pilot test were driven by each subject using the instrumented car. Subjects were verbally guided to a trip origin in the test section. Upon arrival at each destination, the experimenter entered the next destination into the ALI-Scout unit. The order of routes was fixed to minimize excess on-road time. Each subject drove at only one time of day - there were an equal number of subjects for each time. Congestion was not regularly present a night, so the fourth combination needed to separate congestion from lighting effects (congestion at night) could not be examined. Congestion was determined from historic data plus prior knowledge of congestion-inducing events from the traffic control center.

The sample tested included 54 subjects (plus three additional drivers who did not complete the experiment). In each age-sex category there were 9 drivers, with 3 drivers being tested at each of the three time slots. Each subject came for two ses­sions. In session one, the subject used the route guidance system to drive the test route. In session two (which occurred one week after the first at the same time of day), subjects drove the same route a second time (guided by ALI-Scout) to examine the role of familiarity, and a third time (guided by the experimenter) to provide baseline data.

Analysis - Dependent measures of interest fell into three categories: navigation related (trip times, number of wrong turns, number of missed turns), measures of driving behavior (lane variance, mean speed, speed variance, number and duration of eye fixations to the display, mean headway and headway variance), and ratings of safety and ease of use. For each measure, an ANOVA was computed with the main effects being drivers (sex, age, subjects nested within age and sex), destination, section within destination, and time of day (day versus night)/congestion level (none versus high). rFor the subjective ratings, section and destination differences cannot be examined because only one rating per trip was obtained.

Supplemental On-Road Testing

There were two problems with the original on-road test. First, the color cameras for recording the driver's face and road scene lacked the low light sensitivity necessary for nighttime use. Hence, driver eye fixations at night could not be analyzed. Second, changes with the experimental plan meant that insufficient comparison data would be available for assessing the relative safety and usability of the ALI-Scout interface. For that reason, pilot data was collected on the relative usability of the PathMaster/TetraStar interface.

Method - The test route used was identical to that in the previous test except for a minor shift of destination one. As before, subjects drove to four successive destinations and their driving performance was monitored. There were two runs. On the first run, ALI-Scout guidance was used. On the second run (immediately after the first), experimenter verbal guidance (baseline) was utilized as the preliminary analysis had shown there was little difference in driver performance between the first two ALI-Scout runs of the initial on-road experiment.

In the PathMaster pilot effort, four subjects used a PathMaster navigation system that had been installed in the instrumented car. Due to problems with the PathMaster, usable data was obtained from only three drivers.

Analysis - The analysis was similar to that for the initial on-road experiment with dependent measures including navigation-related measures, measures of driving behavior, and ratings of safety and ease of use. For each measure an ANOVA was computed with the main effects being similar to those in the previous on-road experiment. The primary difference was that the comparison was of PathMaster subjects in the supplemental test with ALI-Scout subjects in the original experiment tested at the same time of day. Also, there were only two runs, not three as in the original experiment.

Human Factors Results

The “Human Factors and New Driver Interfaces: Lessons Learned from a Major Research Project” paper (Green, 1998) provides a project overview and summarizes the general lessons learned from the ALI-Scout Human Factors effort. Emphasized are lessons that transcend experiments and have programmatic impact for project managers and sponsors. The goals of the project were to develop 1) human factors guidelines, 2) methods for examining the safety and ease-of-use of driver interfaces, and 3) a driver performance model. Five systems (navigation, traffic information, road hazard warning, vehicle monitoring, and car phones) were examined in 20 experiments. Experiments included surveys at driver licensing offices, response time tasks, driving simulator studies, part-task simulations, and on-the-road evaluations.

A major group of lessons concerned how realistic, inexpensive, and rapidly produced interface prototypes can be, and how to achieve a high level of fidelity. Of the methods explored, there were lessons concerning focus groups, response time tasks and usefulness of the subjects-in-tandem method. The research provided several lessons concerning the inadequacies of low-fidelity driving simulators. Lessons from the on-the-road evaluations related to test vehicle shake down, and determining workload and data reduction. During these evaluations, design guidelines emerged from interface design decisions, not from a summary of the literature. General guidelines and principles, especially consistency, proved very useful in design.

The human factors evaluation of ALI-Scout included data On-The-Road usability experiment and a laboratory test of destination entry and retrieval. The initial usability test compared ALI-Scout baseline data with data collected for the path Master navigation system. This involved collecting ALI-Scout baseline data and an experiment to collect eye fixation data at night for the PathMaster system. The laboratory test of destination entry and retrieval was a static experiment in a driving simulator. Part of this test involved developing a Keystroke-Level model.

The human factors evaluation of ALI-Scout included data On-The-Road usability experiment and a laboratory test of destination entry and retrieval. The initial usability test compared ALI-Scout baseline data with data collected for the path Master navigation system. This involved collecting ALI-Scout baseline data and an experiment to collect eye fixation data at night for the PathMaster system. The laboratory test of destination entry and retrieval was a static experiment in a driving simulator. Part of this test involved developing a Keystroke-Level model.

The On-The-Road experiment involved fifty-four drivers who took three trips to four destinations in an instrumented test vehicle. In two of the trips the drivers were guided by ALI-Scout. In the third trip the drivers were guided by voice. The instrumented test vehicle collected data, on Speed, lateral position, and other measures. While there were four critical incidents, there were no crashes or near misses. Eight percent of the trips had turn errors and thirteen percent of the trips showed uncertainties. Most of the errors and uncertainties were in autonomous mode. There were many significant differences of longitudinal control measures.

The second On-The-Road experiment collected eye fixation data at night for ten ALI-Scout drivers. The driving data for the PathMaster users had no crashes, no near misses, and no critical incidents. The measures of lateral control where more sensitive that longitudinal control. The path Master interface was rated as safer than the ALI-Scout interface.

Thirty-six drivers participated in the ALI-Scout destination entry and retrieval experiment. Their tasks for reviewing an instructional video, practicing retrieval and entry, using the mail to look up destinations, and participating in the main experiment. The main experiment had thirty trials that combined with the light conditions and the two entry and retrieval tasks. The light conditions were dusk, night, and a simulation of dusk. There were five trials in each of the six conditions.

Destination entry and retrieval the performed on a manual keyboard with alphabetical keys. The keyboard was located at the bottom of the ALI-SCOUT driver interface. In order to enter a destination the driver opened a hinged door and exposed the keyboard just below the ALI-Scout display. While in some cases it took nearly a minute or two for the drivers to retrieve the destinations, in most cases it took approximately ten seconds. The median retrieval time was 6.23 seconds and the mean retrieval time was 10.48 seconds. However, destination entry was much slower with a median of entry time of 51.48 seconds and a mean entry time of 64.68 seconds.

In general, the male subjects entered and retrieved destinations faster than the female subject. The men also made fewer destination entry errors. Furthermore, all the subjects were more competent under this simulated night conditions than they were in the real night and dusk conditions. They made nearly twice as many destination retrievals under the simulation condition. They made approximately 30% more destination entries under the simulated condition than they did under the dusk condition.

In conclusion, the ALI-Scout user interface was relatively difficult for the drivers to use. The keying of destination entries took approximately 60 seconds. Destination look up took another 30 to 60 seconds. Destination retrieval took under 10 seconds. Destination entry and retrieval took longer for older drivers and women. It also took slowly longer times for night than dusk. Retrieval under the simulated night conditions was close to the night condition. Suggest that the keyboard to have had serious design problems. Furthermore, it probably was not a good idea to use latitude and longitude for destination identification.

Technical Performance

The purpose of the Technical Performance portion of the evaluation was to assess the actual performance of all portions of the ALI-Scout system during the period in which the Natural Use studies were conducted.

This ALI-Scout evaluation included the performance of hardware, software, databases, and operations. The evaluation covered the total system aspects, as well as each of the three subsystems: 1) the central computer; 2) the vehicle-roadway communications (VRC) via infrared beacon; and 3) the in-vehicle equipment which included dead-reckoning.

The FHWA-recommended operational field test goals emphasized 1) learning all aspects of the user's reaction to the system, and 2) measuring any technical performance items that impacted the evaluation of future deployments of the system. Considering these two goals, the rationale or justification for the Technical Performance items to be described below was based on the following:

• Assuring that the system was operating to specifications during the period that behavioral analysis was being conducted.
• Obtaining an early indication of the reliability/maintainability of the system, keeping in mind that the equipment was a beta-test version while deployed equipment was production quality.
• Measuring system-unique technical performance parameters, which affected evaluation of the future extendibility and compatibility with other ITS services. Such system-specific parameters were needed to analyze and predict the future potential of the system, in terms of advantages and disadvantages, relative to competing approaches.

Evaluation Questions

The technical performance evaluation questions were described in the categories of 1) total system aspects, 2) subsystem aspects and 3) growth and compatibility aspects. In each case the approach to implementing that evaluation question is given.

Total System Performance Evaluation

The evaluation question for the technical performance of the total system emerged after reviewing the entire range of expected benefits of a route guidance system.

• “What fraction of trips in the unit remained in guided mode?”
• “What was the total “down time” experienced for the total system, due to down time of one or more of the subsystems (where down time is defined as a continuing--until fixed--malfunction, as opposed to a sporadic single event error)?”
• “What are the specific times and places where the unit went into autonomous mode between the first and last beacon, for the 10 project vehicles plus the yoked tests?”

Computer Subsystem Performance Evaluation

• “What is the ‘message cycle time’ as a function of time?”
• “What was the Mean Time Between Failure of the computer and all of its peripherals (modems, phone lines, etc.)?”

Vehicle-Roadway Communications Subsystem Performance Evaluation

• “What was the total count (or fraction) of beacon exposures during which the in-vehicle unit failed to receive the downloaded 10 Kbytes of ‘Data Transmission blocks’ needed to compute the route to the next beacon during all yoked and 10 project vehicle (plus test vehicle) use?”
• “What is the error rate for the 10 Kbyte downloading of Data Transmission blocks, and how does this rate vary with physical conditions of speed, rain, snow, dirt, large trucks nearby, etc.?”
• “What was the Mean Time Between Failure of the vehicle-roadside communication equipment (infrared beacon and beacon site controller)?”

In-Vehicle Equipment Subsystem Performance Evaluation

• “In how many cases did the in-vehicle equipment fail to deliver the correct instruction provided by the beacon during the yoked and user-tests with the 10 project-owned vehicles?”
• “What was the mean time between failure for the in-vehicle unit?”

Data Collection Methods

Four sources of data were used to evaluate technical performance: (1) Siemens diagnostic records and maintenance files, (2) data storage in participant vehicles using memory cards, (3) a portable PC along with a Siemens Vehicle Unit Controller in participant vehicles, and (4) a University of Michigan test vehicle. The Siemens diagnostic records and maintenance files recorded data on message cycle times and error rates for uplink probe reporting. Seventeen project vehi­cles were equipped with Vehicle Unit Controllers and memory cards to record data on general route guidance performance, autonomous mode, waiting on blocked links, and in-vehicle unit reception of data transmission blocks. The test vehicle was used for near-continuous driving through the network to assess reliability in beacon-to-vehicle transmissions under various conditions (speed, rain, snow, dirt, large trucks nearby, etc.). Driver calls were monitored to identify obvious equipment or system malfunctions.

Results

The “Technical Performance of ALI-Scout” report (Ristenbatt & Shahine, 1997) describes the technical performance of the ALI-Scout Advanced Traveler Information System (ATIS) based on data collected during most of 1996. The bulk of the data was collected using In-Vehicle Storage units, with additional data taken from ALI-Scout system logs. The technical performance evaluation included the assessment of hardware, software, databases, and operations.

The study found the technical performance of ALI-Scout to be barely adequate, and estimated that it had a slight negative impact on any measured evaluation of the system. About 7% of the beacons were inoperative at any one time, the downlink success rate was in the high nineties, and the uplink success rate was 35%. The beacon-based link reporting used by the system introduced an inherent delay in responding to non-recurrent congestion. Thus, the greatest limitation of ALI-Scout, confirmed by commuter drivers, was the limited ability to provide what the drivers really wanted – a prompt response to the actual current conditions. The study concludes that, under the conditions of low number of probe vehicles, no operator at the console, and trying to serve drivers already expert in the road network, the ALI-Scout system did not deliver benefits commensurate with its cost. The system was turned off on 26 February 1997 after the test had been completed.

Traffic Modeling

The purpose of this task was to assess the impact of ALI-Scout and SCATS on individual users and the system as a whole. The modeling effort addressed questions about ALI-Scout and SCATS deployment that cannot be addressed directly through field observations. The modeling effort was a supplement to the field evaluation effort. The types of impacts investigated included travel times, delays, queue time, stops, pollution, and incident situations, for drivers that use ALI-Scout and for the system as a whole at increasing levels of market penetration.

Modeling was essential in the FAST-TRAC evaluation because it was too expensive to equip the large number of vehicles required to assess the impacts at higher levels of market penetration, where many of the benefits and shortcomings may have proven to be significant. For example, as more vehicles were equipped with the ALI-Scout system, there would have been more vehicles that served as traffic probes. As the number of probes increased, there would have been improved sampling and the value of dynamic link-time updates should have increased. Furthermore, as more drivers used route guidance, there should have been continuing improvements in the overall efficiency of the transportation system. These types of issues are appropriate for traffic modeling.

Furthermore, traffic modeling was important because there was little alternative for observing the impacts of dynamic route guidance at the small levels of planned ALI-Scout deployment. For example, assuming that the field test deployed at most 1000 ALI-Scout vehicles, a fleet that would have accounted for an insignificantly small proportion of daily vehicle-miles travelled in the deployment area, then any measurable impact on traffic could not have been expected. Modeling was the only way to allow for assessing the impact of high levels of market penetration on individuals and all users.

Using traffic modeling to assess the impact of SCATS in the deployed area was of great value. Once a model was developed, SCATS could be simulated over the entire deployed area of Troy, as easily as on a selected corridor or single intersections. Considering a wide deployed area for the simulation of SCATS was particularly important, since it was the only effective and inexpensive way of studying the system level control and coordination. In addition, it could have been used to conduct a “before” and “after” study by allowing controlled and re-configurable experiments to be performed.

Evaluation Questions

Figure 10 presents the evaluation framework for traffic modeling. Impact categories included: travel times, traffic, emissions, energy and safety. Specific evaluation questions to be addressed included:

• Are there any improvements in traffic and individual travel times when using ALI-Scout (i.e., beacon-based dynamic route guidance)?
• Is there any reduction in individual and total intersection delay when using SCATS (i.e., responsive and coordinated signal control)?
• What kinds of impacts (i.e., individual travel times, traffic, environment, and safety) can we expect at higher levels of market penetration?
• How many probe vehicles are required before we can see statistically significant improvements in guidance due to dynamic updates of the database?
• What are the impacts of implementing other types of route guidance features (non-beacon based, non-anticipatory)?

Mesoscopic Traffic Simulation

The plan was to develop and apply a traffic simulation that had the flexibility to represent both SCATS and ALI-Scout. For the initial ALI-Scout analysis (i.e., probe analysis), the INTEGRATION simulation was used. INTEGRATION has some route guidance capabilities that supported a reasonable representation of the major features of ALI-Scout (e.g., anticipatory-based routing). However, in order to accurately evaluate the impact of both ALI-Scout and SCATS, detailed logical modules for the two systems had to be developed. These modules were to be linked with the new simulation developed at the University of Michigan.

The initial analysis was a comparison between the ALI-Scout and the unequipped vehicles at increasing levels of market penetration. The analysis required a comparative assessment on vehicle class and network impact measures.

The second analysis focused on the performance of SCATS on the deployed area. It involved a before and after study to determine and analyze the effect of SCATS implementation on traffic flow conditions. In particular, the individual and the system wide changes in terms of intersection delay and trip time were addressed.

The third analysis involved the study of a generalized route guidance system. The simulation software, coupled with the ALI-Scout module, had enough capabilities to address the following route guidance features:

• In-vehicle route computation,
• static and dynamic travel information,
• route types (e.g., myopic, anticipatory based)
• link weights (e.g., routing on highways only)
• incident delay

The FAST-TRAC evaluation imposed some special requirements on the simulation structure and modeling effort that are outlined below. Chief among these was the need to represent both the route guidance and traffic signal control logic used in FAST-TRAC. This also implied being able to simulate the impacts of ALI-Scout and SCATS on the transportation system.

Time-dependent demand and link flows - This is an essential feature of any traffic simulation used for evaluating ITS. Vehicles enter the network at a specified time and travel from link to link in the network over time. The objective is to describe the time-dependent generation of traffic in the network. This is to be distinguished from static network representations of traffic that are common in most standard transportation planning models.

Vehicle routing - The objective is to route each vehicle individually from its origin to a destination and to collect statistics on the route characteristics. Individual routing is to be distinguished from the end-of-link-turning-percentages approach used in many existing traffic simulations. The drawback of turning percentages is that vehicles do not make complete trips from a specified origin to a specified destination, and therefore routing efficiency statistics cannot be collected.

Multiple routing classes - Given that each vehicle is routed from a specified origin to a specified destination, then it is important to characterize the routing logic guiding each vehicle or class of vehicles. A vehicle class is a group of vehicles that use the same approach to routing. In most cases the routing logic is the heuristic used by the driver as described below. Assuming driver compliance with a route guidance system, then the paths of the equipped vehicles are prescribed by the guidance system. In the FAST-TRAC system, the equipped vehicles were guided by ALI-Scout and there needed to be a way of coding this logic for vehicle guidance.

Behavioral assignment - Most network traffic models assume that drivers take the optimal shortest path from their origin to their destination. However, when evaluating a route guidance system, one cannot make this assumption. In doing so, there leaves no room for improvement using dynamic route guidance.

Drivers who do not use route guidance often take sub-optimal routes due to navigational errors, unfamiliarity with the area, and a host of other reasons. Therefore, the objective is to accurately describe traffic based on routes taken by familiar and unfamiliar drivers making trips of various purposes throughout the simulation period. Routing errors cannot be randomly assigned to vehicles because actual routing errors are unlikely to be random in nature.

Guidance can access current network data- The ALI-Scout dynamic route guidance system had two-way communication between the vehicle and roadside that provided the means to update the traffic database and guide vehicles based on near real-time information. The ALI-Scout vehicles served as traffic probes by collecting data on experienced travel times and uploading this information to the traffic control center where it was used for computing shortest paths. The simulation must have the capacity to model probe updates, which requires that the guidance system have access to the current network link time data.

Regional network - The vehicles that were outfitted with ALI-Scout were distributed throughout the Oakland County area. The simulation network represents this coverage area. This implies that the simulation must be relatively computationally efficient so that it does not become bogged down on a network of this size. This also implies that an event-based simulation approach may be preferred over a time-slice approach, and that at queue-based representation of traffic flow may be preferred over a car-following approach.

Traffic signal control module to accommodate SCATS. Most off-line signal timing plans can be represented by setting the cycles, phases, and offsets of the various signals to the simulation clock. However, the responsive area-wide traffic control system, like SCATS, must be coded into a signal control module so that it can adapt to the traffic flows generated by the simulation. Modular control would help in representing proprietary systems that presumably could be coded to interact with the rest of the simulation.

The traffic signal control options must be relatively complete including controls for isolated signals, arterial intersection control, closed network control, and most importantly, area-wide system control. Control variables include vehicle presence, flow rate, occupancy and density, speed, headway, and queue length. Phases, cycle times, and offsets are fundamental control options. Pedestrian phasing and control are essential. Simple isolated timing heuristics like Webster, and time-actuated control, as well as more advanced on-line responsive signal coordination schemes must be addressable.

Simulation Output - It is important to remember that the purpose of the simulation was to evaluate the impact of Intelligent Transportation Systems under a variety of conditions that for whatever reason cannot be economically addressed in the field trials. It was essential to focus on those measures that are of interest to the ITS and transportation community at large. Therefore, it was important to include standard measures on travel times, stops, emissions, and fuel consumption.

Measures of Effectiveness

This section describes the simulation output measures evaluated. Simulation provides great flexibility in establishing measures of effectiveness. The simulation used in this assessment generated the following measures of effectiveness:

• Link measures: vehicles, discharged by vehicle by class, travel time per vehicle, delay per vehicle, average speed, stops per vehicle, percent stop delay, average saturation, ratio of moving to stopped time.
• Network measures: vehicle miles traveled, vehicle minutes, time in queue, vehicle trips, stops per vehicle, ratio of moving to total trip time, average speed, mean occupancy, average delay per vehicle, total delay, delay per vehicle miles, travel time per vehicle mile, spill-back on links, vehicles entering and exiting the network by class, signal time history.
• Routing class measures: O-D travel times, O-D travel demand, vehicles completing journey.
• Second order measures: vehicles emissions (HC, CO, NOX), fuel consumption, safety (speed variation, accident situations, vehicle miles traveled).

Safety and Second Order Impacts - Estimates of emissions, fuel consumption, and safety were derived from vehicle miles traveled and other related model outputs.

Data Collection

Data was needed to calibrate, validate, and drive the simulation. Most of the data collection involved contacting the appropriate government agencies and obtaining the necessary statistics and parameters. Data obtained this way included the network configuration data, link characteristics, lane characteristics, signal timing parameters, and traffic demand. Other input data like the ALI-Scout route guidance logic and related network data, as well as the SCATS traffic control logic and parameters, were proprietary and obtained from Siemens and AWA.

High-level network data was obtained from the Southeast Michigan Council of Governments (SEMCOG). SEMCOG collects high-level data for use in periodic transportation planning activities. The data have been recorded on tape and had to be manipulated to put in a form for use with our simulation. Detailed information on coordinates, road shapes and classes were obtained from the navigation companies (NavTech) affiliated with the project.

Much of the link characteristic data were obtained from the local traffic agencies and from the Michigan Department of Transportation (MDOT). This data was available in hard copy, so aerial photographs of the area were used, if available, for checking data accuracy. Pedestrian volumes may have were measured in the field.

Data on lane characteristics were obtained from the local agencies and MDOT. Again, aerial photographs were used for checking the data.

Signal timing parameters were obtained from the local agencies for the fixed time signals and for other existing signal plans. SCATS responsive traffic control logic and parameters were obtained from AWA.

The best available data was used for deriving the traffic demand estimates. Data for a seed matrix were obtained from SEMCOG. Historical volumes were obtained from the local agencies. This information was used to synthetically derive the traffic demand.

Results

The “Travel Time Benefits of Centralized Route Guidance: Model-based Evaluation of ALI-Scout in the FAST-TRAC Operational Field Test” paper (Hadj-Alouane, et al. 1998) presents the results of the evaluation of ALI-Scout using traffic modeling and simulation. The simulation shows that three major factors determine the effectiveness of this guidance system: (1) the beacon coverage; (2) the accuracy of link travel times and (3) the level of market penetration. The effectiveness of ALI-Scout is significant when the analysis focuses on selected areas with reasonable beacon coverage. Considering only the area around the I-75 freeway, the benefit accruing to both guided and unguided vehicles amounts to 30% in trip time savings. Overall, the system shows a trend of increasing benefits up to a 15% market penetration. Beyond this level, the system effectiveness tends to decrease because the equipped vehicles are taking the same routes and causing their own congestion.

The objective of the simulation is to perform a quantitative evaluation of the operational impacts of the Ali-Scout system on traffic, under various levels of market penetration. Six levels of market penetration are considered in the simulation: 0%, 1%, 5%, 10%, 15%, and 20%.

The selection of the morning peak hours as the simulation period provides an excellent opportunity to simulate traffic under the condition of recurrent congestion. In addition to this naturally occurring congestion, incident cases are also considered. This provides an opportunity to measure the effectiveness of the Ali-Scout system under different network conditions, where a significant delay is caused for many travelers in the network.

The incident case is designed based on data acquired from MDOT, related to incidents that have occurred during 1995 in the Metro Detroit Area. Three incidents are introduced. They are specified by their location, start time, end time, and the number of lanes affected. The latter represents the reduction in capacity of the link on which the incident takes place. Although these incidents do not necessarily occur at the same time, they are grouped under a single scenario (i.e., one simulation run is needed to capture their effect). Details about incident specifications are shown on Table.

Incident Number	Location	Start Time	End Time	Effective Speed *
1	I-75 NB at Rochester	7:00 AM	7:30 AM	0
2	Rochester NB at Square Lake	8:00 AM	8:30 AM	1/2 SFF
3	696 EB at I-75	7:15 AM	8:05 AM	2/3 SFF

* Effective speed is expressed as a fraction of the free flow speed, SFF.

It is important to note that we are modeling direct incident reporting (e.g., cellular phones, police report) for Ali-Scout. This means that an Ali-Scout vehicle becomes aware of the incident in at most two update cycles (i.e., 10 minutes). The background vehicles, however, respond to incidents with a longer delay.

The scenarios designed for the simulation study have another dimension related to the mode under which the Ali-Scout system is operational. Three modes are considered:

• Static mode,
• Dynamic mode based only on historical data, and
• Dynamic mode based on a weighted combination of historical and real-time information.

Given that the targeted analyses address three attributes: (1) market penetration, (2) Ali-Scout routing mode and (3) network conditions, 32 scenarios are considered in the simulation. These scenarios include the following two baseline cases. The first represents the 0% market penetration level without incident, and the second represents the same level with incidents.

The simulation model generates a number of Measures Of Effectiveness (MOEs), which are given either as aggregate (or averaged) statistics, or as statistics collected by link, time period, O-D pair, and vehicle type. Two vehicle types are considered: Ali-Scout equipped (or guided) vehicles and non-equipped (or unguided) vehicles.

The following three sections present the MOEs related to the individual vehicles, the system as a whole, and selected areas of the network. Each table shows the result of a particular MOE for a selected set of scenarios. Some MOEs are displayed in the form of charts, in order to compare outcomes across various attributes. To simplify the analysis, the results are summarized and only selected scenarios are discussed in details. However, the detailed simulation results can be found in the appendix.

The performance of individual vehicles is assessed through the average trip time, average distance traveled, and average speed. The trip time is defined as the time it takes a vehicle to travel from the its origin to its destination. It includes link travel time, the time spent in the queue at the feeder links (i.e., the links from which vehicles originate) and the time between exit and entrance of consecutive links. These average trip times and the average distance traveled are generated by collecting and averaging the individual MOEs over all vehicles, as well as over unguided and guided vehicles, separately. The average speed is the ratio of the average distance over the average trip time.

The results show that the trip time, distance and speed follow the same general trend. Therefore, the analysis will focus mainly on the trip time. It is important to note that using the system-wide MOEs (i.e., aggregated over all vehicles loaded on the network and all O-D pairs) only gives a general indication on the performance of the Ali-Scout system relative to the baseline. Quantifying the potential benefits, in this case, may not be appropriate given that (1) the beacon area does not cover the entire simulation network, and (2) the simulation scenarios are such that Ali-Scout vehicles may travel between O-D pairs that are not in the FAST-TRAC area. This means that many Ali-Scout vehicles travel under the autonomous mode long before reaching the beacon area.

In order to better estimate the benefits of Ali-Scout, MOEs are also aggregated over some strategic O-D pairs. The targeted O-D pairs in this analysis are such that the vehicles traveling among them are more likely to pass through incident areas. Specifically, the O-D pairs associated with Rochester Road, I-75 and I-696 Freeways. Given the strategic location of the I-75, the analysis will address the performance of vehicles traversing this freeway during their trips.

Tables 17 and 18 summarize the results for the scenarios without incident and with incidents, respectively (refer to Table for a description of the incidents and their locations). For each case, the three Ali-Scout modes are considered: static, dynamic with historical data, and dynamic with historical and real-time data. Also, six levels of market penetration are indicated. Each table shows the average trip time for the unguided vehicles and the Ali-Scout vehicles, as well as the overall average.

Market Penetra-tion	0%	1%	5%	10%	15%	20%
Static
Unguided	28.6	29.1	28.3	29.1	28.6	28.6
Ali-Scout		28.4	27.9	28.5	28.5	28.9
Overall	28.6	29.1	28.3	29.0	28.6	28.6
Historical
Unguided	28.6	29.2	29.2	28.3	28.4	28.9
Ali-Scout		28.0	27.6	27.3	27.5	27.8
Overall	28.6	29.2	29.2	28.2	28.3	28.7
Real-time
Unguided	28.6	29.0	28.3	29.2	28.5	29.0
Ali-Scout		27.4	27.0	27.9	27.5	28.0
Overall	28.6	28.9	28.2	29.1	28.4	28.9

Table 18: Average Trip Time (min) in the Incident Case

Figure 12: Delay Along I-75

It is apparent that the unguided vehicles also benefit from the presence of the guided vehicles. Although their performance is not consistently improving with increasing levels of market penetration, it is clearly better relative to the baseline case. For example, in the incident case, the minimum trip time saving that the unguided vehicles experience is about 25%. This time saving is achieved at only 1% level of market penetration.

Simulation results have illustrated that the effect of Ali-Scout is marginal when the performance of the whole network is evaluated. In this case, the overall benefit of Ali-Scout, in terms of trip time savings, is only around 2% relative to the baseline. This benefit is achieved when Ali-Scout is operating under the dynamic mode, and for both incident and no-incident scenarios. This is not surprising given the beacon area does not cover the entire simulation network; many Ali-Scout vehicles end up traveling under the autonomous mode for a considerable portion of their trip. The effectiveness of Ali-Scout is significant when the analysis focuses on selected areas with a reasonable beacon coverage. Considering only the area around the I-75 freeway, the benefit accruing to both guided and unguided vehicles amounts to 30% in trip time savings.

Although the results show some variability across the scenarios, the general conclusion (see Figure 12) is that the system shows a trend of increasing benefits up to a certain level of market penetration (15% in this case). Beyond this level, the system effectiveness tends to decrease. This implies that the availability of route guidance information may not automatically lead to improvements in traffic conditions. The present low level of market penetration in Oakland County should not be considered a problem. However, if Ali-Scout were to be adopted by more drivers, or should a similar centralized route guidance system catch on with the consumers, then this could become a real limit to individual and collective benefits of route guidance. It is also likely to diminish the market appeal.

We have observed that the reason for the diminishing marginal benefit in the simulation is that the equipped vehicles are taking the same routes and causing their own congestion. In other words, during the update period, vehicles with similar destinations passing by a beacon receive the same route. As the number of equipped vehicles increases, the number of vehicles taking this same route increases, and eventually they start slowing each other down. So it seems that success breeds failure in this example.

However, with a few simple adjustments the above problem can be nearly eliminated. One possible solution is to increase the update frequency while increasing the accuracy of the travel time database. This would be possible as the communication and computational technologies improve. Part of increasing the accuracy would be to update the historical database to take into account the future impacts of recently routed vehicles. The result should be improved routing for each vehicle that passes by a beacon and less beacon originated congestion. Another solution is to implement multi-path routing adjusted to the congestion effects of the routed vehicles. Other techniques may be used, but this problem must be addressed if centralized route guidance is to get beyond the 20% level of market penetration.

It is important to note that there are at least two types of benefits originating from the probing of real-time link times in Ali-Scout. The first type of benefit is a source of data for generating an empirically-based historical travel time profile. The second type of benefit is the use of the real-time data as a supplement to the historical database. Probe reports of divergent link travel times are combined with the historical link travel times so that drivers can respond to changes in traffic flow as they travel.

At low levels of market penetration, there may not be enough probe reports to make a sufficiently rapid impact on the link travel time database. The solutions are conceptually simple, but they may not be practical in a centralized route guidance system. Again, increasing the update frequency of the travel time database is one possible solution. A difference of five minutes can mean a lot in responding to an incident in real time. Another solution is to speed up the response through improvements in surveillance and the synthesis of multiple surveillance sources. In the simulation, we modeled direct incident reporting through, for example, cellular calling. If similar types of incident reports can be used in generating guidance information, then the system will be able to quickly respond to the incidents.

Traffic Control System Evaluation

Accident Analysis

The purpose of this task was to analyze accident data to determine the effectiveness of SCATS in improving highway safety. The system, being both coordinated and adaptive to demand, was expected to reduce certain type of accidents that are liable to occur due to discontinuities in the traffic stream flow, mostly rear-end accidents. Accident frequencies for the total number of accidents and type of accidents were compared before and after SCATS deployment. Moreover, the analyses covered comparisons between different ambient light and weather conditions (e.g., night vs. daylight, and clear vs. adverse weather). Since the initial SCATS-controlled intersections were made operational in June 1992, these analyses were periodic as the accident data accrued and became available for the after period.

Objectives

The main objectives of the accident data analyses were:

• To compare accident frequencies in SCATS-deployed corridors with non-SCATS corridors having similar operational conditions, including data stratification by the type of accident and ambient conditions.
• To compare accident frequencies at selected SCATS-deployed intersections with non-SCATS intersections including the type of accident and ambient conditions.
• To prepare comparative statements, tables and charts disseminating the results of the accident data analyses.

Methodology

The experimental design used in this study was before-after with control.

Procurement of Accident Data - The main source for the procurement of accident data was the Transportation Accident Master from the Traffic and Safety Unit, Michigan Department of Transportation (MDOT). This file is an abridgement of the Michigan Accident Master, developed by the Michigan State Police (MSP) from the original source, Official Michigan Traffic Accident Report (form UD-10). Since abridgement of the data by the MDOT consumed time, interim data were directly procured from the MSP Master for the timely completion of periodic accident analysis.

Creation of Study Data Files - The Transportation Accident Masters for the before period, i.e., 1989 through 1991 are permanently available with the Department of Civil and Environmental Engineering, Michigan State University. As the accident data accrued, the study data files were updated by merging the new (after period) data with the old (before period) data. This updating, on the average, was done every six months subject to availability of the data and covers the past six months.

Identification of SCATS and non-SCATS Corridors - SCATS deployed corridors (and intersections) were identified by the MDOT Physical Road (PR) numbers and the Mile Points (MP) of the road(s). Any physical change in deployment was tracked and incorporated in the analytical program. Areas with similar physical and operational conditions outside the SCATS-deployed area served as control for the experimental design.

Development of Computer Program for Data Analyses - Dependent upon the source and for­mat of the incoming after period accident data, a new computer program was developed for each analysis for the extraction and merging of data, and for undertaking the analysis. The basic idea was to segregate the variables of interest from the master records and to stratify the data by accident type, severity, and ambient light and weather conditions.

Synthesis and Documentation of Results - A written report was developed for each periodic analysis covering the following aspects:

• Preparation of tables and charts to present the numerical results
• Conclusions
• Limitations which affected the results.

Results

The “Safety Impacts of SCATS” (Taylor & Wu, 1995) is one of a series of annual reports on the effects of SCATS on accidents in the city of Troy. The report provides: (1) an update on the accident data to include data through December 1994; (2) a comparison of the accident trends in Troy with two other suburban Detroit cities – Southfield and Farmington Hills; (3) an analysis of the change in accidents in Troy using the Quality Control Chart statistical technique and (4) an analysis of the change in accidents resulting from changes in the left turn control at intersections where SCATS was implemented.

The “Final Report On The Analyses Of Traffic Accidents” (Taylor, Wu & Hein, 1996) includes a trend analysis and a comparison with a control city. The SCATS controllers were installed in 1992 and 1993, and the analyses are based on traffic accident data from 1989 through 1995.

The Road Commission for Oakland County (RCOC) chose two corridors for comparison of accident frequency before and after SCATS installation: John R. and Adams Roads. The intersections along these two corridors are: Maple & John R, Big Beaver & John R, Wattles & John R, Long Lake & John R, Square Lake & John R, South Boulevard & John R, Big Beaver & Adams, Wattles & Adams, Long Lake & Adams, Square Lake & Adams, and South Boulevard & Adams.

The results of the analysis are shown in Table 19 below. The two corridors were virtually identical in the two time periods.

There were small changes in overall accident frequency at various subsets of intersections, but there was no statistically significant increase or decrease in system-wide total or injury accidents after the installation of SCATS.

There was, however, a significant reduction in turn-related accidents following SCATS deployment. For the 39 major intersections in the test area, there was a decrease of 121 turn-related accidents per year (a 60% reduction), as shown in Table 17 below. The 18 intersections where the left-turn control was modified accounted for 75 fewer turn-related accidents per year, and the 11 intersections where the geometry was changed accounted for an additional reduction of 29 turn-related accidents. However, even the ten intersections where there was no change in the geometry of left-turn control strategy experienced a decrease in turn-related accidents. That reduction (46%) is assumed to be attributable to SCATS.

In addition, there was a large, and statistically significant reduction in the severity of injuries sustained in injury accidents. This was caused by the change in accident type from turn accidents to rear-end accidents. The number of severe (incapacitating) injuries was reduced by more than 50% following the deployment of SCATS and the implementation of changes in geometry and left-turn control at selected intersections.

Intersection Delay

The purpose of this study was to determine whether intersection delay was reduced as a result of the implementation of SCATS. The signalized intersection is the most critical location in a traffic system as it causes interruption of traffic flow. Vehicular delay at signalized intersections is a component of the travel time in an urban road network and contributes to vehicle op­erating costs. The Highway Capacity Manual uses delay as the sole criterion for determining level of service (LOS) at signalized intersections, and indirectly for LOS evaluation on urban arterials.

Scope

The area of interest for this study was the portion of Oakland County, Michigan that was affected by the installation of Phase III of the SCATS signal controller system. The “before” study focused on data collected on the status quo condition prior to the activation of the Phase II controllers. The “after” study focused on data collected once the network had reached equilibrium after the activation of the SCATS controllers. Three controller locations were selected as sample intersections for the before and after study based on the following criteria:

• Intersection location on the proposed corridors determined for study in the overall system wide evaluation,
• Left turn characteristics both before and after SCATS (e.g. protected or unprotected),
• Intersection approach geometry (e.g. auxiliary lanes),
• Intersection approach volumes,

Methodology

The method to determine vehicular delay characteristics at a sample intersection used video gathered at the intersection before installation of SCATS signal controller and after installation of the SCATS signal controller. Each approach to the sample intersection was videotaped to provide a clear view of the queue that developed. The camera was positioned above the intersection and observed vehicles entering and leaving the queue. This video was shot with a time indicator overlaying the video image for ease in image processing and documentation.

The videotapes were subsequently reviewed to determine vehicular delay. The method used to determine delay characteristics at a sample intersection considered each lane (both through and auxiliary) of each approach for a given intersection. The delay was measured for each vehicle that leaves the queue during a lane’s green phase. This was done by determining the amount of time each of these vehicles spent in the queue. Then the total delay time was plotted.

Validation of Video Reduction Method

To help validate the video reduction method outlined above, random vehicles were selected during the videotaping and their associated ve­hicular delay was measured. To take these “spot” data points, a technician chose a random car and logged the intersection approach, clock time, vehi­cle make, model and color, and lane number. The technician started a stopwatch when the vehicle entered the queue and stopped the stopwatch when the vehicle crossed the stop bar. This time was logged as the vehicle delay time. These spot data points were then entered into a data­base.

The technicians responsible for reducing the videotape were then given computer printouts containing the intersection approach, clock time, vehicle make, model and color, and lane number of all relevant spot data points. The video reduction technicians then determined the vehicle delay time for each spot data point using the video reduction method outlined above. Since the video reduction technicians were not given the value of vehicle delay that was measured in the field, the two vehicle delay figures could be compared to ensure that the estimate of delay for any given vehicle determined from the videotape was acceptable.

Data Collection Activities

Data collection for the “before” data occurred on the selected intersections in May 1996. The videotaping method was the principal data collection method, supplemented by the spot data collected. The data for both the videotaping and spot data was collected during three time periods: Morning (6:30 to 9:30 am), Afternoon (11:00 am to 12:00 pm and 12:30 pm to 1:30 pm), and Evening (3:30 pm to 6:30 pm). These periods of time included the build up and build down of traffic during the peak period of the day. Analysis of these time periods provided insight into how SCATS distributed intersection delay. Data collection for the “after” period was conducted after the network had reached equilibrium after the activation of the SCATS controllers.

Computer Modeling

Once the set of sample intersections had been analyzed and vehicle delay characteristics determined for each, the entire spectrum of SCATS intersections was modeled. The intersection data was used to predict average speeds and average delay functions within statistically acceptable limits for all intersections in the SCATS signal controller project. In addition, the model(s) was used to predict the accumulated delay for a 24-hour period.

Coordination with Other Activities

The determination of the intersections for study was coordinated with the corridor selected for detailed study as described in the chapter entitled “Corridor Delay.” The intersections selected for study were on a single corridor.

Results

The “Report on Intersection Delay” (Taylor, 1995) aimed to estimate area-wide impacts on intersection delay by analyzing the change in delay at selected intersections due to SCATS operation. However, accurate estimates of delay could not be derived using only the data from SCATS. Nonetheless, with the addition of one parameter, the arrival distribution, reasonably accurate estimates of delay could be formulated. Thus, if a technique for collecting this parameter could be developed and automated, it would be possible to determine the delay at SCATS controlled intersections. If the SCATS output data and an automated arrival distribution data collection procedure could be used prior to initiating the SCATS logic, a before and after comparison of SCATS could also be made.

The City of South Lyon also converted the traffic signals on the street network from fixed time control to SCATS. The objective of the “Final Report on the Analysis on the Impact of Installing SCATS in South Lyon” study (Taylor & Wolshon, 1997) was to analyze and document the changes in delay that occurred following the signal system conversion. The analysis included direct observations of data under SCATS control and simulation of the fixed-time systems using identical approach volumes. The measures of effectiveness included both total intersection delay and the average delay per vehicle on each approach to the intersection.

The implementation of SCATS control resulted in an overall increase in total intersection delay at three of the four intersections included in the study. The change in the delay varied by time of day (and thus volume), with small increases and decreases in delay observed in the midnight to 1:00 am time period and an increase of several seconds per vehicle during the afternoon peak periods. The average delay per approach was decreased by 1.59 seconds per vehicle as a result of the SCATS deployment. This is the result of redistributing the green time to decrease the difference in the degree of saturation across various approaches to an intersection. This results in relatively large decreases in delay to the minor traffic movement accompanied by small increases in delay to the major traffic movement.

Corridor Delay

The primary purpose of this study was to determine whether travel time through the street system was improved as a result of the implementation of SCATS. Travel time is a function of congestion, delay at intersections and other locations, and operating speed (which, in turn, are interrelated).

Measure of Travel Time

Travel time was measured by using videotaping to identify and time-log specific vehicles at both ends of a corridor. The “beginning” of a corridor is defined as the point where a vehicle crosses the stop bar at the entrance intersection of a corridor. The “ending” of a corridor is defined as the point where a vehicle passes the stop bar at the exit intersection of a corridor.

In the videotaping analysis, the traffic stream was videotaped at the beginning and ending points of the test corridor. Subsequently, the videotapes were manually reviewed for “matches” between the two tapes and travel times were recorded. This proved to be an efficient method of collecting reasonable numbers of data from any desired time period (except after dark). About two hours of review time were required to reduce one hour of data collection in the field. Approximately 20% of collected data was “matched.”

Data Collection Activities

The data for the videotaping was collected during three time periods: morning (6:30-9:30 am), noon (11:00 am-1:30 pm), and evening (3:30-6:30 pm). These periods of time included the “build up” of traffic on a corridor from no congestion to total congestion and a “build down” to no congestion. The data from these time periods provided insight into how SCATS distributed corridor delay over the time block.

Coordination with Other Activities

The determination of the corridors was coordinated with the selection of individual intersections for the previously described study. All of the intersections selected for more detailed individual study were on this corridor. This allowed for a determination of the extent to which time-savings at individual intersections was reflected on overall corridor travel times.

Expected Impact of Advanced Traffic Management System

The possible impacts of ATMS were not fully known until the “after” data had been collected and evaluated. Sample data were collected along separate corridors to test the data collection and reduction procedure. The data were reduced into both travel time and average moving speed. The initial analysis was done using individual speeds through the corridor, but the variance proved to be too large. In order to better distinguish overall trends, and smooth the variance, the moving aver­age speed was used. The moving average speed is defined as the average of 15 data points. For ex­ample, (See Figure 13) the first 15 observations are averaged (data point 1 through 15), then the next fifteen are averaged (data points 2 through 16), and so forth. The mov­ing average speeds do vary through a time period. This graph shows two peaks, indicating the high­est speed (and in turn lowest delay) through the corridor: one at about 7:00 am and the other at around 8:15 am. Also, one can note the lowest moving average speed around 7:45 am indicating the maximum delay through the corridor. There is also day-to-day variation of the moving average speed on a corridor-e. g., for West Bound Wattles Road for the period 11:30-1:30 pm, the average speeds were 25.4 mph, 25.6 mph, 37.0 mph on the 13th, 14th, and 17th of May respectively.

Results

The Phase IIA report on “Before and After Videotapes & Special Events Data” contains a review of the methodology used to evaluate the effects of SCATS on corridor travel times (travel speeds), the results obtained from the analysis, procedures studied to automate the data reduction process and plans for the next phase of the analysis.

It was not possible to collect travel time data for all corridors and, even if this were feasible it would still not be possible to determine the total system-wide effect of SCATS. Therefore, a sample set of corridors was selected for this study, with the intent of expanding the results of this sample to estimate system-wide effects.

Five corridors were selected to represent a range of operating conditions. Although reflecting variables such as traffic volume, level of congestion and geometry, the selection process was qualitative rather than quantitative - that is, it was believed to be sufficient to include a range of conditions rather than defining a complete factorial experiment design. The corridors selected represent different characteristics and provide a reasonable range of before and after conditions. The corridors selected for “before SCATS” data collection for Phase II were: South Boulevard, Wattles Road, Maple Road, Coolidge Highway, John R Road.

The intersection accident study and delay study was conducted at all intersections, because the data was available from the MDOT crash tapes and the SCATS output tapes. However, travel time data had to be collected by videotaping the vehicles traversing the corridor and then matching license plates to measure the travel time through the corridor. Since this was too time consuming the five corridors mentioned directly above were selected for analysis. These corridors ranged in providing Level-Of-Service from B to F at the intersections along the corridor.

Travel time dta was collected in May 1993 at all corridors except South Blvd. where it was not possible to set up the video cameras at an angle that allowed data reduction. It was found that the day to day variation in travel time was quite large, as measured by the standard deviation. Both the maximum and minimum travel times, and the 30 minute time period when the travel time was at the maximum value varied by day. This meant that a large sample of data would have to be collected if a statistical analysis were to be conducted.

There were also two confounding factors that were noted. First was the fact that when the SCATS systems was implemented, the left turn phasing at the intersections was changed from “permissive” to “protected” and when this resulted in long queues to “Permissive-Protected”. Thus, a comparison fo before and after data would not be a measure of SCATS alone. The second factor was that the signal timing in the before period had not been optimized for progression, and it was optimized by SCATS in the after period.

For these reasons , the corridor analysis was modified to study a corridor where there would be no changes in the left turn strategy, and where the progression could be optimized in the before period. Orchard Lake Road was selected and studied for this purpose , and there is a full report on this corridor included in the final project report.

System Delay and Capacity

The purpose of this task was to expand the results of the accident study, the intersection delay study and the corridor delay study in order to describe the city-wide impacts of the SCATS system. Ultimately, it is these system-wide benefits that must be compared to the cost of implementing the system to determine overall cost-effectiveness.

A second purpose for this study was to compare the traffic flow on the network with the capacity of the network elements to determine whether (and where) there is capacity available to accommodate traffic diversion.

Evaluation Questions

The accident, intersection delay and corridor delay studies were all conducted on a sample of the intersections and links that comprise the street network in Oakland County. The question to be evaluated in this task was how to expand these sample data points to describe system impacts.

If the accident reduction had been similar at all intersections and links studied, then the aggrega­tion to the network would have been straight forward - one could simply count the in­tersections and links and apply the average re­duction to each element to obtain the system im­pact. However, it was more likely that there would be differential impacts based on differ­ences in volume, geometric configuration and possibly location in the network. This study was used to determine the most appropriate stratifica­tion of these variables to be used to expand the sample results to system safety results.

The same general approach was used to expand the corridor delay and intersection delay sample results to system impacts. The variables used to stratify the data prior to applying reduction factors were probably different for each of the three data sets, but this could not be determined until the results of the three individual studies were available.

The capacity question to be evaluated in this study was the level of service (LOS) at each intersection approach. This information could be useful in determining the potential for re-routing traffic during an incident. If there is no unused capacity during the peak hours, then neither SCATS nor ALI-Scout can contribute significantly to improved performance.

Method

The methodology used in the aggregation from samples to the system was based on expanding a stratified sample of data points from an unknown universe. The stratification was based on combinations of volume, geometrizes and location that define similar intersections or links (based on the outcome of the before-after study of these elements). The stratification criteria for the accident analysis were more rigorous than that for the delay studies, since the sample size was larger.

The methodology employed to determine the LOS was the application of the Highway Capacity Manual Software to each approach to determine the maximum flow at each level of service. These flows were then compared with existing flows (peak period) to determine both the LOS and the unused capacity of each approach or each signal phase.

Results

Early on all the intersections in the City of Troy (42 intersections) were looked at and the level of service at each approach was determined. These results were used to select intersections and corridors for the early before-after studies.

Special Event Analysis

Purpose

The general purpose was to determine whether, and how much, the SCATS system improved traffic conditions during special events such as a football game at the Silverdome.

Methodology

The original plan was to study the before-after traffic performance around football games at the Silverdome.

Results

The special events study could not be conducted as planned. The key intersections were not instrumented until there were only 2 football games left in the season. However, some data was obtained: the delay at six intersections was recorded on those two football game days, starting when the football games finished.

The after study had to be collected on days when the world cup soccer matches were held at the Silverdome. Four days of data collection occurred in this after-period. The problem is the before data was collected on a Sunday afternoon, while the after data was taken on a Tuesday, a Wednesday, a Thursday and a Saturday. The base traffic on the street was thus not comparable. Then the police decided to control the traffic at the signals at the exits to the parking lot, thus metering the flow to the various signals.

Data was accumulated on the length of time each intersection remained congested after each event closing, but the results varied, depending on the day and time. For example, the length of congestion at Adams and Auburn ranged from 7 minutes t0 70 minutes in the after period, and at Opdyke and Walton the range was from 3 minutes to 46 minutes.

Incident Response Analysis

Purpose

The Federal Highway Administration has estimated that congestion caused more than 1.2 billion hours of delay, 1.3 billion gallons of wasted fuel and more than $9 billion in excess road user costs in the United States in 1985. The growth rate of congestion, if continued through the year 2005, would result in a 5 fold increase in these numbers, to a total of $45 billion per year. Other estimates placed the cost of congestion even higher.

Another phenomenon of interest to this study is the relative growth rate of recurring to non-recurring congestion. In 1984, about 62% of delay was incident related (non-recurring). This percentage is expected to increase to over 70% by the year 2005. Thus, the ability to respond to incidents is both a large and increasingly important element of any congestion management plan. These are the two aspects of a management plan where ATMS and ATIS systems can play a significant role.

An incident management plan is presently composed of several elements:

• Detection, verification and classification of an incident;
• Response and remediation of the factor causing the incident;
• Implementation of a traffic management plan to accommodate the change in the traffic pattern;
• Implementation of a motorist communication system to alert the motorist to the existence of the incident, and in some cases to provide the motorist with advice on alternate routes.

The FAST-TRAC test program, with the simultaneous implementation of ALI-Scout and SCATS had the potential to improve these last two elements. In fact, because these systems were designed to function in real time, and to respond with little or no intervention, these two elements could have operated without the first two elements of the management plan. The traffic control logic in SCATS should alter the timing plan to reflect changes in the traffic pattern with no knowledge of the location, type or duration of an incident.

Evaluation Questions

The evaluation in this study was the determination of the impact of the SCATS controller on the duration and magnitude of congestion caused by an incident. It was anticipated that the impact of SCATS (compared to non-SCATS) would depend on the severity, duration, time-of-day and response to each incident. For example, an incident that closes a link of the network will result in a larger traffic shift than a partial closure. The longer the duration of the incident, the greater the distance upstream the impact will be observed. If an incident occurs when the system is at or near capacity, the impact will be larger than if the incident occurs during non-peak periods.

Paradoxically, one could have found that while the congestion impact was greatest in the peak period, the impact of SCATS on the level and duration of the congestion might have been lowest at this time, since the ability to alter the signal timing might have been more limited.

The effectiveness of the driver communication system could also have affected the impact of SCATS. In theory, the communication system might have been non-existent, it may have consisted of radio communication through the Michigan Emergency Patrol (MEP), it may have involved changeable message signs (stationary or mobile) or it could have been through ALI-Scout when the market penetration level was sufficient to impact traffic flow.

Methodology

The research methodology employed in this study was a comparison of the magnitude and duration of the delay caused by an incident with SCATS and with a non-adaptive system of traffic controllers. Since the latter system could not be created in the field, simulation modeling was used.

This study consisted of two approaches. First, simulation was used to estimate the effectiveness of ATMS, ATIS and the combination of these two systems on reducing delay and queue length at varying distances from the incident. Incidents of varying severity and duration were introduced into a network and the resultant delay and queue formation at intersections in the network were recorded.

The first simulation run was conducted with no ATMS or ATIS response. The simulation was then repeated with various ATMS strategies applied to the network intersections (minimization of delay, equal approach saturation flow, equal queue length on alternate paths, etc.) The total system delay, delay to vehicles on the street where the incident occurs and maximum delay encountered by any vehicle was recorded for each run.

The identical incidents were then simulated with various ATIS strategies (diversion in equal increments to 2 or 4 alternate routes, diversion to the path with least delay, etc.) The same output measures listed above were recorded. Finally, combinations of diversion and traffic control response were simulated for the same incidents. The results of this study were included in a project report published in 1996.

The second approach was a field study of the response of SCATS to an incident. Simulation was used to document the delay resulting from the measured response. A network of streets in the Cities of Troy and Rochester Hills was coded into NETSIM. Data from the SCATS controllers at all the intersections in the network was recorded for the non-incident condition. The range of values for volume, cycle length, degree of saturation, etc. was determined from these data, and statistical confidence limits established for each variable.

NETSIM (NETwork SIMulation) was selected for this study because it could be modified to replicate control and drivers characteristics within the ITS environment. NETSIM is a microscopic interval-based simulation model of urban traffic on a surface street network. The model was first developed in the 1970s and has periodically been enhanced. NETSIM version 5.0, which was used in this study, includes many advanced features on traffic signal, driving behavior, and turning movement descriptions and can provide data on the MOEs suitable for the analysis (Federal Highway Administration, 1995).

The traffic engineers from these two cities provided information on scheduled (maintenance) and unscheduled (crashes) events in the network. The value of the variables used to describe non-incident conditions was extracted from the continuous monitoring detectors for the time of the incident and compared to the range of values found for the incident-free conditions. The duration and lateral extent of the “non-normal” condition was noted, and the approach volumes during this time were saved from the SCATS files.

Simulation runs using these volumes combined with a) actual signal timing from the SCATS data, and b) “normal” signal timing representing non-adaptive signal operation were conducted. The difference in delay and queue information for these two runs was used to evaluate the effectiveness of SCATS on adapting to incidents in the network.

An Incident Detection Algorithm

Arrangements were made with the traffic engi­neers from the cities of Troy and Rochester Hills to monitor reports of incidents or accidents on the test network. Once an incident was reported within the SCATS coverage area, the control center was requested to preserve the flow and saturation data from the network signals in an area surrounding the incident. This data was analyzed to determine how long after the incident each of the SCATS controllers recognized the occurrence of the incident as reflected in a timing change.

If the response time using MEP was too long to capture the progression of congestion through the network, police reports were monitored to detect an incident. If this process was still too slow, video images at the control center were monitored to observe these incidents.

The MOEs used in this study include:

• The time between the occurrence of an incident and the first response by SCATS as a function of the distance between the incident and the intersection being studied. By looking at various intersections in the network, the speed at which congestion spreads through the network was determined.
• The time required for the signal system to return to normal operation. This measure was used to determine the efficiency of the SCATS system in dissipating congestion. Using the models, measure of delay, emissions and fuel consumption were compared.

Results

Selecting the most appropriate traffic control strategy for incident congestion management can have a major impact on the extent and duration of the resulting congestion. The “Incident Management” report (Taylor & Narupiti, 1996) investigated the effectiveness of several control strategies on various incident conditions.

A postulated surface street network system was constructed for this study. The network is a grid system, with each intersection one-quarter mile apart. Streets are all two-way with a left turn pocket of 200 ft at each intersection approach.

The research was based on a hypothetical dense grid network with demand characteristics representative of traffic conditions in the City of Troy, Michigan. The normal traffic operations were assumed to be similar to a peak traffic period, with the optimal signal timing plan developed by an off-line calculation. Three types of incidents were considered: a one-lane closure, a two-lane closure, and a reduction of the two-lane capacity to 15 percent of the original capacity.

Three incident durations with various control strategies applied to these scenarios were tested. The incident duration, either 5, 10, or 15 minutes, was limited by the recovery time to normal traffic operation in the most severe case, the 15-minute both-lane closure. With the given size and configuration of the network, level of demand, and location of the incident, traffic operations in the 5 and 10-minute both-lane closure incident recover within one hour after the beginning of the simulation, while it takes about one and a half hours to recover in the 15-minute both-lane closure case.

The selected control strategies representing possible ITS technologies included traffic metering (ATMS), traffic diversion (ATIS), and combined traffic diversion and signal timing modification (ATIS/ATMS).

When an incident was introduced into the network, the evolution and dissipation of congestion were studied. Congestion resulting from an incident in a network without ITS was used as a base case, with traffic performance analyzed for various types of incidents. For the base case, the signal timing was held constant and the impacts of the incidents on specified MOEs were determined.

Several traffic signal control and route diversion strategies were developed for each traffic situation. These strategies were then tested with the simulated network to obtain performance measures in various incident characteristics. Data for the MOEs were collected for each control scheme and the results were compared and discussed. The effectiveness of these control strategies under different demand condition was then determined. Moreover, variations of control strategies were evaluated.

The following tables show some of the results found on the potential impact of alternative control strategies on various incidents.

    Note: *indicates the lowest total travel time for an incident situation

Note: *indicates the lowest total travel time for an incident situation

A sensitivity analysis was performed to obtain the effectiveness of various control strategies under different demand levels. See the table below for details.

Table 23:   Difference In Total Travel Time From The No Control Change Scenarios (10-Minute Both-Lane Incident)

Note: *indicates the lowest total travel time for an incident situation

Note: *indicates the lowest total travel time for an incident situation

Global Evaluation

Cost Evaluation

The purpose of this task was to assemble and analyze cost data for the various components of the SCATS system. These cost items included (when appropriate):

• Purchase price
• Installation cost
• Maintenance cost
• Training cost
• Operating cost
• Replacement cost
• Overhead or administrative cost

Methodology

The cost items were obtained from the records maintained by RCOC. These documents included purchase invoices, daily records of county employees, contracts, utility bills and estimates where necessary (for example, overhead costs and training costs).

The data was organized to determine:

The cost per intersection for comparison with the benefits per intersection

The cost per corridor for comparison with the benefits per corridor

The system cost for comparison with system benefits

Time between failure for various components of the system

In addition to obtaining cost data for the Phase I installation, comparisons were made with Phase IIa installations to determine if the costs were decreasing or increasing as a function of time, experience or number of units purchased. Costs associated with the control center were obtained from the research team analyzing the center and included in the cost estimates for the various units of analysis used in the study.

Results

The “Cost Analysis of ITS Technologies: ATMS/ATIS Integration” study (Levine, Underwood & Torng, 1996) organizes all FAST-TRAC related cost items based on functional categories. The study includes an analysis of direct expenditures as well as other costs induced or avoided system-wide because of the presence of the project.

According to this report, the financial reports provided by RCOC indicated that a total amount of $31, 675, 200 had been spent on the FAST-TRAC project as of September, 1996. This is based on financial reports for Phase I, Phase IIA, and Phase IIB, issued in November, 1996. A summary table of that report (Table 4 on p. 8) breaks out costs according to major subsystems (the total here comes only to $25,855,700), indicating that these costs broken out by major subsystems did not include all the items included in the summary number supplied by RCOC.   Based on the subsystem breakout, and the $25,855,700 figure, it is found that the ATMS (SCATS) systems represented 69% of the cost, the ATIS (ALI-SCOUT) system represented 16.6%, the Traffic Operations Center was 7.6%, the RCOC cost was 2.1%, and the evaluation cost was 4.7%. For further detail on the cost breakouts the reader is referred to the report.

Global Analysis

The global analysis component of the evaluation study was quite different in nature from each of the other study elements. Whereas the other study elements analyzed a particular outcome of the FAST-TRAC system, the global analysis sought to integrate cost and benefit information from each of the other studies. The intent was for this analysis to provide a “road map” to enable users of the evaluation to assimilate and compare the disparate information coming from the rest of the project.

The objective of the global analysis was the integration of results from technical, preference and human factors analyses with cost data in a format that facilitated ready comparison (including marginal cost-effectiveness analysis) of program alternatives, and a delineation of development options desirable from the perspective of various stakeholder groups. The goal of the evaluation was the assessment of outcomes (both costs and benefits) in a manner that:

1.      Was sensitive to phasing, implementation and market penetration issues

2.      Facilitated debate on relative importance of system attributes

3.      Identified likely beneficiaries and losers

4.      Was explicit on private and public benefits

5.      Was consistent on measuring benefits and costs over time, and between routine and incident conditions

6.      Avoided double counting of costs or benefits.

Method

The method was a Multi-attribute Utility Technology (MAUT) style of analysis (Adelman 1992), designed to elicit preferences on system attributes and to delineate desirable alter-natives based on preferences of relevant stakeholder groups. Data supporting this analytical approach was the product of coordination with other study components.

Coordination with Other Evaluation Studies

The global analysis relied on most of the other components of this project. The relationships are described below.

Stakeholder Analysis

For the purposes of the global analysis, the most important outcome from the stakeholder analysis was the relative valuation of system components by various stakeholder groups. Working with the stakeholder analysis component, survey instruments were developed that allowed a quantitative evaluation of the importance the different stakeholder groups placed on system attributes such as time savings, environmental impacts or neighborhood effects.

The dimensions on which on which statements of priority were elicited included:

• Individual Travel Time
• Commercial Travel Time
• Emissions
• Collisions
• Tax Costs
• Energy Consumption
• Driving Difficulty

Two parallel methods for assessing priorities regarding the system dimensions are listed above. The first requested a numerical ranking on the part of the respondent with questions such as the following:

“Intelligent vehicle and highway systems may have a number of benefits in different areas. The way the systems are put together would determine what kind of impact they might have. How important is it to you that a system would: Reduce travel times, Reduce Car Emissions, Etc. (1= Very Important, 5= Not At All Important)?

In addition, systems may have a number of side effects and costs. How important is it to you that a system would: Minimize traffic spillovers in neighborhoods, Minimize safety impacts in neighborhoods, Etc. (1= Very Important, 5= Not At All Important)?”

The second approach to determine weighting was to ask a series of questions supporting a conjoint style of analysis designed to derive implicit preferences for system attributes from comparison of alternatives. These can take the form of a rating or ranking of alternatives (Hair et. al 1992), or a comparison of paired alternatives (Louviere and Hensher 1983).

Traffic and Higher Order Impacts

ATIS-based rerouting of traffic has the potential of lowering environmental quality along the alternative routes, particularly if these run near or through residential neighborhoods. Increased traffic on alternative routes may deteriorate the environment for alternative transportation modes such as public transportation, walking and bicycling (Gordon 1992). Predictions of these effects were developed in conjunction with the Traffic and Higher Order Impacts component.

Together with the traffic and higher order Impacts the following information was developed:

• Predictions on travel time savings over time and at different rates of market penetration
• Predictions on fuel savings  
• Predictions on emissions reductions          
• Predictions on changes in traffic volumes along alternate routes
• In each of these areas, forecasts were needed under:
• ALI-Scout at various levels of market penetration
• ALI-Scout in combination with SCATS
• Alternative operational policies regarding signal operation and diversion instructions
• Effects under routine and incident conditions
• Alternative scenarios regarding inducement of travel
• Varying levels of individual proficiency and willingness to alter travel behavior

Individual Impacts

Driver Behavior, Individual Impacts, User Perceptions And Preferences were integrated from this component including:

• Effect on driver uncertainty, frustration or fatigue - This was presumably self-reported, ordinal data. Approaches were developed to extrapolate these data to alternative system scenarios different from the circumstances of the actual field test.
• Effect on driver behavior - Presumably something less than 100 percent of the population was able to make complete use of ALI-Scout (or was willing to accept route guidance instructions). Information on what segments of the population were able and willing to use the techno-logy at varying levels of efficiency was developed in conjunction with the driver behavior components.
• Valuation of travel time savings - Methods were developed to estimate the relationship between travel time savings and the value of the time saved. In particular, threshold effects and non-linearities had to be examined.

Driver Operation and Interface Design

The safety impacts of route guidance and automated signal control were difficult to assess with any certainty. This stems from the fact that accidents are a relatively rare occurrence, and that surrogate measures such as near misses are in-adequate predictors of accidents. The driver operation and interface design component proposed to study raw measures of eye fixations, lane speeds and variance and navigation errors. To the extent possible these were used to comment, however speculatively, on potential safety impacts of the system.

Technical

Capital and operating cost estimated for the three stages of deployment were developed in conjunction with the Technical component of the evaluation study together with interviews with Oakland County officials.

Instruments

Survey instruments were developed in conjunction with the stakeholder analysis portion and were aimed at analyzing valuation of various system outcomes (e.g., time savings, energy impacts, neighborhood impacts) by various stakeholder groups.

Results

Transportation planning in general, and planning for Intelligent Transportation Systems (ITS) in particular, are notable both for multiple goals and for multiple constituencies. In response to this policy environment, multi-criteria decision analysis has often been utilized to evaluate alternative transportation investments. The report on “Stakeholder Preferences in Urban Transportation Evaluation: A Multiattribute Analysis of Goals for Intelligent Transportation System Planning” (Levine & Underwood, 1995) extends this approach to assess stakeholder valuation of broad goals of an ITS planning process and FAST-TRAC. Representatives of stakeholder groups, ranging from emergency response firm employees to city managers to environmental groups, were interviewed.

Group 1:  Public Sector Transportation	Public sector transportation (highway and transit) professional, School transportation personnel
Group 2:  Private Sector Transportation	Delivery services, Trucking companies, Automotive industry
Group 3:  Education/Media	Educational administrators, University, Newspaper
Group 4:  Safety	Emergency vehicle firms, Police, Fire departments
Group 5:  Other Business	Chambers of commerce, companies, Real Estate Developers
Group 6:  Public Administration	City Managers, Mayors, Planning departments
Group 7:  Citizen Groups	Homeowner/Community Groups, Environmental Groups, Seniors/disabled

Using a modified Analytical Hierarchy Process, implicit preference weights for transportation planning goals were derived, and inter- and intra-group comparisons made.

Overall, collision reduction emerged as a dominant goal, with travel time reduction and energy/environmental impacts each accounting for about 20% of the valuation that stakeholder placed on the range of transportation goals. Much transportation activity in the past decades was premised on travel time reductions as a prime benefit. However, energy savings and emissions reduction are considered by the Oakland County stakeholder groups studied here to be of roughly equal importance to travel time as transportation system goals.

Impacts of stakeholder group affiliation on transportation system preferences appeared most strongly with regards to environmental preferences and reduction in commercial travel time; with regards to other goals, individual interests may dominate those of the ostensible stakeholder group.

The table below presents the ratio of actual to expected populations within each cell of a four cluster by seven stakeholder group matrix. For example, the representation of the Business stakeholder group within the “Safety” cluster is half that which would be expected if stakeholder group membership and cluster membership were independent. In this fashion, it is possible to associate preference-based clusters with stakeholder groups.

The results of this study raise the potential for outcomes associated with traffic diversion and inducement to increase objections to ITS installations. Increased noise and traffic volumes in neighborhoods were universally viewed negatively, while increase traffic volumes along arterials and expressways was viewed negatively by the majority of respondents. Still poorly understood is individuals’ traffic diversion behavior. Presented with a route that takes him or her off the freeway, will a driver attempt to improve on it still by taking short cuts through neighborhoods? Moreover, the negativity with which increase arterial traffic is viewed casts doubt on the political acceptability of traffic diversions as a strategy even of those diverting would avoid neighborhoods. It is important to recognize that public action regarding traffic diversion advisories represents a transportation policy and should be treated as such, rather than left to the vagaries of third party computer software. Thus it may be reasonable for implementation agencies to consider thresholds of congestion below which diversion advisories are not issued, and even modifying the computer encoding of transportation networks to protect potentially vulnerable neighborhoods near freeways and arterials.

Reduction in collision was the goal that virtually all respondents could agree upon in its primacy. While traffic smoothing through advanced traffic management systems may indirectly enhance safety through reduction in speed variance, there is little reason to believe that route guidance would have any positive effect. By diverting the driver’s attention, in-vehicle route guidance could even conceivably be deleterious to traffic safety. An ITS policy that was responsive to stakeholders’ concerns regarding safety would certainly emphasize collision avoidance technology and – perhaps more immediately – speed limit enforcement technology, both for highways and for city streets.

Integrated System Evaluation

Stakeholder Analysis

Stakeholder feedback was used to assess the value of FAST-TRAC technology from the perspective of those whose interests in the community could have been affected by the system. This information enriched the data on which future decisions about FAST-TRAC were based. It told FAST-TRAC partners whether the community believed the system met its needs. It also told FAST-TRAC partners the conditions under which the public would pay for FAST-TRAC technology. FAST-TRAC project managers gained information essential to determining the system’s marketability. In so doing, they developed a group of people able to discuss FAST-TRAC knowledgeably with the public when the system became fully implemented. The specific objectives for this part of the evaluation are as follows:

• To determine what stakeholders viewed as the optimum motor vehicle transportation system, how they viewed the current transportation system, and what changes they would make to the current system to reach their ideal.
• To introduce stakeholders to the FAST-TRAC system.
• To determine the degree to which stakeholders believed the FAST-TRAC system will adequately address today’s traffic problems.
• To monitor shifts in stakeholder perceptions about FAST-TRAC as the Oakland County project progresses.
• To determine stakeholder willingness to pay for the FAST-TRAC system.

Method for Quantifying Stakeholder Values

Interview participants were selected based upon their leadership positions, their familiarity with the communities in the FAST-TRAC deployment area, their ability to assess the pros and cons of FAST-TRAC, and their willingness to follow the progress of the FAST-TRAC project. In interviews, participants were presented a video on the operation of FAST-TRAC. Then they were asked to offer their impressions of their group’s response to such a system. The interviews were semi-structured and the participants responded to a standard set of questions to each groups’ interests and positions vis-a-vis the deployment of ALI-Scout.

Stakeholder Group Representatives

Several stakeholder groups were included in the stakeholder analysis. These groups included:

• Emergency vehicle firms
• Trucking firms
• Delivery services
• Media
• Developers/Real Estate Brokers
• State Government
• Education
• University/Scientific
• Environmental groups
• Automotive companies
• Neighborhood groups
• Consumer advocacy groups
• Businesses and business associations
• Local government (administration, police/fire, planning)
• Special event facilities

Design

The interviews followed the phases of the FAST-TRAC project. Phase I interviews gathered early stakeholder impressions about the FAST-TRAC system as it related to stakeholder interests. Phase IIa and IIb interviews measured changes in stakeholder perceptions and determined levels of stakeholder acceptance of the system.

With input from FAST-TRAC project partners and members of the evaluation team, interviewees were identified. Early interviewees were asked to recommend additional community leaders whose views would be valuable to the project. This approach provided an efficient means of identifying potential interviewees. The goals of this portion of the evaluation were to obtain and measure the assessments of those who represented a variety of community interests. This was not a survey of a randomly selected population.

Interviews followed a format designed to elicit in-depth responses from stakeholders regarding their attitudes toward traffic conditions and FAST-TRAC technology from the perspective of organizations or of the agencies they represented. The interviews were conducted in three stages. In the first, interviewees were asked how traffic conditions related to their organizational interests in an ideal world. Interview subjects were then asked to describe current traffic conditions and the impact of those conditions on their organizations. Ideal and current conditions were given point values, say 100 for the ideal and 65 for current conditions, to quantify the gap between the two as seen by each stakeholder. Interviewees were then asked how they would change the motor vehicle transportation system to reach their ideal and the degree to which their solutions would close the numerical gap.

The second part of the interview obtained data for the Global Analysis evaluation. Stakeholders were asked a number of scaled questions in written form to learn what characteristics they would favor in a new transportation system. They were asked to rate such factors as travel time savings, and emissions savings, safety, and increased travel volume. This information fleshed out stakeholder answers concerning the ideal traffic system. It also established a standard for each stakeholder against which to measure the FAST-TRAC system. Interview Subjects were then asked about their knowledge of the FAST-TRAC project, their sources of information, and which sources they trusted most.

Stakeholders in the third part of the interview viewed a videotape describing the FAST-TRAC project and technology. They were asked how well the program and technology met their goals and how it compared with the solutions they expressed in part one of the interview. Again, points were ascribed to determine the degree to which stakeholders believe FAST-TRAC would close the numerical gap between the ideal and current conditions. Interviewees also were given a second questionnaire concerning transportation system characteristics, this time to determine which traits they believed FAST-TRAC would include. The results from this questionnaire, when compared with results from the previous questionnaire, showed the degree to which stakeholders believed the FAST-TRAC system embodied characteristics they were willing to accept. Finally, stakeholders were questioned about their willingness to pay for the system.

This interview process provided the FAST-TRAC evaluation team with three types of data:

• a portrait of stakeholder goals concerning traffic conditions,
• an assessment of the degree to which stake-holders believe FAST-TRAC met these goals, and
• an assessment of the degree to which stakeholders believed FAST-TRAC would provide valued benefits. This information enabled FAST-TRAC partners to determine the system’s viability from a community stand-point. It provided a gauge of marketability along with ideas for improving or enhancing the system to increase marketability.

For Phases IIa and IIb, stakeholders were divided into three groups. The first group spent time driving cars equipped with the route guidance systems. The second group received detailed, printed information about FAST-TRAC technology and the FAST-TRAC program, but drove no test cars. The third group received no additional stimuli other than that provided by the media and Oakland County promotional efforts. Phase IIa interviews measured stakeholder perception shifts among the groups. The interviews also explored themes and trends developed in the Phase I interviews. Phase IIb interviews measured perception shifts that occurred between the midpoint and the end of the FAST-TRAC project. Interviews in both phases measured stakeholder understanding of the FAST-TRAC program and determined what sources of experience or information were most influential in shaping stakeholder perceptions.

Data collection

Data was collected in two ways. Stakeholder interviews concerning traffic goals, current conditions and potential solutions were recorded on an interview form. The form enabled the interviewer to tabulate point scores ascribed to stakeholder goals, proposed solutions and the FAST-TRAC system as a particular solution. This information was transferred to a grid that showed, at a glance, where each stakeholder group stood regarding FAST-TRAC.

A separate questionnaire was used to record answers regarding global analysis characteristics. Interview subjects checked off their selected answers on the questionnaire itself. Results were presented in a table showing stakeholder characteristics preferences generally and the degree to which FAST-TRAC reflected those traits according to stakeholder perceptions.

Instruments

Interview form, two transportation characteristic questionnaires for before and after videotape presentation, stakeholder grid, videotape, VCR, and monitor.

Facilities

Interviews were conducted at the offices of interview subjects.

Results

The “Report on Stakeholders’ Analysis” (Hansen, Mitchell & Oglesby, 1996) was undertaken to identify and evaluate criteria by which the public, and certain stakeholder groups within the public, would judge the merits of the FAST-TRAC system. Over a period of two years, three surveys were conducted to obtain specific information from stakeholders about various aspects of the system and its impact on the problems associated with adverse traffic conditions.

The majority of stakeholders continued to give the project passing grades; perceptions of certain key aspects of the project, however, fell slightly over its lifespan. Stakeholders continued to support the use of technology for improved transportation but were frustrated by technical malfunctions of the system. Stakeholders were also concerned about the costs of maintaining the system after the experimental phases were over. As the project progressed, fewer stakeholders believed it had a positive impact on the problem of traffic congestion (50% versus 70% a year earlier) and fewer felt the project met their expectations (62% versus 74% a year earlier), but more believed it had a positive impact on traffic safety (66% versus 40% a year earlier).

Institutional Issues

Institutional issues are the non-technical activities that either aided or hindered the deployment of the FAST-TRAC program in Oakland County. The documentation and evaluation of these issues can benefit future implementations of ITS which can build upon the successes and mistakes of these early trials. Having included the institutional issues evaluation in this project, FAST-TRAC can serve as a guide for future deployments of ITS.

In the U.S., the Volpe National Transportation Systems Center (Volpe Center) has developed a case study framework (DeBlasio, 1992) in conjunction with the Federal Highway Administration (FHWA) to ensure that study teams obtain and produce comparable information.

FAST-TRAC was included in this national evaluation. In addition to FAST-TRAC, Volpe also evaluated Help, Advance, TravTek, Transcom and the Advantage-I75 operational field tests. The Volpe evaluation was scheduled through September 1993, but the FAST-TRAC effort continued for the duration of the evaluation.

The Volpe Monitoring Program Framework (DeBlasio, 1993) was the methodology used in the on-going evaluation of FAST-TRAC. The Volpe program was developed for short-term evaluation of the six projects mentioned above, and needed minor modifications for long-term use on this project.

The objectives of the FAST-TRAC Institutional Issues evaluation were similar to the four objectives of the Volpe program. The questions to be answered were:

• What institutional and legal impediments were encountered in establishing partnerships and deploying ITS services and products during the FAST-TRAC project?
• Where in the life cycle of FAST-TRAC did these impediments occur?
• What were the causes of these impediments and how were they overcome?
• What lessons were learned in dealing with these impediments that can be applied to other deployments of ITS products and services?

The most important task was to develop a comprehensive set of lessons learned in a case study format with a historic event timeline which will assist in future deployments of ITS.

Issue Identification Methods

The methodology was based upon the Monitoring Program Framework by Volpe (DeBlasio, 1993). A series of personal and group interviews were conducted to document the history, institutional issues and lessons learned in the early phases of the project. Additional personal and group interviews were conducted to document on-going institutional issues and lessons learned.

Interviewees were selected according to their level of involvement and responsibility with the FAST-TRAC project. A rough time-line and a sample list of institutional issues were presented in a semi-formal and interactive setting. At the start of the interview, the interviewee was told that, although direct quotes may be used in the final report, the names of interviewees would be kept confidential. A facilitator guided the interview by asking a series of questions and recording responses on paper. A second person called the recorder was responsible for taking detailed notes during the interview or a recording device was employed.

The information obtained from the interview was written into a standard format and verified by both the facilitator and recorder. Once the document was reviewed internally, the interviewees were sent a copy of the results and asked to verify the information. Based upon these interviews, key people were selected to monitor continuing issues. These individuals were given a logging device such as a special note tablet or voice recorder to record pertinent information and events. They were then contacted every two months for the ongoing collection of these records.

Other FAST-TRAC documentation such as meeting minutes, contracts, work plans, and reports were reviewed as well. Attendance at meetings such as the Executive Committee and the Evaluation Sub-Committee meeting facilitated this effort.

The case study format incorporated both historical information and institutional issues to present the lessons learned. Historical information was presented as an event time-line. Institutional issues were presented using qualitative tables, charts and matrices. The case studies were meant to be descriptive rather than analytical. They did not address the technical components of the FAST-TRAC study but rather suggested potential solutions to implementation obstacles created by non-technical issues.

Interviews

Intensive interviews were a major component of the case history method. Before the interview, the proper individuals had to be identified and contacted to arrange a two-hour interview. A confirmation letter was sent to all parties.

The individuals were given a sample list of institutional issues and an initial history of events. The interview was conducted with both a facilitator and a recorder or recording device collecting information. These two people jointly wrote a single report. Once the report was completed, the interviewee was sent a thank you letter and a copy of the report in order to verify the information. Once the report was finalized, it became part of the data library that was used for the case study.

Individuals were also selected to participate in the continuous monitoring efforts at this time. If additional people needed to be interviewed, they were added as the need developed.

Questionnaires

In order to augment data collection beyond the scope of interviews and meetings, a questionnaire was used. The questionnaires were issued to individuals who could not be interviewed due to time constraints. The question format closely followed the questions used for the official interviews. The information derived from these interviews was incorporated into the data library and used as a basis for the case study.

Case Study

The data library documented the institutional issues and the processes used to resolve them. The case study summarized the events and the processes in a standard format. The initial individual interviews provided the first set of documents for the library. During the individual, meeting, and document monitoring process, other materials were added. These materials were the basis for the time-line and the production of “lessons-learned” reports. An initial outline for the case study was developed immediately, with changes and expansions occurring throughout the project. The case study was written as part of the final evaluation of the project.

Results

The “FAST-TRAC Evaluation: Report on Institutional Issues – Continuous Monitoring Stage” (Ghosh, Darmofal & Underwood, 1996) documents only the issues or events occurring during the Continuous Monitoring Stage of the project. The primary data for this report are from interviews conducted with most of the project participants. The secondary source of information was project documentation.

The list below is a summary of the recommendations respondents generated for future projects:

• Funding requirements should be assessed and requested well in advance.
• Appropriation of funds should be based on need for a project’s entire duration, not given on an annual basis.
• Contracting procedures need to be expedited.
• A more realistic assessment of what project members and the public can expect needs to be clearly stated at the onset of a project.
• An overall program management plan of the project and the evaluation needs to be established at the onset of the project.
• Project members need to receive some sort of educational training to ensure they understand how the technological systems work.
• The local public needs to be informed about the projects' (goals, objectives, technologies, schedules, etc…) by the project administrators.
• Alternative funding methods, along with a clear explanation of what constitutes a match, need to be developed before the award of a project.
• Evaluation plans should be developed simultaneously with the implementation plans.
• Information regarding these types of projects needs to be disseminated on a national level.
• Plans to account for the various technological equipment used after the project is completed need to be established at the beginning of the project.
• Communication channels among all project members need to remain open throughout the entire project.

The “Institutional Issues Report” (Richeson & Underwood, 1997) presents the following list of “lessons learned:”

• A successful ITS deployment team will learn to adapt and evolve based upon unanticipated events that arise during the course of the project.
• It is important to establish public information sources in order to keep all participants abreast of project changes and accomplishments
• Careful consideration should be given to overall team composition, taking advantage of complimentary capabilities and interests, and promoting cooperative problem solving, to assure that common project goals are eventually achieved.
• In the early stages of the project, it is important to supplement the general vision with clear statements of obligation among the project partners.
• Local government units can effectively administer federal aid highway projects.
• Whereas the success of the project depends on the skills and commitment of individuals, personnel turnover need not pose a risk if talented individuals are continually brought into the project.
• Testing and deployment of large, potentially risky, transportation infrastructure projects may depend on the commitment of federal funds.
• The long process for approving federal funding can result in delays in contract execution, and may result in a need to initiate project tasks before all contracts are in place.
• The evaluation contractor needs to be able to accommodate deployment delays in both the planning and execution of evaluation tasks.
• Large-scale, public infrastructure deployment tests should be evaluated by a single, independent contractor to determine potential benefits and costs.

Traffic Information Management Systems Evaluation

When the FAST-TRAC project evolved from the original “integration of an ATIS and a ATMS system” to a more generalized combination of a large set of systems, the name “Traffic Information Management Systems, (TIMS)” was selected. The goal of TIMS was and is to integrate traffic related information within the project area. Upon integration, the various subsystems of TIMS will become interactively linked, and therefore allow TIMS to act as a central system for data collection and dissemination. TIMS will enable RCOC to monitor traffic conditions throughout Oakland County and t provide travelers, businesses, and agencies with real-time traffic information.

The evaluation here addressed several aspects of TIMS: a Traveler Study and a Systems Integration Study. The traveler study determined how individual travelers with access to the traffic information provided by TIMS respond to it, i.e., how they use it and what they think about it, especially what benefit they believe it provides them. The Systems Integration study describes the systems engineering process; a case study describes tasks and events in the FAST-TRAC system integration process and provides the basis for an integration model that may be used as guidance for future ITS integration projects.

Oakland County Traveler Study

One of the later objectives of the FAST-TRAC project was to investigate new cost-effective ways to survey public opinion by exploiting internet and electronic mail. The Oakland County Traveler Study describes a Web survey tool and e-mail polling strategy to obtain traveler perception about traffic conditions, the effectiveness of the traffic control technologies, and feedback on problems and future traffic improvement measures. As current research suggests, both, e-mail and web-based survey strategies have the potential for cost effective, repeated administration of questionnaires when compared to mail surveys. The main purpose of this study was to see if a tele-survey approach would be cost-effective and easy to repeat.

The traveler study had two main goals: 1) evaluation of ITS technology and traffic conditions and 2) exploration of the use of new survey technologies. In terms of content, the survey is to obtain travelers’ opinions about the traffic situation and traffic problems in the county. Furthermore, the survey provided a follow-up evaluation on user assessment and awareness of the FAST-TRAC system. The primary purpose of the survey was to assess user perceptions and opinions of the:

• implemented transportation control systems and the related technologies,
• impact of these systems and technologies on traffic problems and safety,
• future transportation system improvements including traveler information services, weather warning system, etc., and
• transportation policy in Oakland County.

A second purpose was is to investigate new and innovative survey approaches that would help RCOC reach the public at a reasonable cost- exploiting internet and e-mail technologies. The primary objective for the Road Commission was to support a public survey approach that would be

• cost-efficient and relatively easy to administer,
• facilitate repeated surveys at relatively low cost,
• allow changes to the questionnaire in an ad hoc fashion, and
• exploit technical means for recording data and analyzing results.

The project presents somewhat of a challenge as little research to date has been done on the impact of using internet and e-mail technology for general public opinion surveys for limited geographic regions – in this case Oakland County. To find acceptable solutions to overcome the technical and methodological impediments of internet use was a prime objective of the University research team. The study looked at practical considerations like user access, message transmission speed, and e-mail/web-page format. Of great concern, for example, was how to devise a survey that the RCOC could use for repeated surveys. Another issue was the representativeness of the population with web-access and e-mail accounts and controlling participants selection to ensure generalizable results. Since most of the e-mail and world-wide web surveys conducted to date have been administered within a known community, for example, at a university with a known list of e-mail account holders, the recruitment of participants from an “unknown” community was a crucial issue.

Survey Method

With increasing accessibility to the internet and e-mail for larger segments of the population, and with approximately 6.4 million households in the US connected to the internet, e-mail and web-based surveys have been added to telephone and mail surveys as a means to elicit information from people. E-mail surveys may well become the preferred survey tool in the next decade. E-mail surveys have been found particularly useful in targeting geographically distributed populations and special interest groups (Sell 1997; Walsh et. al. 1992). E-mail and/or web-based surveys are potentially cost efficient since there is no mailing preparation and postage. Moreover, coding of responses and analysis can be automated, at least in part. Questionnaire formats on the World Wide Web (WWW) can include graphics, help in obtaining correct responses and entail adaptive questioning (Gaddis, 1998). Response speed in e-mail surveys is superior to mail surveys and response rates, especially for solicited e-mail questionnaires, are equivalent or higher than for mail surveys. This is due to the ease of communication facilitated by electronic mail. Furthermore, responses using internet based methods have been found to be:

• more thoughtful (longer responses, fewer mistakes); and
• more candid (Kiesler and Sproull 1986).

Problems with WWW and e-mail surveys include issues of representation and access. Due to access constraints, participants in web/e-mail surveys are not an representative sample of the general population. In general, internet users tend to be younger, wealthier and more educated than the US population as a whole. Especially with web-based surveys, there is a chance that users submit a set of responses more than once, which will bias results (Oppermann 1995; Schmidt 1997). Due to a wide range of available browsers and e-mail software, the questionnaire format needs to be tested extensively to avoid technical problems (Hays 1998).

In summary, the literature on e-mail and web-based survey suggests that the technique is promising but more research is needed to gain a good understanding of the consumer behavior and population biases introduced by web-based and e-mail surveys. Computer user population demographics are still skewed toward the younger and more affluent members of society. Thus, caution must be taken to prevent coarse misinterpretation of survey results.

Most currently available web-based surveys depend on individuals browsing and finding a particular web-site. This seemed not very practical for polling the a geographically defined public (Oakland County residents) effectively and broadly. Furthermore, individuals may respond to the questionnaire multiple times, perhaps in response to incidents or other acute traffic problems. It is unclear how frequently people will use this opportunity to make their opinions known.

One strategy to increase traffic at a website is to announce it via e-mail to large groups of internet users. In our case, an e-mail announcement to Oakland County residents would have been an attractive option to lure survey participants to a web-based survey site. However, e-mail address lists are currently not available for purchase from commercial providers. Further, search engines and white pages on the web prohibit the compilation of geographically ordered e-mail addresses.

It was decided that an initial survey would be administered to a representative group of driving adult residents in Oakland county. This initial survey using 3 different modes of communication with previously solicited participation from a randomly selected pool of Oakland County residents was used to establish analysis methods and knowledge about population biases. The three different modes employed are:

• Web-based survey (Mode A)
• E-mail survey (Mode B)
• Paper survey (Mode C)

The latter serves as a control element, which the research team felt was needed due to the lack of knowledge about consumer behavior and population biases for electronic surveys. Biases in terms of demographic, socio-economic status and response rates can then be estimated by comparing the traditional survey responses with the ones from the electronically administered surveys. Questionnaires are identical except for method of distribution. Each questionnaire contains a question about future willingness of the respondent to participate in future repeat surveys. It is hoped that a list of participants can be recruited for a long-term continued evaluation of the traffic situation and transportation policies.

Procedure

The process for the administration of the initial survey consists of 6 major steps:

1.   Purchase of randomly selected Oakland County household addresses.

2.   Invitation letter to participate in traffic survey with return postcard. This mailing was followed by a reminder mailing. Prospective participants were asked to state if they use e-mail and/or the internet on a regular basis and to provide their e-mail address.

3.   Collection of return postcards and classification of respondents in appropriate survey communication modes for participation.

4.   Administration of survey modes (A, B, C)

5.   Analysis and evaluation of results for content and methodology (i.e., mode specific differences)

6.   Hand-over of survey instruments (Web-page survey tool/analysis strategies) to public agency for future survey administration and use.

Participant Selection Results

Solicitation letters to recruit participants for the Traveler Survey (FAST-TRAC Evaluation Phase IV) were sent to 2500 randomly selected household addresses in Oakland County at the beginning of July 1999. A reminder mailing inviting participation with a second return postcard was send in August to raise the response rate. On August 30, at the end of a two month recruitment period, a total of 654 postcards (26.16%) were received from persons willing to participate in the survey. Responses were sorted based on the communication modes available to the prospective participants as indicated on the response cards. Responses reflect the ratio for Oakland County households with computer access which is estimated at 50%; 13% of the positive responses indicating one or more electronic communication modes (e-mail, WWW or both). Coding and distribution of responses for the various categories are shown in Table 27.

Code	Respondent Access to Communication	Frequency	Percent of Total
	Hardcopy only	341	13.64%
2	World Wide Web, no e-mail	68	2.72%
3	e-mail (no WWW)	28	1.12%
4	e-mail and WWW	217	8.68%
8	unusable	15	0.60%
9	non-deliverable postal address	200	8.00%
0	no response	1631	65.24%

Table 27: Coding and Frequencies of Responses

In general, postcards were filled out as expected. One person wrote they would only participate “if compensated.” The respondent was subsequently classified as unusable since no funds for participant compensation are available. In many cases participants checked to use e-mail frequently but did to provide their e-mail address. These potential participants were entered into the WWW only or “hardcopy only” category. A number of respondents indicated that they did not feel comfortable to provide their e-mail address. One respondent noted that his e-mail address should only be used for this particular survey and may not be shared with other institutions or companies. It was never intended to share e-mail addresses. Apparently the confidentiality statement in the solicitation letter was not sufficiently explicit.

Subsequently, participants were assigned to the 3 survey modes. Category 4 participants are the most flexible in terms of communication modes (see Table 1). This group of 217 was split, randomly assigning 72 (73) participants to each of the three survey modes. Participants that indicated no choice were assigned to receive the hardcopy survey. Category 2 and 3 participants were assigned to receive an electronic survey in the mode that was available to them. Resulting frequencies for each survey mode are shown in Table 28.

Survey Type	Description	# of Participants	Communication Mode                   (see “Code” in Table 1)
A	World-Wide-Web	100	3 (28) + 1/3 of CODE 4 (72)
B	E-Mail	140	2 (68) + 1/3 of CODE 4 (72)
C	Hardcopy	414	1 (341) + 1/3 of CODE 4 (73)
	TOTAL	654	all

Table 28 - Assignment to Survey Modes

Mode A participants were notified by e-mail of the URL where the Web-Survey is located and asked to fill out the survey on the web. Mode B participants were send an e-mail with the survey and asked to reply via e-mail. Mode C participants received a hardcopy of the survey by mail (with return envelope).

Web Survey Set-up/functional specification

Although theoretically easy - the administration of web-based and e-mail surveys provide some technical challenges in terms of format, access control and response coding. The more challenging issues may be associated with the web-based survey instrument; these issues will be the focus of this section, including access control and ease of usage.

After developing an extensive list of functional specifications a web-based survey tool was programmed by members of the University of Michigan information technology division. This tool has currently 5 main features:

1.   Access Control and User Tracking. One of the main issues with web-based surveys is that participants could willingly falsify results by completing the survey multiple times. Therefore access and survey completion needs to be tracked. In our case, participants need to have a valid login and valid password to take the survey. Login and password is assigned on the basis of the information provided by them on the response post card (i.e. e-mail address as login etc.). An e-mail with the URL and this information is then send to each participant. Login and Password is also used to track survey completion. After a final submit, the participant is prohibited from taking this particular survey a second or third time.

2.   Administration Support. One of the potential advantages of electronic surveys is that they can be used repeatedly and modified quickly. Our survey tool allows a moderately computer literate person to design a web-based questionnaire or change and alter an existing questionnaire using html-templates. Free-form anwers, selection lists, radio buttons and check boxes for multiple choice questions are available.

3.   Ease of Usage and Consumer friendliness. Use of electronic survey formats can be cumbersome to the participant. Reading on the screen is tiresome and annoying if the network connection is slow or often interrupted. In our framework setup we intended to mimic the use of a paper survey. Thus, the survey tool allows the survey participant to move through the questions forward and backward. Graphics were kept to a minimum to not inhibit transfer speed. The survey structure is based on a single screen which contains a limited number of questions (e.g. 1-3) after which a preliminary submit button moves the participant to the next questions. This reduces time consuming scrolling. The survey facilitates “skips” (that is the participant is moved automatically to the appropriate next question if a particular answer renders a set of questions non-applicable).  The survey can be interrupted without losing the answers to questions already addressed. This feature also saves answers in the case of a computer crash or network interruption. Prior to a final submit (the equivalent of mailing the survey back) - participants can change questions.

4.   Automated Coding. The collection of responses and coding is automated. Survey responses are collected and automatically stored in database format. This represents probably one of the money and time saving potentials of electronic surveys.

5.   Support recruitment of additional subjects. As browsing internet users may find the web survey by chance, visitors can sign-up to become an official survey participant. After providing a user profile (which will then allow tracking of survey completion, see above) applicants can take the survey. This feature is to help with participant recruitment for the long-term survey recipient pool.

Results

The results reported here appear in “FAST-TRAC Traveler Study”, a Phase IV deliverable. Appendix A of that report includes 13 pages listing all the accumulated traffic signals that “most frustrate you (the survey respondent)”. This large response indicates that the respondents evidently appreciated the opportunity to designate a traffic signal which is a problem for them. Next, graphs show what times of the day and the day of the week that problems at signals are endured. As expected, the graphs are correlated with the rush hours, and all days of the week show the same high incidence (~90%). A graph showing “how long do you have to wait at the signal” shows the largest percentage (25%) at over five minutes; this data should be treated as an “estimate only”, since the respondents may not have actually collected data using a stop watch.

About 54 % of the respondents assessed the “safety of traffic near the signal” as either satisfactory or good. When asked “what should be done to improve the intersection you identified”, the largest response (47%) said “make the signal adapt to traffic conditions”. Adding green time and coordinating with other signals received 26% and 28% respectively.

The next six pages showed responses to: “If you wish, please comment on another traffic signal”. When asked “Compared to five years ago, how are traffic conditions in Oakland County”, 43% said “much worse”, and 32% said somewhat worse. These responses indicate that the growth in Oakland County is producing a lot of angst in transportation, in spite of the efforts by RCOC to ameliorate it. When asked “how do you rate the management of traffic in Oakland County”, 27% said “needs lot of improvement”, and 47% said “needs some improvement”. This result can be interpreted to mean that RCOC and all its contractors need to continue to find ways to improve the over-crowded situation.

About 72% of the respondents said they knew about the RCOC’s efforts to install high-tech traffic management systems at many major intersections. When asked for their assessment of how effective the new traffic signals were in improving congestion and safety, the results appear as shown in Fig. 14. As seen, about 47% of respondents thought improvement in congestion occurred, with an additional 30% saying “don’t know”.

Figure 15 shows that there is substantial support, among this set of respondents, for installing more high-tech traffic signals.

Figure 15 Support of more traffic signals

When asked about different ways to pay for maintenance and improvements, the gas tax and developer fees were the most recommended. Responding to “would you support a tax increase if it were used only to improve the roads?”, 37% said Yes, and 63% said No.

Fig. 16 shows the results when asked for their preference among different ways of distributing traffic information.

Figure 16 Preferred Alternate Delivery Methods

Appendix E of this report contains 13 pages of “Respondent Comments”. Overall, this report plus its six Appendices contains a great deal of information that should be valuable to RCOC.

System Integration Case Study

A report “System Integration Case Study” by Frank, Reed, LeBay, and Underwood, dated May 18, 1999 is summarized here. The case study focuses on the systems integration aspect of the Transportation Information Management Systems (TIMS), which is the tool developed under the direction of RCOC to integrate various Intelligent Transportation System (ITS) technologies in Oakland County (Michigan). The TIMS is the center of a communications network that integrates advanced traffic control with traffic surveillance and traveler information systems, and solicits and distributes traffic information across jurisdictional levels of local and state traffic agencies.

The core of the TIMS is a UNIX server which is housed at the TOC. The TIMS UNIX-server hosts a relational database and various applications which represent the interface between the various traffic management and control applications and data collection and dissemination functions. The workstations host a graphical user interface (GUI) that aids traffic operators in accessing, managing, and disseminating traffic information as well as operating traffic management and control applications.

A major subsystem of TIMS is AUTOSCOPE, which uses cameras over intersections to provide traffic flow data. The SCATS traffic control subsystem is another major component of TIMS. An originally-intended subsystem, ALI-Scout, was to be interfaced with TIMS to provide link-time data, but that was abandoned when the ALI-Scout system was shut down. TIMS is also linked to other systems: MDOT’S “Michigan Intelligent Transportation Systems Center (MITSC)”; and to the regional bus system—Suburban Mobility Authority for Regional Transportation (SMART). The “Lessons Learned” are the major item in this report, and are similar to items produced in the Institutional Issues in a prior section of this report. They are summarized as:

• Identify and empower a project champion
• Get the right parties involved
• Secure long and short-term funding
• Establish appropriate project control mechanisms, and
• Follow standard systems integration practices

The report notes that throughout the course of the FAST-TRAC project, RCOC was transformed. Staff expertise grew significantly. It is anticipated that growing success in traffic management and positive feedback from road users will ensure that traffic management will be incorporated as a standard component in the Road Commission’s task list. Further, systems integration needs to be viewed as a continuous effort rather than a one-time event.

System Integration Model

As noted before, the concept of creating a model for system integration was to draw general principles from the case  study and incorporate these with other known deployment factors in a format designed to help public sector project managers and transportation planners across the nation deploy ATMS/ATIS. The resulting model is described in a report: “A Model of Systems Integration to Facilitate ITS Deployment”, by Reed and LeBay, Phase IV Deliverable.

The model is described in terms of a Graphical Spiral, shown in Figure 17. The process starts in the outer ring and proceeds clockwise, and consists of 32 separate cells which consist of discrete steps in the building/integration process.

Fig. 17. Graphial Spiral Form of the Systems Integration Model

In addition to the 32 cells portrayed, the model also includes three significant overarching issues: champions, communication, and cooperation. These issues affect each cell of the model and must receive due consideration throughout the process for the systems integration effort to be successful.

The model also includes the three levels of stakeholders that are involved in the systems integration process, which are depicted in Figure 18. The operating agency (or agencies), the systems integrator, and the vendors are directly involved. A second level includes the service providers (public or private), other local agencies, and state and regional agencies.

               Fig. 18 Stakeholders in the Systems Integration Process

The third level of stakeholders is the citizenry, whether directly or indirectly affected by the system being implemented.