ITS - Intelligent Transportation Systems Report ITS Home Page
Table of Contents

3. THE CARTA SYSTEMS ENGINEERING PROCESS

3.1 A BRIEF HISTORY AND TIMELINE OF CARTA'S ITS

Prior to 2002, CARTA did not employ significant ITS assets. Although the agency used computerized systems for accounting, timekeeping, and payroll and a GIS software application to manage paratransit operations, these systems were not integrated. Operating fareboxes and headsigns required manual input from drivers, and Incline Rail ticket sales were still recorded in manual logs. This situation began to change in 2002 when CARTA began to plan their use of ITS.

3.1.1 Defining the Vision

In 2002, CARTA prepared a grant application under the ITS Integration Program. This resulted in grants under the FY 2002 and 2003 ITS Integration Programs. The grant application identified a long-term vision for ITS at CARTA that considered the needs that could be met through the deployment of proven ITS technologies. The planned deployments included:

The grant applications identified the general scope of ITS plans for CARTA, established funding for deploying that ITS, and established a milestone for completing the deployment by March 2009.

In 2003, CARTA reviewed the existing Regional ITS Architecture and mapped its needs to the User Services and Market Packages defined in that architecture. CARTA used a regional stakeholders workshop to begin prioritizing its needs. This workshop helped identify the existing organizational practices at CARTA, the infrastructure in place to support those practices, the outstanding needs that could be met using ITS, and the priorities for meeting those needs. This workshop helped finalize the vision for ITS at CARTA by defining the capabilities that would be provided by ITS technologies. CARTA next conducted a review of available ITS technologies that could be used to meet those needs and documented the results of this review in a November 2003 report. At about the same time, CARTA elected to hire a Manager of Technology. The Manager of Technology would be dedicated to the development, operation, and maintenance of ITS at CARTA.

After reviewing both the agency's needs and the technologies available, CARTA identified the technologies to include in its ITS program and documented this in the System Overview Update – 2004 report completed in January 2004. A key feature of this report was the system overview diagram (see Figure 2), which depicted how the various technologies would be integrated into an overall system. It was this diagram that summarized CARTA's long-term vision for its ITS program. This diagram identified the following as existing technologies deployed at CARTA:

Complex diagram depicting the data interactions on CARTA's ITS system.
Figure 2. CARTA 2005 ITS System Overview Diagram

This diagram also depicted two government-owned systems with which CARTA's ITS would integrate - the city traffic signal (for transit signal priority) and the GPS (for AVL). The other elements of this diagram identified CARTA's long-term vision for its ITS program.

3.1.2 Planning the Deployments

The System Overview Update also defined the plan for deploying the technologies it described. It grouped the technologies into a collection of individual deployment projects and defined the general strategy for the deployment procurements. In April 2004, CARTA identified the sequence in which the ITS projects would be deployed, and also identified infrastructure enhancements (e.g., network upgrades) needed to support the projects. CARTA established the deployment schedule for these projects in May 2004, along with the procurement procedures they would follow in pursuing these deployments. CARTA also began updating the System Overview Update report annually, describing the existing and planned ITS projects.

CARTA also developed an ITS implementation schedule in 2005 and integrated this schedule into the System Overview document, beginning in 2006.

3.1.3 Deploying Technologies

CARTA's first step in its ITS program was to upgrade its infrastructure to prepare for the ITS deployments. This included deploying the CARTA Data Warehouse, deploying new timekeeping and payroll software, and integrating the new software with the data warehouse. With each deployment, CARTA developed a project deployment plan (refer to Figure 2), which included the following non-technical topics:

The technical topics related to each project were addressed through a series of standard systems engineering documents - a specifications document, a design review, test procedures, test results, etc.

3.1.4 Maintaining the Vision

As technologies were deployed, CARTA's vision for its ITS program evolved. As a result, CARTA updated its System Overview annually to reflect (a) technologies that were already deployed and (b) changes to the technologies that were planned for deployment. Maintaining this document helped CARTA ensure that its vision for ITS remained consistent with evolving technologies and that its ITS deployment activities remained consistent with its vision for ITS.

3.2 CARTA'S SYSTEMS ENGINEERING PROCESS AND THE "V" MODEL

FHWA has noted that the "V" model, as shown in Figure 3, is a standard way to represent systems engineering for ITS projects.

Diagram showing the elements of the system engineering process in the shape of a 'V.'
Figure 3. The Systems Engineering "V" Diagram

CARTA followed this general approach, but tailored the approach as indicated in Figure 2 to better suit the scale of their organization and the incremental approach used to develop the overall ITS program through a sequence of individual project deployments. For example, the subsystem and system verification steps were combined in an overall acceptance testing process based on the modest scope of each individual deployment project. There were also opportunities to involve key CARTA end users throughout the project development and implementation process to validate that the system would meet their needs. The concept of operations was incrementally updated when needed to reflect the specific effects of individual projects as they were deployed.

Diagram shows the elements of a modified systems engineering V diagram that depicts the tailored approach CARTA used.
Figure 4. CARTA's Implementation of the Systems Engineering "V" Diagram

In Figure 4, green shading indicates activities that were conducted for their entire ITS program and blue shading indicates activities that were conducted for each project. Purple shading indicates activities performed primarily by project implementation contractors in a collaborative approach involving CARTA review and feedback.

CARTA began its process with a review of the existing Regional ITS Architecture. This included a review of CARTA operations/organization/infrastructure to help identify needs that could be addressed through the deployment of proven ITS technologies and related these needs to the User Services and Market Packages in the ITS Architecture. These efforts helped CARTA explore and define an overall ITS program vision - a system concept overview for how existing and additional technologies could be best integrated to address the agency's needs and situation over the course of developing the overall ITS program. This system overview document defined the ongoing and near-term procurement packages and provided general descriptions of medium- and long-term plans for procurement packages. It also served as both a concept of operations for CARTA ITS and a planning document.

As individual project procurements were being initiated, CARTA developed a project deployment plan. These plans defined the scope of the individual projects and helped CARTA prepare to successfully transition each project into revenue service once the deployment was completed. It addressed aspects such as organizational impacts, operations and maintenance, monitoring and evaluation, and outreach. The project requirements were documented in the procurement package for the project, including an acceptance matrix that served as the basis for the design review and acceptance testing.

The selected project implementation contractor completed a collaborative design documentation and review process with CARTA, prior to developing and deploying the system hardware, software and integration. The project implementation contractor was responsible for in-house testing prior to the start of formal acceptance testing, which was witnessed by CARTA for each subsystem and the overall system. The testing followed procedures that were collaboratively planned in advance with the vendor so as to formally verify all acceptance matrix requirements.

CARTA also involved its end users throughout the process, in developing procurement specifications, contractor selection, design review, acceptance testing, review of training materials and documentation, and the transition into operations and maintenance. This all served to help validate that the project was being developed and implemented so as to address CARTA's needs. As each project was completed, the System Overview was updated to reflect the post-deployment situation. This provided another form of validation, as the review of CARTA's post-deployment situation helped CARTA assess the extent to which the project had helped it move towards the vision described in that document.

This step and the process of regularly updating the system overview also helped CARTA prepare for eventual changes, upgrades, retirements, and replacements that may be necessary. The process of preparing the annual updates to the system overview will provide CARTA with the opportunity to identify the need for such activities and include those needs in their long-term ITS plans. As with their ITS deployments, CARTA's needs for changes, upgrades, retirements, and replacements to their IT systems would be documented in the system overview and, from there, would follow the path of their ITS deployments.

3.3 KEYS TO CARTA'S SUCCESS

CARTA faced many challenges while deploying ITS technologies to support its operations. Some projects were delayed and others did not meet expectations and required rework. When new opportunities arose, CARTA had to re-plan its deployment schedule to accommodate the opportunities. Despite these challenges, CARTA remained focused on the ITS deployments and continued to deploy ITS technologies that had positive impacts on its operations. This section of the report describes approaches CARTA used to manage its systems engineering processes that helped lead to the success of its ITS deployments.

3.3.1 Developing and Maintaining a Long-Term Vision for ITS

As summarized in the previous section, one of the most important steps CARTA undertook for the ITS program was development of a System Overview Update report in 2004 to document a long-term vision of how CARTA wanted to use ITS. The report included:

The table of contents from this document is shown in Figure 5.

Screenshot of the Table of Contents of CARTA's 2004 System Overview Update.
Figure 5. The System Overview Update Table of Contents

CARTA updated this report annually to reflect changes that occurred in the previous year and plans for the upcoming years, so that this document always provided CARTA with a road map of where the agency was and where it was headed with its ITS program. This provided CARTA with some notable benefits.

Documenting the long-term vision for ITS helps ensure that long lead-time activities are completed in time to support future plans.

First, understanding the long-term goals helped CARTA ensure that all the necessary preliminary activities were completed to support the long-term goals. This was particularly important with regard to long lead-time items with a long lifetime, such as bus purchases. For example, in 2006 CARTA added requirements that bus purchases include multiplex systems to better support the agency's plans for AVM. (The multiplex system allows the AVM to perform enhanced monitoring of more devices than a bus without a multiplex system. The AVM system on older CARTA buses monitors fewer devices than the system on those purchased after 2006.)

Photo of a bus arrival time sign at the University of Tennessee bus stop.
Figure 6. A CARTA Bus Arrival Time Sign

A documented long-term vision also helped CARTA take advantage of short-term opportunities that arose. For example, CARTA was approached by the University of Tennessee at Chattanooga with funding to support the installation of arrival time signs at several bus stops on CARTA routes on the UT Chattanooga campus. At the time that this opportunity became available, CARTA did not have all of the systems in place to support real-time bus arrival time information. Realizing that real-time arrival time information was consistent with its long-term plans, CARTA re-organized its planned deployment activities to fast-track those items needed to support arrival time signs. (More information on this re-organization is in the next section.)

Taking advantage of opportunities that arise may require being flexible regarding the deployment schedule.

3.3.2 Proper Sequencing of ITS Deployments

CARTA also strove to introduce changes incrementally, avoiding the temptation to do too much too fast. Doing too much too fast can increase the deployment risks because problems in developing one system can impact development of a second system dependent on the first. A review of the CARTA deployment schedule indicates that CARTA avoided simultaneously deploying systems that included strong dependencies between them.

Avoid the temptation to do too much too fast.

An exception to this rule occurred in 2008, when CARTA fast-tracked several system elements needed to support the real-time bus arrival time signs. At the time that the University of Tennessee at Chattanooga approached CARTA with the real-time bus arrival time sign opportunity, many system elements necessary to support real-time bus arrival time information were already in place; onboard location and route tracking equipment, ability to load the onboard run databases from Trapeze FX via the WLAN, and network connectivity to the buses. Other needed system elements were not in place, however. For example, the CleverCAD system was going to be used to access bus location archives needed to estimate run times and the real-time bus schedule adherence data needed to estimate arrival times. Also needed were elements from the BusTime system, including the arrival time website server and the signs themselves.

Be willing to accept schedule delays when needed to help manage deployment risks.

So, in this particular case, CARTA elected to deploy the BusTime system, the arrival time website server, and the bus stop dynamic message signs in 2008 despite the original intention to deploy these elements after the CAD AVL software deployment was well underway. To offset the risks from accelerating the deployment on specific elements of the program, CARTA deferred the rollout of the AVM and APC and slowed the CAD/ AVL deployment to allow the agency and its contractors to focus on elements needed for the bus arrival time predictions system. Thus, even when CARTA needed to adjust the schedule to accelerate some elements, it simultaneously accepted delays on other elements to help offset the risks of simultaneous deployments.

Using a data warehouse can reduce overall system complexity by reducing the number of systems that must be integrated.

The early deployment of the data warehouse provides another example of sequencing projects to reduce complexity. Early in its ITS deployment process, CARTA recognized the need to integrate data from different sources to support CARTA operations. An example of this is integrating fuel and maintenance cost data to report on the total cost of vehicle operations in order to identify the most cost-effective vehicles. One approach for doing this would have been to interface each of the ITS applications with every other ITS application for which integration was needed. Deploying a data warehouse allowed CARTA to achieve the same capabilities by integrating each ITS application with only a single other application – the data warehouse.

3.3.3 Manage the Limited IT Resources

CARTA's IT resources were limited – the agency had a Manager of Technology who was responsible for most IT functions at CARTA, from maintaining desktop computers to overseeing ITS deployments. With limited IT resources, CARTA took steps to control the maintenance burden that would be placed on the Manager of Technology as systems were brought online. The following list summarizes some of the steps CARTA used to help ensure that the Manager of Technology did not become overburdened as new ITS applications were deployed.

Using virtual servers and ensuring that all applications use a single database engine can reduce time required to maintain the IT infrastructure.

Taken together, these approaches allowed CARTA to effectively deploy and maintain its ITS technologies with the limited IT support staff that was available.

3.3.4 Involving Stakeholders

CARTA recognized the importance of stakeholder involvement early in its ITS program. One of the most vital phases in each ITS deployment was development of a project-specific deployment plan, with the last chapter of each plan devoted to outreach. In general, the outreach chapter for each project addressed the following three stakeholder groups:

Including a broad range of stakeholders, including those skeptical of ITS, can improve the final product and its level of acceptance.

A good example of CARTA's concern for stakeholder involvement was the formation of ITS Oversight Committees to provide feedback on plans for the deployment of onboard systems, which included paid time for employees to participate in committee meetings. These committees included union representatives and many drivers, as well as several employees who had preconceived negative opinions regarding these particular technologies. These committees provided useful recommendations regarding the ITS rollout, and this approach also resulted in better buy-in for the ITS deployments among those employees.

Photo of a CARTA staff person developing a schedule.
Figure 7. CARTA Scheduler at Work

Another example of CARTA being sensitive to user needs was the change in plans regarding the timing for introducing the flex-route operations management software. CARTA originally planned to begin using the software to support these services in 2006, in conjunction with the rollout of new operations management software for fixed-route and paratransit operations. However, feedback from flex-route dispatchers indicated that the planned system could not efficiently support dispatch services until an overall CAD/AVL system was available to provide real-time bus location information. In response to these concerns, CARTA delayed this deployment and made plans to include acquisition of alternative flex-route operations management software incorporating additional software features to better satisfy user needs.

3.3.5 Planning for Demonstrations of Success

Another facet of managing stakeholders is planning projects so that successful demonstrations and observable benefits occur regularly. These demonstrations can help maintain program momentum in two ways. First, they build enthusiasm for ITS by regularly providing benefits to stakeholders - stakeholders that are benefiting from ITS are more likely to continue to support it. Second, they help build confidence that ITS will be deployed successfully and, once deployed, will generate benefits. With this in mind, CARTA chose as its first ITS projects those that could be completed in a relatively short time and would generate easy-to-see benefits:

Search for easy-to-implement projects that will produce easy-to-see benefits.

CARTA's ITS staff were aware of the need to demonstrate the benefits being achieved through the ITS technologies being deployed. The project-specific deployment plans included a section on monitoring and evaluation that described the types of benefits anticipated and how those benefits might be identified. And, CARTA ITS staff monitored the systems to identify benefits. For example, when the Evaluation Team discussed free Wi-Fi usage with CARTA, CARTA staff accessed monitoring screens that allowed them to identify the number of users presently accessing the Wi-Fi system.

3.3.6 Evaluation and Testing

Another feature of CARTA's systems engineering approach was a strong commitment to thoroughly testing the ITS technologies before introducing them to support operations. CARTA's general approach worked as follows. All the requirements for a system were documented in an acceptance matrix. This matrix included a requirement ID, the requirement description and amendments to that description, the requirement status, and any remarks related to the requirement and the verification and validation performed. For example, the acceptance matrix for the CAD/AVL deployment was maintained in an ExcelŽ worksheet that included nearly 800 lines of requirements information.

The deployments themselves were divided up into a series of incremental demonstrations, with each demonstration exhibiting more of the final functionality required. Before each demonstration, the contractor was required to submit for CARTA approval an acceptance test procedure that identified the requirements that would be met by that demonstration and the method used to verify the requirements were met. CARTA and the contractor would follow the test procedures during the demonstration, and CARTA would update the requirements matrix with information about the results of the testing and the status of each requirement tested. This approach provided a mechanism for (a) ensuring that all requirements were satisfactorily met and (b) monitoring progress towards meeting the requirements.

But having a mechanism for testing is not sufficient without a commitment to thorough testing, and there were numerous examples that demonstrated CARTA's commitment to thorough testing. First was CARTA's recognition of the importance of testing. During discussions with the evaluators, CARTA identified testing as an important part of their system engineering process. Second was CARTA's willingness, if necessary, to change plans based on testing results. For example, during testing of the new operations software for its flex-route operations, CARTA determined that the planned new software would not provide the expected benefits. So, CARTA stopped the introduction of that software, selected an alternative that would better meet operational needs, and rescheduled the deployment to take place after the CAD/AVL system was in place.

Commit to thoroughly testing systems before introducing them to operations.

An example of the thoroughness of CARTA's testing is related to the rollout of the next-stop announcement system in 2008. In late 2008, CARTA completed the onboard database and was capable of activating onboard audio and dynamic message sign (DMS) announcements. Rather than proceed with operations after verifying the general functionality of the system, CARTA elected to conduct a comprehensive test of the system's functionality for of their routes. CARTA, using a test bus set up with the test database, drove each route variation in the system with an observer onboard to note any errors in the messages displayed and announcements made. During this testing, CARTA discovered and corrected several errors in the system, including incorrect announcements and announcements triggered at incorrect locations.

This end-use testing also allowed CARTA to identify enhancements to the system that would improve its operation. The system was originally configured to display a stop announcement then display the date/time on the interior DMS prior to displaying information about the next stop. During testing, CARTA discovered that the time spent displaying the date/time sometimes delayed the next-stop display, particularly when stops were closely spaced. Thus, performing end-use testing of the system prevented exposing the public to these errors, which could have reduced public confidence. It also enabled CARTA to identify ways to improve the way the system operated.

The 2009 roll out of the AVM system provided another example of CARTA's commitment to testing. The core infrastructure needed to support AVM – the onboard equipment, the WLAN for daily data uploads, and the AVM software – was in place early in 2008. However, it had been decided to focus 2008 ITS resources on implementing systems that would deliver the most direct and visible benefits most directly and visibly to riders, such as the next-stop announcements and the next-arrival-time predictions. After these systems came online in December 2008, CARTA shifted focus to rolling out AVM. By January 2009, the AVM system elements were integrated and the AVM system was receiving daily uploads of data from the buses. CARTA then conducted internal testing of the AVM system to confirm it was operating correctly before releasing it for use by the mechanics in March 2009.

Initially, some contractors were resistant to the extra time CARTA required for testing. This resistance was typically overcome during the initial subsystem implementations, when CARTA first applied its thorough testing processes with that contractor. CARTA required that test procedures fully demonstrate all the specified requirements for the systems and that these systems be planned in advance. CARTA reinforced its commitment to testing by refusing to accept systems that did not pass the documented test procedures. After working with CARTA's testing process, most contractors recognized that this approach, while it might delay final acceptance, helped prevent errors from being found after the system was in use when corrections are more difficult and costly to make.

Previous | Next