5. Lessons Learned
5.1 Overall
MST learned several lessons regarding deployment process management, vendor management and the significance of adopting a flexible approach while overseeing numerous simultaneous project implementations over the last 7 years. Some of these lessons learned are summarized below.
5.1.1 Process Management
Agencies should "own" the project by having one (or more) project champions and should not solely rely on vendors to conduct the implementation successfully. Agencies should have commitment from management at the highest possible level (e.g., at General Manager level). For example, at MST, both the current General Manager/Chief Executive Officer and his predecessor were actively involved in the ITS Augmentation Project and provided full support and commitment to the deployment of each system within the Project.
Agencies also should appoint a Project Manager that can devote full time to the project.
5.1.2 Staffing
Agencies should be willing to increase the number of staff as needed. For example, MST needed an additional full-time staff member to monitor revenue service throughout the service day using the ACS system. Another example is that MST had to increase the number of shifts in its Communications Center in order to monitor the ACS system adequately.
MST believes that it is critical to recruit "right" staff members for the project implementation and later for operations and maintenance of each system. The recruited staff should have the right talent, interest and should be receptive to new ideas. MST has increased its staff since the technology implementation, but recognizes that it probably needs even more staff. For example, MST added an IT Director position, an IT hardware/software management position, and an ITS technician position. But the agency believes that it also needs a person for performing GIS analysis in the planning area.
MST recognizes that finding the right person to "get the job done" can be a challenge. On the vendor side, MST's experiences suggest that the vendor's project manager should understand the project thoroughly and have a competent project team to support him/her. For example, MST experienced many issues with the configuration of the ACS, and it took considerable time for the project team to figure out the best solution. These issues were the reason, in part, for the late implementation of the system.
5.1.3 Flexibility
Agencies should be flexible in their expectations regarding the benefits that they can achieve from a technology. MST recognizes that sometimes it is a challenge to meet the original expectations due to several issues (e.g., technical failures, operational restrictions, and issues with institutional agreements). Further, agencies should be open to negotiating with vendors to obtain something else in exchange of the technology or component that cannot be delivered (and was identified in the original project scope). For example, MST was able to get additional licenses and spare parts in exchange for the functions that were promised by the vendor and not delivered as part of ACS.
Agencies should maintain a good personal relationship with vendors while, at the same time, being persistent about their expectations. This "good customer" attitude often leads to success for both the vendor and the agency.
5.1.4 Innovation
MST's experience is that innovative and perhaps unconventional systems can save a significant amount of money. For example, MST decided not to implement a traditional automated passenger counting (APC) system, since it was skeptical about the reliability of APC systems in the market. Also, the agency thought it was not cost-effective to install a fully-functional APC system, so it decided to implement an interface for the coach operator to enter the number of boardings at each stop. MST developed this approach internally and was able to use the money that they saved on another project. This innovative solution has provided MST with highly accurate passenger counts.
5.1.5 Implementation Management
MST thinks that an adequate amount of time should be allowed for implementing a technology and should not be rushed.
Further, agencies should ensure that they have the right tools to operate and maintain the system. For example, MST is not able to upload the current route structure into the GIS interface of the ACS system. As a result, the mapping function displays an old route layer, with the current route traces not matching the old route traces. The ACS vendor provided a map interchange program as part of the ACS, but it was never able to provide an accurate display with current route traces.
MST believes that ITS vendors should also analyze how the system would interact with other existing systems while implementing their technologies. This issue of interaction and integration with legacy systems can cause problems in the implementation process. The vendors and agency should also save mission-critical data before any software upgrade. At MST, the ACS vendor did a software upgrade in 2005, which resulted in a significant loss of data. MST was not able to restore that information.
Finally, agencies should be aware of the operations and maintenance (O&M) requirements for each system. This often gets overlooked when the focus of a project is on initial implementation. The recurring costs for operations and maintenance can be a significant financial burden on an agency.
5.1.6 Forward Thinking Approach
MST believes that agencies should be forward thinking. Once a system implementation is complete, they should start thinking about what could be done in the future. The exploration of new technologies should be critical to an agency's strategic plan as technologies change rapidly and the current systems may become obsolete 5 years from the original implementation. For example, MST management will start exploring a new CAD/AVL system that is based on newer technologies. The ACS system has been deployed at MST for only 6 years, but it is based on much older technology.
The following sections describe lessons learned by specific departments based on their experience with deployed systems.
5.2 Planning and Operations
5.2.1 Data Utilization
MST believes that anecdotal information obtained from field supervisors and the operational data logged by the ACS system should be combined together to obtain a complete picture about an event or incident. Prior to the implementation of the ACS, the source of the majority of information was anecdotal. Now, data from the ACS system combined with information from field supervisors and coach operators can be used to make fact-based decisions.
Along with the database information, the ACS system generates control logs that provide information on the chronological sequence of operational events throughout each day. Even though the control log provides information about missed and cancelled trips, these events are not currently logged in the database under separate data columns. Currently, MST staff needs to search the control log using keyword searches (e.g., missed trip) to find the occurrence of these specific events.
5.2.2 Training
MST had some issues with the timing of the training of dispatchers on the ACS system. Training was conducted while the vendor was still trying to resolve problems with the system. Since the system was still in the state of flux, not only was the dispatchers' trust in the system lost, but additional training was needed once the problems were resolved. MST has learned that training should be conducted once the system is fully setup and working reliably.
Agency staff should be provided ample time to learn the system. MST believes that vendors should have direct users of the system more involved in the implementation process, and that agency staff should understand both the front-end (e.g., graphical user interface) and back-end (e.g., database) aspects of the system in order to have comprehensive knowledge of the system.
5.2.3 Implementation Management
Agencies should be patient during the implementation process since it often takes some time for systems to stabilize. MST had a number of "unknowns" at the beginning of the ACS installation. Most of its route surveys were incorrect and needed to be re-done. The survey errors resulted in a loss of data at the beginning of the implementation since the arrival zones were smaller than they should have been. MST adjusted timepoint boundaries (or arrival zones) before 2005 in part to fix the problem.
5.3 Maintenance
MST learned that it needs to make many more inquiries of vendors before contracting with them. In addition, after the implementation, MST learned that it has to be more pro-active and cannot just rely on vendors to maintain a stable operation. For example, one of the selling points of the ACS system for MST was the remote access capability for field supervisors. However, the vendor did not really have that module developed and operational yet. This lack of capability led MST to implement two shifts since there was only one workstation that could be used for monitoring by dispatchers; field supervisors did not have access.
MST has become more cautious while evaluating vendor products, but is willing to embrace a technology if it is satisfying a specific need. The agency thinks that it helped to do some research before buying specific systems, including conducting one or more site-visits to locations where the vendor's product(s) was operational.
5.4 Information Technology
5.4.1 Training
MST did not have any challenges with training per se, but found that getting people to use the system in the manner that it was intended was a big challenge. Further, as mentioned earlier, the timing of training was critical. For example, for the ACS, MST has a "train the trainer" program, but the system was not available for their use after the training. So they lost whatever they had learned during the training since they could not apply what they had learned on a regular basis.
5.4.2 Culture of Change
The "change culture" is very important for implementing technology in an organization. Many staff members are very familiar with the way an older system works, and may not be amenable to accepting new systems. For example, since MST personnel are used to earlier maintenance and financial management systems, it is challenging for them to work with the user interfaces of new systems.
However, the deployment of other ITS systems such as ACS, HASTUS, and DDAM were accepted well in the organization. The primary reason for this was that their implementation resulted in a decrease in the volume of manual effort required by MST employees to perform their functions.
5.4.3 Standardization
MST believes that database and technology platforms should be standardized across agency systems. Hence, they are building other systems around the ACS. The standard platform for all technologies is Microsoft Windows, and SQL Server is used for all databases except for MMS, which uses an Oracle database.
5.5 Safety and Security
5.5.1 Procurement Process
MST believes that agency staff should visit other sites that have already installed systems similar to those that they are considering for deployment. If on-site visits are not possible, agencies can participate in the American Public Transportation Association (APTA) EXPO, in which many of the technology vendors display and can demonstrate their technology. In this forum, agencies can speak with vendor representatives directly and possibly get a "feel" for the systems that they are considering.
Agencies should ensure that they specify the functional requirements and the number of units of hardware and software according to their specific needs. Also, they should be persistent with the vendor to ensure that the system they are purchasing is what was specified. This means that a rigorous implementation management approach should be used (e.g., conducting design reviews, overseeing vendor installations, conducting testing according to the specifications, etc.). The vendor alone should not be relied upon to ensure a successful implementation.
If at all possible, agencies should order system components when they identify a need. Initially, MST did not deploy exterior cameras that would have provided specific views for security monitoring. The agency believes that the lack of these specific cameras cost MST $3 million; if they had had video from those specific views, a costly lawsuit may have been dismissed.
MST's experience with exterior cameras suggests that such footage is especially beneficial in the event of accidents or incidents.
5.5.2 Technology Upgrade
MST recognizes the importance of keeping its systems up to date as much as possible. MST had to upgrade its DVRs twice when the old DVR technologies became obsolete. Also, MST's experience suggests that agencies should ensure the consistency of various system platforms (e.g., DOS or Windows).