5. Summary of Findings

The previous chapter explained in detail the results of the evaluation activities of the CRRAFT system. Table 5-1 shows the key findings for each of the hypothesis developed during the early stages of this project. Overall, the CRRAFT system seems to have had a more positive impact on the NMDOT's PTPB than on the transit agencies. Also, those agencies with higher ridership and demand-responsive service were found more likely to find it unsuccessful.

Transit agencies generally agreed that the CRRAFT system is useful for tracking ridership and generating invoices/reports for submission to NMDOT. However, the CRRAFT system (which, at the time of this evaluation did not have the IC Transit Smartcard operational) has presented several obstacles to complete acceptance of the system by transit agencies. Although about half of the transit agencies thought that overall CRRAFT was successful, transit agencies that provided a large number of demand responsive trips tended to be dissatisfied with the overall success. Based on our surveys and interviews it appears that much of the dissatisfaction is related to usability issues with the most commonly mentioned issue being the time required to manually enter trips into the scheduler. In addition to improving the scheduler module, other desired features included improvements to allow transit agencies to query their data and develop custom reports.

Table 5-1. Summary of Findings
Number	Hypothesis	Findings
1*	Use of the system saves transit providers time invoicing and reporting to funding agencies	Not True. On average, the use of CRRAFT has not saved transit providers time invoicing and reporting to the PTPB. In fact, Transit agencies with higher ridership and demand responsive service may have had the opposite experience and are spending more time preparing invoices after the implementation of CRRAFT.
2*	Use of the system results in funding agencies having faster access to reports	Not True. On average, the use of CRRAFT has not resulted in funding agencies having faster access to invoices and reports. With the online system however, funding agencies may be able to monitor the numbers that transit agencies are entering into the system along the month.
3*	Reports created by the system are accurate and reliable. Use of the system reduces the time funding agencies spend checking and correcting reports and reduces money incorrectly allocated or invoiced	True. The use of CRRAFT has resulted in more accurate invoices and has saved time from funding agencies during the reviewing process. The fact that transit agencies know at all times their remaining balance in each line item seems to have helped reduce the number of incorrect amounts on invoices.
4	Use of the system reduces the time funding agencies spend researching and collecting information	True. The use of CRRAFT has in fact reduced the time funding agencies spend researching and collecting information
5*	Use of the system reduces the overall time required for transit providers to schedule demand response trips	Not True. The use of CRRAFT has increased the time to schedule demand response trips for a majority of transit agencies and the impact is particularly evident for Agencies entering schedule data for many trips.
6	Use of the system results in more efficient schedules for demand response trips	Mixed. For most users CRRAFT did not have a positive impact on the efficiency of the scheduled route or the development and use of the demand response schedule, but may have improved the efficiency for a few smaller transit agencies.
7	Use of the system reduces the number of unauthorized trips	Mixed. CRRAFT did not have a clear and decisive impact on the number of unauthorized trips.
8	Use of the system reduces the number of in-service breakdowns	Little/no impact. CRRAFT did not have an impact on the number of in-service vehicle breakdowns.
9	Use of the system reduces the operating cost of transit services	Mixed. For the providers, CRRAFT may result in higher operational costs for larger transit agencies that enter many demand response trips. However, the data analysis did not provide conclusive results about the relationship of CRRAFT with changes in operating cost alone or operating cost per trip.
10	Use of a Web-based system has minimized the time and cost of deployment, support, and maintenance	Mixed. CRRAFT appears to be useful for generating invoices, supporting auditing activities, but has resulted in many transit agencies doing additional work to use CRRAFT in support of NMDOT reporting/invoicing requirements.
11	Transit providers and funding agencies perceive that the benefits of the system outweigh its costs	Mixed. NMDOT and New Mexico Human Services Department (NMHSD) are generally pleased with the benefits of CRRAFT and generally agree that the benefits outweigh the costs. The transit agencies have mixed views, however larger Agencies, particularly those providing demand response service, were more likely to indicate that CRRAFT has been unsuccessful and that the costs outweigh the benefits.
12	Use of a single system improves communication between diverse agencies	True. For NMDOT, CRRAFT has resulted in better communication and coordination with transit agencies. For transit agencies, communication and coordination remained about the same or better.
Table Notes: * Key hypothesis

Previous | Next