PART II: FINDINGS
Part II of this evaluation report addresses the findings of the NT-T/SP test.
1. BACKGROUND
This section of Part II provides background on the project, describes the applications involved in the test, and gives background on the evaluation including information on the evaluation objectives, hypotheses, measures of effectiveness, and activities.
1.1 NETWORKED TRAVELER – TRANSIT/SMART PARKING TEST
The Networked Traveler – Transit / Smart Parking (NT-T/SP) test consists of real-time transit information and trip planner applications that serve travelers along the US-101 corridor in San Francisco, California. While the original scope of the test was to only provide parking and transit information for trips along the US-101 corridor, it was later expanded to include most transit agencies throughout the San Francisco Bay Area. The Bay Area is an ideal test bed for understanding what type of information is useful to travelers. It is an area of significant population density, traffic congestion, and alternate transportation options, especially along the US-101 corridor which provides access to San Francisco from other major metropolitan areas such as San Mateo, Palo Alto, and San Jose. The primary purpose of the test is to provide information to travelers in real-time across all transit agencies that serve the US-101 corridor. This "one-stop shop" for traveler information was designed to help travelers make better pre-trip planning decisions in terms of mode selection and to serve as a tool for planning transit trips from any origin to any destination considering all transit options available. The NT-T/SP test is unique in that it integrates a broad range of real-time multi-modal transportation information, including transit and parking information. In fact, the multi-modal trip planner website is the first to integrate real-time information for most transit agencies in the Bay Area.
The NT-T/SP test was developed by the California Partners for Advanced Transit and Highways (PATH)4 and the University of California at Berkeley in partnership with the California Department of Transportation (Caltrans). PATH is administered by the UC Berkeley Institute of Transportation Studies, with a mission to develop solutions to the problems of California's surface transportation systems through cutting edge research.
It is important to note that this test was re-scoped towards the end of 2009 to address concerns on distracted driving. Previously, PATH had planned to integrate real time traffic, transit, and parking information to compare the fastest mode from the drivers' locations to their destinations during congested conditions. In situations where Caltrain, the commuter rail service in the US-101 corridor, was projected to be faster than driving, even after considering the time required to divert to a station and park, PATH planned to make mode shift alerts available to drivers through their dash-mounted smart phones. The mode shift alerts would have directed drivers to a specific train at a nearby railway station where real time parking status information indicated if parking spaces were available. Having switched to rail mode, the system would continue to provide real time transit status and connection information to drivers' destinations. After receiving an alert, drivers could send immediate feedback to PATH on the value of the alert by responding to simple questions on their smart phones.
The re-scoping of the test resulted in a greater focus on pre-trip information for drivers and real time transit information via the smart phone application versus "en route" mode-shift alerts that could be a distraction to drivers. Alerts on transit status are provided by the smart phone application, but a technique referred to as "geo-fencing" was introduced to prevent the use of the smart phone application while driving. Traditional geo-fencing typically intends to provide alerts when individuals or assets deviate from a pre-defined route, zone, or time period while other smart phone applications attempt to prevent use of mobile phones while driving. Instead, PATH developed a geo-fencing technique which attempted to determine which mode the traveler was using, in order to allow transit users to continue to receive updates while on the move but prevent drivers from using the information while driving. In this way, the potential for distraction to the driver is substantially reduced, while service is maintained to transit riders.
With the support of several regional partners, PATH developed a set of web-based and smart phone-based applications branded as "PATH2Go." PATH made the applications available via its "Networked Traveler" project website5 where users can access the multi-modal trip planner website, the transit smart phone application, and the transit and parking traveler information website. For the purposes of the evaluation and for overall clarity, the evaluation team is using the project team designated names for the two web-based applications and the smart phone application. These are the Trip Planner, the Web Based Traveler Information, and the Smart Phone Application, respectively, which together make up the suite of tools branded PATH2Go. The PATH2Go applications provide a variety of travel and transit information across most Bay Area transit agencies. The transit agencies available in all three of the PATH2Go applications are:
- Caltrain,
- SF Muni,
- VTA,
- BART, and
- SamTrans
The Networked Traveler website also provided detailed information about what each of the applications has to offer. Visitors to the project website had the option to register as a user by signing up and creating an account. Users were required to register in order to download the PATH2Go Smart Phone Application, but could use either of the web-based applications with or without registering. Users who created an account were asked to create a user name and password as well as fill out a short, optional survey with demographic, travel pattern, and marketing questions. The user registration survey is available in Appendix A.
1.1.1 PATH2Go Trip Planner
The PATH2Go Trip Planner website6 provides multi-modal trip planning information for travel along the US-101 corridor in the San Francisco Bay Area. Users accessing the website can plan a trip by entering an origin and destination in the form of a physical address, landmark, transit station/stop, or any other location recognizable by the system, which uses a combination of GoogleŽ Map API and its own transit database developed by PATH.
In return, the system provides detailed information about the different transportation options available for the trip, including transit only, driving to transit, driving only, and bicycling. Users can compare itineraries for several different transportation mode options with the ability to make a decision based on time, cost, and carbon emissions. For each trip option available, the system provides the total time required to complete the trip, the estimated total cost of the trip, and the expected carbon emissions or savings as the result of the trip; allowing travelers to make a decision based on which factor is most important to them for that trip. Figure 1 1 below is a screenshot of the output presented on the PATH2Go Trip Planner website after an example trip from Palo Alto, CA to San Francisco, CA has been planned. As shown below, the trip summary information allows users to consider the cost/benefit of choosing one option compared to another. For driving to transit trips, the system provides the estimated cost of fuel for the driving leg of the trip, the actual cost of parking when the user will transfer to transit, and the actual fare for the transit leg of the trip. Users can even enter their vehicle's fuel efficiency to update their itinerary with a more accurate estimate of fuel consumption. To help users consider the potential savings of considering alternate modes, the system displays a graphical representation of work/relax time versus driving time and driving emissions versus emissions saved by taking public transit for each trip option available.
Figure 1-1. Screenshot of PATH2Go Trip Planner Website.
The system also allows users to view the detailed itinerary specific to each trip option by clicking on any of the trip options in the trip summary. For example, if a user clicks on a bicycling trip, the system will provide turn-by-turn directions for that trip. Alternately, clicking on the transit trip will provide users with detailed information about the start and end transit station/stop and any transfer points as well as real-time arrival and departure times for each leg of the trip. As shown in the figure above, a window with detailed travel times for the route selected as well as additional departure times from that location in case the user misses the first train/bus is displayed when a user selects a specific leg of any transit trip.
While the functionality presented above is accessible to non-registered users, there are added benefits to being a registered user when using the trip planner website in conjunction with the smart phone application. Registered users can plan a trip using the website and choose to send the itinerary for that trip to their smart phone in order to use the smart phone application functionality for the transit portion of the trip. With access to real-time arrival and departure times on-the-go using their smart phone, users can potentially reduce wait-time at a transit stop/station. Other additional functionality for all users of the trip planner website includes the ability to view color-coded traffic congestion information on roadways where its available using the Google Maps API or the same information on El Camino Real specifically using sensor data provided by Caltrans. The congestion information allows users to assess traffic conditions on the roadways used for the driving or bus segments of a planned trip.
The PATH2Go Trip Planner and PATH2Go Web Based Traveler Information applications access information from two different servers, but both are actually hosted on the same website. Although different URLs are available to navigate to each specifically, the applications are accessible from each other by selecting between the two tabs available, the "Plan a trip" tab and the "Transit Routes" tab, respectively. A trip planner server stores all of the information needed to plan a trip on the PATH2Go Trip Planner website and the PATH2Go Smart Phone Application. A traveler information server stores all of the real-time transit information accessed by all three of the applications.
1.1.2 PATH2Go Web Based Traveler Information
The PATH2Go Web Based Traveler Information website7 provides real-time, detailed transit information for all Bay Area transit agencies that serve the US-101 corridor. The website provides a variety of transit information to travelers for train, commuter rail, light rail, BRT, and bus routes including real-time schedule information for most transit agencies in the Bay Area.
The primary purpose of the PATH2Go Web Based Traveler Information website is to provide real-time arrival and departure times for all transit routes where available as well as real-time parking availability information for several instrumented Caltrain lots along the US-101 corridor. This allows transit riders to browse transit options and plan trips across several providers versus going to each transit agency website individually. Users can view real-time information versus just schedule information, which could impact user perception of the convenience of riding transit.
In order to implement this type of information into one system, PATH had to work closely with the transit agencies involved to establish a constant stream of up-to-date schedule information, route information, and changes in service. While SF Muni and BART provide open source data to the general public, the other transit agencies involved either did not provide the data to the public or did not have it available. PATH put forth a considerable effort to coordinate retrieving this data or gathering this data themselves. Some of their efforts included implementing GPS trackers on trains and buses where not available, building route shapes to identify bus routes and stations/stops, and creating algorithms to estimate real-time arrivals and departures based on real-time and historical data. Where real-time arrival and departure times were not available, automatic vehicle locator (AVL) data was not available, or GPS trackers could not be installed; the project team used normal schedule information to feed the traveler information server. Table 1-1 below presents the source for real-time tracking information collected by PATH for each of the transit agencies involved in the test.
| Transit Agency | Data Source | Notes |
|---|---|---|
| Caltrain | GPS devices installed by PATH. | PATH installed GPS devices on the full fleet of Caltrain locomotives to allow for real-time tracking. |
| SF Muni | Open Source. | SF Muni provides open source data for real-time arrival and departures times across the whole system. |
| VTA | GPS devices installed by PATH or schedule information. | PATH installed GPS devices on the full fleet of 22 buses on the VTA 522 route in Palo Alto, which is a high use bus-rapid transit route. For all other VTA routes, PATH used static schedule information. |
| BART | Open Source. | BART provides open source data for real-time arrival and departures times across the whole system. |
| SamTrans | GPS devices installed by PATH and SamTrans AVL data or schedule information. | PATH installed GPS devices on 15 SamTrans buses. For all other SamTrans routes, PATH used AVL data or schedule information. |
Unlike the trip planner website which seeks to connect users from origin to destination using transit or other modal options, the traveler information website allows users to do the trip planning themselves by searching or browsing all transit routes available from the transit agencies covered by the system. When accessing the website, users have the option to select a specific transit agency and route from drop-down menus, use the search bar to find a specific transit route, search for a transit route using a nearby address, or select from a list of recently viewed routes. All transit routes and stations/stops provided by each transit agency are available using the search bar or drop-down menus.
Additionally, the website offers parking lot capacity or real-time parking availability information for Caltrain Park-and-Ride lots to travelers who use Caltrain. As a test, four parking lots along the Caltrain route were instrumented with sensors to count the number of vehicles entering and exiting the lot in order to provide the available number of parking spots in real-time. Figure 1-2 below shows a 3G transmitter at a lot in Redwood City that sends information from the PATH-installed sensors to its server.
Figure 1-2. Caltrain Smart Parking 3G Transmitter in Redwood City, CA.
Photo Courtesy of PATH.
Many of the Caltrain parking lots along this busy commuter corridor are generally at or near capacity on any given day. To best serve commuters who may consider taking an alternative mode by parking their car and riding transit for the remainder of their trip, the PATH team did a survey of all Caltrain parking lots in September 2008. The study showed that about half of all Caltrain parking facilities operate at or near capacity during the middle of the day. However, the team identified ten lots that had excess parking available and decided to instrument four of these lots at Caltrain stations with hopes to advertise the excess parking to commuters who are willing to switch modes. Table 1-2 below was provided by PATH and presents the details of the instrumented lots.
| Station | Lot | Technology | Instrumented By | Notes |
|---|---|---|---|---|
| Millbrae | Caltrain Lot | Sensys sensors | PATH | 182 spots total |
| Redwood City | Surface Lot | Sensys sensors | PATH | 157 spots total |
| Redwood City | Underground Lot | Aldis Video Camera | ParkingCarma | 515 spots total |
| Menlo Park | North Lot | Sensys sensors | PATH |  |
| Menlo Park | South Lot | Sensys sensors | PATH | |
| Palo Alto | West Lot | Sensys sensors | PATH | |
Once a user has identified a transit agency and a transit route, the website displays a full list of the stops along that route. Figure 1 3 below presents a screen shot of the PATH2Go Web Based Traveler Information website with Caltrain information including real-time arrivals and departures for the South San Francisco station as well as the list of all stations along the route with parking availability at each. A user can use the list of station/stop names or the map to select a specific stop and view all upcoming arrival and departures times for that stop in real-time. Because most transit trips have a direction associated with the route, users can also choose to reverse the route to view the schedule information in the opposite direction. Additionally, users can elect to view the real-time information or view the route schedule for the entire day by selecting the "Arrivals for now" or "Schedules for all day" buttons, respectively. As shown in the figure, the instrumented lots display the total number of parking spaces available at that location in real-time while other lots simply list total capacity. The times highlighted yellow in the South San Francisco station window represent "Baby Bullet" or express trains that only stop at specific stations to lower long-distance commute times.
Figure 1-3. Screenshot of the PATH2Go Web Based Traveler Information Website.
As with the PATH2Go Trip Planner website, there are added benefits to being a registered user when using the traveler information website in conjunction with the smart phone application. Registered users can set-up alerts to be sent to their smart phone application on a regular basis. By identifying specific transit routes, users can view real-time schedule information for that transit trip on their smart phone for a specific time of day (i.e., a user's morning or evening commute). With access to real-time arrival and departure times on-the-go, users can potentially reduce wait-time at a transit stop/station. Other functionality available to all users of the traveler information website includes the ability to view color-coded traffic congestion information on roadways where its available using the Google Maps API or the same information on El Camino Real specifically using sensor data provided by Caltrans. The congestion information allows users to assess traffic conditions on the roadways they use to travel to transit stops/stations. Users can also access a list of nearby connecting transit routes from the same or other transit agencies as well as directions to or from any transit stop/station using the trip planner website at the click of a button.
1.1.3 PATH2Go Smart Phone Application
The PATH2Go Smart Phone Application is a multi-agency transit trip planner that provides real-time arrival and departure information and user location-based trip details for transit available in the Bay Area along the US-101 corridor. The application uses the GPS functionality of smart phones to help users identify transit options and guide them step-by-step through a planned transit trip itinerary. The primary purpose of the application is to provide real-time transit information across multiple transit agencies on-the-go.
The application allows users to view real-time transit schedules, plan transit trips by selecting nearby stops (based on their current GPS location), or create itineraries by providing origin and destination transit stops or locations. Users can select their current location or enter an origin and destination in the form of a physical address, landmark, transit station/stop, or any other location recognizable by the system, which uses a combination of GoogleŽ Map API and its own transit database developed by PATH. As mentioned before, the smart phone application uses both the trip planner and traveler information servers developed by PATH to compile information and present it to the user.
The PATH2Go Smart Phone Application was available for download on smart phones with an iPhone, Android, or Windows Mobile software platform. iPhone and Android smart phone users can download the application named "PATH2Go" from the Apple iTunes App Store or Google Android Market, respectively. Windows Mobile users have to download the application from PATH's Networked Traveler website after registering as a user. Although iPhone and Android users can technically download the application before registering, all smart phone application users have to register on the Networked Traveler website before they can log-in to the smart phone application (see Figure 1-4). Once registered, users can begin sending transit trips planned with the PATH2Go Trip Planner website or the PATH2Go Web Based Traveler Information website to the smart phone application. Smart phone users receive a notification within the application when it was time for the first leg of the planned trip to begin. Functionality between the website applications and the smart phone application gives users the benefit of working on a computer (i.e., bigger screen and faster internet speeds) while planning a trip or exploring transit options as well as the benefit of taking the detailed itinerary with them to reference throughout their transit trip via their smart phone.
Figure 1-4. PATH2Go Smart Phone Application Login Screen on Apple iPhone.
In addition to the compatibility between the websites and smart phone application, users can also use the smart phone application on its own to plan trips or view transit itineraries. Although the functionality and design of the smart phone application varies across the different software platforms, the basic information provided by the application is essentially the same for each. Figure 1-5 below shows the PATH2Go Smart Phone Application on a Windows Mobile phone, Android phone, and iPhone (from left to right, respectively).
Figure 1-5. PATH2Go Smart Phone Application.
Photo courtesy of PATH.
Using the Apple iPhone as an example, Figure 1-6 below shows screenshots of the PATH2Go Smart Phone Application user interface and the steps required to plan a transit trip on the application. The first screenshot on the left shows the default screen displayed after a user logs into the application. The application uses the smart phone's GPS capabilities to identify the user location when it is opened, which is represented by the green pin in the screenshot. Next, the user has three options for choosing a trip origin: select a nearby stop identified using the user location, choose from previous locations selected, or select an origin from the list of transit stops/stations by transit agency. The middle screenshot shows the interface for selecting a transit stop/station from the list. Users also have the option to search for a specific stop/station using the search bar. After a user has identified an origin for their transit trip, they can then choose a destination by using their trip history, by selecting from the list of transit stops/stations by transit agency, or by typing a physical address or landmark into an address bar. Once an origin and destination are established, the user can view the planned itinerary before finalizing the trip. The screenshot on the far right shows the resulting view after a trip has been planned and submitted. The application displays the first segment of the transit trip planned and provides real-time arrival and departure times. In addition to the schedule information, the map on the screen displays the current user location, the trip origin, the trip destination, and the current location of the bus/train using color-coded pins. The application automatically displays the first segment of the transit trip, but the user can scroll through any additional segments in the trip by swiping the top portion of the screen to the left. The three dots below the top portion of the screen indicate that there are three segments to the current planned trip. From this screen, a user can follow their trip progress throughout the entire trip as their current location pin updates automatically and moves along the route highlighted in green on the map. Once a trip is planned, the application begins providing alerts to the user when their train/bus is approaching their initial boarding stop and as the train/bus approaches their next stop and/or destination. Alerts are provided for each trip segment and schedule and location information is updated constantly throughout the planned trip.
Figure 1-6. Apple iPhone User Interface for the PATH2Go Smart Phone Application.
Although the application is designed for use while traveling on transit, it is possible that many users access transit by driving and parking at or near a transit stop/station. As mentioned above, geo-fencing functionality was implemented into the PATH2Go Smart Phone Application design when the project was re-scoped to discourage use of the application while driving. The geo-fencing design compares transit route information stored on the traveler information server with speed and location data provided by the smart phone GPS to determine whether a user is traveling along a planned transit route or driving a vehicle. If the system determines that a user is driving, a warning message that reads "Warning! Application Disabled While Driving" appears and blocks users from the information provided on the application (see Figure 1-7). To better understand the effectiveness of the geo-fencing functionality in preventing distracted driving, the evaluation team conducted a test of the geo-fencing design. The geo-fencing design and evaluation activities are presented in greater detail in the following section of the report.
Figure 1-7. PATH2Go Smart Phone Application – Distracted Driving
Alert.
Photo courtesy of PATH.
1.2 EVALUATION APPROACH
The evaluation approach was driven by a series of objectives that align with the USDOT's goals for the SafeTrip-21 initiative – see Table 1-3.
| SafeTrip-21 Goal | Evaluation Objectives | Hypothesis |
|---|---|---|
| Expand / accelerate research in vehicle connectivity with the wireless communications environment | Observe the consumer response to the NT-T/SP application | Use of smart phones for delivery of transit and mode shift information will be adopted, accepted, and will expand over time. |
| Build on ITS research | Understand the technical and institutional issues associated with distributing multi-modal information to smart phone users | Lessons learned through the development of the NT-T/SP application will build on current knowledge / understanding of the use of smart phones for providing in-vehicle mode shift alerts and en-route transit information. |
| Build on ITS research | Test the ability of geo-fencing as a method to prevent distracted driving | Geo-fencing is an effective means to prevent the use of mobile devices while driving |
| Build on ITS research | Understand the development process and institutional issues associated with implementing a server-based geo-fencing method versus a client-based method on mobile devices | A server-based geo-fencing method is efficient and effective, but has limitations in user capacity and only limits mobile device use at the application-level |
| Explore / validate benefits of real-time, smart phone mode shift alerts and transit information | Measure usage of the NT-T/SP application | Usage of the NT-T/SP application will be an indication of user acceptance. Usage will increase over time. |
| Explore / validate benefits of real-time, smart phone mode shift alerts and transit information | Analyze the perceived accuracy and usefulness of mode shift alerts and en-route transit information | Mode shift alerts and en-route transit information will be perceived to be accurate and useful. |
Each objective was supported by corresponding hypotheses and measures of effectiveness, which in turn were used to identify specific data sources for the key activities for the evaluation. These data sources provide a detailed bank of knowledge relevant to the application, and a comprehensive look at lessons learned and the success of NT-T/SP. To achieve the objectives above, the evaluation team implemented the following key activities:
- Documented performance of the geo-fencing capabilities of the PATH2GO Smart Phone Application by conducting test scenarios across all transit agencies with various smart phones - discussed in Part II, Section 2;
- Analyzed usage statistics using information provided by PATH – discussed in Part II, Section 3;
- Analyzed ratings of the attributes (timeliness, accuracy, usefulness, acceptance, etc.) using information provided by registered users – discussed in Part II, Section 4; and
- Conducted interviews with deployment and operational partners – discussed in Part II, Section 5.
4 Link to the PATH's website: http://www.path.berkeley.edu [ Return to note 4. ]
5 Link to the Networked Traveler project website: http://www.networkedtraveler.org/ [ Return to note 5. ]
6 Link to the PATH2Go Trip Planner website: http://tlab.path.berkeley.edu/dpiVII/?p=true [ Return to note 6. ]
7 Link to the PATH2Go Web Based Traveler Information website: http://tlab.path.berkeley.edu/dpiVII/?p=false [ Return to note 7. ]