2. PATH2GO Smart Phone Application Geo-Fencing Test
As described in the Background section of the report, the PATH2Go Smart Phone Application was outfitted with geo-fencing functionality as the result of a project re-scope to address distracted driving concerns. This section of the report begins with a discussion of the concept of geo-fencing as it relates to the NT-T/SP test, specifically the PATH2Go Smart Phone Application. Next, the geo-fencing design developed by the project team is described in detail. Last, this section of the report presents the evaluation findings of the geo-fencing test conducted by the evaluation team including lessons learned regarding the prevention of distracted driving.
2.1 GEO-FENCING 101
A geo-fence is a virtual perimeter or boundary line for a real-world geographic area. Typically, the act of geo-fencing generates a notification when a location-aware device enters or exits a geo-fenced area, or crosses a geo-fence. Under this definition, geo-fencing does not directly prevent the user of a location-aware device from crossing a geo-fence, or restrict the functioning of the device. Geo-fencing has been available for several years, although not necessarily in a mass market form, e.g. using an on-board GPS device in a truck to track high value freight and to detect the possibility of theft if the truck deviates from a predetermined corridor or route. Past uses of the term geo-fencing referred to tracking while new uses of the term seek to restrict functionality.
Various technologies have emerged more recently that are designed to address the issue of distracted driving, by preventing the use of cell phones while driving. Such technologies rely on the ability to detect the movement of the location-aware device at speeds faster than walking. PATH has prepared an inventory of these technologies, including an overview of each. These technologies are not geo-fencing products, as defined above, in that they do not require a virtual perimeter or boundary to be defined, and they do not generate notifications when such geo-fences are crossed. Instead, their intent is to restrict the normal functionality of a device under certain circumstances, e.g. texting, making calls, emailing while driving. As such, the user of the location-aware device will experience a temporary loss of service when traveling above walking speed.
As required by USDOT, PATH has implemented a technique (also referred to as geo-fencing) to minimize the possibility that a smart phone user can access the mobile application for the NT-T/SP test while driving. PATH's approach, as detailed more fully below, shares some functional similarity with geo-fencing and the distracted driving technologies described in the preceding paragraphs. It too relies on the ability to detect the movement of the location-aware device at speeds faster than walking. It also uses the ability to identify the device's proximity to transit routes and railroads. However PATH's technique uses a more sophisticated approach that attempts to identify the mode of travel involved. In doing so, the NT-T/SP mobile application can (with some exceptions) be accessed on a smart phone while traveling on buses and trains, but not in cars. It is noted that PATH's technique only applies to the NT-T/SP mobile application (PATH2Go Smart Phone Application). It does not otherwise restrict the normal functionality of a mobile device while driving.
The NT-T/SP test redefines the meaning of the term geo-fencing actually restricting use of the application based on user mode of transportation. However, for the purpose of consistent referencing, the term geo-fencing will be used when referring to PATH's method of preventing drivers from using the PATH2Go Smart Phone Application throughout this document.
2.2 GEO-FENCING DESIGN
In order to evaluate how the geo-fencing design functions within the PATH2Go Smart Phone Application, it is important to fully understand the server structure used to develop the NT-T/SP system. The following is a description of how the Smart Phone Application interacts with the NT-T/SP system and how the geo-fencing design functions within that system.
As mentioned above, the primary goal of implementing geo-fencing functionality is to detect driving and prevent use of the application by drivers. Therefore, the geo-fencing design was integrated into the PATH2Go Smart Phone Application, which provides real-time transit information to users on the move. With any application design for smart phones, developers can decide whether to host the code and source information for certain application functionality on the server-side of the application or the client-side of the application. In other words, the decision-making can either take place on servers hosted by the developers or on the smart phone itself.
- Client-based functionality requires wrapping up all of the code and sources into the application download, which can be demanding on the smart phone in terms of application size, processor speed, and battery life depending on complexity.
- With server-based functionality, all of these processes, or decisions, can take place on the system's server and then can be wirelessly communicated to the client, or smart phone application.
The geo-fencing design was integrated into the PATH2Go Smart Phone Application using server-side logic, which allowed for a thin client-side design. Implementing a server side geo-fencing design prevented the design team from having to address differences in the operating systems of Windows Mobile, iPhone, and Android smart phones that could have an effect on geo-fencing performance. Figure 2-1 below was developed by PATH to explain the basic design of geo-fencing into the system.
Figure 2-1. PATH2Go Smart Phone Application - Geo-fencing System Design.
Courtesy: PATH
If PATH had decided to design their geo-fencing functionality using a client-based design, the processes that take place on the NT-T/SP server and the communication with third party services as shown in figure above would all have to be carried out by the smart phone application, which could be quite burdensome on the phone's operating system as mentioned above.
As for the decision process for blocking users from the application, the geo-fencing design uses a combination of GPS data provided by the smart phone and trip information provided by the system server to determine whether or not a user is driving. As mentioned in the Background chapter, the PATH2Go Smart Phone Application displays a warning message to block users from accessing the information on the application when geo-fencing is engaged. The decision whether or not to block users from the application is initially dependent on these two primary factors: 1) the GPS location and speed of the smart phone, and 2) whether a trip has been planned by the user. From there, a number of other thresholds/constraints and decision processes are assessed to determine whether a user is riding transit or driving a vehicle. Figure 2-2 shows a simplified version of the decision process used to determine whether or not to display the distracted driving message.
Figure 2-2. Geo-fencing Design Decision Process.
The geo-fencing design is driven by a set of thresholds and/or constraints at each step in the decision process presented in the figure above. The GPS speed and location serve as a primary source for threshold and constraint checks while the application is being used. When the application is open, the status of the geo-fencing is continuously checked and updated using the decision process above until certain conditions are met. The rate at which the geo-fencing status is updated is dependent on the smart phone model. The Android platform updates the geo-fencing status every thirty seconds while the iPhone and Windows Mobile platforms update every second and every five seconds, respectively.
The primary constraint controlling the geo-fencing status initially is the GPS location and speed of the smart phone. If the data is not available from the smart phone, the server assumes the user is out of satellite range and most likely not driving a vehicle realizing that there are exceptions (i.e., tunnels, satellites blocked by skyscrapers). In this situation, the user can access the application (Condition A). If the GPS location and speed are available, the server logic looks next at whether or not a trip has been planned. If a trip has not been planned, the geo-fencing status is dependent on the speed of the smart phone. If it is determined that a user is traveling 5 mph or greater, then the application is blocked. This speed constraint is considered in every remaining step of the decision process until specific conditions are met (Condition B).
If a trip has been planned, the server then goes into a more complicated decision process to determine whether or not a user is on a transit route. This process involves determining the user location and its relation to the route identified in the planned trip. First, the system uses the relevant station/stop location data for the planned trip to verify that a user has arrived at the start location of that trip. The user location must meet certain time and distance thresholds in relation to the beginning station/stop location in order to confirm that the user is actually beginning a transit trip. Distance thresholds are dependent on the transit station/stop, but can be as high as a 1,000 foot radius around the station/stop. The time threshold requires a user to be located at the beginning stop for at least 1 minute. If users do not meet these initial time and location constraints, they will be blocked from the application based on their speed (Condition A or Condition C).
While the geo-fencing status continues to be dependent on the speed of the user, the time and distance thresholds are continuously checked and updated along with the geo-fencing status. This allows the application to block users while they are driving off the transit route or driving to the transit stop/station (Condition C). However, once a user arrives at the transit stop/station and the beginning stop conditions are met, they can board the correct bus or train and will not be blocked from the application during their transit trip (Condition A). Although the application is not blocked after the initial conditions are met, the geo-fencing status is still continuously checked and updated to ensure the user is actually traveling on transit and did not just go to the beginning stop before driving again (Condition C). To confirm travel along the transit route, the user's GPS location is compared to the GPS/AVL data on the transit vehicle or the transit route shape data for the planned trip. At this point, the geo-fencing status is dependent on user GPS location in relation to route matching (i.e., GPS/AVL or route shape data) and the trip history collected since the beginning of the trip (i.e., server requests). If the server receives several consecutive confirmations that a user is traveling along the planned transit route, the geo-fencing status is set to leave the application open and is not checked again until the user arrives at a transfer stop or plans another trip.
2.3 GEO-FENCING TEST
This section of the report details the evaluation activities conducted by the evaluation team to test the capabilities of the geo-fencing design integrated into the PATH2Go Smart Phone Application. The geo-fencing test conducted by the evaluation team was limited to assessing the geo-fencing functionality as it relates to preventing distracted driving.
2.3.1 Test Approach
Based on the server logic for the geo-fencing design identified in the previous section, the PATH project team developed a list of test scenarios that the smart phone application is capable of identifying when determining whether or not to block a user from the application. Presented in Table 2-1 below, the list of scenarios was tested by PATH using various transit trips covered by the PATH2Go Smart Phone Application along the US-101 corridor. The results of the test conducted by the project team are presented in the "Identifiable" column with corresponding notes for further clarification.
| Scenarios | Identifiable | Notes |
|---|---|---|
| No trip planned on Smart Phone Application (Pre-Trip) | ||
| 1. Determine whether a user is 1) driving or 2) not driving. |  |
The system uses GPS data including location and speed to determine whether or not a user is driving. |
| 2. Determine whether a user 1) is near a bus/train stop when not driving or 2) is not near a bus/train stop when not driving. | |
Using the smart phone GPS, the system takes user location relevant to bus/train stops into consideration when deciding whether or not to block application use. |
| 3. Distinguish between 1) a user who is riding a bus/train, and 2) one who is driving. |  |
Because GPS traces are not available unless a trip is planned, the system cannot identify whether a user is on a transit route or driving. |
| Trip planned on Smart Phone Application (En-Route) | ||
| 4. Distinguish between 1) a user who is walking to a bus/train station, and 2) one who is driving to a bus/train station. | |
The system tracks speed and location history and uses a speed threshold of 5 mph to distinguish between driving (>5mph) and walking (<5mph). |
| 5. Distinguish between 1) a user who is waiting at a bus/train stop, and 2) one who is passing a bus/train stop while driving. | |
The system compares the location of the user to the bus/train routes in the planned trip using GPS traces to differentiate mode. |
| 6. Distinguish between 1) a user who is riding a bus/train, and 2) one who is driving along a bus/train route. | (with
constraints) |
Because GPS traces are used to determine user mode, the system may allow users who closely follow the bus/train on a planned transit route via car to use the application. |
Because the geo-fencing test was limited to testing geo-fencing functionality as it relates to preventing distracted driving, only scenarios 1 and 4 – 6 were tested. The test was focused on assessing the ability of users to access information on the application while traveling in a vehicle. While conducting this test, no member of the evaluation team attempted to use a mobile device while operating a vehicle. The scenarios below that involve driving were tested by the passenger who attempted to access the information available on the application with a smart phone while the vehicle was being driven by another member of the evaluation team.
To assess the scenarios above, the evaluation team ran test trips by traveling (i.e., driving in a vehicle, walking, or riding transit) to specific transit routes or along specific transit routes while using the PATH2Go Smart Phone Application. The PATH2Go Smart Phone Application is available for download on Apple iPhones, Android phones, and Windows Mobile phones. All test trips were conducted using each type of phone. The test trips covered all transit agencies that have information available on the application including:
- Caltrain
- SF Muni
- VTA
- BART
- SamTrans
The test trips also covered all types of transit available from each transit agency (i.e., train, light rail, BRT, bus). Again, the focus of this test was to evaluate the ability of the geo-fencing design to prevent distracted driving. Therefore, the majority of the test trips involved driving to a transit route or along a transit route to assess whether or not the user was blocked from using the application while in a vehicle on a roadway. Each test trip presented in the itinerary is associated with a specific transit route. Table 2-2 below lists the transit routes identified for the test.
| Transit Route | Transit Agency | Transit Type | Plan a Trip From | Plan a Trip To | Route |
|---|---|---|---|---|---|
| A | Caltrain | Train | SF 4th St & King | 22nd St | San Francisco to San Jose/Gilroy |
| B | SF Muni | Light Rail | 3rd St & Marin St | 4th St & King St | K/T Ingleside Outbound |
| C | SF Muni | Bus | 4th St & Townsend St | 3rd St & Market St | 30 – Stockton Outbound |
| D | VTA | BRT | Palo Alto Caltrain Station | El Camino & Showers | 522 – Eastridge – Palo Alto |
| E | VTA | Bus | El Camino & Showers | El Camino & Hollenbeck | 22 – Eastridge – Palo Alto/Menlo Park |
| F | VTA | Light Rail | Mountain View Station | Whisman Station | 902 – Mountain View – Winchester |
| G | BART | Commuter Rail | Millbrae | San Bruno | Millbrae/Daly City – Richmond |
| H | SamTrans | Bus | Millbrae Transit Center | San Bruno BART | 391 North |
In order to test for and identify inconsistencies, the itinerary was designed to test scenarios 1 and 4 - 6 at least three times each within one run of the itinerary. Any test trips that showed anomalies in the geo-fencing design were run additional times. The three tables presented in Appendix B provide the three sets of the itinerary used for the geo-fencing test. Each step of the itinerary lists the scenarios tested with that step and the transit route and mode of transportation used to test the scenarios. The goal of the test was to push the geo-fencing design to its limits to understand exactly how it works and whether or not it is effective in preventing distracted driving. After several discussions with the PATH project team, the evaluation team gained a solid understanding of the geo-fencing design, its capabilities, and its limitations and how those related to the test scenarios. Ultimately, the evaluation team set out to assess how the system could be "tricked" into not blocking the information available on the application while driving and whether or not those scenarios are likely to be repeated by actual users. Essentially, the thresholds and constraints related to test scenarios 1 and 4 - 6 are what determine whether or not a user is blocked from the application.
As discussed in section 2.2 above (Geo-fencing Design), these constraints and thresholds include:
- Geo-fencing as the result of GPS speed (Test Scenarios 1 and 4).
- Threshold - users traveling 5 mph or greater are considered to be driving.
- Geo-fencing as the result of GPS location and speed in relation to route planned (Test Scenarios 5 and 6).
- Threshold/Constraints - the geo-fencing status is dependent on user GPS location and speed in relation to route matching (i.e., GPS/AVL or route shape data) and the trip history collected since the beginning of the trip (i.e., server requests). Users confirmed to be on a transit trip can access the application; otherwise, the application is blocked.
The itinerary developed primarily involves running trips to test whether or not users can gain access to the application by imitating transit trips while driving a car, which tests scenario 6. The geo-fencing test was conducted over two consecutive days beginning on November 6, 2010.
2.3.2 Findings
Table 2-3 presents the data and observations collected during the geo-fencing test by each step of the itinerary. For each step of the itinerary, the evaluation team recorded general observations of scenarios where the application was blocked and scenarios where the user was able to access the information available on the application, designated as the application being "open". The evaluation team also kept track of differences in smart phone functionality as it relates to the geo-fencing design. For the driving steps in the itinerary, the evaluation team recorded the primary roadway type traveled for that step to best understand under which driving conditions the geo-fencing functionality was most effective. During steps where the evaluation team attempted to gain access to the application while driving by mimicking a transit rider planning a transit trip and following the transit route closely by car, detailed observations were collected to better understand what aspects of the server logic produced those results (highlighted red in the table below).
| Step | Individual | Scenario Tested | Mode of Transportation | Transit Route | Trip Instructions for Application | Observations |
|---|---|---|---|---|---|---|
| 1 | Driver & Passenger | 4 | Driving | A | Plan trip A prior to departing start address. | Interstate/Highway Trip. Berkeley to San Francisco. No GPS signal for WM for 16 minutes of 40 minute trip. Application blocked while traveling on highway and GPS signal available. Application open when stopped at traffic signals. |
| Passenger | 4 | Walking | A | Upon arrival at end address, exit car and walk to transit station. | Application open. | |
| 2 | Passenger | 6 | Riding transit | A | Board Caltrain – San Francisco to San Jose/Gilroy and ride one stop to 22nd Street. | Application open. |
| Driver | - | Driving | - | Depart Caltrain station and drive to 22nd St & Penn Ave. Pick up passenger. | – | |
| 3 | Driver & Passenger | 4 | Driving | B | Plan trip B prior to departing start address. | Urban Arterial Trip More difficulties with WM GPS signal while traveling to station. Application open when stopped at traffic signals. |
| Passenger | 4 | Walking | B | Upon arrival at end address, exit car and walk to transit stop. Return to car without boarding. | Application open. | |
| 4 | Driver & Passenger | 5, 6 | Driving | B | After identifying SF Muni light rail, attempt to follow the route of the train/bus. | Following light rail on 4 lane urban arterial. Allowed complete access to application while driving throughout the entirety of following the train. However, following light rail required non-typical driving behavior (i.e., slowing, stopping, etc.). Even after leaving the route, application still allowed access and seemed to still be linked to light rail. Was phone GPS disabled and system relying on light rail GPS? |
| 5 | Driver & Passenger | 1 | Driving | C | Do not plan a trip before departing start address. Take 4th St away from the water to Townsend St. Upon arriving, plan trip C. | Urban City Streets Trip Application blocked except when driving slowly or stopped at traffic signals. |
| Passenger | 4, 5 | Walking | C | Upon arrival at end address, exit car and walk to transit stop. Return to car without boarding. | Application open. | |
| 6 | Driver & Passenger | 5, 6 | Driving | C | After identifying SF Muni bus, attempt to follow the route of the train/bus. | Following bus on urban city streets. The application/system seemed to think that we were on the bus, but we were still receiving the geo-fencing message when we reached greater speeds despite following the bus very closely. The traffic was stop and go in many cases, so it was difficult to see if we were just seeing short instances of the geo-fencing message or if the application was being blocked because we weren't actually on the bus. |
| 7 | Driver & Passenger | 1 | Driving | D | Do not plan a trip before departing start address. From US-101 S, take exit 404B to Willow Rd. Turn left onto Middlefield Rd. Turn right at University Ave. Turn right toward Mitchell Ln. Take second left onto Mitchell Ln. | Interstate/Highway Trip ending with local streets. Berkeley to Palo Alto. Application blocked except when driving slowly or stopped at traffic signals. |
| Passenger | 4, 5 | Walking | D | Upon arrival at end address, exit car, plan trip D, and walk to VTA 522 BRT stop. | – | |
| 8 | Driver & Passenger | 5, 6 | Driving | D | After identifying VTA 522 BRT, attempt to follow the route of the bus. | – |
| 9 | Passenger | 4, 5 | Walking | E | Exit car upon arrival at El Camino & Showers stop. Plan trip E and identify VTA 22 bus stop. Return to car. | – |
| 10 | Driver & Passenger | 5, 6 | Driving | E | After identifying a VTA 22 bus, attempt to follow the route of the bus. Drive east on El Camino Real towards Hollenbeck Ave. | Following bus on arterials and urban city streets. The application/system seemed to think that we were on the bus, but we were still receiving the geo-fencing message when we reached greater speeds despite following the bus very closely. The traffic was more free flow in this case, so we were thinking the system realized we were not on the bus. At one of the stops, we exited the car and boarded the bus to see if we still received the geo-fencing message. Despite being on the bus, we still received the geo-fencing message at speeds above 5 mph. |
| 11 | Driver & Passenger | 4 | Driving | F | Plan trip F prior to departing start address. | Highway and Urban Arterial Trip. Application blocked except when driving slowly or stopped at traffic signals. |
| Passenger | 4, 5 | Walking | F | Upon arrival at Mountain View VTA Station, exit car and walk to station. | Application open. | |
| 12 | Driver & Passenger | 4 | Driving | G | Plan trip G prior to departing start address. Take US-101 | Interstate/Highway and Urban Arterial Trip. Application blocked except when driving slowly or stopped at traffic signals. |
| Passenger | 4, 5 | Walking | G | Upon arrival at Milbrae Transit Center, exit car and walk to BART station. | Application open. | |
| 13 | Passenger | 4, 5 | Walking | H | Before departing Milbrae Transit Center. Plan trip H and identify SamTrans 391 bus stop. Return to car. | Application open. |
| 14 | Driver & Passenger | 5, 6 | Driving | H | After identifying a SamTrans 391 North bus, attempt to follow the route of the bus. | Following bus on arterials and urban local streets. The application/system seemed to think that we were on the bus, but we were still receiving the geo-fencing message when we reached greater speeds despite following the bus very closely. |
| 15 | Driver & Passenger | 1 | Driving | - | Return back to the hotel after final trip without trip planned. | Interstate/Highway and Urban Arterial Trip. Application blocked except when driving slowly or stopped at traffic signals. |
| Scenarios | Tested by Itinerary Step | Result | Notes |
|---|---|---|---|
| No trip planned on Smart Phone Application (Pre-Trip) | |||
| 1. Determine whether a user is 1) driving or 2) not driving. | 5,7, 15 | (with constraints) |
App not blocked when driving slowly or stopped. |
| 2. Determine whether a user 1) is near a bus/train stop when not driving or 2) is not near a bus/train stop when not driving. | – | – | Not Tested |
| 3. Distinguish between 1) a user who is riding a bus/train, and 2) one who is driving. | – | – | Not Tested |
| Trip planned on Smart Phone Application (En-Route) | |||
| 4. Distinguish between 1) a user who is walking to a bus/train station, and 2) one who is driving to a bus/train station. | 1, 3, 5, 7, 9, 11, 12, 13 | |
Confirmed. Based on GPS signal. |
| 5. Distinguish between 1) a user who is waiting at a bus/train stop, and 2) one who is passing a bus/train stop while driving. | 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 | (with constraints) |
Confirmed with constraints |
| 6. Distinguish between 1) a user who is riding a bus/train, and 2) one who is driving along a bus/train route. | 2, 4, 6, 8, 10, 14 | (with constraints) |
Confirmed with constraints |
2.3.2.1 Results of Test Scenario 1
For test scenario 1, the evaluation team used itinerary steps 5, 7, and 15 to confirm that using GPS data on the smart phone to control geo-fencing status is a sufficient way to prevent users from accessing the smart phone application while driving. The 5 mph threshold set by the PATH project team appeared to block the application for all smart phones when the GPS signal was available. However, the evaluation team observed several instances where the Windows Mobile phone was not able to obtain a GPS signal for significant portions of time. During this time, the application was open for use. Although this was primarily due to limitations in the smart phone capabilities, this situation as well as others where satellite connections are limited does identify a disadvantage to designing geo-fencing functionality that relies heavily on smart phone GPS data to prevent distracted driving.
Additionally, the evaluation team observed that the geo-fencing design with a 5 mph threshold is more effective at preventing distracted driving on certain types of roadways versus others. As mentioned in the Geo-fencing Design section, the geo-fencing status is constantly being checked and updated within the smart phone application by the system server. On roadways where there is mostly uninterrupted, free-flow traffic like interstates and major highways, the geo-fencing design is very effective at preventing users from accessing information on the application because driving speeds tend to not vary as much and rarely drop to less than 5 mph, which would open the application for use. However, on arterials and local roads where speeds are more variable due to greater occurrences of red lights at intersections, congestion, or stop and go traffic; the smart phone application is constantly being blocked and unblocked as driving speeds fluctuate above and below 5 mph. Although driver distraction may be less dangerous at lower speeds or when a vehicle is stopped, an unblocked smart phone application is another temptation among many that could attract driver attention to somewhere other than the roadway. While it was known that the geo-fencing design would not be able to block the smart phone application at speeds less than 5 mph, it is still important to note that the ability to access information on the application while operating a vehicle at any speed is distracting. Moreover, it is possible that users could begin to recognize the opportunity to access the application at speeds less than 5 mph or when stopped, which could cause an even greater level of distraction if users adjust driving behavior to gain access to the application. While this specific scenario is likely far-fetched, the above observations suggest that completely eliminating distraction due to a smart phone application likely requires completely eliminating driver access to that application.
2.3.2.2 Results of Test Scenario 4
For test scenario 4, the evaluation team used itinerary steps 1, 3, 5, 7, 9, 11, 12, and 13 to confirm that the geo-fencing design can indeed distinguish between users driving to a transit station and users walking to a transit station. The geo-fencing design's ability to identify this scenario relies on GPS speed data from the user smart phone. When the user was traveling at or above 5 mph while en route to a transit stop/station, the application was blocked. Alternately, when the user was walking to a transit stop/station, the application was not blocked. However, the same concerns about distracted driving in regard to the limitations of the geo-fencing design in terms of the 5 mph threshold discussed in test scenario 1 apply to test scenario 4 as well.
2.3.2.3 Results of Test Scenario 5
Scenario 5 was tested using itinerary steps 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, and 14. As described previously, the geo-fencing design uses a time and a distance constraint to determine whether or not a user is waiting at a transit station/stop. The evaluation team observed 10 instances where the geo-fencing design successfully distinguished between users waiting at a station/stop versus users driving past a station/stop and one instance where it did not. Seven of the test itinerary steps assessed the geo-fencing design's ability to recognize that a user was waiting at a transit station/stop while four steps tested whether or not a user could gain access to the application by driving near the beginning stop of a planned trip. All seven of the waiting steps and three of the driving steps confirmed the test scenario while the driving trip test in step 4 resulted in access to the smart phone application.
Purposely, the testers waited in the car near the beginning stop of a planned trip on light rail and followed the train just as it was boarding and departing the stop. The location of the smart phone being tested must have successfully met the time and distance constraints required for that transit station/stop. The testers in the car drove alongside the train, which was using a median right-of-way, closely for several blocks and were allowed complete access to the application. This means the smart phone test in the car must have also consistently met the route matching and trip history requirements of the transit trip planned, which resulted in the geo-fencing status being set to allow application access and not check status again. Even after the testers left the train route, the smart phone application on each phone continued providing updates to the transit trip. Although the testers were successful in gaining access to the smart phone application while driving by mimicking a light rail transit trip, it is highly unlikely that a normal user would a) know enough about the server logic to know geo-fencing exceptions, b) go to such lengths to access the application while driving, or c) stumble across this scenario during normal travel behavior. For those reasons, the one exception to test scenario 5 is not concerning in terms of distracted driving.
It is important to note that the other three driving trip tests successfully blocked the smart phone application despite the best efforts of the testers to follow the bus closely and mimic the transit trip while driving. It appears that the varying distance constraints for the different beginning transit stations/stops was the primary factor in confirming or denying test scenario 5. The bigger the distance constraint, the easier it was for the testers to meet the initial distance and time criteria for mimicking a transit trip while driving.
2.3.2.4 Results of Test Scenario 6
For test scenario 6, the evaluation team used itinerary steps 2, 4, 6, 8, 10, and 14 to confirm that the geo-fencing design can indeed distinguish between users driving along a transit route versus users taking transit. In the first itinerary step, the testers took a transit trip on Caltrain and confirmed that geo-fencing design could successfully allow actual transit riders to access the application while taking transit. The remaining itinerary steps resulted in the same observations discussed in test scenario 5 above. These were trips where testers attempted to mimic a transit trip by waiting near a transit station/stop and then following a bus/train along the transit route while driving in a car. As discussed, the evaluation team observed one scenario where the user was allowed access to the smart phone application while driving along a transit route. Generally, the time and distance constraints as well as the route matching and trip history requirements implemented into the geo-fencing design successfully prevent drivers from mimicking transit trips to gain access to the applications. Again, the likelihood of users going to these lengths to gain access to the smart phone application is probably low. It does not seem likely that distracted driving would occur as the result of test scenario 6.
In several other transit trips not listed in the test itinerary, the evaluation team did observe several instances where the smart phone application was blocked while truly riding transit. Although unrelated to distracted driving, implementing a geo-fencing design into the smart phone application that primarily provides transit information may detract from the user experience of actual transit riders.