Executive Summary
Introduction
This executive summary presents an outline of the assessment of the ITS standard involved with Dynamic Message Sign (DMS) communications as deployed by the Virginia Department of Transportation (VDOT). The standard evaluated by this report is:
- NTCIP 1203 v2.25, Dynamic Message Signs (DMS), January 15 2004.
The DMS standard is derived from the architecture flows identified in the National ITS Architecture Version 4.0. The DMS standard concentrates primarily on the interface between the Traffic Management Center (VDOT) and the roadside controller adjacent to where DMS signs are installed.
Test Methodology
From the ITS Standards Test Team (ISTT) perspective, the testing of NTCIP 1203 presented a unique opportunity to leverage the test activities and associated findings collected as part of the ITS Standards Deployment Program effort performed under separate contract to USDOT by the Virginia DOT (VDOT). Under this program, VDOT, along with the Virginia Tech Transportation Institute (VTTI), Cambridge Systematics and Trevilon, specified, procured, tested and integrated a DMS sign and controller and Central System software from two different sources. Details of this activity are available in the VTTI submitted Final Report – Deployment and Testing of an Updated Dynamic Message Sign (DMS) Standard, dated April 24, 2007. As such, it was the charge of the ISTT to monitor and participate in these test activities and to then subsequently conduct an independent evaluation and test of the standard, using both the results of the VDOT activities, as well as separately developed criteria and tests established by the ISTT. This process can be summarized in four phases.
The first phase involved the participation of the ISTT in the various activities associated with the VDOT deployment. This included reviewing test procedures and test cases, and participating in the procurement and test workshops, along with supporting on-site testing.
The second phase involved the collection and assessment of the body of the standard and the vendor documentation, specifications, and test results data as it related to the VDOT deployment. This examination included a detailed read; search for consistency, completeness, and compatibility in the standard; and an analysis and evaluation of any issues or concerns discovered. As part of this step, a determination of the coverage of the VDOT testing as compared to the entirety of standard was made. Any exceptions were noted and further examined to determine if these could indeed be further evaluated or tested. This step was referred to as the static analysis.
The third phase involved generating and conducting a detailed questionnaire to investigate issues identified during the static analysis phase, probing the experiences and issues encountered by the stakeholders, and assessing any non-testable technical features. These interviews were conducted with VDOT as well as the system developers and integrators. The texts of the interviews are attached in Appendix A of this report. Any findings of note were also identified in Section 4.0 of this document.
The final phase of the testing process involved the field-testing of the deployed system and capture of test data for analysis. Testing was conducted in parts. The first part was performed automatically using the Device Tester software to exercise each data object defined in the standard. The coverage and result (pass/fail) was then compared with the coverage and results of the VDOT testing, and any exceptions were noted. The second part of the testing involved communication with the DMS controller by systematically exercising a selected sample of standard dialogs. The results of these test cases are contained in Appendix B of this report.
Deployment and Coverage
The results of this analysis indicated that, with the exception of some findings documented in both the VDOT final report and herein, the VDOT deployment strongly adheres to the ITS standards and shows both a commitment to use of the features of the standards as defined in the standard and well as the success in using standards to integrate separately developed components into a functional system, which satisfies the end-user functional requirements with minimal post-development adjustments. The following table summarizes the NTCIP 1203 coverage by the VDOT deployment.
| NTCIP 1203 Features | Clauses | Total in NTCIP 1203 v2.25 | Total Used by VDOT | Coverage of Std |
|---|---|---|---|---|
| User Needs | 2.3.2.x, 2.4.1.x-2.4.3.x | 32 | 23 | 72% |
| Functional Rqmts. | 3.3.x.x, 3.4.1.x - 3.4.3.x | 122 | 106 | 87% |
| Supplemental Rqmts. | 3.5.x | 84 | 56 | 67% |
| Dialogs | 4.3.1.x - 4.3.3.x | 33 | 29 | 88% |
| Interfaces | 4.4.1.x-4.4.12.x | 122 | 111 | 91% |
| Objects | 5.x | 233 | 195 | 84% |
Summary of Results
Testing was successfully conducted in Columbus, OH during August 2007 at the offices of Battelle. A test system was set up as shown in the diagram below. The system allowed the test team to communicate with the DMS controller using both the management station software as well as the Device Tester software, while using the serial test ComProbe to capture the data transmissions.
The testing phase yielded a large body of data that are recorded and available on the companion CD accompanying this report. The test cases produced the following inventory of results:
- A total of 390 unique data object identifier (OID) were captured.
- A total of 3,083 tests were carried out on these data objects.
Overall Findings
All information collected by the static analysis, questionnaire interviews, and field testing was compiled into a knowledge base. For each issue identified, a determination was made if it represented a genuine finding against the standards or was an artifact of some other influence such as versioning, legacy concerns, local requirements, misinterpretations, etc. Additionally, all of the observations, findings and recommendations from VTTI, their integrator, Trevilon, and the vendors were also The entirety of these findings are included in Section 4.0.
Conclusion
The NTCIP 1203 v2.25 standard for DMS communication tested was assessed and evaluated to be generally suitable, effective, and contributed positively to the interoperability and interchangeability for communication and control with Dynamic Message Signs, except as discussed in the findings stated in this report.
The conclusion of the testing team is that the portions of the standard deployed are relatively mature and has allowed for a successful deployment. There were no findings that would be considered to be critical in nature. The findings annotated in this report mostly consist of issues of interest that should be considered to address improvements in the clarity, usability and flexibility of the standard, rather than issues that render the standard ineffective. As noted in the findings, there are areas of the standard that are open to interpretation, which can lead to interoperability and interchangeability issues. These areas should be addressed by the working group for clarification.