A Report to the ITS Standards Community
ITS Standards Testing Program

FINAL TEST REPORT

For NTCIP 1203 v2.25 – Dynamic Message Signs (DMS)
as deployed by the Virginia Department of Transportation (VDOT)

By
Battelle Memorial Institute
505 King Avenue
Columbus, OH 43201

Prepared for
US Department of Transportation (USDOT)

Battelle Work Order: BA34012
Task No.: 8

April 25, 2008

PDF Version 3.3 MB

Get Adobe PDF Reader Here

---

Notice

The U.S. Department of Transportation provides high-quality information to serve Government, industry, and the public in a manner that promotes public understanding. Standards and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its information. USDOT periodically reviews quality issues and adjusts its programs and processes to ensure continuous quality improvements.

---

TECHNICAL REPORT DOCUMENTATION PAGE

Table of Contents

• EXECUTIVE SUMMARY
  • Introduction
  • Test Methodology
  • Deployment and Coverage
  • Summary of Results
  • Overall Findings
  • Conclusion
• 1.0 INTRODUCTION
• 2.0 BACKGROUND
  • 2.1 ITS Standards Testing Program
  • 2.2 ITS National Architecture
  • 2.3 Standards Baseline
  • 2.4 VDOT Deployment
  • 2.5 Requirements
• 3.0 TESTING PROCESS METHODOLOGY
  • 3.1 Scope of Test
  • 3.2 Testing Goals
    • 3.2.1 Suitability
    • 3.2.2 Effectiveness
    • 3.2.3 Interoperability and Interchangeability
  • 3.3 Testing Process Outline
  • 3.4 Establish and Verify Standards Baseline
  • 3.5 DMS Standard Coverage
  • 3.6 Participation of VDOT Test Activities
  • 3.7 Interview Product Vendor/Developers
  • 3.8 Evaluate the Purity and Integrity of the External Interfaces
  • 3.9 Conduct Exception Testing
  • 3.10 Test Approach
  • 3.11 Test Results
• 4.0 OBSERVATIONS AND FINDINGS
  • 4.1 VTTI Findings
    • 4.1.1 General
    • 4.1.2 Specification Guide
    • 4.1.3 Problems Experienced in Specification Development and Procurement
    • 4.1.4 Testing
      • 4.1.4.1 Test Workshop
      • 4.1.4.2 Test Plan Development
      • 4.1.4.3 Unit Testing of DMS
      • 4.1.4.4 DMS Testing Process
      • 4.1.4.5 Diagnostic Testing
      • 4.1.4.6 Summary of DMS Unit Testing Results
      • 4.1.4.7 Central Systems Software Testing
      • 4.1.4.8 Developing the Software Testing Procedures
      • 4.1.4.9 Summary of Software Testing Results
      • 4.1.4.10 Suggestions for Improving the Testing Process
      • 4.1.4.11 VTTI Conclusions on Testing
      • 4.1.4.12 VTTI Overall Conclusions
  • 4.2 ISTT Findings
  • 4.3 Trevilon Findings
  • 4.4 LedStar Findings
  • 4.5 IBI Group Findings
  • 4.6 Minor Typographical Errors
• 5.0 CONCLUSION
  • 5.1 ISTT Observations
    • 5.1.1 Suitability
    • 5.1.2 Effectiveness
    • 5.1.3 Interoperability and Interchangeability
    • 5.1.4 Application of Systems Engineering Process
    • 5.1.5 Overall
  • 5.2 VTTI Observations
  • 5.3 Trevilon Observations
    • 5.3.1 Suitability
    • 5.3.2 Effectiveness
    • 5.3.3 Interchangeability/Interoperability
    • 5.3.4 Additional Observations
  • 5.4 LedStar Observations
    • 5.4.1 Suitability
    • 5.4.2 Effectiveness
    • 5.4.3 Interchangeability/Interoperability
    • 5.4.4 Additional Observations
  • 5.5 IBI Group Observations

List of Appendices

• APPENDIX A: INTERVIEW QUESTIONNAIRE
• APPENDIX B: TEST CASES AND RESULTS

List of Tables

• Table 2.1. Standard of Interest
• Table 3.1. Test Process Steps
• Table 3.2. DMS Standard Coverage
• Table 3.3. Global Objects Features Coverage
• Table 3.4. DMS Stakeholders
• Table 3.5. Objects from Failed Functional Requirements
• Table 3.6. Dialogs Tested
• Table 3.7. DMS Test Result Summary
• Table 4.1. NTCIP Knowledge Level Requirement
• Table 4.2. PRL Entry for Requirement 3.5.6.2.5.1
• Table 4.3. Round 1 Ledstar Unit Testing Results
• Table 4.4. Round 2 Ledstar Unit Testing Results
• Table 4.5. Example of Keystroke Mapping
• Table 4.6. User Need 2.4.2.1
• Table 4.7. Test Log for User Need 2.4.2.1
• Table 4.8. User Need 2.4.2.3.1
• Table 4.9. Round 3 Testing Results – IBI Software
• Table 4.10. Round 4 Testing Results

List of Figures

• Figure 2.1. ITS Physical Architecture
• Figure 2.2. ATMS06 Market Package
• Figure 3.1. Test Configuration
• Figure 3.2. DMS Test System
• Figure 4.1. Schematic of PRL Information Flow

---

U.S. Department of Transportation
ITS Joint Program Office, HOIT
Washington, DC 20590
Toll-Free "Help Line" 866-367-7487
www.its.dot.gov

EDL 14475

FHWA-JPO-09-041

ITS Home | US DOT Home | Privacy | Feedback

United States Department of Transportation - ITS