4 Hardware Testing.......................................................................................

4.1 Scope.........................................................................................................

4.2 Basis of Hardware Testing........................................................................

4.2.1 Testing Focus and Applicability..............................................................

4.2.2 Hardware Provided by Vendor................................................................

4.3 Test Conditions..........................................................................................

4.4 Test Log Data Requirements....................................................................

4.5 Test Fixtures..............................................................................................

4.6 Non-operating Environmental Tests..........................................................

4.6.1 General...................................................................................................

4.6.1.1 Pretest Data.........................................................................................

4.6.1.2 Preparation for Test.............................................................................

4.6.1.3 Mechanical Inspection and Repair.......................................................

4.6.1.4 Electrical Inspection and Adjustment..................................................

4.6.1.5 Operational Status Check...................................................................

4.6.1.6 Failure Criteria.....................................................................................

4.6.2 Bench Handling Test..............................................................................

4.6.2.1 Applicability..........................................................................................

4.6.2.2 Procedure............................................................................................

4.6.3 Vibration Test..........................................................................................

4.6.3.1 Applicability..........................................................................................

4.6.3.2 Procedure............................................................................................

4.6.4 Low Temperature Test...........................................................................

4.6.4.1 Applicability..........................................................................................

4.6.4.2 Procedure............................................................................................

4.6.5 High Temperature Test...........................................................................

4.6.5.1 Applicability..........................................................................................

4.6.5.2 Procedure............................................................................................

4.6.6 Humidity Test..........................................................................................

4.6.6.1 Applicability..........................................................................................

4.6.6.2 Procedure..........................................................................................

4.7 Environmental Tests, Operating..............................................................

4.7.1 Applicability...........................................................................................

4.7.2 Temperature and Power Variation Tests.............................................

4.7.3 Power Disturbance...............................................................................

4.7.4 Electromagnetic Radiation...................................................................

4.7.5 Electrostatic Disruption........................................................................

4.7.6 Electromagnetic Susceptibility.............................................................

4.7.7 Electrical Fast Transient.......................................................................

4.7.8 Lightening Surge...................................................................................

4.7.9 Conducted RF Immunity.......................................................................

4.7.10 Magnetic Fields Immunity...................................................................

4.7.11 Accept/Reject Criteria.........................................................................

4.7.12 Data Accuracy....................................................................................

4.7.13 Maintainability Test..............................................................................

4.7.14 Reliability Test.....................................................................................

4.7.15 Availability Test...................................................................................

 

4                                                                                       Hardware Testing

This section contains a description of the testing to be performed by the ITAs to confirm the proper functioning of the hardware components of a voting system submitted for qualification testing. It describes the scope and basis for functionality testing, required test conditions for conducting hardware testing, guidance for the use of test fixtures, test log data requirements, and test practices for specific non-operating and operating environmental tests.

This section addresses the focus and applicability of hardware testing, and specifies the vendor’s obligations to produce hardware to conduct such tests.

ITAs shall design and perform procedures that test the voting system hardware requirements identified in Volume I, Section 3. Test procedures shall be designed and performed by the ITA for both operating and non-operating environmental tests:

¨       Operating environmental tests apply to the entire system, including hardware components that are used as part of the voting system telecommunications capability.

¨       Non-operating tests apply to those elements of the system that are intended for use at poll site voting locations, such as voting machines and precinct counters. These tests address environmental conditions that may be encountered by the voting system hardware at the voting location itself, or while in storage or transit to or from the poll site.

Additionally, compatibility of this equipment with the voting system environment shall be determined through functional tests integrating the standard product with the remainder of the system.

All hardware components custom-designed for election use shall be tested in accordance with the applicable procedures contained in this section. Unmodified COTS hardware will not be subject to all tests. Generally such equipment has been designed to rigorous industrial standards and has been in wide use, permitting an evaluation of its performance history. To enable reduced testing of such equipment, vendors shall provide the manufacturers specifications and evidence that the equipment has been tested to the equivalent of the Standards.

The specific testing procedures to be used shall be identified in the Qualification Test Plan prepared by the ITA. These procedures may replicate testing performed by the vendor and documented in the vendor’s TDP, but shall not rely on vendor testing as a substitute for hardware testing performed by the ITA.

The hardware submitted for qualification testing shall be equivalent, in form and function, to the actual production versions of the hardware units. Engineering or developmental prototypes are not acceptable unless the vendor can show that the equipment to be tested is equivalent to standard production units in both performance and construction.

Qualification tests may be performed in any facility capable of supporting the test environment. Preparation for testing, arrangement of equipment, verification of equipment status, and the execution of procedures shall be witnessed by at least one independent, qualified observer who shall certify that all test and data acquisition requirements have been satisfied.

When a test is to be performed at “standard” or “ambient” conditions, this requirement shall refer to a nominal laboratory environment at prevailing atmospheric pressure and relative humidity.

Otherwise, all tests shall be performed at the required temperature and electrical supply voltage, regulated within the following tolerances:

a.       Temperature of +/- 4 degrees F; and

b.       Electrical supply voltage +/- 2 VAC.

The ITA shall maintain a test log of the procedure employed. This log shall identify the system and equipment by model and serial number. Test environment conditions shall be noted.

In the event that the ITA deems it necessary to deviate from requirements pertaining to the test environment, the equipment arrangement and method of operation, the specified test procedure, or the provision of test instrumentation and facilities, the deviation shall be recorded in the test log. A discussion of the reasons for the deviation and the effect of the deviation on the validity of the test procedure shall also be provided.

The use of test fixtures or ancillary devices to facilitate hardware qualification testing is encouraged.  These fixtures and devices may include arrangements for automating the operation of voting devices and the acquisition of test data.

The use of a fixture to ensure correctness in casting ballots by hand is recommended. Such a fixture may consist of a template, with apertures in the desired location, so that selections may be made rapidly. Such a template will eliminate or greatly minimize errors in activating test ballot patterns, while reducing the amount of time required to cast a test ballot.

For systems that use a light source as a means of detecting voter selections, the generation of a suitable optical signal by an external device is acceptable. For systems that rely on the physical activation of a switch, a mechanical fixture with suitable motion generators is acceptable.

To speed up the process of testing and to eliminate human error in casting test ballots the tests may use a simulation device with appropriate software. Such simulation is recommended if it covers all voting data detection and control paths that are used in casting an actual ballot. In the event that only partial simulation is achieved, then an independent method and test procedure must be used to validate the proper operation of those portions of the system not tested by the simulator.

If the vendor provides a means of simulating the casting of ballots, the simulation device is subject to the same performance, reliability, and quality requirements that apply to the voting device itself so as not to contribute errors to the test procerss.

This section addresses a range of tests for voting machines and precinct counters, as such devices are stored between elections and are transported between the storage facility and polling site.

Environmental tests of non-operating equipment are intended to simulate exposure to physical shock and vibration associated with handling and transportation of voting equipment and precinct counters between a jurisdiction’s storage facility and precinct polling site. These tests additionally simulate the temperature and humidity conditions that may be encountered during storage in an uncontrolled warehouse environment or precinct environment.  The procedures and conditions of these tests correspond generally to those of MIL-STD-810D, “Environmental Test Methods and Engineering Guidelines,” 19 July 1983. In most cases, the severity of the test conditions has been reduced to reflect commercial, rather than military, practice.

Systems exclusively designed with system-level COTS hardware whose configuration has not been modified in any manner and are not subjected to this segment of hardware testing. Systems made up of individual COTS components such as hard drives, motherboards, and monitors that have been packaged to build a voting machine or other device will be required to undergo the hardware testing.

Prior to each test, the equipment shall be shown to be operational by means of the procedure contained in Subsection 4.6.1.5.  The equipment may then be prepared as if for actual transportation or storage, and subjected to appropriate test procedures outlined. After each procedure has been completed, the equipment status will again be verified as in Subsection 4.6.1.5.

The following requirements for equipment preparation, functional tests, and inspections shall apply to each of the non-operating test procedures.

The test technician shall verify that the equipment is capable of normal operation. Equipment identification, environmental conditions, equipment configuration, test instrumentation, operator tasks, time-of-day or test time, and test results shall be recorded.

The equipment shall be prepared as for shipping or storage, with any protective enclosures or internal restraints normally used for transportation and handling between the storage facility and the polling location.

After the test has been completed, the devices shall be removed from their containers, and any internal restraints shall be removed.  The exterior and interior of the devices shall be inspected for evidence of mechanical damage, failure, or dislocation of internal components.  Devices shall be adjusted or repaired, if necessary.

After completion of the mechanical inspection and repair, routine electrical maintenance and adjustment may be performed, according to the manufacturer's standard procedure.

When all tests, inspections, repairs, and adjustments have been completed, normal operation shall be verified by conducting an operational status check.

During this process, all equipment shall be operated in a manner and environmental conditions that simulate election use to verify the functional status of the system.  Prior to the conduct of each of the environmental hardware non-operating tests, a supplemental test shall be made to determine that the operational state of the equipment is within acceptable performance limits.

The following procedures shall be followed to verify the equipment status:

Step 1:     Arrange the system for normal operation.

Step 2:     Turn on power, and allow the system to reach recommended operating temperature.

Step 3:     Perform any servicing, and make any adjustments necessary, to achieve operational status.

Step 4:     Operate the equipment in all modes, demonstrating all functions and features that would be used during election operations.

Step 5:     Verify that all system functions have been correctly executed.

Upon completion of each non-operating test, the system hardware shall be subject to functional testing to verify continued operability. If any portion of the voting machine or precinct counter hardware fails to remain fully functional, the testing will be suspended until the failure is identified and corrected by the vendor. The system will then be subject to a retest.

The bench handling test simulates stresses faced during transport and handling of voting machines and ballot counters.

All systems and components, regardless of type, shall meet the requirements of this test.  This test is equivalent to the procedure of MIL-STD-810D, Method 516.3, Procedure VI.

Step 1:     Place each piece of equipment on a level floor or table, as for normal operation or servicing.

Step 2:     Make provision, if necessary, to restrain lateral movement of the equipment or its supports at one edge of the device.  Vertical rotation about that edge shall not be restrained.

Step 3:     Using that edge as a pivot, raise the opposite edge to an angle of 45 degrees, to a height of four inches above the surface, or until the point of balance has been reached, whichever occurs first.

Step 4:     Release the elevated edge so that it may drop to the test surface without restraint.

Step 5:     Repeat steps 3 and 4 for a total of six events.

Step 6:     Repeat steps 2, 3, and 4 for the other base edges, for a total of 24 drops for each device.

Step X:    Remove the test item from its transit or combination case and verify its continued operability.

The vibration test simulates stresses faced during transport and handling of voting machines and ballot counters.

All systems and components, regardless of type, shall meet the requirements of this test.  This test is equivalent to the procedure of MIL-STD-810D, Method 514.3, Category 1- Basic Transportation, Common Carrier.

Step 1:     Install the test item in its transit or combination case as prepared for delivery to the polling site.

Step 2:     Attach instrumentation as required to measure the applied excitation.

Step 3:     Mount the equipment on a vibration table with the axis of excitation along the vertical axis of the equipment.

Step 4:     Apply excitation as shown in MIL-STD-810D, Method 514.3-1, “Basic transportation, common carrier, vertical axis”, with low frequency excitation cutoff at 10 Hz, for a period of 30 minutes.

Step 5:     Repeat steps 2 and 3 for the transverse and longitudinal axes of the equipment with the excitation profiles shown in Figures 514.3-2 and 514.3-3, respectively. (Note:  The total excitation period equals 90 minutes, with 30 minutes excitation along each axis.)

Step X:    Remove the test item from its transit or combination case and verify its continued operability.

The low temperature test simulates stresses faced during transport and storage of voting machines and ballot counters.

All systems and components, regardless of type, shall meet the requirements of this test.  This test is equivalent to the procedure of MIL-STD-810D, Method 502.2, Procedure I-Storage.  The minimum temperature shall be -4 degrees F.

Step 1:     Arrange the equipment as for storage.  Install it in the test chamber.

Step 2:     Lower the internal temperature of the chamber at any convenient rate, but not so rapidly as to cause condensation in the chamber, and in any case no more rapidly than 10 degrees F per minute, until an internal temperature of -4 degrees F has been reached.

Step 3:     Allow the chamber temperature to stabilize.  Maintain this temperature for a period of 4 hours after stabilization.

Step 4:     Allow the internal temperature of the chamber to return to standard labo­ratory conditions, at a rate not exceeding 10 degrees F per minute

Step 5:     Allow the internal temperature of the equipment to stabilize at laboratory conditions before removing it from the chamber.

Step 6:     Remove the equipment from the chamber and from its containers, and inspect the equipment for evidence of damage.

Step 7:     Verify continued operability of the equipment.

The high temperature test simulates stresses faced during transport and storage of voting machines and ballot counters.

All systems and components, regardless of type, shall meet the requirements of this test.  This test is equivalent to the procedure of MIL-ST