1
24-083
Testing of Ground Truth Instruments for Use in Evaluating
Haul Truck Collision Warning and Avoidance Systems
J. Bickson, Chenming Zhou, Yongjun Zhang, J.R. Srednicki,
J.L. Carr, M. Girman, J. A. Galanko, C.R. DeGennaro, and B. Lambie
CDC NIOSH, Pittsburgh, PA
ABSTRACT
Between 2005 and 2021, surface mining haul trucks
were involved in 54 fatal incidents in the United States
[1]. Collision warning and avoidance systems (CXS) can
help haul truck operators navigate their route safely. The
evaluation of CXS object detection performance for surface
mining haul trucks relies on the positional accuracy of the
ground truth instrument. As part of a holistic approach,
researchers from the National Institute for Occupational
Safety and Health (NIOSH) characterized the accuracy of
a global navigation satellite system (GNSS) that serves as
the ground truth instrument to determine object position
and velocity in CXS object detection performance testing.
We used precision surveying equipment to establish ground
truth points for comparison with GNSS data collected for
static positional measurements and reduced-scale straight-
line vehicle dynamic tests. We conducted these tests with
real-time kinematics (RTK) and then satellite-based aug-
mentation systems (SBAS). For the dynamic tests, we mea-
sured a distance error of 1.34 m (4.40 ft) using RTK and
1.50 m (4.92 ft) using an SBAS. This research will provide
CXS manufacturers and CXS researchers a basis for evalu-
ating the positional accuracy of CXS. Note that we did not
evaluate any CXS in this experiment.
INTRODUCTION
Background
For surface mine haul trucks, the main function of collision
warning and avoidance systems (CXS) is to assist drivers
at avoiding accidents in their travel that can lead to col-
lateral damages, injuries, or fatalities. Because detection
performance is critical, the ground truth instrument (GTI)
used to assess positional accuracy of CXS must be reli-
able. Modern GNSS receivers can be considered as a GTI
because of their reported centimeter level of accuracy,
under static condition [2]. Researchers from the National
Institute for Occupational Safety and Health (NIOSH)
designed an experiment to validate the positional accuracy
of GNSS receivers for our intended purpose which is to use
GNSS receivers as ground truth to assess detection of CXS,
specifically while GNSS receivers are in motion.
Limited literature exists describing dynamic test mea-
surements of GNSS receivers. However, two relevant stan-
dards exist that discuss the accuracy of GNSS receivers for
static measurements or testing methods for GNSS while in
motion. These are ISO 12188-1 and ISO 12188-2. These
standards provide detailed instructions on how to test the
positional accuracy of GNSS receivers in the agricultural
industry. ISO 12188-1 specifies common parameters to
assess and compare different GNSS receivers in dynamic
conditions [3]. ISO 12188-2 covers how to assess auto-
mated guidance systems based on GNSS technologies for
agricultural vehicles [4]. NIOSH researchers modified the
static and dynamic test methods described in these stan-
dards to make them more suitable to our application. Our
test method differed in terms of test course requirement,
test procedure and test report and calculations.
Approach in the Current Study
Using ISO 12188-1 and ISO 12188-2 as reference, we
took measurements to evaluate the positional accuracy of
GNSS receivers we used in a separate experiment to assess
24-083
Testing of Ground Truth Instruments for Use in Evaluating
Haul Truck Collision Warning and Avoidance Systems
J. Bickson, Chenming Zhou, Yongjun Zhang, J.R. Srednicki,
J.L. Carr, M. Girman, J. A. Galanko, C.R. DeGennaro, and B. Lambie
CDC NIOSH, Pittsburgh, PA
ABSTRACT
Between 2005 and 2021, surface mining haul trucks
were involved in 54 fatal incidents in the United States
[1]. Collision warning and avoidance systems (CXS) can
help haul truck operators navigate their route safely. The
evaluation of CXS object detection performance for surface
mining haul trucks relies on the positional accuracy of the
ground truth instrument. As part of a holistic approach,
researchers from the National Institute for Occupational
Safety and Health (NIOSH) characterized the accuracy of
a global navigation satellite system (GNSS) that serves as
the ground truth instrument to determine object position
and velocity in CXS object detection performance testing.
We used precision surveying equipment to establish ground
truth points for comparison with GNSS data collected for
static positional measurements and reduced-scale straight-
line vehicle dynamic tests. We conducted these tests with
real-time kinematics (RTK) and then satellite-based aug-
mentation systems (SBAS). For the dynamic tests, we mea-
sured a distance error of 1.34 m (4.40 ft) using RTK and
1.50 m (4.92 ft) using an SBAS. This research will provide
CXS manufacturers and CXS researchers a basis for evalu-
ating the positional accuracy of CXS. Note that we did not
evaluate any CXS in this experiment.
INTRODUCTION
Background
For surface mine haul trucks, the main function of collision
warning and avoidance systems (CXS) is to assist drivers
at avoiding accidents in their travel that can lead to col-
lateral damages, injuries, or fatalities. Because detection
performance is critical, the ground truth instrument (GTI)
used to assess positional accuracy of CXS must be reli-
able. Modern GNSS receivers can be considered as a GTI
because of their reported centimeter level of accuracy,
under static condition [2]. Researchers from the National
Institute for Occupational Safety and Health (NIOSH)
designed an experiment to validate the positional accuracy
of GNSS receivers for our intended purpose which is to use
GNSS receivers as ground truth to assess detection of CXS,
specifically while GNSS receivers are in motion.
Limited literature exists describing dynamic test mea-
surements of GNSS receivers. However, two relevant stan-
dards exist that discuss the accuracy of GNSS receivers for
static measurements or testing methods for GNSS while in
motion. These are ISO 12188-1 and ISO 12188-2. These
standards provide detailed instructions on how to test the
positional accuracy of GNSS receivers in the agricultural
industry. ISO 12188-1 specifies common parameters to
assess and compare different GNSS receivers in dynamic
conditions [3]. ISO 12188-2 covers how to assess auto-
mated guidance systems based on GNSS technologies for
agricultural vehicles [4]. NIOSH researchers modified the
static and dynamic test methods described in these stan-
dards to make them more suitable to our application. Our
test method differed in terms of test course requirement,
test procedure and test report and calculations.
Approach in the Current Study
Using ISO 12188-1 and ISO 12188-2 as reference, we
took measurements to evaluate the positional accuracy of
GNSS receivers we used in a separate experiment to assess