4
Although it had multiple successful runs, one of the
challenges while testing was that the sensors and hard hat
had fixed dimensions, and the users’ head sizes and eye dis-
tances to the sensors were different. This made a slight off-
set for the visual enhancement interface. Since the search
and rescue operations are time-critical, we added a wire-
less gaming controller for real-time alignment for the visual
enhancement stream. The alignment helped users to fit the
enhancement in three degrees of freedom.
CONCLUSIONS AND FUTURE WORK
In this work, we combined a thermal imaging camera
with a LiDAR sensor and visualized a real- time world con-
struction as an AR interface. The feasibility and progressive
nature of the device were tested in an experimental under-
ground mine located in Idaho Springs, Colorado. The
results showed that combining the technologies we used
enables faster, safer, and more effective disaster response
for mine rescue operations. Not only does it allow the
responders to search the environment more rapidly, but
it also enables them to detect unexpected hazards before
they become imminent threats. Moreover, the utility of
the developed system is far-reaching, for example, for first
responders searching smoke-filed burning structures. In the
future, it might also enable autonomous systems to navigate
these occluded environments effectively and enable disaster
response to focus on the rescue in search-and-rescue.
We are currently designing a future human subject
study to test the visual variable color. The design is already
submitted to the institutional review board of the Colorado
School of Mines. The application is approved as an exempt
study under 45 CFR 46.104(d)(3) (July 19, 2018).
ACKNOWLEDGMENTS
This study was sponsored by the Alpha Foundation for the
Improvement of Mine Safety and Health, Inc. (ALPHA
FOUNDATION). The views, opinions and recommen-
dations expressed herein are solely those of the authors
and do not imply any endorsement by the ALPHA
FOUNDATION, its Directors and staff.
Table 1: Display types with their respective number of vertices and triangles, and performances
Display Type
Number of
Vertices
Number of
Triangles
Qualitative
2D performance Visualization
Triangulated mesh: Triangle tessellation
using gd3. Provides a smooth surface but is
unpredictable between scans.
1,253 1,671 Poor-Good
Square Surface Patch Square surface
representation requires minimal triangles and
vertices and provides a reasonable level of fidelity.
996 1,992 Good-Excellent
Triangle Surface Patch Triangle surfaces
minimizes the number of triangles to send and
voxels, but is disconnected and only provides
moderate fidelity.
476 1,428 Good
Cube Occupancy Grid: Cube occupancy. Very
consistent but overly conservative and blocks
much of the view.
5,712 3,808 Poor
Cuboid Surface Representation: Cuboid provides
a good view of the surroundings but requires a
lot of vertices and triangles.
3,070 2,456 Good
Ellipsoid Surface Representation: Ellipse
representation has the best fidelity (note the
hanging pipes clearly visible in the upper right)
but requires a lot of vertices and
triangles.
9,600 5,760 Excellent
Although it had multiple successful runs, one of the
challenges while testing was that the sensors and hard hat
had fixed dimensions, and the users’ head sizes and eye dis-
tances to the sensors were different. This made a slight off-
set for the visual enhancement interface. Since the search
and rescue operations are time-critical, we added a wire-
less gaming controller for real-time alignment for the visual
enhancement stream. The alignment helped users to fit the
enhancement in three degrees of freedom.
CONCLUSIONS AND FUTURE WORK
In this work, we combined a thermal imaging camera
with a LiDAR sensor and visualized a real- time world con-
struction as an AR interface. The feasibility and progressive
nature of the device were tested in an experimental under-
ground mine located in Idaho Springs, Colorado. The
results showed that combining the technologies we used
enables faster, safer, and more effective disaster response
for mine rescue operations. Not only does it allow the
responders to search the environment more rapidly, but
it also enables them to detect unexpected hazards before
they become imminent threats. Moreover, the utility of
the developed system is far-reaching, for example, for first
responders searching smoke-filed burning structures. In the
future, it might also enable autonomous systems to navigate
these occluded environments effectively and enable disaster
response to focus on the rescue in search-and-rescue.
We are currently designing a future human subject
study to test the visual variable color. The design is already
submitted to the institutional review board of the Colorado
School of Mines. The application is approved as an exempt
study under 45 CFR 46.104(d)(3) (July 19, 2018).
ACKNOWLEDGMENTS
This study was sponsored by the Alpha Foundation for the
Improvement of Mine Safety and Health, Inc. (ALPHA
FOUNDATION). The views, opinions and recommen-
dations expressed herein are solely those of the authors
and do not imply any endorsement by the ALPHA
FOUNDATION, its Directors and staff.
Table 1: Display types with their respective number of vertices and triangles, and performances
Display Type
Number of
Vertices
Number of
Triangles
Qualitative
2D performance Visualization
Triangulated mesh: Triangle tessellation
using gd3. Provides a smooth surface but is
unpredictable between scans.
1,253 1,671 Poor-Good
Square Surface Patch Square surface
representation requires minimal triangles and
vertices and provides a reasonable level of fidelity.
996 1,992 Good-Excellent
Triangle Surface Patch Triangle surfaces
minimizes the number of triangles to send and
voxels, but is disconnected and only provides
moderate fidelity.
476 1,428 Good
Cube Occupancy Grid: Cube occupancy. Very
consistent but overly conservative and blocks
much of the view.
5,712 3,808 Poor
Cuboid Surface Representation: Cuboid provides
a good view of the surroundings but requires a
lot of vertices and triangles.
3,070 2,456 Good
Ellipsoid Surface Representation: Ellipse
representation has the best fidelity (note the
hanging pipes clearly visible in the upper right)
but requires a lot of vertices and
triangles.
9,600 5,760 Excellent