5
stereo-pair is input to the trained network, the output is a
disparity map as shown in Figure 9.
A. Evaluation Metric
To evaluate these predictions, the Frobenius norm of the
difference between the prediction and the ground truth
image is used.
e d d
/2
F ij ij
j
n
i
m 2
1 1
1
=-
==
e o //(3)
where d is the ground truth depth and d predicted depth.
These prediction images can be used to re-project
images taken by the event cameras into 3D world coordi-
nates. This is used to compute a surface rep- resentation of
the roof.
B. Data Sets Collected
Data was collected from four different areas of the Edgar
Mine -two of these were reserved for unseen testing, with
the LiDAR off to prevent unintentional triggering of the
event cameras. These areas already have bolted roofs, with
straps and meshes in all align- ments. The lighting condi-
tions are all single-source, point lighting. Stereo pairs of
accumulated event-based images are input to the trained
network and a predicted depth image is the output. Ten %
of the labeled data is reserved as a validation set. For this
subset of the data, ground truth is available to evaluate the
output of the network. Figure 9, from left to right, shows
the right event image, the ground-truth disparity image,
and the predicted disparity image.
Two models with the same architecture were instan-
tiated and trained on two different training data sets. The
first was a data set from one adit of the mine called the
Miami. The distance to the roof was kept within a range of
one meter to two meters. The focal length of the event cam-
eras was adjusted appropriately to ac- count for this range
and the cameras were intrinsically calibrated afterwards.
The features in the data set were horizontal straps and fairly
regular gneiss rock. The light source was a single-point
headlamp. Ego motion was limited to handheld motion,
mimicking a rigid mount to a drill rig.
The second data set was from a different adit called the
Army. The distance to the roof was a wide range from one
meter to ten meters. Therefore, the images from the data
set are not necessarily in focus for the duration of the col-
lection run. The data set is comprised of a variety of fea-
tures -straps in various orientations, mesh, gneiss rock with
discontinuities, and a desk-sized rolling toolbox. The light
Figure 7. Shows the sensor rig used during mobile data
collection operations in the mine. The stereo event cameras
are mounted to the top of the rig, while the L515 lidar
camera is mounted to the bottom rail
Figure 8. Shows the time-synced accumulated event images as taken by the left event camera and right event camera
stereo-pair is input to the trained network, the output is a
disparity map as shown in Figure 9.
A. Evaluation Metric
To evaluate these predictions, the Frobenius norm of the
difference between the prediction and the ground truth
image is used.
e d d
/2
F ij ij
j
n
i
m 2
1 1
1
=-
==
e o //(3)
where d is the ground truth depth and d predicted depth.
These prediction images can be used to re-project
images taken by the event cameras into 3D world coordi-
nates. This is used to compute a surface rep- resentation of
the roof.
B. Data Sets Collected
Data was collected from four different areas of the Edgar
Mine -two of these were reserved for unseen testing, with
the LiDAR off to prevent unintentional triggering of the
event cameras. These areas already have bolted roofs, with
straps and meshes in all align- ments. The lighting condi-
tions are all single-source, point lighting. Stereo pairs of
accumulated event-based images are input to the trained
network and a predicted depth image is the output. Ten %
of the labeled data is reserved as a validation set. For this
subset of the data, ground truth is available to evaluate the
output of the network. Figure 9, from left to right, shows
the right event image, the ground-truth disparity image,
and the predicted disparity image.
Two models with the same architecture were instan-
tiated and trained on two different training data sets. The
first was a data set from one adit of the mine called the
Miami. The distance to the roof was kept within a range of
one meter to two meters. The focal length of the event cam-
eras was adjusted appropriately to ac- count for this range
and the cameras were intrinsically calibrated afterwards.
The features in the data set were horizontal straps and fairly
regular gneiss rock. The light source was a single-point
headlamp. Ego motion was limited to handheld motion,
mimicking a rigid mount to a drill rig.
The second data set was from a different adit called the
Army. The distance to the roof was a wide range from one
meter to ten meters. Therefore, the images from the data
set are not necessarily in focus for the duration of the col-
lection run. The data set is comprised of a variety of fea-
tures -straps in various orientations, mesh, gneiss rock with
discontinuities, and a desk-sized rolling toolbox. The light
Figure 7. Shows the sensor rig used during mobile data
collection operations in the mine. The stereo event cameras
are mounted to the top of the rig, while the L515 lidar
camera is mounted to the bottom rail
Figure 8. Shows the time-synced accumulated event images as taken by the left event camera and right event camera