6
depth camera and each of the event cameras found from
extrinsic calibration.
The event image E I, is time synced with the color
image, L I which is then input to the trained network. The
prediction is a real-time label for the event image and can
be added to the data set to further diversify it.
RESULTS
The testing set is validated by the time-synced and mapped
corresponding color image prediction as shown in Figures
11, 12, and 13.
Color Segmentation
The performance of the trained model on color images is
evaluated using the metrics outlined above. The validation
set has 612 test image and ground-truth label pairs available
but was not used to train the network.
Ground-truth Label
Rock Not Rock
Rock 0.997 0.003
Not Rock 0.069 0.931
Accuracy 0.964
Figure 10. Shows the calibration rig usedto extract the
relative sensor poses
Figure 11. Shows a severe lighting scene with shadows. The
model predicts a support strap from an event image, while
the prediction from the color image misses it
Figure 12. Shows the result of predicting a semantic mask
using the network trained and validated on the color data set
Pr
Label