8
The blue wire as seen in Figure 13 is an unseen fea-
ture to the network. Despite the network’s generalization
capability, where the color prediction correctly categorizes
the feature as Not Rock, the event prediction exhibits some
false positive pixels. This scenario is noteworthy for several
reasons. Primarily, it emphasizes the model’s capacity to
extrapolate knowledge from the training set and accurately
predict features even when encountering previously unseen
elements. The successful classification of the blue wire as
Not Rock in the color prediction indicates the model’s abil-
ity to leverage feature-based information to make informed
decisions about unfamiliar objects or structures. However,
the occurrence of false positive pixels in the event predic-
tion suggests that the network, when relying on event-based
data, may encounter challenges in precisely identifying
novel features that differ significantly from the character-
istics present in the training set. This underscores the need
for quick and inexpensive data augmentation techniques as
presented by this work.
The incorporation of a self-supervised, hierarchical
training pipeline makes a valuable contribution by quickly
enhancing the event camera-based dataset. This approach
involves a method where the system refines and improves
its own learning through various stages, optimizing the
dataset for better performance. The significance lies in the
accelerated readiness it brings for automated operations in
novel environments. By efficiently augmenting the dataset,
not only saves time but also minimizes costs associated with
initiating automated processes in new settings. Overall, the
utilization of such a training pipeline facilitates a more
seamless and cost-effective transition to automated opera-
tions across diverse environments.
This work was able to demonstrate a self-supervised
approach to the semantic segmentation task which proved
to be effective in the harsh lighting and environmental
conditions of an active mine. In fact, the vibratory load
expected on a drill rig will trigger the event cameras. While
these vibrations would cause motion blur in traditional opti-
cal cameras, the event cameras generate images with more
dense information. Additionally, the ability to use a model
trained on color images, and then predict using event-based
images is valuable as pre-existing data sets can be leveraged
to accelerate the performance of novel sensors. While there
are large data sets for urban driving like Cityscapes [23] and
KITTI [24], semantic recognition networks struggle with
generalization [25]. This leads to industry-wide specializa-
tion in urbanized environments in developed countries.
This work can be used to fill in the gaps of representation
at a much lower cost. In essence, this approach promotes a
seamless transition from established imaging technologies
to emerging ones, ensuring a smoother integration of novel
sensors into existing systems while optimizing the use of
available data resources.
CONCLUSION
This work demonstrates a pipeline that uses inexpensive
semantic hand labels and self-supervision methods to gen-
erate a generalized data set. Using this pipeline, this paper
presents the capability to use event cameras to solve the
semantic segmentation task in a mine during a drilling ses-
sion. Additionally, the capability to benchmark against tra-
ditional color cameras is presented and shows that event
cameras are able to perform the task.
REFERENCES
[1] JJ Sammarco, A Podlesny, EN Rubinstein, and
B Demich. An analysis of roof bolter fatalities and
injuries in us mining. Transactions of Society for
Mining, Metallurgy, and Exploration, Inc, 340(1):11,
2016.
[2] Mark Aldrich. Engineers attack the “no. one killer” in
coal mining: The bureau of mines and the promotion
of roof bolting, 1947–1969. Technology and culture,
pages 80–118, 2016.
[3] Christopher Mark. The introduction of roof bolting
to us underground coal mines (1948–1960): a cau-
tionary tale. 2002.
[4] Ronald C Althouse, Michael J Klishis, and R Larry
Grayson. Microanalysis of roof bolter injuries. Applied
occupational and environmental hygiene, 12(12):851–
857, 1997.
[5] Fred C Turin. Human factors analysis of roof bolting
hazards in underground coal mines. 1995.
[6] Syd Peng. Roof bolting and underground roof falls.
Geohazard Mechanics, 1(1):32–37, 2023.
[7] Jing Li, Yaru Qin, Cheng Guan, Yanli Xin, Zhen
Wang, and Ruikang Qi. Lighting for work: a study
on the effect of underground low-light environment
on miners’ physiology. Environmental Science and
Pollution Research, pages 1–10, 2022.
[8] Yongqin Xian, Subhabrata Choudhury, Yang He,
Bernt Schiele, and Zeynep Akata. Semantic projec-
tion network for zeroand few-label semantic segmen-
tation. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition (CVPR),
June 2019.
[9] Zhaoning Sun, Nico Messikommer, Daniel Gehrig,
and Davide Scaramuzza. Ess: Learning event-based
semantic segmentation from still images. In European
The blue wire as seen in Figure 13 is an unseen fea-
ture to the network. Despite the network’s generalization
capability, where the color prediction correctly categorizes
the feature as Not Rock, the event prediction exhibits some
false positive pixels. This scenario is noteworthy for several
reasons. Primarily, it emphasizes the model’s capacity to
extrapolate knowledge from the training set and accurately
predict features even when encountering previously unseen
elements. The successful classification of the blue wire as
Not Rock in the color prediction indicates the model’s abil-
ity to leverage feature-based information to make informed
decisions about unfamiliar objects or structures. However,
the occurrence of false positive pixels in the event predic-
tion suggests that the network, when relying on event-based
data, may encounter challenges in precisely identifying
novel features that differ significantly from the character-
istics present in the training set. This underscores the need
for quick and inexpensive data augmentation techniques as
presented by this work.
The incorporation of a self-supervised, hierarchical
training pipeline makes a valuable contribution by quickly
enhancing the event camera-based dataset. This approach
involves a method where the system refines and improves
its own learning through various stages, optimizing the
dataset for better performance. The significance lies in the
accelerated readiness it brings for automated operations in
novel environments. By efficiently augmenting the dataset,
not only saves time but also minimizes costs associated with
initiating automated processes in new settings. Overall, the
utilization of such a training pipeline facilitates a more
seamless and cost-effective transition to automated opera-
tions across diverse environments.
This work was able to demonstrate a self-supervised
approach to the semantic segmentation task which proved
to be effective in the harsh lighting and environmental
conditions of an active mine. In fact, the vibratory load
expected on a drill rig will trigger the event cameras. While
these vibrations would cause motion blur in traditional opti-
cal cameras, the event cameras generate images with more
dense information. Additionally, the ability to use a model
trained on color images, and then predict using event-based
images is valuable as pre-existing data sets can be leveraged
to accelerate the performance of novel sensors. While there
are large data sets for urban driving like Cityscapes [23] and
KITTI [24], semantic recognition networks struggle with
generalization [25]. This leads to industry-wide specializa-
tion in urbanized environments in developed countries.
This work can be used to fill in the gaps of representation
at a much lower cost. In essence, this approach promotes a
seamless transition from established imaging technologies
to emerging ones, ensuring a smoother integration of novel
sensors into existing systems while optimizing the use of
available data resources.
CONCLUSION
This work demonstrates a pipeline that uses inexpensive
semantic hand labels and self-supervision methods to gen-
erate a generalized data set. Using this pipeline, this paper
presents the capability to use event cameras to solve the
semantic segmentation task in a mine during a drilling ses-
sion. Additionally, the capability to benchmark against tra-
ditional color cameras is presented and shows that event
cameras are able to perform the task.
REFERENCES
[1] JJ Sammarco, A Podlesny, EN Rubinstein, and
B Demich. An analysis of roof bolter fatalities and
injuries in us mining. Transactions of Society for
Mining, Metallurgy, and Exploration, Inc, 340(1):11,
2016.
[2] Mark Aldrich. Engineers attack the “no. one killer” in
coal mining: The bureau of mines and the promotion
of roof bolting, 1947–1969. Technology and culture,
pages 80–118, 2016.
[3] Christopher Mark. The introduction of roof bolting
to us underground coal mines (1948–1960): a cau-
tionary tale. 2002.
[4] Ronald C Althouse, Michael J Klishis, and R Larry
Grayson. Microanalysis of roof bolter injuries. Applied
occupational and environmental hygiene, 12(12):851–
857, 1997.
[5] Fred C Turin. Human factors analysis of roof bolting
hazards in underground coal mines. 1995.
[6] Syd Peng. Roof bolting and underground roof falls.
Geohazard Mechanics, 1(1):32–37, 2023.
[7] Jing Li, Yaru Qin, Cheng Guan, Yanli Xin, Zhen
Wang, and Ruikang Qi. Lighting for work: a study
on the effect of underground low-light environment
on miners’ physiology. Environmental Science and
Pollution Research, pages 1–10, 2022.
[8] Yongqin Xian, Subhabrata Choudhury, Yang He,
Bernt Schiele, and Zeynep Akata. Semantic projec-
tion network for zeroand few-label semantic segmen-
tation. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition (CVPR),
June 2019.
[9] Zhaoning Sun, Nico Messikommer, Daniel Gehrig,
and Davide Scaramuzza. Ess: Learning event-based
semantic segmentation from still images. In European