2
associated with roof bolting process on roof bolters includ-
ing fatal and nonfatal injuries [17]. The fatal and nonfatal
injuries associated with roof bolting are caused by six tasks
with the following risk indices: bolting, handling of materi-
als, temporary roof support, drilling, tramming, and tra-
versing [18]. Airborne dust exposure is also a health hazard
associated with roof bolting which has not been addressed
jointly with the previous safety hazards in the literature
[19].
Based on the safety risks of roof bolters during roof
bolting operations and based on a recommendation by
the National Institute for Occupational Safety and Health
(NIOSH), the authors conduct this research to improve the
safety of roof bolters by proposing a computer vision-based
machine learning model to automate roof bolting which
will save roof bolters and reduces the cost.
MACHINE LEARNING FOR ROOF
BOLTING
The application of machine learning in the mining industry
is still growing compared to other fields [20]. Leveraging
machine learning in the mining industry is essential in driv-
ing the mining industry to be an autonomous and more
technologically advanced sector [21]. Deep learning is
a subfield of machine learning that enhances the perfor-
mance of machine learning algorithms especially in com-
puter vision-based tasks. Autonomous technologies in the
mining industry offer numerous advantages by minimiz-
ing workers’ exposure to dangerous conditions, increasing
safety standards, lowering costs, and enhancing efficiency
[22]. An autonomous system needs to understand the scene
and localize objects of interest. Therefore, the first task to
automate roof bolting is the perception task to classify rocks
from non-rocks and locate where to drill. Manual methods
are still used for the localization of bolts in underground
roof bolting [23]. In the literature [23],[24], researchers
used machine learning algorithms to detect roof bolts that
have been already installed. However, none of the exist-
ing work has addressed the automatic detection of where
to drill the holes for roof bolting. The automation of the
roof bolting process using machine learning requires the
detection of rock from anything that is non-rock such as
power cables of existing straps. Figure 1 shows a scenario
of the working environment where it is required to identify
rock areas from a strap and a power cable. In addition, it is
required to identify the small holes inside the strap to drill
and insert the bolts.
Computer vision tasks are mainly classified into image
classification, object detection and recognition, and image
segmentation. However, one of the key challenges for
computer vision tasks in underground mines is the poor
lighting conditions and the dust which degrade the qual-
ity of images. As a result, the performance of the machine
learning algorithm is degraded. Prepossessing for image
enhancement can be used to improve the performance of
deep learning algorithms [25]. Image pre-processing is also
useful for unsupervised domain adaption where a deep
learning model that has been trained on images captured
by traditional cameras (also known as frame-based cam-
eras) can be used for sparse images captured by event-based
cameras. In this research, deep learning-based binary image
semantic segmentation is used to classify pixels that belong
to rocks from pixels that belong to non-rock objects as the
first step toward autonomous roof bolting. Figure 2 shows
a sample of a color image and its corresponding segmented
mask.
EVENT CAMERAS
Event cameras are bio-inspired cameras that capture the
changes of pixel intensity asynchronously as a stream of
events instead of collectively at fixed frame rates which are
streamed by regular cameras. An event e(x, y, t) as
,
,
,
,
e th
y, th
y, th
y, th
1 0
0
0 0
L
DL
D
DL
=
=
^x,
^x,
^x,
^x,
Z
[–1
\
]]
]
]
(1)
where e(x, y, t) is the event value of a pixel located at (x,
y) at timestamp t and ΔL(x, y, t) is the change of the pixel’s
intensity between t − 1 and t. Event cameras are charac-
terized by high dynamic range, high temporal resolution,
and low blur for fast motion [26]. The high dynamic range
Figure 1. A scenario of rock, strap, and power cable area in
the Edgar Mine in Idaho Springs, CO where it is required to
identify the drillable regions from non-rock regions
associated with roof bolting process on roof bolters includ-
ing fatal and nonfatal injuries [17]. The fatal and nonfatal
injuries associated with roof bolting are caused by six tasks
with the following risk indices: bolting, handling of materi-
als, temporary roof support, drilling, tramming, and tra-
versing [18]. Airborne dust exposure is also a health hazard
associated with roof bolting which has not been addressed
jointly with the previous safety hazards in the literature
[19].
Based on the safety risks of roof bolters during roof
bolting operations and based on a recommendation by
the National Institute for Occupational Safety and Health
(NIOSH), the authors conduct this research to improve the
safety of roof bolters by proposing a computer vision-based
machine learning model to automate roof bolting which
will save roof bolters and reduces the cost.
MACHINE LEARNING FOR ROOF
BOLTING
The application of machine learning in the mining industry
is still growing compared to other fields [20]. Leveraging
machine learning in the mining industry is essential in driv-
ing the mining industry to be an autonomous and more
technologically advanced sector [21]. Deep learning is
a subfield of machine learning that enhances the perfor-
mance of machine learning algorithms especially in com-
puter vision-based tasks. Autonomous technologies in the
mining industry offer numerous advantages by minimiz-
ing workers’ exposure to dangerous conditions, increasing
safety standards, lowering costs, and enhancing efficiency
[22]. An autonomous system needs to understand the scene
and localize objects of interest. Therefore, the first task to
automate roof bolting is the perception task to classify rocks
from non-rocks and locate where to drill. Manual methods
are still used for the localization of bolts in underground
roof bolting [23]. In the literature [23],[24], researchers
used machine learning algorithms to detect roof bolts that
have been already installed. However, none of the exist-
ing work has addressed the automatic detection of where
to drill the holes for roof bolting. The automation of the
roof bolting process using machine learning requires the
detection of rock from anything that is non-rock such as
power cables of existing straps. Figure 1 shows a scenario
of the working environment where it is required to identify
rock areas from a strap and a power cable. In addition, it is
required to identify the small holes inside the strap to drill
and insert the bolts.
Computer vision tasks are mainly classified into image
classification, object detection and recognition, and image
segmentation. However, one of the key challenges for
computer vision tasks in underground mines is the poor
lighting conditions and the dust which degrade the qual-
ity of images. As a result, the performance of the machine
learning algorithm is degraded. Prepossessing for image
enhancement can be used to improve the performance of
deep learning algorithms [25]. Image pre-processing is also
useful for unsupervised domain adaption where a deep
learning model that has been trained on images captured
by traditional cameras (also known as frame-based cam-
eras) can be used for sparse images captured by event-based
cameras. In this research, deep learning-based binary image
semantic segmentation is used to classify pixels that belong
to rocks from pixels that belong to non-rock objects as the
first step toward autonomous roof bolting. Figure 2 shows
a sample of a color image and its corresponding segmented
mask.
EVENT CAMERAS
Event cameras are bio-inspired cameras that capture the
changes of pixel intensity asynchronously as a stream of
events instead of collectively at fixed frame rates which are
streamed by regular cameras. An event e(x, y, t) as
,
,
,
,
e th
y, th
y, th
y, th
1 0
0
0 0
L
DL
D
DL
=
=
^x,
^x,
^x,
^x,
Z
[–1
\
]]
]
]
(1)
where e(x, y, t) is the event value of a pixel located at (x,
y) at timestamp t and ΔL(x, y, t) is the change of the pixel’s
intensity between t − 1 and t. Event cameras are charac-
terized by high dynamic range, high temporal resolution,
and low blur for fast motion [26]. The high dynamic range
Figure 1. A scenario of rock, strap, and power cable area in
the Edgar Mine in Idaho Springs, CO where it is required to
identify the drillable regions from non-rock regions