3
This paper is laid out with a discussion of the planned
approach, a description of the experimental setup, a presen-
tation of the results, a discussion grounding the results in
the context of the project, and a conclusion.
METHODOLOGY
This effort is to enable the prediction of semantic segmenta-
tion masks given an event image taken during roof bolting
as shown in Figure 2.
Since the goal is to minimize the per-pixel error between
an event-based image and its corresponding ground-truth
segmentation mask, any supervised detection or segmen-
tation model can be used, given a diverse data set. To do
so, color images are labeled twice. Once with all the Rock
classification pixels removed and once with all the Not Rock
pixels removed. This can be combined into a trimap. Using
a background removal algorithm, a ground-truth mask
can be computed. This combined with the original color
image is used to train a supervised segmentation network.
To test, time-synced event and color images are input to the
trained network. The color prediction can serve as a labeled
image for the event image. This process can be iterated until
the accuracy of the event images asymptotes to that of the
color images.
A. Developing a Data Set for Color Classification
The color pictures, like the one in Figure 3 are hand-labelled
into a pair of images. One with all Rock pixels and one with
all Not Rock pixels. This can be done within a much larger
margin of error than would be possible if this were to result
in a ground-truth image directly.
Figure 4 shows that the human-generated labels are
often a lower fidelity because it is difficult to accurately
label each pixel. Due to the allowance of an area of uncer-
tainty, even though this is a binary classification problem, a
data set can be created more quickly.
These images are then combined using bitwise logical
operations as in 1 to create an area of uncertainty. This is
then combined with the all rock image to generate a tri-
map as seen in Figure 5. This is an image with an area of
certain rock, certain not rock, and a band of uncertainty
between them.
T R R Nh ,+=^(1)
where,
T is the set of all pixels that create the trimap.
R is the set of all white pixels belonging to Rock,
N is the set of all black pixels belonging to Not Rock.
Figure 3. Shows an image of the roof from a typical point-
of-view of a roof bolting drill. It is critical to be able to
differentiate the strap from the rock
Figure 4. Left Shows a hand-labeled image of the roof with
all the pixels belonging to the Rock classification removed.
Right Shows a hand-labeled image of the roof with all the
pixels belonging to the Not Rock classification removed
Figure 5. Shows the generated tri-map. Here the white pixels
represent Rock labels, the black pixels represent Not Rock label
and the grey pixels represent the area of uncertainty
This paper is laid out with a discussion of the planned
approach, a description of the experimental setup, a presen-
tation of the results, a discussion grounding the results in
the context of the project, and a conclusion.
METHODOLOGY
This effort is to enable the prediction of semantic segmenta-
tion masks given an event image taken during roof bolting
as shown in Figure 2.
Since the goal is to minimize the per-pixel error between
an event-based image and its corresponding ground-truth
segmentation mask, any supervised detection or segmen-
tation model can be used, given a diverse data set. To do
so, color images are labeled twice. Once with all the Rock
classification pixels removed and once with all the Not Rock
pixels removed. This can be combined into a trimap. Using
a background removal algorithm, a ground-truth mask
can be computed. This combined with the original color
image is used to train a supervised segmentation network.
To test, time-synced event and color images are input to the
trained network. The color prediction can serve as a labeled
image for the event image. This process can be iterated until
the accuracy of the event images asymptotes to that of the
color images.
A. Developing a Data Set for Color Classification
The color pictures, like the one in Figure 3 are hand-labelled
into a pair of images. One with all Rock pixels and one with
all Not Rock pixels. This can be done within a much larger
margin of error than would be possible if this were to result
in a ground-truth image directly.
Figure 4 shows that the human-generated labels are
often a lower fidelity because it is difficult to accurately
label each pixel. Due to the allowance of an area of uncer-
tainty, even though this is a binary classification problem, a
data set can be created more quickly.
These images are then combined using bitwise logical
operations as in 1 to create an area of uncertainty. This is
then combined with the all rock image to generate a tri-
map as seen in Figure 5. This is an image with an area of
certain rock, certain not rock, and a band of uncertainty
between them.
T R R Nh ,+=^(1)
where,
T is the set of all pixels that create the trimap.
R is the set of all white pixels belonging to Rock,
N is the set of all black pixels belonging to Not Rock.
Figure 3. Shows an image of the roof from a typical point-
of-view of a roof bolting drill. It is critical to be able to
differentiate the strap from the rock
Figure 4. Left Shows a hand-labeled image of the roof with
all the pixels belonging to the Rock classification removed.
Right Shows a hand-labeled image of the roof with all the
pixels belonging to the Not Rock classification removed
Figure 5. Shows the generated tri-map. Here the white pixels
represent Rock labels, the black pixels represent Not Rock label
and the grey pixels represent the area of uncertainty