2
(mobile and stationary) in the environment, b) realistic,
deterministic, and comprehensive projection of the paths
and activities those objects will have as a function of time
(including probability fields), and c) rapid, meaningful
evaluation of projected interactions in expected and alter-
nate futures. This paper focuses on the second element—
Path Projection.
PATH PROJECTION
Path projection is a prediction of the motion and behav-
ior of an object, with corresponding confidence (inverse
of variance) in that prediction. In its simplest form, track
projection would project the position and velocity of a pro-
jectile influenced only by gravity and air friction. In more
practical applications, it would project the position, veloc-
ity, orientation, and auxiliary activity (dumping, spraying,
drilling, etc.) of a piece of mobile equipment as a function
of time. The associated time horizon depends on what time
the equipment requires to intervene on behalf of safety.
This projection is not simply dynamics and kinematics, it
is heavily dependent on the context of an object’s motion
and state. Some contributors to context are external phys-
ics (road conditions, weather conditions, slope, curvature,
etc.), internal physics (braking and acceleration ability,
turning potential, center of gravity, weight, etc.), current
activity (driving, dumping, drilling, waiting, etc.), expected
behavior (road follower, typical speed versus. slope, etc.),
and intent (change tasks, destination, changing speed, etc.).
Each of these contexts contribute to the calculation of pro-
jected path and associated variance (inverse of confidence).
For MSA to work, the perception engine needs to pass
on object identification (what is it), current location (where
it is) and a profile of current direction and speed (where it’s
heading), and the current activity context (what it’s doing).
Most equipment on a mine site is distinct and known as
opposed to automotive environment. This enables recogni-
tion of a discrete set of objects that are well known and rela-
tively understood and predictable (baseline behavior and
capabilities). On a mine site, the MSA system would have
a database of the characteristics of all the equipment types
in its environment.
Introducing Context
Utilizing context is fundamentally a function that adjusts
the calculation based on the current state. Context can
change a set of conditions and/or applicable algorithms,
or it can adjust the parameters and coefficients of dynamic
and kinematic calculations. When making decisions about
what context to include in MSA, anything that can materi-
ally affect the action of an object must be modeled. Objects
are anything in the environment that an MSA equipped
machine can interact with (pickup trucks, people, boulders,
other equipment, etc.). Machines can also interact with
features of the environment such as berms, slopes, edges,
roads, etc.), but the topography is typically static, at least
in the short term. For an example, road condition is a con-
text that can affect how the machine will interact with the
environment (stopping distance, turn radius, maximum
safe speed, etc.).
EXTENSION TO PERCEPTION
The human parallel to Path Projection in an MSA system
is the way we subconsciously calculate what movements
will be made by objects in our immediate world and evalu-
ate choices to avoid an incident. We do this while driving,
walking down a street, pushing a shopping cart, playing
sports, and virtually every task in which we engage involv-
ing movement. We usually do not have conscious thoughts
about what we are doing. It is just automatic. The goal of
this project is to develop an approach that emulates that
ability by harnessing modern sensor technology, advances
in computing potential, and emergence of new mathemati-
cal strategies that make this within reach.
Path projection must take all we know about objects or
environmental features and create a best guess about what
will happen in the near future and relay the level of confi-
dence in that projection. It must tolerate sensor inaccura-
cies, lag times, conflicting data, environmental influences,
object behavior and characteristics, etc. It must evaluate the
probability profile surrounding it and other objects. This
is all because any safety intervention system cannot have
false trips or anomalous responses. The gateway to industry
acceptance is that the system perform better than a human
would in preventing unwanted interactions [3].
To have adequate information to model a path more
than a couple seconds into the future, mathematics would
need to accommodate position and 3 derivatives: speed,
acceleration, and jerk [4]. The system would also need to
calculate the variance for position as a function of time.
That variance would include anything in the system or
environment that would contribute to inaccuracy (e.g.,
sensor input) or path deviation (e.g., sudden change in
object acceleration or direction). Given position variance
handed from the perception engine, the path model must
be smoothed using a 4th order (to model jerk) curve fit.
Whatever model is used, it must also produce variances for
each variable it is reporting. This would typically be derived
from the co-variance matrix that is derived during matrix
calculation.
(mobile and stationary) in the environment, b) realistic,
deterministic, and comprehensive projection of the paths
and activities those objects will have as a function of time
(including probability fields), and c) rapid, meaningful
evaluation of projected interactions in expected and alter-
nate futures. This paper focuses on the second element—
Path Projection.
PATH PROJECTION
Path projection is a prediction of the motion and behav-
ior of an object, with corresponding confidence (inverse
of variance) in that prediction. In its simplest form, track
projection would project the position and velocity of a pro-
jectile influenced only by gravity and air friction. In more
practical applications, it would project the position, veloc-
ity, orientation, and auxiliary activity (dumping, spraying,
drilling, etc.) of a piece of mobile equipment as a function
of time. The associated time horizon depends on what time
the equipment requires to intervene on behalf of safety.
This projection is not simply dynamics and kinematics, it
is heavily dependent on the context of an object’s motion
and state. Some contributors to context are external phys-
ics (road conditions, weather conditions, slope, curvature,
etc.), internal physics (braking and acceleration ability,
turning potential, center of gravity, weight, etc.), current
activity (driving, dumping, drilling, waiting, etc.), expected
behavior (road follower, typical speed versus. slope, etc.),
and intent (change tasks, destination, changing speed, etc.).
Each of these contexts contribute to the calculation of pro-
jected path and associated variance (inverse of confidence).
For MSA to work, the perception engine needs to pass
on object identification (what is it), current location (where
it is) and a profile of current direction and speed (where it’s
heading), and the current activity context (what it’s doing).
Most equipment on a mine site is distinct and known as
opposed to automotive environment. This enables recogni-
tion of a discrete set of objects that are well known and rela-
tively understood and predictable (baseline behavior and
capabilities). On a mine site, the MSA system would have
a database of the characteristics of all the equipment types
in its environment.
Introducing Context
Utilizing context is fundamentally a function that adjusts
the calculation based on the current state. Context can
change a set of conditions and/or applicable algorithms,
or it can adjust the parameters and coefficients of dynamic
and kinematic calculations. When making decisions about
what context to include in MSA, anything that can materi-
ally affect the action of an object must be modeled. Objects
are anything in the environment that an MSA equipped
machine can interact with (pickup trucks, people, boulders,
other equipment, etc.). Machines can also interact with
features of the environment such as berms, slopes, edges,
roads, etc.), but the topography is typically static, at least
in the short term. For an example, road condition is a con-
text that can affect how the machine will interact with the
environment (stopping distance, turn radius, maximum
safe speed, etc.).
EXTENSION TO PERCEPTION
The human parallel to Path Projection in an MSA system
is the way we subconsciously calculate what movements
will be made by objects in our immediate world and evalu-
ate choices to avoid an incident. We do this while driving,
walking down a street, pushing a shopping cart, playing
sports, and virtually every task in which we engage involv-
ing movement. We usually do not have conscious thoughts
about what we are doing. It is just automatic. The goal of
this project is to develop an approach that emulates that
ability by harnessing modern sensor technology, advances
in computing potential, and emergence of new mathemati-
cal strategies that make this within reach.
Path projection must take all we know about objects or
environmental features and create a best guess about what
will happen in the near future and relay the level of confi-
dence in that projection. It must tolerate sensor inaccura-
cies, lag times, conflicting data, environmental influences,
object behavior and characteristics, etc. It must evaluate the
probability profile surrounding it and other objects. This
is all because any safety intervention system cannot have
false trips or anomalous responses. The gateway to industry
acceptance is that the system perform better than a human
would in preventing unwanted interactions [3].
To have adequate information to model a path more
than a couple seconds into the future, mathematics would
need to accommodate position and 3 derivatives: speed,
acceleration, and jerk [4]. The system would also need to
calculate the variance for position as a function of time.
That variance would include anything in the system or
environment that would contribute to inaccuracy (e.g.,
sensor input) or path deviation (e.g., sudden change in
object acceleration or direction). Given position variance
handed from the perception engine, the path model must
be smoothed using a 4th order (to model jerk) curve fit.
Whatever model is used, it must also produce variances for
each variable it is reporting. This would typically be derived
from the co-variance matrix that is derived during matrix
calculation.