International Mining | SEPTEMBER 2022 (Paul Moore)
Autonomy in surface mining is changing and it is changing fast. Up until now, and still today, the market more or less 100%
consists of automation of large primary rigid haul truck fleets, working with either Caterpillar or Komatsu and their Command for Hauling or FrontRunner systems respectively, with a couple of exceptions – Hitachi Construction Machinery (HCM) having its system running at one site in Australia, and ASI Mining close to handing over its solution that it has supplied to Roy Hill via an overall mine contract with Epiroc.
These systems are by nature expensive, and increasingly require the latest network infrastructure such as 4G LTE to function efficiently, depend on controller input in certain situations, plus with the exception of the Roy Hill project still lock the user into a particular fleet management system – Cat AHS still requires the use of MineStar and Komatsu requires use of Modular. But they are very well proven and the benefits have been enormous both from a safety and efficiency point of view. And it isn’t just about a few first mover operations today – Caterpillar told IM it now has 540 autonomous mining trucks operating (this is an increase of 40% from 2020). This is across 23 sites and with 13 separate customers. Komatsu’s numbers are similar.
But there is also a second market for autonomy – mines want the benefits of autonomy for mixed fleets, for ancillary machines and for smaller machines. There are several reasons.
Some big miners are looking at using larger fleets of smaller trucks to reduce maintenance
not restricting trucks to the pit itself; while smaller miners and quarry operators want a more flexible and open solution. Of course the big OEMs are working on this, and some details are given later, but a raft of new autonomy players are working on it too – companies like Wenco with Oxbotica, ABD Solutions, Steer, SafeAI, Pronto, xtonomy and Auto-mate.
ASI’s technology is also aimed at a more open solution, an example being it forming part of Liebherr’s open autonomy offer, which includes interoperable autonomy products for mixed fleet applications. Hexagon technology, including its autonomous mission management system, will also be used in the mine automation options offered by Liebherr to customers.
The market is also changing from level 4 automation to level 5 – where no controller interaction is needed at all and where the trucksmake AI-based decisions, such as how to react to an unknown obstacle themselves. This is happening in two ways – Cat and Komatsu are already making updates to their systems to allow machines to react on the fly, while the new players are aiming to embed these capabilities from the outset.
Then the elephant in the room is how to marry autonomy with the transition away from diesel. Again this is being worked on – mainly by the big OEMs currently as they know they have to get there to make the zero emissions solutions work in the first place, accounting for battery charge management. Even autonomous trucking under trolley is something not yet achieved in the market.
So the mining truck market is becoming more and other costs, including for longer hauls and open and customised in terms of power source, but also in terms of autonomy. The big miners will likely stick with the established OEM AHS systems for new mines and new fleet replacements. But for existing fleets, mixed fleets, ancillary machines, new projects involving smaller class trucks, they are likely to also consider other options – both the OEMs and the new players.
But too much diversity across different operations even within the same company is also likely to create inefficiencies – these new technologies have to introduced only where it makes economic and operational sense to do so. The reality is that there remains a lot of uncertainty as to how the mining industry will look by 2030 and that applies to autonomy, power source, battery strategy (charging or swapping) if using battery, the right size of truck…the list goes on. For some mines there is no doubt that a better option may be to go truckless with IPCC if it is economic. For others the solution may be different set-ups at different operations.
A new level playing field
Wenco International Mining Systems, owned by HCM, is one company that is confident of the new value proposition level 5 open autonomy brings to mining’s new level playing field. President and CEO Andrew Pyne told IM: “We committed ourselves to drive meaningful change to the autonomy paradigm back in 2017 because our Wencomine FMS customers wanted to start their autonomous journey but didn’t want to be forced in to using another FMS. We decided to go down the open autonomy route in a way that was right for the industry and would allow us to compete on our merits. We set out to be an innovator and disruptor, and some major miners that had little interaction with us have already told us they see us as a next generation autonomy provider. While it might seem altruistic, we believe we will be rewarded in time by focussing on new value creation for the customer. What started as a concept has moved on to some intensive engineering collaborations, but this is new technology so the projects will take time to mature.”
Its active projects also include the ASI/Epiroc– led project at Roy Hill, as Wencomine is the in- place FMS, which has provided invaluable experience as part of the world’s most advanced mixed fleet and OEM agnostic AHS project to date.
It has also been closely involved in testing and trials relating to the option of using larger fleets of smaller trucks in West Australian Iron Ore mines. Pyne told IM: “The value proposition is that for the next generation of their iron ore mines, the big players need to either build more overland conveyors, extend railheads with added crushing infrastructure, or long haul the ore from the mines. And for the latter option, switching to a fleet of smaller trucks that are capable of electrification and have a level 5 autonomy system on them could represent very significant cost savings.”
Back to open autonomy and Wenco continues to work closely with Oxbotica, with whom it signed a partnership agreement to work on open autonomy solutions for mining back in 2020.
Pyne told IM: “We perceive, based on quite a lot of research, that they still have the most technically capable autonomous-vehicle software, but also the right group of experienced people, to deliver something that is truly transformational.” He also referred to the fact that these emerging autonomy systems designed for smaller trucks and ancillary machines, retrofittable to any brand, as well as incorporating more AI, will still have to get regulatory approval when used outside of established AHS hubs like WA. Then there is the fact they will have to handle climate extremes including dust, fog, snow, temperature highs and lows etc.
But Pyne is also well aware that Wenco is just one of a group of next generation autonomy solution providers but argues that the capabilities of new options are being somewhat overstated. “I think despite what you read from companies in this space, we are all still a couple of years away from saying – ‘yes we can operationalise a large fleet with this technology.’ Why is this? First off while it is relatively easy to automate one truck and trial it, this is leagues behind orchestrating an autonomous fleet. Plus there are more specific aspects. Look at the excavator operator that currently calls in and kicks out the large autonomous rigid trucks – that role is already very challenging for them and it isn’t widely known but in some mines they are using two operators on one shift as it is too much for one operator to handle the AHS panel and their other tasks. The big OEMs are already being asked to semi-automate parts of the loading my solutions for mining back in 2020 cycle – increasingly the digging part will remain manual as that’s where the operator feel really exists, but the swing and load elements will be automated.”
In terms of the number of trucks increasing to many hundreds of smaller class machines, Pyne also believes that the use of mobile surge loading – in the form of the patented MMD Fully Mobile Surge Loader – will become more important, to decouple the excavator from the smaller trucks. This will of course allow mine operators to keep their existing large excavator fleet, while avoiding excessive queueing and represent more of a continuous loading process. What about the fleet management layer? “This is a big part of it – replacing 60 trucks with say 300 is a very different mine to manage and control. We need to re-architect our system so we can horizontally scale up in hardware terms – the cost per unit per truck has to come down for example. And when they become battery powered instead of diesel, how do we build that into the dispatching algorithms to account for battery charging or swapping?”
Based on what Wenco has done already and commitments that customers have already made, Pyne said he is confident that there will be major mines running larger fleets of smaller class autonomous trucks, certainly by 2030 and maybe even earlier – but probably not before 2026.
While NDAs mean details cannot be released, Wenco and Oxbotica have an autonomy pilot running at a major mine in North America, currently on a 220 t unit and going through multi- phase testing but with a view longer term of using it on smaller class electric machines. Pyne said the value proposition was the ability to retrofit and get the benefits of autonomy from existing fleets, but also to be able to keep control of your mine and allow you to buy whatever truck you want. “It also provides the customer with a lot more data than they are currently getting off the OEM autonomy systems. Plus the nature of the sensor and localisation technology means you don’t actually have to fit every vehicle on the mine with the autonomy system – today with level 4 OEM systems, the system doesn’t just involve the big mining trucks themselves – every single machine in the pit has to be fitted with all sorts of sensors, antennae and software, at great cost. A big mine might have over 200 light vehicles alone.” Then there is the remaining controller intervention needed when a truck stops due to an obstacle that it doesn’t recognise. It usually requires someone from a dedicated pit team to go and check it out and give the OK for the truck to restart. With the Wenco system all this would be handled by the truck’s onboard AI. It responds more like a human brain – the truck can sense where it is, understand its localisation compared to everything around it, and then make a decision to proceed or not, and which path to take around the obstacle. Plus you won’t have a tether of GPS either so are able to operate in GPS-denied environments. “The truck is only using a combination of radar, LiDAR and cameras with an AI layer over the top. It will choose the right combination of sensors based on the conditions and what you are trying to do. This ability to not rely on GPS also means you have a much larger addressable market.”
Moving on to another topic, what about Wenco’s being owned by Hitachi Construction Machinery? HCM of course has its own AHS running at Whitehaven Coal’s Maules Creek in which Wenco has been closely involved. HCM has also recently introduced the ConSite® Mine remote machine analytics solution on which it has worked with Wenco. Pyne suggested that the two companies are now working much more closely and strategically beyond Wenco just supplying the autonomy layer to HCM’s solutions and becoming more of a close partner in customer projects. Whilst the HCM AHS is a level 4 AHS system (like the other OEM systems) and still largely designed to operate with Hitachi equipment it is available now for customers to adopt. As the Wenco level 5 open autonomy solution becomes commercially available over the next few years, it gives HCM an option for its customers for small and ancillary vehicles, non- Hitachi rigid truck fleets and longer haul trucks. Overall, Pyne stated that it is fair to say that on autonomous mining strategy, HCM and Wenco are converging more than ever before. Cat on leading automation’s evolution Autonomy for mobile mining fleets is anything but a fixed or set system – it is constantly evolving with technology upgrades and functionality additions. Caterpillar Inc. through its Site 17 facility at its Tucson Proving Ground
typically puts out on average two software updates for its Command for hauling system every year – ensuring that its mining customers have access to the best that autonomy can offer.
On a recent visit to the TPG, IM had a unique insight into the depth of R&D that goes into Command for hauling. It included discussions with Michael Vance, Engineering Manager; Jag Samaraweera, Command for hauling Commercial; and Joe Forcash, Electronic Engineering Manager, Autonomy & Automation.
One of the areas Samaraweera talked about the importance of autonomy in Caterpillar’s journey towards zero emissions mining trucks – with the first battery trucks planned for rollout at multiple customer sites before 2030. “Command will play a key role in managing battery trucks. With diesel trucks, MineStar can assign the truck to the fuel bay, ‘once you’re at 20% fuel level you need to go to the fuel bay.’ With battery trucks you will need to know the battery level, and at a set point send the truck to charge or adjust its route to charge its battery so it always has enough power to complete its haul cycle.
Autonomy is a crucial part of the journey of getting to battery electric – in many ways the best route will be to start with a good foundation of running autonomy as is, then the transition to battery will be a lot easier. It will also make the change management a lot smoother. As many operations migrate toward autonomy and fleet
electrification we believe the combination of autonomy, connectivity, and electrification, when knit together, provides customers the safest and Autonomous Caterpillar 794 AC trucks at Anglo American’s Quellaveco copper minemost productive sites.”
Vance talked about the industry desire for a ‘lighter weight’ autonomy solution for some mine sites. “At one end of the spectrum, you have Command for hauling that is tech heavy, with high-end sensors like LiDAR. At the other end you have smaller sites which require a less tech heavy solution to get similar functionality on sites with less infrastructure, for example, relying more on cameras and being able to bring in the latest technology from automotive. The two will converge somewhat and the solution will really depend on the individual customer site – based on the need and the application. Command for hauling didn’t get where it is by accident – the big mines need the extra functionality and capability that it has – but other mines may not.”
What about other new Command for hauling functionalities that are being developed? Vance states: “We are working to add more automation to the process to make the trucks smarter. Today we are in the final testing stages of technology that is camera based – the camera allows the operations team to see what the obstacle is so if it is something like a contractor vehicle that happens to not be MineStar connected andshouldn’t be in that area, then they can intervene. The technical challenge is merging the camera image with the LiDAR image, but we have done this.”
On the largest Command for hauling sites, there are over 70 autonomous trucks and over 300 other vehicles including light vehicles, water trucks, graders, dozers, wheel loaders and even other non- autonomous rigid trucks. All those vehicles are on MineStar and have a radio transmitter, high-precision GPS and a display. The operators of those vehicles must know where the autonomous trucks are – on an autonomous site, the autonomous trucks have the right of way and the manned machine operators can see where they are. Blue lanes denote active autonomous truck routes where those trucks own that lane. Green is just a lane that exists in the mine model, and yellow means the autonomous truck has been assigned there but doesn’t yet have permission. These colour codings are very useful at intersections so manned vehicle operators know if they can go or wait for the autonomous truck to go past.
But the market for application of autonomy onto quarry class and smaller mining class machines is also a space where Caterpillar also has big plans and Forcash gave some insight into this project.
In June 2020, Caterpillar acquired select assets and hired employees from Marble Robot, Inc., a San Francisco, California based robot and autonomy technology solutions company. The acquisition was part of the company’s automation and autonomy strategy to meet the needs of smaller mines as well as the construction and quarrying, plus industrial and waste industries. The first machine to be equipped with the new autonomy system is a Cat 299D3 Compact Track Loader (CTL), based on a first customer in solar power. The CTL delivers solar panels into position. At a typical solar farm during the construction phase, there are solar modules and other components that need delivering across a large area.
Caterpillar takes the customer site design files and develops available routes for autonomy, which are then integrated with a tablet which is in wireless communication with the CTL. Using remote control, the operator can pick up a pallet then tell the CTL where it wants it delivered on the map, switching to autonomous mode in which the CTL can deliver and return with no interaction and no further network input needed as it has all the necessary information onboard. With this technology, one operator can manage up to four CTLs at the solar farm site.
Forcash stated: “It’s a nice introduction to a smaller machine – and is applicable to ancillary machines in mining but right now is focussed on a semi-autonomous, remote control solution for a solar farm site which is very isolated, where you have no network access. Over time we intend to integrate the technology into MineStar so you would have the ability to control it via a Command station. It is similar to what we have today with our semi-autonomous tractor system for our crawler dozers plus Command suite solutions for small wheel loaders and excavators that are already being used in port material handling.” What about its evolution up into larger wheel loaders, articulated dump trucks and other non- mining-class equipment? “Right now we are supporting customers at two ends of the spectrum – we have light construction, exemplified by the CTL, and Command for
Hauling for large miners. Our engineers are working toward leveraging our capabilities at both ends to provide a scalable solution across multiple customer applications,” Forcash continued. “The more repetitive the machine movements the more ready it will be for autonomy. This lighter weight approach to autonomy with the CTL allows Cat to bring autonomy to operations with a different cost structure – quarries and construction sites don’t operate with the same cost approach to big mines. Mines generally have all the network infrastructure already in place, so that every machine on the site can be connected relatively easily. The CTL can run fully autonomous without any network so is a lot easier for a quarry application from a long-term point of view. And where it makes sense, we will also drive the best elements of this technology into mining.”
He adds: “The solution is a combination of three things. First, what we have learned and developed for autonomous mining, including components and some of the software. Second is our Command remote control technology. And third, what we acquired with Marble. Those three pieces have really come together. It has been designed not as a one off but as a technology we can integrate and grow into the Cat product line. Automotive progress has also helped drive down cost in terms of hardware to help us be able to bring these Command solutions to smaller equipment – it would not have been possible even five years ago.”
AHS and longer lasting parts While big productivity gains combined with safety are the most obvious upsides to autonomous haulage, there are others – and one less obvious one is the extended component lifecycle that automating equipment brings with it. IM spoke at length to Tony Wickenheiser, Komatsu Mining Product Manager, Autonomous Systems about this phenomenon. He commented: “Operating the equipment within the design parameters is critical to optimising production but also for longer life of parts. When you automate the equipment, the machine is operating within the design parameters, which brings extended tyre, cylinders, motor components, and various equipment assembly parts. This automated practice also reduces waste with fluids and other consumables for the parts. All of these seemingly ancillary benefits add up to become big cost savings and big opportunities for decreasing waste for mining companies.”
Wickenheiser said that Komatsu did an initial deep dive into this subject going back a couple of years – based on the biggest bank of data that it had which was out of the Pilbara. “And it really correlates with what has been seen on tyre life and AHS. You see an increase in tyre life because you see an increase in change management when it comes to autonomous operation – such as better road maintenance and traffic management. Autonomous trucks drive in a manner that doesn’t push the limits of the truck – things like maybe not taking a corner as fast as an operator would. We see extended parts lifecycle with AHS mostly in chassis parts – in the front and rear suspensions – spindle hub and brake, and park brake.” He says Komatsu mining truck customers are typically changing out these parts depending on the application anywhere from 10,000 to 16,000 hours. With autonomy it is seeing up to a 40% extended parts life on these components.
But of course there are always other aspects to consider – because the truck doesn’t have an operator to provide ‘operator feel’ feedback and just runs – visual inspections like walkarounds become much more important, which are typically done during the fuelling. Though this also means the truck doesn’t have to be brought in for issues like the windscreen wipers not working or the windows not rolling down – these can be addressed in scheduled maintenance instead of immediately.
So how is maintenance handled differently for AHS? “Overall it is similar. But there is an electrical component that changes on our trucks which is part of our IP that is devoted to autonomous troubleshooting and is monitored by our technicians. The rest of the truck is dealt with as normal, including the use of distributors or the mine’s own maintenance crews for truck repairs and the use of the onboard KOMTRAX system for both proactive and preventative maintenance. Some mining customers are also adding further sensors of their own to provide even more diagnostics in specific areas such as
turbos to monitor temperatures. In many ways AHS is paving the way for the most advanced remote health monitoring yet achieved in mining.”
With no operator, which is normally the first line of defence, maintenance scheduling becomes very important, such as for the mentioned visual inspections for leaks and any other abnormalities. In addition, the KOMTRAX data also becomes critical in understanding any faults before they become serious. But using data analytics in the right way also means components in some cases can be run longer but still safely. “Taking rear suspensions as an example, if the scheduled replace and repair point is 18,000 hours, but there are no leaks and they’re holding pressure, then you can go well beyond that. And the AHS customers are doing that – pushing those component lives out. Less scheduled maintenance also means lower labour costs.” Komatsu has a dedicated analytics team that is actively quantifying and collating all of this lifecycle data from its growing number of AHS sites and fleets, to produce a meaningful cloud-based database “to help the customer quantify performance, productivity and truck health.”
When you look at engine life, this is based on volumes of fuel consumed. There are fuel savings as it is running more continuously so overall you are moving more tonnes per litre with a higher utilisation but from a fuel consumption point of view the AHS versus operator truck is about neutral.
It is also worth pointing out the economic benefit from extended parts and overall truck life with AHS is somewhat balanced by the fact that maintaining roads at the highest quality, also being wider (in some cases to allow for multiple tracks to avoid rut development) and with wider turns and wider load and dump areas to enable AHS comes with significant associated higher cost. Also, autonomy comes with its own additional expenses beyond just road maintenance – such as network upgrades and the fact every AHS truck has other vehicles supporting it – equipped with what Komatsu calls EMV kits – these include light vehicles for pit patrollers, graders, dozers and shovel operators.
On the wider autonomous market, Wickenheiser also confirmed that Komatsu is actively looking at bringing autonomy to its smaller truck classes as well as developing autonomy to manage its new power agnostic truck platform. “Smaller gold and coal mines, plus quarries and industrial minerals operations are all looking to the advantages of autonomy but they have both smaller trucks and smaller fleets – we just have to find a way to make the AHS proposition economic for these operators, and of course competitive and profitable for us. But it’s a challenge, especially in emerging markets where many mines use smaller trucks but have a totally different cost structure.”
The advantage of route path planning Auto-mate – another one of the leading “new autonomy” innovators says putting a brain on the mining truck gives you a Route Path Planning (RPP) advantage. “While that certainly sounds impressive – here is what it means for you; if you’re a CEO of a mining company, it means a faster return on your investment with minimised risk of downtime, if you’re an operations manager, it means meeting your targets more efficiently, as an operator, it means confidence and ease of use, and as an HR manager it means a simple transition for the team to autonomy.”
For Auto-mate, the recently held Diggers & Dealers Conference in Kalgoorlie in Western Australia included a range of conversations on autonomy – including the RPP advantage.
“You’re a haul truck controller, yes that’s right, not the driver, and you’re very excited to be part of the new automation team – you’ve been allocated the position of planning and monitoring the routes for the autonomous trucks at a particular mining operation and you’re looking forward to watching as the automated vehicles do what they should. The mine site has just completed a transition to automation for three of its Cat 775 trucks, and as a supporter of this transition, you’re looking forward to reporting on the success of the move to automation to your operations manager and CEO.” The company continues: “You take your time
on planning the route, taking care in ensuring the waypoints are accurate. It takes a little longer than anticipated as the route includes some inclines and declines which need to be considered in determining the waypoints as you will need to specify the Z axis (height) as the site scan file came back corrupted this morning. The significance of this plan is not lost on you – the CEO is keen to report back to the board on the savings and efficiencies gained, illustrating the value for money invested in automation, to the board. Finally, when the automated trucks are on their way, you can sit back and enjoy the comfort and convenience of the operations centre located a way away from the hot and dusty environment in which the truck is operating, autonomously.”
You can see a digital rendering of the operation, and you can see the autonomous truck making its way along a mine road as planned. But then the truck stops, and as a result, all autonomous vehicles that need the lane it was using will need to wait and there are strict mine safety regulations in place for anyone approaching the stopped truck. The sensors on the truck have detected an unexpected obstacle.
Today, this takes a little time to resolve, as a person is required to visually inspect the environment and to determine the safest way to resolve the issue and enable the truck to proceed on its autonomous path. “Finally, you’re informed that the obstacle was a medium sized rock from a
previous spill event and it has now been cleared from the lane. Frustratingly the inspection reports that there was more than enough space for the truck to pass it to the side and safely continue on its route and it was a pity there wasn’t someone in the cab to make the decision and continue on. However, it is now on its way.”
Your bubble of excitement is fast deflating… considering the downtime of not only the truck, but the other assets held up behind it, as well as the negative tone from the inspection team, this is not going to be the planned, very impressive, kick off report to the operations manager, let alone the CEO.
So what does RPP do and why is it a key element? “With RPP the waypoints are there merely to show the planned route, however, this planned route includes a corridor within which the vehicle will find the optimal path. The width of the corridor, the area within which the vehicle can choose its optimal path, is adjustable too, allowing maximum flexibility for the vehicle.
Should an obstacle be detected, the onboard intelligence of the autonomous solution will assess if it is safe to pass within the autonomous corridor, and if so, it will continue on its planned route with no stopping or downtime.”
Auto-mate describes itself as a unique mining automation provider that offers asset level automation which determines optimum pathing and executes missions. “Auto-mate’s RPP considers 100+ factors constantly to determine the optimum path for assets to travel, avoiding obstacles, navigating terrain and complying with all mission parameters.”
xtonomy and the rise of radar
xtonomy’s mantra is to provide “autonomy everywhere.” It says its AHS offering for trucks finds application in mining environments, autonomously hauling material and interacting with semi-automated shovels, loaders and crushers, while utilising a robust radar-based perception systems for GNSS-denied and rough environments. Within its automation ecosystem it also provides starting points towards full autonomy, like operator assistance, tele- operation and collision avoidance systems.
The company was founded in 2011 as a R&D subsidiary of indurad GmbH – the industrial radar company. The full xtonomy spin-off from indurad has recently been launched with the long- standing experience of its robotics team. “The objective of this new venture is to bring our unique and proven autonomy ecosystem to market as a stand-alone company that has envisioned to provide autonomy everywhere. We have extensive experience in mobile machine automation based on radar perception, applied in underground and surface mining as well as industrial applications. We work in a consultative and solution-oriented manner in close cooperation with our customers.”
xtonomy says its modular technology platform enables differing levels of automation, from proximity detection and collision avoidance to full-fleet mixed autonomy. The technology platform leads to increased operator safety, machine availability and performance while reducing costs for clients. The systems are retrofittable to any vehicle and the solution can be progressively upgraded to meet new operational requirements. CEO Christian Riedel: “The modular architecture enables automation of all mobile machines in off-highway applications including their work functions. Besides haulage trucks or loaders in surface, underground or hybrid applications, the system enables automation of a great variety of machines, such as multi-service vehicles in tunneling or utility vehicles. Based on +15 years of experience in R&D robotic domains, xtonomy has developed and utilises a unique belief-aware reasoning-AI for a highly dynamic, adaptive and smart mission planning. The AI mimics the way a person would flexibly reason about his tasks, while being fully comprehensible and deterministic.”
He adds: “Our verifiable risk-aware autonomous decision-making AI enables a high recovery capability from failure modes leading to highest OEE and is capable of adaptively planning even very complex tasks, allowing autonomous operation in partially unknown and changing environments/situations. This advanced networked belief-aware fleet traffic optimisation maximises productivity of the autonomous fleet operation in a holistic context such as loading and unloading equipment or manual participants. In practice this means that the system automatically resolves bottlenecks in the material flow process from face to crusher.
xtonomy’s verifiable risk-aware autonomous decision-making AI in action
Furthermore, the autonomous haulage trucks continuously adapt its loading position to the current shovel position.”
xtonomy says it is implementing the latest machine learning techniques for the development of highest detection and classification capabilities in its radar and vision perception modules. This means that even smallest objects, such as rocks in highly complex and unstructured environments may be detected and consequently avoided by the autonomous vehicle. Furthermore, this enables highly reliable radar driveability mapping of entire mines but also critical work areas such as the truck spotting or crusher entry zone. “Consequently, tyre damage may be also minimised for manual vehicles and berms may be monitored for safety compliance.”
Riedel concluded: “The xtonomy AHS is designed to be adaptive for different mine sizes and ever-changing mine conditions, and thus works not only in large, but also for highly dynamic smaller and medium sites. It has not only proven to work on pre-mapped paths, but also in very unstructured environments, where loading zones, drivable area, routes, etc are constantly changing. By using robust radar onboard mapping and object detection it continuously perceives the environment. Its motion planners freely plan the best routes and dynamically react to changes. Furthermore, it handles the interaction of multiple trucks in such unstructured environments. This is only possible through leveraging Machine Learning (ML) and AI for a very high degree of autonomy at highest system levels.”
ABD Solutions launches Indigo Drive
Described as a complete solution to add autonomy to existing vehicles; “Indigo Drive is an ecosystem of technology that provides existing vehicles with autonomy, quickly and cost- effectively,” says Matthew Price, Managing Director of ABD Solutions. “The mining industry is continually looking to extend the lifecycle of its existing fleets and automation is a critical part of this. Not only does vehicle automation have clear safety benefits, it also allows for significant cost savings.”
Indigo Drive offers all hardware and software required to provide a complete, integrated, automation solution, available from a single supplier.
Retrofitting existing vehicles with autonomy requires a flexible solution that can be tailored to a specific vehicle, environment and operational scenario. A core modular, certified and secure software eco-system is used to build the various elements required for automation; including vehicle management, vehicle control actuation, communication, sense and detect, health and diagnostics and third-party integration.
This modular approach enables Indigo Drive to integrate with core and third-party technologies to ensure that specific requirements are met. The result is almost any vehicle can be retrofitted with autonomy and then fully integrated into any end user’s existing operational/fleet management system.
There are several major benefits to automating vehicles in the mining industry; it reduces risk to personnel in dangerous environments, increases efficiencies, reduces emissions, and maximises the life and versatility of legacy vehicle platforms and infrastructure.
In a mining environment where tasks are often repetitive, all the vehicles involved can be synchronised. Task planning software can ensure that all the vehicles take the most efficient route, reducing fuel consumption and the time to complete the operation. Vehicles that have the ability to detect obstructions can communicate them to the fleet management system to prevent others from encountering the same issue. Once the obstruction has been flagged the vehicle management system can generate an alternative route or a human can remote into the vehicle to immediately assess the situation.
One of the welcome side effects of autonomy is the digitisation of information from the vehicle. It creates new data streams that are invaluable to fleet operators. The vehicle’s speed, location, fuel level, diagnostic status, distance to target etc. can all be relayed via the vehicle management toolset. Data analytics can process this information to assist with operational decisions, fleet maintenance scheduling and procurement and route planning.
ABD Solutions is a member of the AB Dynamics Group and says it will benefit from its knowledge of developing and supplying thousands of driving robots to the world’s leading vehicle manufacturers, proving grounds and regulatory authorities. The Group has decades of experience in vehicle autonomy and simulation.
“AB Dynamics has been automating vehicles in one way or another for development and test purposes for decades,” continues Price. “Now, through ABD Solutions and the development of Indigo Drive, the mining industry can take advantage of this expertise to automate their existing fleets today.”
Knowing what you don’t know
One of the newest entrants to the mining autonomous haulage space is Pronto, which many in the industry first encountered at MINExpo 2021. Having cut its teeth in state of the art on-road autonomous driving, the San Francisco-based company has approached mining AHS from a new point of
Ognen Stojanovski, COO & Co-Founder, who before Pronto helped launch Otto and Uber Freight, both of which focused on developing technology for the trucking industry, told IM: “We didn’t start off looking to build a mining or even an off-road autonomy system. Much of our leadership team has been closely involved with on-road autonomous vehicles for some time – almost 20 years in fact – making a lot of breakthroughs with new technologies. We started out with some advanced driver assist solutions for on-road trucks and we had a customer with a commercial trucking fleet but which also had some material haulage contracts in mines and wanted to use the same tech in what was a much more controlled environment and not only that, use it to its fully autonomous capability. The more we learned about mining, the better fit we saw it having with our technology.”
Stojanovski said the Pronto team initially assumed from what they had seen the large OEMs doing that autonomous mining was already solved, “but we quickly realised it was hardware heavy and essentially restricted to ultraclass truck primary haulage and that our software and AI heavy approach could offer a much more general autonomy for mining, able to handle a much greater range of mining environments with a lot fewer sensors, much lower costs and much less operational restrictions. We seriously started looking at mining in late 2019 and by October 2020 we were already running loads unmanned in quarries.” Today Pronto has already deployed not just for R&D and trials – vehicles equipped with its system are hauling ore in daily production and it believes it was the first in the US to do so. It also believes it was the first to be in regular mine production using autonomous vehicles in the smaller equipment classes. While he wouldn’t give specifics, he said that it involves multiple sites; still mainly today quarry environments with auto ADTs but also on smaller rigid trucks. IM has also previously published information on its growing partnership with South Africa-headquartered global ADT major Bell Equipment while Pronto says it also has a lot of new projects in the works.
Stojanovski adds on the potential: “You are looking at a whole other part of the market. Many of our customers don’t even have a fleet management system – quarries and small mines with five or six, or in some cases only two trucks. You can manage our system with just a phone app and sufficient connectivity through existing public LTE – which we can also upgrade in-house if needed as we have our own private LTE company.”
From the mining perspective, Advisory Specialist Owain Morton commented: “From my career in mining, we had mixed fleets which was one major issue, but also quickly realised how deep the investment had to go with the big truck OEM AHS providers into hardware to support the AHS plus a very long term commitment to that one supplier which at the time we just weren’t ready to do and we weren’t the only major miner taking that position. We were focused on mine production and our metals revenue stream. Not only that but these autonomous systems were adding to already huge data lakes which were causing their own headaches. Pronto gives a simpler solution.”
He believes that for the tier 2 large scale mines but also for all the other small mines out there that Pronto hits the target from an outside in perspective. “For all the operators you can take out of smaller than ultraclass trucks – it could be possible to redeploy and re-certify in many other areas in mining where there are huge skills shortages. Plus, there are many regions where there are already shortages of truck operators – especially in contracting companies. Many tier 2 miners don’t have the required fleet management system in place or the depth of relationship with the big OEMs to move complex AHS deals forward.”
Plus, despite the current Pronto operations being mainly US-based, the company has global ambitions. So far the focus has been on North America, but Pronto is in discussions about other projects in Europe and South America, with updates to be given in due course. Ognen said that the first deployment outside the US is set to start operation in September 2022.
What does the on-road experience bring to mining? Morton says: “On-road has much faster traffic, meaning AI decision making has to be faster, you are closer to a larger number of vehicles and they are less predictable, plus required stopping distances are shorter. You have more delineated lanes, crossroads, roundabouts, signs to be followed. Even from a collision avoidance and object detection perspective, we feel our solution is more flexible, creates less false positives and doesn’t interrupt production. In fact, the stack needed for our collision avoidance alone can effectively drive the truck autonomously. There has been resistance in mining to using technology developed for on- road with the argument that it is too different – but truck operators in mines when they finish their shift are getting in their private cars that have more advanced systems and many times more AI than the mining vehicles have that they just drove, so why not have some of that in the mine vehicles where it is applicable?”
But how does the Pronto tech actually work? The big difference to anything else out there, and this also relates to the on-road experience, is that it is camera vision based with no radar or LiDAR. Cat Culkin, Pronto CTO: “The benefits of camera-based systems are many in mining. First off, they are relatively speaking much lower in cost as we use automotive grade cameras; plus they are very robust in dusty and high vibration environments. There aren’t moving parts that can get damaged by dust or water ingress. Our software allows us to upgrade our digital imagery capabilities on the fly, unlike existing AHS requiring the latest sensors which often need to be upgraded by replacement. Neural networks processing data from cameras are also very good at classification of things in the vehicle’s path such as correctly identifying whether it is debris or a tyre or a person. But they are also good at knowing what they don’t know. They see the world more like a person does – seeing where the berms are, assessing the driveable area on a muddy, wet and changeable road. In foggy or snowy conditions, the cameras perform slightly better than the human eye and at night can just use the headlights to light up the road as a manually driven vehicle would. Our main front-facing camera behind the windscreen will maintain its clear view by coordinating with wipers just as in manned equipment.”
Redundancy is provided by V2V on other vehicles. The software analyses the imagery in real time and can decide the safe speed to drive given the conditions while still maximising efficiency. “If something comes into the driveable space like an animal which then clears the space, the truck will retard or slow down but won’t need to stop. And again, almost eliminating false positives that other sensors are affected by is a big advantage. So our system is really good at stopping for things that need to be stopped for and for ignoring those that don’t.” Current AHS trucks mainly follow a set route or factor in slight deviations to that route each time to avoid rut generation. Pronto’s system can do this but also has dynamic path planning, not only for wear levelling but also to allow for the excavator having just moved along the face, for example. The loader operator still brings the trucks in and sends them out but the truck itself is best able to determine an optimum path to the loader in the first place. This also applies to following the best path to a dumping area whether that be a waste dump, stockpile or crushing station. “Manned vehicles entering our autonomous vehicle zones just need a small transponder. The loader still manages trucks coming in and out but just has to hold up the bucket which will tell the system the spotting point automatically.” Pronto says it aims to deliver full autonomy to any truck, in any mine, anywhere, and is deployable today; simplifying autonomy for the industry.