Mixed reality could play a significant role in the future of mining, especially in the post-Covid-19 world where reduced travel and physical distancing are likely to continue for some time. Now more than ever, allowing for mining experts to be able to continue to guide and support field service staff in remote areas or hazardous conditions is of vital importance.
One solution, the RemoteSpark software platform, built by Canadian augmented and mixed reality support company Kognitiv Spark, has been created to enable remote workers to establish a secure, low-bandwidth mixed reality connection, between people anywhere in the world. It creates a shared audio and video connection while allowing experts to share and use images, 3D assets, and various documents, which appear as holograms in the users’ real-life environment.
Go deeper with GlobalData
Capabilities of Kognitive Spark’s ‘RemoteSpark’ mixed reality tool
As digital connectivity in mines is becoming essential, Kognitiv Spark chief technology officer and co-founder Ryan Groom thinks that the ability to use mixed reality underground is increasingly improving.
“Before, we used to go into the dark zone – your Wi-Fi didn’t work, your cell phone didn’t work, but now the mines are being wired for internet connectivity so [miners] can go to the deepest and darkest parts of a mine and still be able to get signal and get assistance,” he says.
RemoteSpark provides a video feed from HoloLens, which is a Microsoft-manufactured mixed reality headset. The HoloLens spatial computing system uses multiple sensors, advanced optics, and holographic processing to create holograms that can be used to display information and blend with the real world.
The feed is taken from the remote user to the expert, which ensures that the first person to observe a potential mining incident can do it hands free, which is quite critical. This initial step of the process is not mixed reality itself, it’s what comes next that is truly interesting.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataGroom explains: “In addition, we have enough horsepower left to be able to do a true mixed reality call, which means we can create holograms while doing a call. Most products can do 2D, so you have video call and a PDF or a photo, while we can put 3D holograms right in front of you, whatever you’re working on. These holograms can be annotated and animated.”
In cases where companies are working with complex machinery, if they can rely on a 3D model version instead of a 400-page manual, this would ensure a quicker and more efficient learning experience.
Increased safety and real-life testing of the technology
RemoteSpark can communicate with live sensor data from mining sites in order to improve operational safety and focuses on things that cannot be easily seen with a naked eye, like air quality or temperature.
The internet of things (IoT) capabilities of the mixed reality platform could inform workers if a given system is operating outside of its baseline and requires repair, or if a system anomaly could create a safety risk to workers on the job site.
“If you have a propane leak sensor that can detect a concentration of propane, you put a headset on and actually visualise that, or before you send a crew in to do a repair you can see what part the system has highlighted. It’s early days for that, but through holograms we get the most natural way to consume data,” Groom says.
When it comes to testing of the system, Kognitiv Spark has worked with partners to test network bandwidth capability and figure out how low-connectivity environments impact the platform. Testing has proven to be especially effective in rural Canada where there are plenty of spots with poor Internet connectivity and it’s easy to detect how bad a network gets before the video signal breaks up.
Groom shares that, before Covid-19, much of the mixed reality testing was quite immobile as it took place in a boardroom or an office.
“But now people need it immediately, so the testing is actually having to deal with real situations. We’ve had people in South Dakota take it out to oil and gas sites where connectivity is very poor and they put on their headset and do real live testing. We can test it here in our own backyard, but the best test is when the customer takes it into harsh condition and it works. We find that’s a great test,” he says.
Technical challenges and future plans
The creation of the technology has not gone without challenges and hardships, especially in the initial building stages of the system.
While a large percentage of people that build for HoloLens or mixed reality use a popular gaming engine application called Unity, the team found that it needed more horsepower for the application. They decided to build their own rendering engine, which proved to be no mean feat for a small team.
“To be able to balance all those very heavy workloads and have that run on the HoloLens, which is just a mobile device, and then on an ARM processor…For that, we needed to be as efficient as possible,” Groom recalls.
When it comes to the future plans of development of Remote Spark, the company is looking to take sensor data from the real world as inputs so as to render holograms in the highest fidelity possible.
For example, monitoring the illumination of an area where the hologram is rendered could give realistic lighting, reflections, and shadows on the hologram, thus bringing a new level of realism to the mixed reality environment.
“Through immersion, when this is coupled with the increased context provided by industrial IoT sensor data, we’re looking at a fully realised, intuitively understood evolution in computing systems,” Groom says.
Since its creation, the RemoteSpark mixed reality tool has been used by a number of industrial and defence organizations including the Royal Canadian Navy, Valcom, Surepoint Group, and Rogers Electric and Machine for remote worker support.
