FMTech Insights: How Augmented Reality (AR) Really Becomes a Game Changer

FMTech Insights is a series where subject matter experts and thought leaders in FMTech share their knowledge and opinions on the topic of their expertise; helping the industry learn, innovate and grow. The series will cover every sector in FM – from workplace management, operations and maintenance, energy management, waste management and everything in between.

Adrian Merkel is the Managing Director at Framence GmbH, a hybrid model digital twin platform. With Framence, its more than just pictures as any information from external sources can be integrated into the platform, resulting in very accurate digital twins that replace the need for most physical on-site inspections.

FMTech Insights: How Augmented Reality (AR) Really Becomes a Game Changer

Since Microsoft launched the first HoloLens in 2016, many companies have realized that augmented reality, that is, the expansion of reality with digital content, can become a game-changer in many areas. Unfortunately, none of these use cases have really hit the market and have progressed beyond the status of being research or test projects. There are some exciting applications, especially in the consumer area. However, these applications cannot be transferred to commercial or industrial applications.

There are some reasons for the lack of market penetration of AR technology. In many cases, specialized glasses are already used today to blend digital information with reality. However, these glasses are relatively expensive making them widely unavailable in practice. They  also significantly restrict the user's field of vision, rendering them incompatible with occupational safety. Finally, the glasses are quite heavy, which makes long-term use rather difficult.

The technological approach that is used for AR often stands in the way of its practical use. In most cases, the device used creates a 3D model of the environment in which the virtual content is then located and displayed. However, this focus on the end device prevents open systems into which ad hoc information can be imported or data can be exchanged with other devices. A change of hardware or collaborative work is almost impossible.

Due to the problems with the glasses, methods of integrating cell phones or tablets have been developed.

The information is positioned via the location of the device used. This localization is carried out using markers of similar size QR codes that are attached in the physical environment. However, this is very time-consuming in terms of preparation and subsequent maintenance and is therefore not economically feasible in practice.

What approach can therefore be developed to solve the existing problems and makes AR economically viable?

Until the glasses make a serious technological leap, we can only use tablets and smartphones as end devices. An advantage to this is since these devices are already in daily use in every company, the question of availability does not even arise. In addition, they are inexpensive and are already available in many different designs for industrial environments: outdoor, rugged environments, or even explosion-proof. Concepts such as "bring-your-own-device" can be considered with this approach, which benefits a simple and, above all, fast implementation.

In a technical environment in particular, information must be displayed very precisely at the right location within reality. Information projected on the wrong pipe can lead to very expensive problems!

To ensure the exact localization and avoid the problems of physical markers, the use of "dynamic markers"  that are automatically generated by an AI-supported process based on the images available. A corresponding algorithm on the tablet recognizes these feature points in the camera live stream and can then position itself accordingly.

The time-consuming process of attaching and positioning markers is no longer necessary.

In addition to these technical refinements, a hybrid data model also offers another, very decisive advantage – 3D models are no longer necessary for AR. As mentioned above, as the exact position of the device is known, we can also precisely position the camera live stream as well. The live stream of the device then simply replaces the existing images and takes their place. All information from the hybrid model is automatically available in AR. Existing points of information, 3D objects, live data from sensors and measuring devices or BIM models and floor plans can be localized in the correct positions without any further action. This complete digital twin can therefore be easily transported into reality.

With live access to the central model, all changes, or enhancements, regardless of where they are made, are immediately available. This makes collaborative work by distributed colleagues possible without any problems. A specialist in the back office can, for example, pass on information to colleagues on-site on an ad hoc basis. The on-site technician can also enter important information directly via voice input or ask colleagues for advice.

At Framence we believe that AR is and will be a game-changer. All of the above functionalities and features - the combination of hardware (i.e., cell phones & tablets) available on the market, latest IT technology, and a hybrid model can be done with Framence. We have liberated augmented reality from the laboratories and made it usable in practice. 

With Framence, you can implement AR applications in your company quickly, easily, and above all cost-effectively.

Disclaimer: the opinion of the author does not necessarily reflect that of the Global Alliance of Facility Management Innovators.