Keywords

Industry 4.0; Augmented reality; Assembly; Human-robot-collaboration; Training; Product control quality; Building monitoring

Introduction

Nowadays Augmented Reality (AR) is a renowned concept but its origins can be dated back to the sixties when Sutherland proposed the first acknowledged AR prototype, based on a Head Mounted Device (HMD) [1-53], in 1968. After Sutherland, more than twenty years passed before the term Augmented Reality was officially coined by Tom Caudell and David Mizell, two scientists employed at Boeing Corporation, which developed an experimental AR system with the purpose of simplifying the manufacturing process of the air company [9]. The concept of AR has been formalized only in 1994 by Milgram and Kishino [39] as the relationship among real space, virtual space and all the intermediate forms of mixed space. In 1999, the ARToolKit [1] was publicly disclosed at SIGGRAPH [5], the first time a working AR system had been seen outside of research labs in the US, heralding the start of the AR industry, released as open source software after only two years.

Until some years ago, the lack of cost-affordable devices was the main barrier to a wide adoption of AR applications. Nowadays, the widespread adoption of mobile devices has removed this limitation, as smartphones and tablets feature all the sensors and processing units needed to develop and deploy AR applications. Moreover, the technology innovations that affect mobile devices can produce new challenging products, commonly referred to as wearables, and industries are moving forward with new categories of AR devices, such as Emacula contact lenses from Innovega [2], the Vuzix Blade 3000 AR glasses [6] or the Meta 2 AR headset [4]. The global market for augmented reality is growing fast and the pervasive adoption of AR technologies implies an undeniable impact on the society.

Overall, the application of AR to the industry domain is relevant since it greatly improves the communication in product design and production development: it helps to identify and avoid design errors in early stages of the development process; it reduces the number of physical prototypes and saves time and cost for enterprises. AR is considered as a valuable tool for improving and accelerating product and process development in many industrial applications.

It is now possible to identify at least five major areas of application for AR in the industry domain: Human-Robot Collaboration, maintenance-assembly-repair, training, products inspection and building monitoring. In the Human-Robot Collaboration domain, AR is used to create efficient interfaces to interact with industrial robots. In the maintenance-assemblyrepair tasks, AR improves their own productivity. In training operations, users can find in the AR a powerful solution to enhance their skills. In the products inspection, controllers can notice any discrepancies of items using powerful and versatile AR systems. Finally, in the building monitoring operations, AR highlights any error or deviation of a facility in a simple and intuitive manner.

Human-Robot Collaboration

The fourth industrial revolution is bringing new technological challenges. The capability of industrial robots is steadily increasing, together with the expectation of stronger cooperative interaction. Operators need to work in a safe environment that enhances their trust in the robots. To create a system in which robots work side by side with humans, new interfaces must be developed to allow users to interact with them in the most natural way. For these reasons, new scientific disciplines are emerging. Human Robot Collaboration (HRC) is a new scientific discipline that tries to understand how to improve the human-robot collaboration using innovative interfaces. Creating a safe and trustworthy human-robot system is a complex challenge. A human-human collaboration system is considered safe because one human can naturally understand the intention of another human. In the case of robots, the same behaviour can be achieved showing in advance to users the movements or forces that robots are going to apply in the real environment: in this way the operator can clearly understand the robot’s intentions. AR can indeed be used to achieve this purpose, since it is able to show information contextualized in the real environment, improving the operators awareness of the system. AR, among other applications, is a promising technology that can enhance the user’s ability to understand:

• The movements of a mobile robot;

• The movements of a robotic arm;

• The forces applied by a robot.

The remaining part of this section is dedicated to the explanation of some systems that take advantage of AR for understanding the features introduced above.

Mobile robot movement

Industries often employ Automated Guided Vehicles (AGV), instead of using human skilled laborers, for material transportation. AGV are robots that can move independently and they are often used to transport equipment around a manufacturing facility. Most of the time, an AGV follows a predefined path that on the one hand makes it easy for workers to be able to predict the robots intentions, on the other hand it imposes some limitations on the type of task the AGV can perform. The next generation of AGV will be capable of moving without following a predefined path and it will be able to decide, in real time, what is the best trajectory to follow in a given environment [54-61]. This behaviour introduces some degree of uncertainty and for this reason the communication of the vehicles intention must be as clear as possible. In fact, a way to improve the safety of these systems might be giving to the robot the ability to understand and predict human motion. Creating such an intelligent system is however a complex task, due to the fact that humans are highly unpredictable. A reasonable alternative lies into shifting the burden of understanding the others intentions from the robot to the human. To make this system safe, the robot has to explicitly express its upcoming intended movements: since sight is one of the most developed human senses, making explicit an intended action through a visualization system is certainly an excellent choice. Moreover, the visualization system can use the AR technology to show the robots intended motion projected on the real environment. Several projects [11,13,37] have been following this approach: for example, Chadalavada et al. [11] added a standard projector (Optoma ML 750) to a Linde CitiTruck AGV. The projector is used to display the motion of the robot directly on the floor, allowing humans to detect its future moves on a time horizon of several seconds.

Robot’s arms movement

In a factory different tasks are accomplished, one of them is represented by the pick and place action or assembly procedure. These activities are usually performed by the so called “arm robots”, which are capable of grabbing objects and placing them in specific areas. As for the AGV’s movement, understanding in advance the path that will be taken by the robot’s arms is crucial to allow humans to predict the robot’s intention. Some researches [8,38] are exploring the use of AR for visualizing the robot’s arm movement projected in the real environment. Ameri et al. [8] developed an AR system that improves the interaction between workers and industrial robots. Using this AR interface, a worker is aware not only of which objects will be taken, but also of which trajectories the robot will follow: in this way users can decide to cancel wrong commands that they had given to the robot before executing them.

Showing information about the robot’s intention can improve the collaboration system, but it must be taken into account not only the type of information, but also when the information has to be displayed. Understanding when data has to be displayed is fundamental, since the worker has to know the right information at the right moment in order to exactly understand what the robot is doing and consequently feel safe. E Ruffaldi et al. [45] presented a system in which a stereo Augmented Reality eye-wear is integrated in a working helmet for HRC with a humanoid robot for collaborative applications (Baxter robot, from Rethink Robotics [7]). The Authors’ goal is to show information about what the robot is doing at certain times, without overloading the operator with data. Specifically, if the object recognized by the robot is in the field of view of the operator, the object is augmented in the head-up display. Moreover, a three-dimensional arrow is placed near the endeffector to show the direction of motion of the robot.

Robot’s force

Monitoring the robots movement is useful to understand its intentions, but it lacks information about how much strength the robot is employing in performing its task. Mateo et al. [36] developed an Android based application for programming industrial robots. The task can be monitored by overlapping real time information through AR. For example, while the robot is performing the task, the force component of the tool center point in X, Y and Z are displayed as well as the resulting vector using a 3D CAD model. Furthermore, the components are coloured with different colors depending on the intensity of the force.

Although much work has still to be done in order to guarantee a safe interaction with the system by interpreting the robot’s intentions, AR appears to be a promising technology for this purpose: it makes the user more aware of what the robot is doing, improving the safety sensation of the worker and the system performance. The main challenges are most of the times due to the quality of the tracking system and to occlusion problems.

Maintenance, Assembly and Repair

Maintenance, repair and assembly tasks represent another strategic research field for AR, since cost reduction is a key goal for many industries. One of the problems posed by this kind of tasks is related to their complexity: the technicians might need to refer to instructions manuals to correctly complete the assigned procedures. The continuous switch of attention between the device involved in the procedure and the manual may involve an high cognitive load to the technicians.

Interactive Electronic Technical Manuals (IETMs) have been proposed to address the inaccuracies and difficulties related to a standard Technical Manual (TM) and their effectiveness in improving the performances of both expert and unexperienced technicians have been proved [17]. However, IETMs are not completely part of the interaction process between the technician and the machine and it has been proved that this separation increases both the time and cost of the tasks and the cognitive load on the technicians [25].

AR can easily and efficiently tackle the separation problem posed by TMs and IETMs [54] and the benefits that AR can bring to maintenance, repair and assembly tasks have been throughly analyzed [24]. AR-based documentation has been proved to produce a reduction of the cost up to 25% and an improvement in performances up to 30% [55]. The International Data Corporation (IDC) forecasts that on-site assembly and safety would attract investments in AR and VR technologies up to $362 million in the next four years and by the end of 2021 industrial maintenance is expected to be the largest industry use case with up to $5.2 billion of investments [27].

AR applications for maintenance and repair consist of a set of virtual assets which provides indications, aids or suggestions to the technicians. The most common assets comprehend audio tracks with instructions, animated 3D models which visually describe what to do and textual labels which provides details on the task to be performed. Graphical assets are overlaid and aligned with respect to the machine to be maintained, enabling the technicians to take advantage of them while performing the procedure. Moreover, this kind of AR applications often incorporate a telepresence system, which enables a remote technician to interactively support maintainers when AR aids are not sufficient.

Starting with the early experiments to support technicians in sim-ple maintenance procedures with a head-worn AR prototype by Feiner et al. [16], the first attempts of conveying maintenance in-structions to technicians by AR-based systems can be dated back to 1990s, as researchers began to investigate the benefits and appli-cations of AR to maintenance, repair and assembly tasks [40,41]. One of the first example of teleassistance through AR, also called tele-maintenance, was exploited in 2000 as a result of the Etal¨a¨ project [23]: an ARbased system that could establish a communica-tion channel between maintainers and remote experts. The STAR-MATE project was one of the first to investigate and implement multimodal interaction in AR, combining a virtual pointing device and voice-based commands [50]. In other projects the aim is to provide to the users the ability to perform AR procedures moving in an industrial site or plant, instead of operating on a stationary work-bench, implementing the so called mobile AR [22,43,47]. Even if the spread of personal devices such as tablets and smartphones in the last decade has fueled Handheld Augmented Reality (HAR) [48,57], one of the main limitation of its use in industry is that technicians often need hand-free devices to accomplish assigned tesks. Moreover, observing the object to be mantained or assembled through a camera might involve safety issues for the technician. Even so, some examples of HAR-based maintenance exists such as the HAR solution proposed by Kahn et al. to support the overall life-cycle of constructions and facilities managment [28]. Examples of AR applications for assembly tasks through AR tehcnologies compre-hend the project MOON (asseMbly Oriented authOring augmeNted reality), developed by AIRBUS Military: the aim of the project was to use 3D information from the industrial Digital Mock-Up to gener-ate assembly instructions for the aerospace industry to be deployed through AR technology [51]. A research by Hou et al. investigates the cognitive gains that can be derived from using AR for assembly tasks [26].

At the present time, one of the problems that limits the spread of AR in maintenance and repair tasks consist of the time needed to create, change and improve the AR procedures [32]. Another difficulty is posed by the lack of a clear and accessible workflow to design and develop an AR application for the industry: this problem is addressed by Manuri et al. [35], which propose an evaluation of markerless tracking systems for maintenance procedures since environment-independent solutions can be obtained without markers. Furthermore, the user interface of wearable devices provides other difficulties when applied to the industry domain: the robustness of the interfaces poses a major requirement, eventually even more important than intuitiveness or easiness of use, due to the dangerous-ness that some procedures may pose to the technicians [34]; on the other hand, since minimizing the user’s cognitive load during the interaction has been proven to be crucial for cost reduction, Lamberti et al. [31] investigated a framework for automatically generating speech interfaces for wearable AR application using semantics.

Training

The usage of AR technologies for training purposes is strictly related to maintenance, assembly and repair tasks, since they are usually the object of learning for an user from the industry domain. AR techniques for improving traditional learning approaches have been deeply investigated through the years, since teachers, instructors and trainers are always searching for new methods to enhance the learning experience of their students and to develop innovative learn-ing and training paths. Multimedia contents can not only deliver a rich sensory experience that can enhance both user-machine and user-user interactions but also because multimedia contents can in-crease the motivation and interest of the reader or viewer, as stated by Chignell and Waterworth [60]. Different researches investigated the causes behind maintenance-related reports regarding procedural errors, illustrating that some failures in maintenance are not due to the lack of proper information on the task, and AR is believed to be a valuable tool for supporting task execution due to its capability to increase the user’s motivation [56]. Another important benefit related to the usage of AR technologies for training is that AR allows to simulate dangerous or perilous tasks or even destructive events with no risk of the students.

First examples of AR systems for supporting and training techni-cians through computer-generated instructions can be dated back to early 1990s, as reported by Nee et al. [40] and Ong et al. [41]. AR technologies are now used for training and supporting technicians pertaining a large number of industry domains, such as industrial plants [21,42], aerospace [14,15] and automotive [52,58]. A Flexible AR-based Training System for Industrial Maintenance is proposed by Sanna et al. [46] to address the difficulty for teachers to create AR contents [30]. The system allows the instructor to generate an AR-based training procedure, easily “tuning” its complexity accord-ing to the students’ skills. Moreover, the teacher can provide remote assistance through an integrated telepresence system: the instruc-tor can see what the student’s camera is framing, interact with the student, identify which step of the procedure is preventing the user from completing their task. Eventually, the teacher can also modify the procedure, thus sending back to students an updated version.

Product Control Quality

Creating a product is a complex task to accomplish. Product manu-facturing goes through several phases, such as conception, design and actual realization. Once a product is realized, it is inspected to check both that no errors occurred during the creation phase and that there are no differences with respect to what had been estimated. For efficiency reasons, the overall process should be completed as quickly and accurately as possible. There is a growing tendency to satisfy a standard of perfection both in the company management and in the actual manufacture of the product. At the end of the production chain, quality controls are high; in order to place on the market efficient products that best satisfy the end user’s expectations. Regarding commercial assets, products are examined visually using a list that contains unacceptable products defects. This procedure is commonly called inspection. Inspection can be done in different areas, such as manufacturing, business, government, mechanics and it consists in an organized examination of a particular equipment or process. Since the variety of products and their details increases, the inspection task becomes more complex. Thus, the inspection could become less effective because of the cognitive limitations of human inspectors. AR appears clearly to be a promising technology in set-ting up the inspection process, because it allows a direct comparison between the real object and an ideal model. Indeed, using wearable devices, the operator can visualize a 3D representation of the ideal object directly superimposed on the product that is being inspected. This procedure is often called ”discrepancy check” [59].

An example of AR system used to inspect an industrial product is proposed by Ramakrishna et al. [44]. In the suggested sys-tem, a printer is analyzed using some Android devices (Cardboard with mobile phone, Google Glass and Tablet) that can extract in-formation about the object by detecting a QR code attached to the printer. The gathered information regards some relevant details of the printer (type, year of manufacture, inspection history, etc.) along with the check-list to be performed during inspection. Afterwards, instructions and manuals are displayed on the user device screen, in this way the inspector can complete the operation having all the necessary information. In the process, different objects are tracked and specific information is thus sent to the operator. To improve the system performance, a speech recognition engine has been integrated in order to allow the user to give comments. In the approach proposed by Chung [12], an AR system to analyzed some small industrial object is presented. The goal is to understand which is the most effective way to inspect a real product. Four different modal-ities of inspection are compared. The manual modality consists into taking measurement directly on the real object. With the 2D and 3D modalities, only the relevant data relative to the real object are shown to the user. The only difference between 2D and 3D modalities consists into how data are visualized on the monitor (2D perspective figures and isometric perspectives, respectively). Finally, the AR modality uses the same approach of the 3D modality, but the subject is equipped with a see-through head-mounted display (HMD), in which the 3D model is superimposed over the real object together with the measuring information. Results deduced from the comparison among these four modalities, show that the AR method provides the best performance by being the fastest approach. Further, the AR system shows the least number of errors since the operator has to accomplish fewer tasks than with the other three modalities. Finally, this system helps the operator to understand which attributes need to be measured. More advanced systems can reconstruct the 3D scene in real time using depth cameras. Wasenmuller at al. [59] developed an AR system that can construct in real time a 3D representation of any object and it allows an instant inspection. The algorithm detects the geometry of a given object and it compares the 3D model with the real one. The discrepancies are computed with an accuracy of 0.01 m.

Building Monitoring

Building environments are complex structures, made up by systems that are composed by different machines that require a suitable infrastructure. Building such an environment is a difficult task; each system is composed by different sub-systems (columns, tube, rooms, etc.) that have to be positioned in the appropriate place. Normally, these sub-systems are represented using 3D models and the factory is pre-visualized using Computer Graphics. In this way it is possible to position every sub-system in the appropriate place, checking either for any collisions or other sprites irregularities [49].

Construction process monitoring

Verifying the construction process is a complex challenge, detection of actual or potential schedule delay in field construction activities is vital to project management. Indeed, the current method presents several drawbacks described as follow:

• Manual collection required;

• Manually collected data may be low;

• The collected information is interpreted on what needs to be measured;

• Existing representations of progress monitoring are visually complex.

Consequently, a lot of time is spent to realize this process, reducing the overall productivity and increasing the outflow of resources. Certainly, a real time feedback and monitoring system should be based on solutions that can avoid or reduce the problems introduced above. AR is undoubtedly a legitimate candidate, users can visualize the construction process directly on the real environment and it is possible to detect any deviation from the original plan. Two similar systems [19,20] use AR to superimpose 3D models of the future building over timelapsed photographs. As the 3D models are superimposed on the real environment, the software determines if there are differences between what is being constructed and what had been planned. If any deviation is detected in some regions, the 3D models of these specific sites are coloured in red, while if the construction is proceeding as planned, the 3D model is coloured in green. These systems show again the potential of AR, since all the individuals involved in the construction process (e.g., from the owners and architects to subcontractors and trades foremen) can keep under control in real time the progress of work visualizing if there are any problems in a simple and intuitive way.

Facility Monitoring

Once the real facility is built, verification and control processes are implemented to verify if the final product is different from what has been planned. There are several ways of checking an environment: the traditional approach consists into verifying by hand directly in the building, using geodetic devices and laser scanners. The main disadvantages are related to the absence of an automatic process that translates the points measured by the devices (laser scanner, etc.) into a 3D model that can be compared in situ with the real environment. Thanks to the ability of AR to be used in the real environment, this challenge can be overcome. Several projects [18,29,33] use AR to improve the recognition of problems concerning the pipe system. George et al. [18] developed a system that can detect some unique features proper of each environment for figuring out any problem regarding pipe configuration. Unique features are represented by Anchor-Plates that are metallic structures embedded in the walls. Once these elements are tracked correctly, the software superimposes the pipe’s 3D CAD model over the real pipe and the user is able to detect any discrepancies between them. In a similar way, Lee et al. [33] developed an AR system that shows a pipe 3D CAD model along with temperature/pressure measures and the pipe distribution diagram. Test results show that workers can inspect properly the pipeline comparing the CAD model with the real one.

Another stimulating use of AR to monitor complex building has been developed by Zollmann et al. [61]. The system proposed consists into a combination of AR with Unmanned Aerial Vehicles (UAV), aircrafts without a human pilot aboard. The degree of autonomy of these aerial vehicles can vary: they can be controlled either remotely by a human operator or autonomously by on board com-puters. Zollmann et al. [61] use UAVs to capture a set of aerial images of an area of interest (e.g., from houses to factories), then these images are used to reconstruct in real time a 3D representation of the construction site. Finally, an user, who is placed at the site, can use the AR component to visualize differences between the real building and its 3D representation by superimposing the 3D model. Moreover, it is possible to take spatial annotation directly on the AR view, in this way the construction staff can store suggestions directly on the object to be modified. This information can be visualized in real time, improving the execution times and the efficiency of the inspection.

Conclusion

The terms “Industry 4.0” and “smart factory” are often associated with the concept of Internet of Things (IoT), which refers to net-worked devices that can exchange data. IoT is one of the main technology of Industry 4.0 but many others are becoming part of it. It is no longer unimaginable to think of a factory where not only everything is connected, but it is also viewable and interactive. The effectiveness of AR does not lie in the visualization process itself (data can be visualized in many ways): it is how data are visualized that makes AR a very powerful technology. Its ability to enhance the real space has been proved several times and the usage of this technology in a complex environment, such as the one of a fac-tory, can indeed improve the industry productivity. AR improves reliability and safety of robotic systems showing to workers the intentions of robots, it reduces costs and improves performances of maintenance systems or it shows precisely any discrepancies of products superimposing models on the real object. AR will be definitely one of the key technologies of Industry 4.0, it will enrich both the managers/supervisors’ job and the workers’ one. Industry 4.0 represents indeed one of the major technological revolutions. Beside the improvements that it will certainly lead to, difficulties regarding data processing may arise. In a context where everything is monitorable and viewable, the protection of the user’s privacy must be considered very carefully. Data are gathered using multiple sources positioned in different places all around the factory and in-formation is continuously processed, saved and controlled. Gestures, voices, facial expressions (and many other features) are detected and stored. Sensible data, such as life-parameters or health-status, can be monitored on the legitimate grounds of preventing any physical illness. Moreover, any mistakes made during a generic operation can be detected in real time and promptly managed. Undoubtedly, these procedures may be implemented for improving the efficiency of the factory, but it must be given the opportunity to know how these data are processed and who can visualize them. Data processing is becoming one of the most discussed themes and organizations and government authorities are trying to find common rules to regulate it. For what concerns Europe, the General Data Protection Regulation (GDPR) will enter in force on the 25th of May 2018 with the intent to harmonize data privacy laws across Europe, to protect and empower all EU citizens data privacy and to reshape the way organizations across the region approach data privacy [3]. Inspired by the seven Foundational Principles of Privacy by Design [10], the GDPR will maybe change how data are collected. Technologies which compose the basis for the Industry 4.0 will maybe change their own way of working by modifying the very nature of Industry 4.0.

Acknowledgment

This work is co-funded by the regional project HuManS (Human centered Manufactoring Systems).

References

1. AR Toolkit (2018) Accessed.
2. Emacula, enhanced retina technologies (2018) Accessed.
3. Gdpr portal: Site overview (2018) Accessed.
4. Meta 2 ar headset (2018) Accessed.
5. SIGGRAPH (2018) Accessed.
6. Vuzix AR3000 Smart Glasses (2018) Accessed.
7. With our Smart, Collaborative Robot Pioneer (2018) Accessed.
8. Akan B, CB, et al. (2010) Augmented reality meets industry: Inter-active robot programming. Linkoping University Electronic Press, Vsters 52: 55-58.
9. Caudell TP, Mizell DW (1992) Augmented reality: An application of heads-up display technology to manual manufacturing processes. Kauai 2: 659-669.
10. Cavoukian A (2009) Privacy by design. Information and privacy commis-sioner.
11. Chadalavada RT, Andreasson H, Krug R, Lilienthal AJ (2015) That’s on my mind! robot to human intention communication through on-board projection on shared floor space. Lincoln 1-6.
12. Chung KH (2002) Application of augmented reality to dimensional and geometric inspection, Virginia Tech. Blacksburg.
13. Coovert MD, Lee T, Shindev, Sun Y (2014) Spatial augmented reality as a method for a mobile robot to communicate intended movement. Computers in Human Behavior 34: 241-248.
14. Datcu D, Cidota M, Lukosch S, Oliveira DM, Wolff M (2014) Virtual co-location to support remote assistance for inflight maintenance in ground training for space missions. In Proc. CompSysTech 134-141.
15. De Crescenzio F, Fantini M, Persiani F, Di Stefano L, Azzari P, et al (2011) Augmented reality for aircraft maintenance training and operations support. Computer Graphics and Applications 31: 96-101.
16. Feiner S, Macintyre B, Seligmann D (1993) Knowledge-based aug-mented reality. Communications of the ACM 36: 53-62.
17. Fuller JJ (1994) IETMs: From research to reality. CALS Expo’94 Interna-tional Proceedings.
18. Georgel P, Schroeder P, Benhimane S, Hinterstoisser S, Appel M, et al. (2007) An industrial augmented reality solution for discrepancy check. In Proc. ISMAR Computer Society 1-4.
19. Golparvar-Fard M, Pena-Mora F, Arboleda CA, Lee S (2009) Visual-ization of construction progress monitoring with 4D simulation model overlaid on time-lapsed photographs. J Computing in Civil Engineering 23: 391-404.
20. Golparvar-Fard M, Pena-Mora F, Savarese S (2009) D4AR–a 4-dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication. J Information Technol Construction 14: 129-153.
21. Goose S, Sudarsky S, Zhang X, Navab N (2003) Speech-enabled augmented reality supporting mobile industrial maintenance. IEEE Pervasive Computing 2: 65-70.
22. Haritos T, Macchiarella ND (2005) A mobile application of augmented reality for aerospace maintenance training. Washington. In Proc. DASC 1: 5-B.
23. Harmo P, Halme A, Virekoski P, Halinen M, Pitkanen H (2000) Etal¨a¨-virtual reality assisted telepresence system for remote control and maintenance. IFAC Proceedings Volumes 33: 1011-1016.
24. Henderson S, Feiner S (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics 17: 1355-1368.
25. Henderson SJ, Feiner SK (2007) Augmented reality for maintenance and repair (armar). Technical report, DTIC Document.
26. Hou L, Wang X, Bernold L, Love PE (2013) Using animated aug-mented reality to cognitively guide assembly. J Comput Civil Eng 27: 439-451.
27. IDC (2018) Worldwide spending on augmented and virtual reality expected to double or more every year through 2021, according to IDC. Accessed.
28. Kahn S, Olbrich M, Engelke T, Keil J, Riess P, et al. (2012) Beyond 3d“ as-built” information using mobile are enhancing the building lifecycle management. IEEE In Proc. CW 29-36.
29. Kahn S, Wuest H, Stricker D, Fellner DW (2010) 3d discrepancy check via augmented reality. Seoul. In Proc. ISMAR 241-242.
30. Kerawalla L, Luckin R, Seljeflot S, Woolard A (2006) “Making it real”: exploring the potential of augmented reality for teaching primary school science. Virtual Reality 10: 163-174.
31. Lamberti F, Manuri F, Paravati G, Piumatti G, Sanna A (2017) Using semantics to automatically generate speech interfaces for wearable virtual and augmented reality applications. IEEE T Hum-Mach Syst 47: 152-164.
32. Lamberti F, Manuri F, Sanna A, Paravati G, Pezzolla P, et al. (2014) Challenges, opportunities, and future trends of emerging tech-niques for augmented reality-based maintenance. IEEE T Emerg Top Comput 2: 411-421.
33. Lee JM, Lee KH, Kim DS, Kim CH (2010) Active insp ection supporting system based on mixed reality after design and manufacture in an offshore structure. J Mech Sci Technol 24: 197-202.
34. Manuri F, Sanna A, Lamberti F, Paravati G (2016) Vocal one switch (VOS) selection interfaces for virtual and augmented reality hands-free tasks. Eurographics Association, Genova. In Proc. STAG 79-87.
35. Manuri F, Sanna A, Lamberti F, Paravati G, Pezzolla Pb (2014) A workflow analysis for implementing ar-based maintenance procedures. Springer, Lecce. In Proc. AVR 185-200.
36. Mateo C, Brunete A, Gambao E, Hernando M (2014) Hammer: An android based application for end-user industrial robot programming. In Proc. MESA. Senigallia 1-6.
37. Matsumaru T (2006) Mobile robot with preliminary-announcement and display function of forthcoming motion using projection equipment. In Proc. ROMAN 443-450.
38. Michalos G, Karagiannis P, Makris S, Tokcalar O, Chrys-solouris G (2016) Augmented reality (AR) applications for supporting human-robot interactive cooperation. Procedia CIRP 41: 370-375.
39. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77: 1321-1329.
40. Nee A, Ong S, Chryssolouris G, Mourtzis D (2012) Augmented reality applications in design and manufacturing. CIRP Annals-manufacturing technology 61: 657-679.
41. Ong S, Yuan M, Nee A (2008) Augmented reality applications in manufacturing: a survey. International journal of production research 46: 2707-2742.
42. Pantoja G, Garza LE, Mendivil EG (2014) Augmented reality in pneumatic conveying system: fuller pump dry material line charger. Barcelona. In Proc. CISTI 1-5.
43. Pentenrieder K, Bade C, Doil F, Meier P (2007) Augmented reality-based factory planning-an application tailored to industrial needs. In Proc. MAR. IEEE, Nara 31-42.
44. Ramakrishna P, Hassan E, Hebbalaguppe R, Sharma M, Gupta G, et al. (2016) An ar inspection framework: Fea-sibility study with multiple ar devices. In Proc. ISMAR-Adjunct. IEEE, Merida 221-226.
45. Ruffaldi E, Brizzi F, Tecchia F, Bacinelli S (2016) Third point of view augmented reality for robot intentions visualization. Springer, Otranto. In Proc. AVR 471-478.
46. Sanna A, Manuri F, Piumatti G, Paravati G, Lamberti F, et al. (2015) A flexible ar-based training system for industrial maintenance. Springer, Lecce. In Proc. AVR 314-331.
47. Savioja P, Jarvinen P, Karhela T, Siltanen P, Woodward C (2007) Developing a mobile, service-based augmented reality tool for modern maintenance work. In Proc. Springer, Beijing. ICVR 554-563.
48. Schmalstieg D, Wagner D (2007) Experiences with handheld augmented reality. In Proc. MAR 3-18.
49. Schonfelder R, Schmalstieg D (2008) Augmented reality for industrial building acceptance. In Proc. VR 83-90.
50. Schwald B, Figue J, Chauvineau E, Vu-Hong F, Robert A, et al. (2001) STARMATE: Using augmented reality technology for computer guided maintenance of complex mechanical elements. E-work and ECommerce 1: 196-202.
51. Servan J, Mas F, Menendez J, Rios J (2012) Assembly work instruction deployment using augmented reality. In Proc. KEM, Trans Tech Pub 502: 25-30.
52. Stanimirovic D, Damasky N, Webel S, Koriath D, Spillner A, et al. (2014) [Poster] a mobile augmented reality system to assist auto mechanics. In Proc. ISAMR. IEEE, Munich 305-306.
53. Sutherland IE (1968) A head-mounted three dimensional display. In Proc. AFIPS. ACM, San Francisco 757-764.
54. Tang A, Owen C, Biocca F, Mou W (2003) Comparative effectiveness of augmented reality in object assembly. In Proc. SIGCHI, ACM, Ft. Lauderdale 73-80.
55. Terenzi G, Basile G (2014) Smart maintenance: An augmented re-ality platform for training and fields operations in the manufactur-ing industry. ARMEDIA Augmented Reality blog . Accessed on 2018.
56. Veinott ES, Kanki BG, Shafto MG (1995) Identifying human factors issues in aircraft maintenance operations.
57. Wagnerc D (2007) andheld augmented reality. PhD thesis, Graz University of Technology, Institute of Computer Graphics and Vision, Graz.
58. Wang J, Feng Y, Zeng C, Li S (2014) An augmented reality based system for remote collaborative maintenance instruction of complex products. In Proc. CASE. IEEE, Taipei 309-314.
59. Wasenmuller O, Meyer M, Stricker D (2016) Augmented reality 3d discrepancy check in industrial applications. In Proc. ISMAR, IEEE, Merida 125-134.
60. Waterworth JA, Chignell MH (1997) Multimedia Interaction.
61. Zollmann S, Hoppe C, Kluckner S, Poglitsch C, Bischof H, et al. (2014) Augmented reality for construction site monitoring and documentation. Proceedings of the IEEE 102: 137-154.