Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaboration

Simple item page
dc.contributor.authorGurevich, Pavel
dc.contributor.authorLanir, Joel
dc.contributor.authorCohen, Benjamin
dc.date.accessioned2020-06-06T13:06:38Z
dc.date.available2020-06-06T13:06:38Z
dc.date.issued2015
dc.date.issued2015
dc.description.abstractTeleAdvisor is a versatile projection-based augmented reality system designed for remote collaboration. It allows a remote expert to naturally guide a local user in need of assistance in carrying out physical tasks around real-world objects. The system consists of a small projector and two cameras mounted on top of a tele-operated robotic arm at the worker’s side, and an interface to view the camera stream, control the point-of-view and gesture using projected annotations at the remote expert’s side. TeleAdvisor provides a hands-free, mobile, low-cost solution that supports gesturing by the remote expert while minimizing the cognitive overhead of the local worker. We describe the challenges, design considerations and implementation details of the two phases of the TeleAdvisor prototype, as well as its evaluation and deployment at an industrial manufacturing center. We summarize our understandings from our experiences during the project and discuss the general implications for design of augmented reality remote collaboration systems.de
dc.identifier.doi10.1007/s10606-015-9232-7
dc.identifier.pissn1573-7551
dc.identifier.urihttp://dx.doi.org/10.1007/s10606-015-9232-7
dc.identifier.urihttps://dl.eusset.eu/handle/20.500.12015/3860
dc.publisherSpringer
dc.relation.ispartofComputer Supported Cooperative Work (CSCW): Vol. 24, No. 6
dc.relation.ispartofseriesComputer Supported Cooperative Work (CSCW)
dc.subjectAugmented reality
dc.subjectMobile projector
dc.subjectMobility
dc.subjectRemote assistance
dc.subjectRemote collaboration
dc.titleDesign and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaborationde
dc.typeText/Journal Article
gi.citation.endPage562
gi.citation.startPage527
gi.citations.count57
gi.citations.elementPeng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Xiangyu Zhang, Shuxia Wang, Weiping He, Yuxiang Yan, Hongyu Ji (2021): AR/MR Remote Collaboration on Physical Tasks: A Review, In: Robotics and Computer-Integrated Manufacturing, doi:10.1016/j.rcim.2020.102071
gi.citations.elementCatarina G. Fidalgo, Yukang Yan, Hyunsung Cho, Maurício Sousa, David Lindlbauer, Joaquim Jorge (2023): A Survey on Remote Assistance and Training in Mixed Reality Environments, In: IEEE Transactions on Visualization and Computer Graphics 5(29), doi:10.1109/tvcg.2023.3247081
gi.citations.elementBernardo Marques, Samuel Silva, António Teixeira, Paulo Dias, Beatriz Sousa Santos (2022): A vision for contextualized evaluation of remote collaboration supported by AR, In: Computers & Graphics, doi:10.1016/j.cag.2021.10.009
gi.citations.elementCedric Kervegant, Julien Castet, Juliette Vauchez, Charles Bailly (2021): Distant Assist Cursor (DAC): Designing an Augmented Reality System to Facilitate Remote Collaboration for Novice Users, In: Companion Proceedings of the 2021 Conference on Interactive Surfaces and Spaces, doi:10.1145/3447932.3490520
gi.citations.elementPhilipp Ladwig, Bastian Dewitz, Hendrik Preu, Mitja Säger (2019): Remote Guidance for Machine Maintenance Supported by Physical LEDs and Virtual Reality, In: Proceedings of Mensch und Computer 2019, doi:10.1145/3340764.3340780
gi.citations.elementPaola Fantini, Marta Pinzone, Marco Taisch (2020): Placing the operator at the centre of Industry 4.0 design: Modelling and assessing human activities within cyber-physical systems, In: Computers & Industrial Engineering, doi:10.1016/j.cie.2018.01.025
gi.citations.elementXiangyu Zhang, Xiaoliang Bai, Shusheng Zhang, Weiping He, Shuxia Wang, Yuxiang Yan, Quan Yu, Liwei Liu (2023): A novel MR remote collaboration system using 3D spatial area cue and visual notification, In: Journal of Manufacturing Systems, doi:10.1016/j.jmsy.2023.02.013
gi.citations.elementSung Ho Choi, Minseok Kim, Jae Yeol Lee (2018): Situation-dependent remote AR collaborations: Image-based collaboration using a 3D perspective map and live video-based collaboration with a synchronized VR mode, In: Computers in Industry, doi:10.1016/j.compind.2018.06.006
gi.citations.elementJiawei Yu, Tengyue Wang, Yunze Shi, Liangjing Yang (2022): MR Meets Robotics: A Review of Mixed Reality Technology in Robotics, In: 2022 6th International Conference on Robotics, Control and Automation (ICRCA), doi:10.1109/icrca55033.2022.9828895
gi.citations.elementJozef Husár, Lucia Knapčíková (2023): Implementation of Augmented Reality in Smart Engineering Manufacturing: Literature Review, In: Mobile Networks and Applications 1(29), doi:10.1007/s11036-023-02121-x
gi.citations.elementYu Zhu, Jun Yang, Hongwei Zhang, Wenmin Zhu, Jie Wang, Zelin Zhou (2022): Intelligent assembly assistance for hull structure construction based on optical projection, In: Assembly Automation 2(42), doi:10.1108/aa-05-2021-0061
gi.citations.elementLucy Whitfield (2023): Watching me … watching you: Training and assessment in a virtual world, In: Laboratory Animals 2(57), doi:10.1177/00236772221146824
gi.citations.elementIñigo Fernández del Amo, John Erkoyuncu, Rok Vrabič, Romain Frayssinet, Cristina Vazquez Reynel, Rajkumar Roy (2020): Structured authoring for AR-based communication to enhance efficiency in remote diagnosis for complex equipment, In: Advanced Engineering Informatics, doi:10.1016/j.aei.2020.101096
gi.citations.elementShuo Feng, Yizhe Liu, Qianrui Zhang, Weiping He, Xiaotian Zhang, Shuxia Wang, Mark Billinghurst (2023): Parallel or Cross? Effects of Two Collaborative Modes on Augmented Reality Co-located Operations, In: International Journal of Human–Computer Interaction 15(40), doi:10.1080/10447318.2023.2202574
gi.citations.elementYue Wang, Peng Wang, Zhiyong Luo, Yuxiang Yan (2022): A novel AR remote collaborative platform for sharing 2.5D gestures and gaze, In: The International Journal of Advanced Manufacturing Technology 9-10(119), doi:10.1007/s00170-022-08747-7
gi.citations.elementPeng Wang, Shusheng Zhang, Mark Billinghurst, Xiaoliang Bai, Weiping He, Shuxia Wang, Mengmeng Sun, Xu Zhang (2019): A comprehensive survey of AR/MR-based co-design in manufacturing, In: Engineering with Computers 4(36), doi:10.1007/s00366-019-00792-3
gi.citations.elementGitte Pedersen, Konstantinos Koumaditis (2020): Virtual Reality (VR) in the Computer Supported Cooperative Work (CSCW) Domain: A Mapping and a Pre-study on Functionality and Immersion, In: Lecture Notes in Computer Science, doi:10.1007/978-3-030-49698-2_10
gi.citations.elementNicolas Kahrl, Michael Prilla, Oliver Blunk (2021): The Influence of Spatial Representation on Remote Peer Consultation, In: Mensch und Computer 2021, doi:10.1145/3473856.3473868
gi.citations.elementJin Uk Kwon, Jae-In Hwang, Jaeyoung Park, Sang Chul Ahn (2019): Fully Asymmetric Remote Collaboration System, In: IEEE Access, doi:10.1109/access.2019.2912397
gi.citations.elementLucas Pometti, Charles Bailly, Julien Castet (2023): AHO-Guide: Automatically Guiding the Head Orientation of a Local User in Augmented Reality to Realign the Field of View with Remote Users, In: Lecture Notes in Computer Science, doi:10.1007/978-3-031-42280-5_1
gi.citations.elementPeng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Sili Wei, Guangyao Xu, Weiping He, Xiangyu Zhang, Jie Zhang (2020): 3DGAM: using 3D gesture and CAD models for training on mixed reality remote collaboration, In: Multimedia Tools and Applications 20(80), doi:10.1007/s11042-020-09731-7
gi.citations.elementMfundo A. Maneli, Omowunmi E. Isafiade (2023): A Multifactor Comparative Assessment of Augmented Reality Frameworks in Diverse Computing Settings, In: IEEE Access, doi:10.1109/access.2023.3242238
gi.citations.elementPeng Wang, Shusheng Zhang, Xiaoliang Bai, Mark Billinghurst, Weiping He, Li Zhang, Jiaxiang Du, Shuxia Wang (2018): Do You Know What I Mean? An MR-Based Collaborative Platform, In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), doi:10.1109/ismar-adjunct.2018.00038
gi.citations.elementEleonora Bottani, Giuseppe Vignali (2019): Augmented reality technology in the manufacturing industry: A review of the last decade, In: IISE Transactions 3(51), doi:10.1080/24725854.2018.1493244
gi.citations.elementDavid Anton, Gregorij Kurillo, Ruzena Bajcsy (2018): User experience and interaction performance in 2D/3D telecollaboration, In: Future Generation Computer Systems, doi:10.1016/j.future.2017.12.055
gi.citations.elementYuXiang Yan, Xiaoliang Bai, Weiping He, Shuxia Wang, XiangYu Zhang, Peng Wang, Liwei Liu, Bing Zhang (2023): Can you notice my attention? A novel information vision enhancement method in MR remote collaborative assembly, In: The International Journal of Advanced Manufacturing Technology 3-4(127), doi:10.1007/s00170-023-11652-2
gi.citations.elementGuillaume Bataille, Abdelhadi Lammini, Jean-Rémy Chardonnet (2023): ARPuzzle: Evaluating the Effectiveness of Collaborative Augmented Reality, In: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), doi:10.1109/ismar59233.2023.00079
gi.citations.elementZhuo Wang, Yang Wang, Xiaoliang Bai, Xiangyu Huo, Weiping He, Shuo Feng, Jie Zhang, Yueqing Zhang, Jinzhao Zhou (2021): SHARIDEAS: a smart collaborative assembly platform based on augmented reality supporting assembly intention recognition, In: The International Journal of Advanced Manufacturing Technology 1-2(115), doi:10.1007/s00170-021-07142-y
gi.citations.elementXiangyu Zhang, Xiaoliang Bai, Shusheng Zhang, Weiping He, Shuxia Wang, Yuxiang Yan, Peng Wang, Liwei Liu (2024): A novel mixed reality remote collaboration system with adaptive generation of instructions, In: Computers & Industrial Engineering, doi:10.1016/j.cie.2024.110353
gi.citations.elementBernardo Marques, Samuel Silva, Joao Alves, Tiago Araujo, Paulo Dias, Beatriz Sousa Santos (2022): A Conceptual Model and Taxonomy for Collaborative Augmented Reality, In: IEEE Transactions on Visualization and Computer Graphics 12(28), doi:10.1109/tvcg.2021.3101545
gi.citations.elementBernardo Marques, Samuel Silva, Paulo Dias, Beatriz Sousa Santos (2022): Evaluating Augmented Reality Based Remote Collaboration: A Contextualized Approach, In: Automation, Collaboration, & E-Services, doi:10.1007/978-3-031-10788-7_16
gi.citations.elementMfundo A. Maneli, Omowunmi E. Isafiade (2023): A Comparative Evaluation of Augmented Reality Frameworks: A Plane Mapping and Resource Utilisation Perspective, In: 2023 IST-Africa Conference (IST-Africa), doi:10.23919/ist-africa60249.2023.10187850
gi.citations.elementAna Villanueva, Zhengzhe Zhu, Ziyi Liu, Feiyang Wang, Subramanian Chidambaram, Karthik Ramani (2022): ColabAR: A Toolkit for Remote Collaboration in Tangible Augmented Reality Laboratories, In: Proceedings of the ACM on Human-Computer Interaction CSCW1(6), doi:10.1145/3512928
gi.citations.elementZahraa Bassyouni, Imad H. Elhajj (2021): Augmented Reality Meets Artificial Intelligence in Robotics: A Systematic Review, In: Frontiers in Robotics and AI, doi:10.3389/frobt.2021.724798
gi.citations.elementTaimur Kausar Khan, Margit Pohl, Philipp Schworm, Matthias Klar, Achim Ebert (2024): Towards sustaining human-centered part manufacturing under workforce restrictions, In: Proceedings of the European Conference on Cognitive Ergonomics 2024, doi:10.1145/3673805.3673841
gi.citations.elementJin Uk Kwon, Eun Sun Lee, Sang Chul Ahn (2016): Projector-camera based remote assistance system for the elderly: Design issues and implementation, In: 2016 IEEE/SICE International Symposium on System Integration (SII), doi:10.1109/sii.2016.7844003
gi.citations.elementAzin Semsar, Hannah McGowan, Yuanyuan Feng, H. Reza Zahiri, Adrian Park, Andrea Kleinsmith, Helena Mentis (2019): How Trainees Use the Information from Telepointers in Remote Instruction, In: Proceedings of the ACM on Human-Computer Interaction CSCW(3), doi:10.1145/3359195
gi.citations.elementPeng Wang, Shusheng Zhang, Xiaoliang Bai, Mark Billinghurst, Li Zhang, Shuxia Wang, Dechuan Han, Hao Lv, Yuxiang Yan (2019): A gesture- and head-based multimodal interaction platform for MR remote collaboration, In: The International Journal of Advanced Manufacturing Technology 7-8(105), doi:10.1007/s00170-019-04434-2
gi.citations.elementZenglei Wang, Shusheng Zhang, Xiaoliang Bai (2021): A mixed reality platform for assembly assistance based on gaze interaction in industry, In: The International Journal of Advanced Manufacturing Technology 9-10(116), doi:10.1007/s00170-021-07624-z
gi.citations.elementAzin Semsar, Hannah McGowan, Yuanyuan Feng, H. Reza Zahiri, Adrian Park, Andrea Kleinsmith, Helena M. Mentis (2020): Quality of and Attention to Instructions in Telementoring, In: Proceedings of the ACM on Human-Computer Interaction CSCW2(4), doi:10.1145/3415236
gi.citations.elementPeng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Dechuan Han, Mengmeng Sun, Zhuo Wang, Hao Lv, Shu Han (2020): Haptic Feedback Helps Me? A VR-SAR Remote Collaborative System with Tangible Interaction, In: International Journal of Human–Computer Interaction 13(36), doi:10.1080/10447318.2020.1732140
gi.citations.elementPhilipp Ladwig, Christian Geiger (2018): A Literature Review on Collaboration in Mixed Reality, In: Lecture Notes in Networks and Systems, doi:10.1007/978-3-319-95678-7_65
gi.citations.elementKenan Bektaş, Adrian Pandjaitan, Jannis Strecker, Simon Mayer (2024): GlassBoARd: A Gaze-Enabled AR Interface for Collaborative Work, In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, doi:10.1145/3613905.3650965
gi.citations.elementA. Golubkin, R. Pirmagomedov (2021): ANALYTICAL REVIEW OF AUGMENTED AND MIXED REALITY SYSTEMS IN THE CONTEXT OF INDUSTRY 4.0, In: Telecom IT 4(9), doi:10.31854/2307-1303-2021-9-4-1-27
gi.citations.elementAlexander Schäfer, Gerd Reis, Didier Stricker (2022): A Survey on Synchronous Augmented, Virtual, andMixed Reality Remote Collaboration Systems, In: ACM Computing Surveys 6(55), doi:10.1145/3533376
gi.citations.elementZhanat Makhataeva, Huseyin Varol (2020): Augmented Reality for Robotics: A Review, In: Robotics 2(9), doi:10.3390/robotics9020021
gi.citations.elementLe Luo, Dongdong Weng, Jie Hao, Ziqi Tu, Haiyan Jiang (2023): Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System, In: Sensors 8(23), doi:10.3390/s23084113
gi.citations.elementTakafumi Asao, Takeru Kobayashi, Kentaro Kotani, Satoshi Suzuki, Kazutaka Obama, Atsuhiko Sumii, Tatsuto Nishigori (2021): Development of a Drawing Application for Communication Support during Endoscopic Surgery and Its Evaluation Using Steering Law, In: Applied Sciences 10(11), doi:10.3390/app11104505
gi.citations.elementBernardo Marques, Samuel Silva, João Alves, António Rocha, Paulo Dias, Beatriz Sousa Santos (2022): Remote collaboration in maintenance contexts using augmented reality: insights from a participatory process, In: International Journal on Interactive Design and Manufacturing (IJIDeM) 1(16), doi:10.1007/s12008-021-00798-6
gi.citations.elementNelusa Pathmanathan, Tobias Rau, Xiliu Yang, Aimée Sousa Calepso, Felix Amtsberg, Achim Menges, Michael Sedlmair, Kuno Kurzhals (2024): Eyes on the Task: Gaze Analysis of Situated Visualization for Collaborative Tasks, In: 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), doi:10.1109/vr58804.2024.00098
gi.citations.elementMelanie Zimmer, Ali Al-Yacoub, Pedro Ferreira, Ella-Mae Hubbard, Niels Lohse (2022): Experimental study to investigate mental workload of local vs remote operator in human-machine interaction, In: Production & Manufacturing Research 1(10), doi:10.1080/21693277.2022.2090458
gi.citations.elementShihui Xu, Like Wu, Wenjie Liao, Shigeru Fujimura (2024): Enhancing Collaborative Shopping Experience Through Interactive Personalized Avatars and Shared Gaze in a Multi-User Augmented Reality Environment, In: International Journal of Human–Computer Interaction, doi:10.1080/10447318.2024.2381923
gi.citations.elementMatt Gottsacker, Mengyu Chen, David Saffo, Feiyu Lu, Blair MacIntyre (2023): Hybrid User Interface for Audience Feedback Guided Asymmetric Immersive Presentation of Financial Data, In: 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), doi:10.1109/ismar-adjunct60411.2023.00046
gi.citations.elementTaimur K. Khan, Philipp Schworm, Moritz F. Glatt, Catherinemary Ugoji, Achim Ebert, Jan C. Aurich (2024): Engaging human-centered design to maintain part manufacturing under reduced workforce restrictions, In: Production Engineering, doi:10.1007/s11740-024-01275-1
gi.citations.elementBernardo Marques, António Teixeira, Samuel Silva, João Alves, Paulo Dias, Beatriz Sousa Santos (2022): A critical analysis on remote collaboration mediated by Augmented Reality: Making a case for improved characterization and evaluation of the collaborative process, In: Computers & Graphics, doi:10.1016/j.cag.2021.08.006
gi.citations.elementYuxiang Yan, Xiaoliang Bai, Weiping He, Shuxia Wang, XiangYu Zhang, Liwei Liu, Qing Yu, Bing Zhang (2025): Less gets more attention: A novel human-centered MR remote collaboration assembly method with information recommendation and visual enhancement, In: Robotics and Computer-Integrated Manufacturing, doi:10.1016/j.rcim.2024.102898
gi.citations.elementKeisuke Mizuno, Keito Kamimura, Tadashi Ebihara, Naoto Wakatsuki, Keiichi Zempo (2021): 2D-3D Transformation of Pointing Objects Based on Skeletal Body Surface Models for Remote Collaboration, In: 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), doi:10.1109/gcce53005.2021.9621900

Files

Collections

JCSCW Vol. 24 (2015)