Journal Article

Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaboration

Thumbnail Image

Fulltext URI

Document type

Text/Journal Article

Additional Information

Date

2015, 2015

Authors

Journal Title

Journal ISSN

Volume Title

Source

Computer Supported Cooperative Work (CSCW): Vol. 24, No. 6

Publisher

Springer

Abstract

TeleAdvisor is a versatile projection-based augmented reality system designed for remote collaboration. It allows a remote expert to naturally guide a local user in need of assistance in carrying out physical tasks around real-world objects. The system consists of a small projector and two cameras mounted on top of a tele-operated robotic arm at the worker’s side, and an interface to view the camera stream, control the point-of-view and gesture using projected annotations at the remote expert’s side. TeleAdvisor provides a hands-free, mobile, low-cost solution that supports gesturing by the remote expert while minimizing the cognitive overhead of the local worker. We describe the challenges, design considerations and implementation details of the two phases of the TeleAdvisor prototype, as well as its evaluation and deployment at an industrial manufacturing center. We summarize our understandings from our experiences during the project and discuss the general implications for design of augmented reality remote collaboration systems.

Description

Gurevich, Pavel; Lanir, Joel; Cohen, Benjamin (2015): Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaboration. Computer Supported Cooperative Work (CSCW): Vol. 24, No. 6. DOI: 10.1007/s10606-015-9232-7. Springer. PISSN: 1573-7551. pp. 527-562

Keywords

Augmented reality, Mobile projector, Mobility, Remote assistance, Remote collaboration

Citation

DOI

10.1007/s10606-015-9232-7

URI

URI

http://dx.doi.org/10.1007/s10606-015-9232-7
 https://dl.eusset.eu/handle/20.500.12015/3860

Collections

JCSCW Vol. 24 (2015)

Endorsement

Review

Supplemented By

Referenced By

Number of citations to item: 57

  • Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Xiangyu Zhang, Shuxia Wang, Weiping He, Yuxiang Yan, Hongyu Ji (2021): AR/MR Remote Collaboration on Physical Tasks: A Review, In: Robotics and Computer-Integrated Manufacturing, doi:10.1016/j.rcim.2020.102071
  • Catarina G. Fidalgo, Yukang Yan, Hyunsung Cho, Maurício Sousa, David Lindlbauer, Joaquim Jorge (2023): A Survey on Remote Assistance and Training in Mixed Reality Environments, In: IEEE Transactions on Visualization and Computer Graphics 5(29), doi:10.1109/tvcg.2023.3247081
  • Bernardo Marques, Samuel Silva, António Teixeira, Paulo Dias, Beatriz Sousa Santos (2022): A vision for contextualized evaluation of remote collaboration supported by AR, In: Computers & Graphics, doi:10.1016/j.cag.2021.10.009
  • Cedric Kervegant, Julien Castet, Juliette Vauchez, Charles Bailly (2021): Distant Assist Cursor (DAC): Designing an Augmented Reality System to Facilitate Remote Collaboration for Novice Users, In: Companion Proceedings of the 2021 Conference on Interactive Surfaces and Spaces, doi:10.1145/3447932.3490520
  • Philipp Ladwig, Bastian Dewitz, Hendrik Preu, Mitja Säger (2019): Remote Guidance for Machine Maintenance Supported by Physical LEDs and Virtual Reality, In: Proceedings of Mensch und Computer 2019, doi:10.1145/3340764.3340780
  • Paola Fantini, Marta Pinzone, Marco Taisch (2020): Placing the operator at the centre of Industry 4.0 design: Modelling and assessing human activities within cyber-physical systems, In: Computers & Industrial Engineering, doi:10.1016/j.cie.2018.01.025
  • Xiangyu Zhang, Xiaoliang Bai, Shusheng Zhang, Weiping He, Shuxia Wang, Yuxiang Yan, Quan Yu, Liwei Liu (2023): A novel MR remote collaboration system using 3D spatial area cue and visual notification, In: Journal of Manufacturing Systems, doi:10.1016/j.jmsy.2023.02.013
  • Sung Ho Choi, Minseok Kim, Jae Yeol Lee (2018): Situation-dependent remote AR collaborations: Image-based collaboration using a 3D perspective map and live video-based collaboration with a synchronized VR mode, In: Computers in Industry, doi:10.1016/j.compind.2018.06.006
  • Jiawei Yu, Tengyue Wang, Yunze Shi, Liangjing Yang (2022): MR Meets Robotics: A Review of Mixed Reality Technology in Robotics, In: 2022 6th International Conference on Robotics, Control and Automation (ICRCA), doi:10.1109/icrca55033.2022.9828895
  • Jozef Husár, Lucia Knapčíková (2023): Implementation of Augmented Reality in Smart Engineering Manufacturing: Literature Review, In: Mobile Networks and Applications 1(29), doi:10.1007/s11036-023-02121-x
  • Yu Zhu, Jun Yang, Hongwei Zhang, Wenmin Zhu, Jie Wang, Zelin Zhou (2022): Intelligent assembly assistance for hull structure construction based on optical projection, In: Assembly Automation 2(42), doi:10.1108/aa-05-2021-0061
  • Lucy Whitfield (2023): Watching me … watching you: Training and assessment in a virtual world, In: Laboratory Animals 2(57), doi:10.1177/00236772221146824
  • Iñigo Fernández del Amo, John Erkoyuncu, Rok Vrabič, Romain Frayssinet, Cristina Vazquez Reynel, Rajkumar Roy (2020): Structured authoring for AR-based communication to enhance efficiency in remote diagnosis for complex equipment, In: Advanced Engineering Informatics, doi:10.1016/j.aei.2020.101096
  • Shuo Feng, Yizhe Liu, Qianrui Zhang, Weiping He, Xiaotian Zhang, Shuxia Wang, Mark Billinghurst (2023): Parallel or Cross? Effects of Two Collaborative Modes on Augmented Reality Co-located Operations, In: International Journal of Human–Computer Interaction 15(40), doi:10.1080/10447318.2023.2202574
  • Yue Wang, Peng Wang, Zhiyong Luo, Yuxiang Yan (2022): A novel AR remote collaborative platform for sharing 2.5D gestures and gaze, In: The International Journal of Advanced Manufacturing Technology 9-10(119), doi:10.1007/s00170-022-08747-7
  • Peng Wang, Shusheng Zhang, Mark Billinghurst, Xiaoliang Bai, Weiping He, Shuxia Wang, Mengmeng Sun, Xu Zhang (2019): A comprehensive survey of AR/MR-based co-design in manufacturing, In: Engineering with Computers 4(36), doi:10.1007/s00366-019-00792-3
  • Gitte Pedersen, Konstantinos Koumaditis (2020): Virtual Reality (VR) in the Computer Supported Cooperative Work (CSCW) Domain: A Mapping and a Pre-study on Functionality and Immersion, In: Lecture Notes in Computer Science, doi:10.1007/978-3-030-49698-2_10
  • Nicolas Kahrl, Michael Prilla, Oliver Blunk (2021): The Influence of Spatial Representation on Remote Peer Consultation, In: Mensch und Computer 2021, doi:10.1145/3473856.3473868
  • Jin Uk Kwon, Jae-In Hwang, Jaeyoung Park, Sang Chul Ahn (2019): Fully Asymmetric Remote Collaboration System, In: IEEE Access, doi:10.1109/access.2019.2912397
  • Lucas Pometti, Charles Bailly, Julien Castet (2023): AHO-Guide: Automatically Guiding the Head Orientation of a Local User in Augmented Reality to Realign the Field of View with Remote Users, In: Lecture Notes in Computer Science, doi:10.1007/978-3-031-42280-5_1
  • Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Sili Wei, Guangyao Xu, Weiping He, Xiangyu Zhang, Jie Zhang (2020): 3DGAM: using 3D gesture and CAD models for training on mixed reality remote collaboration, In: Multimedia Tools and Applications 20(80), doi:10.1007/s11042-020-09731-7
  • Mfundo A. Maneli, Omowunmi E. Isafiade (2023): A Multifactor Comparative Assessment of Augmented Reality Frameworks in Diverse Computing Settings, In: IEEE Access, doi:10.1109/access.2023.3242238
  • Peng Wang, Shusheng Zhang, Xiaoliang Bai, Mark Billinghurst, Weiping He, Li Zhang, Jiaxiang Du, Shuxia Wang (2018): Do You Know What I Mean? An MR-Based Collaborative Platform, In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), doi:10.1109/ismar-adjunct.2018.00038
  • Eleonora Bottani, Giuseppe Vignali (2019): Augmented reality technology in the manufacturing industry: A review of the last decade, In: IISE Transactions 3(51), doi:10.1080/24725854.2018.1493244
  • David Anton, Gregorij Kurillo, Ruzena Bajcsy (2018): User experience and interaction performance in 2D/3D telecollaboration, In: Future Generation Computer Systems, doi:10.1016/j.future.2017.12.055
  • YuXiang Yan, Xiaoliang Bai, Weiping He, Shuxia Wang, XiangYu Zhang, Peng Wang, Liwei Liu, Bing Zhang (2023): Can you notice my attention? A novel information vision enhancement method in MR remote collaborative assembly, In: The International Journal of Advanced Manufacturing Technology 3-4(127), doi:10.1007/s00170-023-11652-2
  • Guillaume Bataille, Abdelhadi Lammini, Jean-Rémy Chardonnet (2023): ARPuzzle: Evaluating the Effectiveness of Collaborative Augmented Reality, In: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), doi:10.1109/ismar59233.2023.00079
  • Zhuo Wang, Yang Wang, Xiaoliang Bai, Xiangyu Huo, Weiping He, Shuo Feng, Jie Zhang, Yueqing Zhang, Jinzhao Zhou (2021): SHARIDEAS: a smart collaborative assembly platform based on augmented reality supporting assembly intention recognition, In: The International Journal of Advanced Manufacturing Technology 1-2(115), doi:10.1007/s00170-021-07142-y
  • Xiangyu Zhang, Xiaoliang Bai, Shusheng Zhang, Weiping He, Shuxia Wang, Yuxiang Yan, Peng Wang, Liwei Liu (2024): A novel mixed reality remote collaboration system with adaptive generation of instructions, In: Computers & Industrial Engineering, doi:10.1016/j.cie.2024.110353
  • Bernardo Marques, Samuel Silva, Joao Alves, Tiago Araujo, Paulo Dias, Beatriz Sousa Santos (2022): A Conceptual Model and Taxonomy for Collaborative Augmented Reality, In: IEEE Transactions on Visualization and Computer Graphics 12(28), doi:10.1109/tvcg.2021.3101545
  • Bernardo Marques, Samuel Silva, Paulo Dias, Beatriz Sousa Santos (2022): Evaluating Augmented Reality Based Remote Collaboration: A Contextualized Approach, In: Automation, Collaboration, & E-Services, doi:10.1007/978-3-031-10788-7_16
  • Mfundo A. Maneli, Omowunmi E. Isafiade (2023): A Comparative Evaluation of Augmented Reality Frameworks: A Plane Mapping and Resource Utilisation Perspective, In: 2023 IST-Africa Conference (IST-Africa), doi:10.23919/ist-africa60249.2023.10187850
  • Ana Villanueva, Zhengzhe Zhu, Ziyi Liu, Feiyang Wang, Subramanian Chidambaram, Karthik Ramani (2022): ColabAR: A Toolkit for Remote Collaboration in Tangible Augmented Reality Laboratories, In: Proceedings of the ACM on Human-Computer Interaction CSCW1(6), doi:10.1145/3512928
  • Zahraa Bassyouni, Imad H. Elhajj (2021): Augmented Reality Meets Artificial Intelligence in Robotics: A Systematic Review, In: Frontiers in Robotics and AI, doi:10.3389/frobt.2021.724798
  • Taimur Kausar Khan, Margit Pohl, Philipp Schworm, Matthias Klar, Achim Ebert (2024): Towards sustaining human-centered part manufacturing under workforce restrictions, In: Proceedings of the European Conference on Cognitive Ergonomics 2024, doi:10.1145/3673805.3673841
  • Jin Uk Kwon, Eun Sun Lee, Sang Chul Ahn (2016): Projector-camera based remote assistance system for the elderly: Design issues and implementation, In: 2016 IEEE/SICE International Symposium on System Integration (SII), doi:10.1109/sii.2016.7844003
  • Azin Semsar, Hannah McGowan, Yuanyuan Feng, H. Reza Zahiri, Adrian Park, Andrea Kleinsmith, Helena Mentis (2019): How Trainees Use the Information from Telepointers in Remote Instruction, In: Proceedings of the ACM on Human-Computer Interaction CSCW(3), doi:10.1145/3359195
  • Peng Wang, Shusheng Zhang, Xiaoliang Bai, Mark Billinghurst, Li Zhang, Shuxia Wang, Dechuan Han, Hao Lv, Yuxiang Yan (2019): A gesture- and head-based multimodal interaction platform for MR remote collaboration, In: The International Journal of Advanced Manufacturing Technology 7-8(105), doi:10.1007/s00170-019-04434-2
  • Zenglei Wang, Shusheng Zhang, Xiaoliang Bai (2021): A mixed reality platform for assembly assistance based on gaze interaction in industry, In: The International Journal of Advanced Manufacturing Technology 9-10(116), doi:10.1007/s00170-021-07624-z
  • Azin Semsar, Hannah McGowan, Yuanyuan Feng, H. Reza Zahiri, Adrian Park, Andrea Kleinsmith, Helena M. Mentis (2020): Quality of and Attention to Instructions in Telementoring, In: Proceedings of the ACM on Human-Computer Interaction CSCW2(4), doi:10.1145/3415236
  • Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Dechuan Han, Mengmeng Sun, Zhuo Wang, Hao Lv, Shu Han (2020): Haptic Feedback Helps Me? A VR-SAR Remote Collaborative System with Tangible Interaction, In: International Journal of Human–Computer Interaction 13(36), doi:10.1080/10447318.2020.1732140
  • Philipp Ladwig, Christian Geiger (2018): A Literature Review on Collaboration in Mixed Reality, In: Lecture Notes in Networks and Systems, doi:10.1007/978-3-319-95678-7_65
  • Kenan Bektaş, Adrian Pandjaitan, Jannis Strecker, Simon Mayer (2024): GlassBoARd: A Gaze-Enabled AR Interface for Collaborative Work, In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, doi:10.1145/3613905.3650965
  • A. Golubkin, R. Pirmagomedov (2021): ANALYTICAL REVIEW OF AUGMENTED AND MIXED REALITY SYSTEMS IN THE CONTEXT OF INDUSTRY 4.0, In: Telecom IT 4(9), doi:10.31854/2307-1303-2021-9-4-1-27
  • Alexander Schäfer, Gerd Reis, Didier Stricker (2022): A Survey on Synchronous Augmented, Virtual, andMixed Reality Remote Collaboration Systems, In: ACM Computing Surveys 6(55), doi:10.1145/3533376
  • Zhanat Makhataeva, Huseyin Varol (2020): Augmented Reality for Robotics: A Review, In: Robotics 2(9), doi:10.3390/robotics9020021
  • Le Luo, Dongdong Weng, Jie Hao, Ziqi Tu, Haiyan Jiang (2023): Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System, In: Sensors 8(23), doi:10.3390/s23084113
  • Takafumi Asao, Takeru Kobayashi, Kentaro Kotani, Satoshi Suzuki, Kazutaka Obama, Atsuhiko Sumii, Tatsuto Nishigori (2021): Development of a Drawing Application for Communication Support during Endoscopic Surgery and Its Evaluation Using Steering Law, In: Applied Sciences 10(11), doi:10.3390/app11104505
  • Bernardo Marques, Samuel Silva, João Alves, António Rocha, Paulo Dias, Beatriz Sousa Santos (2022): Remote collaboration in maintenance contexts using augmented reality: insights from a participatory process, In: International Journal on Interactive Design and Manufacturing (IJIDeM) 1(16), doi:10.1007/s12008-021-00798-6
  • Nelusa Pathmanathan, Tobias Rau, Xiliu Yang, Aimée Sousa Calepso, Felix Amtsberg, Achim Menges, Michael Sedlmair, Kuno Kurzhals (2024): Eyes on the Task: Gaze Analysis of Situated Visualization for Collaborative Tasks, In: 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), doi:10.1109/vr58804.2024.00098
  • Melanie Zimmer, Ali Al-Yacoub, Pedro Ferreira, Ella-Mae Hubbard, Niels Lohse (2022): Experimental study to investigate mental workload of local vs remote operator in human-machine interaction, In: Production & Manufacturing Research 1(10), doi:10.1080/21693277.2022.2090458
  • Shihui Xu, Like Wu, Wenjie Liao, Shigeru Fujimura (2024): Enhancing Collaborative Shopping Experience Through Interactive Personalized Avatars and Shared Gaze in a Multi-User Augmented Reality Environment, In: International Journal of Human–Computer Interaction, doi:10.1080/10447318.2024.2381923
  • Matt Gottsacker, Mengyu Chen, David Saffo, Feiyu Lu, Blair MacIntyre (2023): Hybrid User Interface for Audience Feedback Guided Asymmetric Immersive Presentation of Financial Data, In: 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), doi:10.1109/ismar-adjunct60411.2023.00046
  • Taimur K. Khan, Philipp Schworm, Moritz F. Glatt, Catherinemary Ugoji, Achim Ebert, Jan C. Aurich (2024): Engaging human-centered design to maintain part manufacturing under reduced workforce restrictions, In: Production Engineering, doi:10.1007/s11740-024-01275-1
  • Bernardo Marques, António Teixeira, Samuel Silva, João Alves, Paulo Dias, Beatriz Sousa Santos (2022): A critical analysis on remote collaboration mediated by Augmented Reality: Making a case for improved characterization and evaluation of the collaborative process, In: Computers & Graphics, doi:10.1016/j.cag.2021.08.006
  • Yuxiang Yan, Xiaoliang Bai, Weiping He, Shuxia Wang, XiangYu Zhang, Liwei Liu, Qing Yu, Bing Zhang (2025): Less gets more attention: A novel human-centered MR remote collaboration assembly method with information recommendation and visual enhancement, In: Robotics and Computer-Integrated Manufacturing, doi:10.1016/j.rcim.2024.102898
  • Keisuke Mizuno, Keito Kamimura, Tadashi Ebihara, Naoto Wakatsuki, Keiichi Zempo (2021): 2D-3D Transformation of Pointing Objects Based on Skeletal Body Surface Models for Remote Collaboration, In: 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), doi:10.1109/gcce53005.2021.9621900
Please note: Providing information about citations is only possible thanks to to the open metadata APIs provided by crossref.org and opencitations.net. These lists may be incomplete due to unavailable citation data.source: opencitations.net, crossref.org