Benchmarking analysis of human pose estimation solutions for virtual television sets

  1. Arenas, Rubén 2
  2. Méndez, Roi 2
  3. Pedraza, Luis 1
  4. Flores, Julian 2
  1. 1 UNIR, Spain
  2. 2 Universidade de Santiago de Compostela, Spain
Actas:
Proceedings of the XXIV International Conference on Human Computer Interaction

Año de publicación: 2024

Tipo: Aportación congreso

DOI: 10.1145/3657242.3657244 GOOGLE SCHOLAR lock_openAcceso abierto editor

Resumen

In recent years, the use of virtual television sets (VTS) has grown in traditional TV productions, online broadcasting shows and streaming for both professional and amateur applications. Nevertheless, the interaction between actor or television anchors and the virtual scene is very limited because human body tracking is a complex problem that requires expensive equipment and high-performance software to be developed in real time. On the other hand, Human Pose Estimation (HPE) by low-cost devices, has been a hot topic of research due to its wide range of applications from sport visualization, security, medicine and so on. The objective of this paper is to determine if the modern technologies of human pose estimation can be used as interface between users, actors, presenters or speakers and a scene in a VTS. A comprehensive comparative of the different technologies is developed to determine those solutions that can be used in VTS for broadcasting and streaming, allowing to improve the communicative capacities of the modern VTS.

Referencias bibliográficas

  • 10.1007/s11042-019-08064-4
  • [Brainstorm. 2022. Brainstorm. https://www.brainstorm3d.com.
  • 10.1109/TPAMI.2019.2929257
  • 10.1109/ICCV.2017.137
  • 10.1109/VR.2017.7892322
  • 10.26599/TST.2018.9010100
  • Google MediaPipe. 2023. MediaPipe. . https://mediapipe.dev/
  • 10.1109/ICIP.2015.7351586
  • 10.1007/s10489-020-01918-7
  • 10.5594/JMI.2016.2632398
  • Ginés Hidalgo Martínez, Yaser Sheikh, Kris Kitani, and Aayush Bansal. 2019. OpenPose: Whole-Body Pose Estimation. Master Thesis April (2019).
  • Paul Kruszewski and Thomas Jan Mahamad. 2018. The AI Powered Magic Mirror: Building Immersive AR/VR Experiences with Only Webcams and Deep Learning. In ACM SIGGRAPH 2018 Virtual, Augmented, and Mixed Reality, (2018)
  • MediaPipe. 2023. Pose - MediaPipe. https://google.github.io/mediapipe/solutions/pose .
  • 10.1007/s10484-005-6381-3
  • 10.1109/VR.2003.1191132
  • 10.1109/3DV.2017.00064
  • 10.1145/3072959.3073596
  • 10.1007/s10209-017-0586-0
  • Nobuyasu Nakano Tetsuro Sakura Kazuhiro Ueda Leon Omura Arata Kimura Yoichi Iino Senshi Fukashiro and Shinsuke Yoshioka. 2020. Evaluation of 3D Markerless Motion Capture Accuracy Using OpenPose With Multiple Video Cameras. Front Sports Act Living 2 (2020). https://doi.org/10.3389/fspor.2020.00050
  • 10.1007/978-3-030-20205-7_25
  • 10.1007/978-3-030-01264-9_17
  • M., Jaksic, B., Spalevic, P., Petrovic, I., Dakovic Petrovic, B Jaksic, P Spalevic, I. Petrovic, and Dakovic V. 2012. The analysis background on the effect of chroma-key in virtual tv studio. INFOTECH 12 (2012), 973–941.
  • M. J. Schuemie P. Van der Straaten M. Krijn and C. A.P.G. Van der Mast. 2001. Research on presence in virtual reality: A survey. Cyberpsychology and Behavior 4. https://doi.org/10.1089/109493101300117884
  • 10.1109/11.40835
  • 10.1109/CVPR.2015.7298594
  • Town of Ocean City. 2023. Patrol Semaphore. https://oceancitymd.gov/pdf/ocbpsemaphore.pdf.
  • Alfred N.; Goldsborough T.R. Goldsmith. 1937. US2073370A Television system – Google Patents.
  • 10.1109/93.664742
  • WU Y. (2022). Detectron2. https://github.com/facebookresearch/detectron2.
  • Yuliang Xiu, Jiefeng Li, Haoyu Wang, Yinghong Fang, and Cewu Lu. 2019. Pose flow: Efficient online pose tracking. In British Machine Vision Conference 2018, BMVC 2018,
  • 10.1007/978-3-030-58526-6_14
  • R. Méndez J. Flores E. Castelló and J. R. R. Viqueira 2019 “Natural interaction in virtual TV sets through the synergistic operation of low-cost sensors ” Univers. Access Inf. Soc. 18 17-29 doi: 10.1007/s10209-017-0586-0.