Mixed Reality and Artificial Intelligence: A Holistic Approach to Multimodal Visualization and Extended Interaction in Knee Osteotomy

被引:1
|
作者
Moglia, Andrea [1 ]
Marsilio, Luca [1 ]
Rossi, Matteo [1 ,2 ]
Pinelli, Maria [3 ]
Lettieri, Emanuele [3 ]
Mainardi, Luca [1 ]
Manzotti, Alfonso [4 ]
Cerveri, Pietro [1 ,2 ]
机构
[1] Politecn Milan, Dept Elect Informat & Bioengn, Milan I-20133, Italy
[2] Ist Auxol Italiano IRCCS, I-20149 Milan, Italy
[3] Politecn Milan, Dept Management Econ & Ind Engn, Milan I-20133, Italy
[4] Hosp ASST FBF Sacco, Milan I-20157, Italy
关键词
Lower limb osteotomy; surgical planning; image segmentation; deep neural networks; OSTEOARTHRITIS; EPIDEMIOLOGY;
D O I
10.1109/JTEHM.2023.3335608
中图分类号
R318 [生物医学工程];
学科分类号
0831 ;
摘要
Objective: Recent advancements in augmented reality led to planning and navigation systems for orthopedic surgery. However little is known about mixed reality (MR) in orthopedics. Furthermore, artificial intelligence (AI) has the potential to boost the capabilities of MR by enabling automation and personalization. The purpose of this work is to assess Holoknee prototype, based on AI and MR for multimodal data visualization and surgical planning in knee osteotomy, developed to run on the HoloLens 2 headset. Methods: Two preclinical test sessions were performed with 11 participants (eight surgeons, two residents, and one medical student) executing three times six tasks, corresponding to a number of holographic data interactions and preoperative planning steps. At the end of each session, participants answered a questionnaire on user perception and usability. Results: During the second trial, the participants were faster in all tasks than in the first one, while in the third one, the time of execution decreased only for two tasks ("Patient selection" and "Scrolling through radiograph") with respect to the second attempt, but without statistically significant difference (respectively p = 0.14 and p = 0.13, p<0.05). All subjects strongly agreed that MR can be used effectively for surgical training, whereas 10 (90.9%) strongly agreed that it can be used effectively for preoperative planning. Six (54.5%) agreed and two of them (18.2%) strongly agreed that it can be used effectively for intraoperative guidance. Discussion/Conclusion: In this work, we presented Holoknee, the first holistic application of AI and MR for surgical planning for knee osteotomy. It reported promising results on its potential translation to surgical training, preoperative planning, and surgical guidance. Clinical and Translational Impact Statement - Holoknee can be helpful to support surgeons in the preoperative planning of knee osteotomy. It has the potential to impact positively the training of the future generation of residents and aid surgeons in the intraoperative stage.
引用
收藏
页码:279 / 290
页数:12
相关论文
共 50 条
  • [1] A holistic approach to implementing artificial intelligence in radiology
    Kim, Bomi
    Romeijn, Stephan
    van Buchem, Mark
    Mehrizi, Mohammad Hosein Rezazade
    Grootjans, Willem
    [J]. INSIGHTS INTO IMAGING, 2024, 15 (01)
  • [2] A holistic approach to implementing artificial intelligence in radiology
    Bomi Kim
    Stephan Romeijn
    Mark van Buchem
    Mohammad Hosein Rezazade Mehrizi
    Willem Grootjans
    [J]. Insights into Imaging, 15
  • [3] The Combination of Artificial Intelligence and Extended Reality: A Systematic Review
    Reiners, Dirk
    Davahli, Mohammad Reza
    Karwowski, Waldemar
    Cruz-Neira, Carolina
    [J]. FRONTIERS IN VIRTUAL REALITY, 2021, 2
  • [4] Exposure to Extended Reality and Artificial Intelligence-Based Manifestations: A Primer on the Future of Hip and Knee Arthroplasty
    Shaikh, Hashim J. F.
    Hasan, Sayyida S.
    Woo, Joshua J.
    Lavoie-Gagne, Ophelie
    Long, William J.
    Ramkumar, Prem N.
    [J]. JOURNAL OF ARTHROPLASTY, 2023, 38 (10): : 2096 - 2104
  • [5] A Holistic Approach to Implementing Artificial Intelligence in Lung Cancer
    Haghighikian, Seyed Masoud
    Shirinzadeh-Dastgiri, Ahmad
    Vakili-Ojarood, Mohammad
    Naseri, Amirhosein
    Barahman, Maedeh
    Saberi, Ali
    Rahmani, Amirhossein
    Shiri, Amirmasoud
    Masoudi, Ali
    Aghasipour, Maryam
    Shahbazi, Amirhossein
    Ghelmani, Yaser
    Aghili, Kazem
    Neamatzadeh, Hossein
    [J]. INDIAN JOURNAL OF SURGICAL ONCOLOGY, 2024,
  • [6] Multimodal Interaction with Gaze and Pressure Ring in Mixed Reality
    Wang, Zhimin
    Sun, Jingyi
    Hu, Mingwei
    Rao, Maohang
    Ge, Yangshi
    Song, Weitao
    Lu, Feng
    [J]. 2024 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES ABSTRACTS AND WORKSHOPS, VRW 2024, 2024, : 841 - 842
  • [7] Technologies for Multimodal Interaction in Extended Reality-A Scoping Review
    Rakkolainen, Ismo
    Farooq, Ahmed
    Kangas, Jari
    Hakulinen, Jaakko
    Rantala, Jussi
    Turunen, Markku
    Raisamo, Roope
    [J]. MULTIMODAL TECHNOLOGIES AND INTERACTION, 2021, 5 (12)
  • [8] Artificial Intelligence, Computational Simulations, and Extended Reality in Cardiovascular Interventions
    Samant, Saurabhi
    Bakhos, Jules Joel
    Wu, Wei
    Zhao, Shijia
    Kassab, Ghassan S.
    Khan, Behram
    Panagopoulos, Anastasios
    Makadia, Janaki
    Oguz, Usama M.
    Banga, Akshat
    Fayaz, Muhammad
    Glass, William
    Chiastra, Claudio
    Burzotta, Francesco
    LaDisa, John F., Jr.
    Iaizzo, Paul
    Murasato, Yoshinobu
    Dubini, Gabriele
    Migliavacca, Francesco
    Mickley, Timothy
    Bicek, Andrew
    Fontana, Jason
    West, Nick E. J.
    Mortier, Peter
    Boyers, Pamela J.
    Gold, Jeffrey P.
    Anderson, Daniel R.
    Tcheng, James E.
    Windle, John R.
    Samady, Habib
    Jaffer, Farouc A.
    Desai, Nihar R.
    Lansky, Alexandra
    Mena-Hurtado, Carlos
    Abbott, Dawn
    Brilakis, Emmanouil S.
    Lassen, Jens Flensted
    Louvard, Yves
    Stankovic, Goran
    Serruys, Patrick W.
    Velazquez, Eric
    Elias, Pierre
    Bhatt, Deepak L.
    Dangas, George
    Chatzizisis, Yiannis S.
    [J]. JACC-CARDIOVASCULAR INTERVENTIONS, 2023, 16 (20) : 2479 - 2497
  • [9] Artificial Intelligence For Air Quality Control Systems: A Holistic Approach
    Bazzi, Tony
    Zohdy, Mohamed
    [J]. 2018 TWENTIETH INTERNATIONAL MIDDLE EAST POWER SYSTEMS CONFERENCE (MEPCON), 2018, : 25 - 32
  • [10] Jointly Optimizing Sensing Pipelines for Multimodal Mixed Reality Interaction
    Rathnayake, Darshana
    de Silva, Ashen
    Puwakdandawa, Dasun
    Meegahapola, Lakmal
    Misra, Archan
    Perera, Indika
    [J]. 2020 IEEE 17TH INTERNATIONAL CONFERENCE ON MOBILE AD HOC AND SMART SYSTEMS (MASS 2020), 2020, : 309 - 317