An Accessible, Browser-Based Gestural Controller for Web Audio, MIDI, and Open Sound Control

被引：0
|
作者：
Lim, Monica [1 ]
Kotsani, Natalia [2 ]
机构：
[1] Univ Melbourne, Fac Fine Arts & Media, Grattan St, Melbourne, Vic 3010, Australia
[2] Natl Tech Univ Athens, Sch Elect & Comp Engn, Zografou Campus,9 Iroon Polytech str, Athens 15772, Greece
来源：
COMPUTER MUSIC JOURNAL | 2023年 / 47卷 / 03期
关键词：
D O I：
10.1162/COMJ_a_00693
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
Handmate is a browser-based hand gestural controller for Web Audio, MIDI, and Open Sound Control (OSC), using open-source pose-estimation technology from Google MediaPipe. The Web browser was chosen as the platform to execute all computational processes so that users could access the latest computer vision technology for music-making without requiring any specialized hardware, software, or technical expertise. By combining preexisting technologies in a browser environment, we enable accessible, interactive sound-making using hand movement. Three controllers are described: (1) Handmate Effects, which receives microphone or user sound input and provides a choice of audio effects that can be controlled by hand movement; (2) Handmate MIDI, which outputs MIDI notes or control values; and (3) Handmate OSC, which outputs OSC messages for users to access hand-movement data in other applications.
引用
收藏
页码：6 / 18
页数：13
相关论文
共 50 条
  • [41] JS']JSDoop and TensorFlow.js']js: Volunteer Distributed Web Browser-Based Neural Network Training
    Morell, Jose A.
    Camero, Andres
    Alba, Enrique
    IEEE ACCESS, 2019, 7 : 158671 - 158684
  • [42] Web Service Access Control Based on Browser Fingerprint Detection
    Liu Hui
    He Xudong
    Gao Fan
    Wang KaiLun
    Yuan Enze
    JOURNAL OF WEB ENGINEERING, 2021, 20 (05): : 1587 - 1621
  • [43] Tesseract-Medical Imaging: Open-Source Browser-based Platform for Artificial Intelligence Deployment in Medical Imaging
    Sedghi, Alireza
    Hamidi, Soheil
    Mehrtash, Alireza
    Ziegler, Erik
    Tempany, Clare
    Pieper, Steve
    Kapur, Tina
    Mousavi, Parvin
    MEDICAL IMAGING 2019: IMAGE-GUIDED PROCEDURES, ROBOTIC INTERVENTIONS, AND MODELING, 2019, 10951
  • [44] IMAGE: An Open-Source, Extensible Framework for Deploying Accessible Audio and Haptic Renderings of Web Graphics
    Regimbal, Juliette
    Blum, Jeffrey R.
    Kuo, Cyan
    Cooperstock, Jeremy R.
    ACM TRANSACTIONS ON ACCESSIBLE COMPUTING, 2024, 17 (02)
  • [45] Starting to think digitally: Evaluation of web browser-based presentation of images at a Rad Path conference in a traditional film-based environment
    Hirschorn, DS
    Sider, LF
    JOURNAL OF DIGITAL IMAGING, 2000, 13 (02) : 236 - 237
  • [46] Starting to think digitally: Evaluation of web browser-based presentation of images at a rad path conference in a traditional film-based environment
    David S. Hirschorn
    Lee F. Sider
    Journal of Digital Imaging, 2000, 13 : 236 - 237
  • [47] Starting to think digitally: Evaluation of web browser-based presentation of images at a Rad Path conference in a traditional film-based environment
    Hirschorn, David S.
    Sider, Lee F.
    Journal of Digital Imaging, 2000, 13 (2 SUPPL. 1) : 236 - 237
  • [48] Data collection and web browser of embedded control system based on Internet
    Ai, H
    Wang, J
    Li, ZX
    Wavelet Analysis and Active Media Technology Vols 1-3, 2005, : 929 - 934
  • [49] Browser-based Crypto Mining and EU Data Protection and Privacy Law: A Critical Assessment and Possible Opportunities for the Monetisation of Web Services
    Mondschein, Christopher F.
    JOURNAL OF THE BRITISH BLOCKCHAIN ASSOCIATION, 2020, 3 (02): : 27 - 35
  • [50] ERP-System With networks create Islands of trust Browser-based user interface makes Application flexible and on Intranet and Internet accessible throughout the world
    Vossmann, Helmut
    FLEISCHWIRTSCHAFT, 2011, 91 (04): : 42 - 43
12345