In preparation AI4Science
Strokes of Sonic Tradition
Team member, University of Michigan, PI Prof. Herman Dong
The guqin, a seven-string zither on UNESCO’s list of Intangible Cultural Heritage, was the instrument of China’s literati from the Song through the Qing dynasties. Between 30,000 and 100,000 historical pieces survive in jianzipu, a shorthand tablature. Each character is built from components that say where the hand goes and how the string is plucked, but none of them say how long a note lasts or how a phrase moves: that was passed on from teacher to student.
Strokes of Sonic Tradition builds AI that can read this corpus and connect it to living practice. It develops data-efficient optical music recognition shaped around the way jianzipu characters are composed (Figure 1), an annotation studio that detects the strokes of each character, and music-information-retrieval and computer-vision tools that align historical scores with modern audio and video of guqin players.
The two-year project received $343K from Schmidt Sciences’ Humanities and AI Virtual Institute (HAVI), one of 19 teams in the program. It is led by Prof. Hao-Wen (Herman) Dong with David Sears, Nathan Martin and Joseph Lam at the University of Michigan, Eduardo Veas at Graz University of Technology and Ming Tak Ted Hui at the University of Oxford; the team also includes Ming Wilson, Tristan Repolusk, Yuxuan Xin, Yikang Hong and me. A paper is in preparation.
Figures
