Submission Date
7-24-2020
Document Type
Paper
Department
Computer Science
Faculty Mentor
Christopher J. Tralie
Project Description
Sonification is the process of deriving an audio representation of a time series which conveys important information about that time series. Otology and vision science have established that humans process audio information more quickly than visual information, and sonification can convey data to the visually impaired. In our work, we implement pipelines using Python/Numpy, and we handle both ordinary 1D time series and multivariate time series. For 1D time series, we find that using data to modulate the pitch or timing of preselected sounds (such as sine waves) simply and effectively captures repeating patterns and anomalies/outliers within the data. To capture similar patterns in multivariate time series, we explore three geometric approaches in which we treat the time series as a curve shape. First, we generalize the pitch modulation to multivariate time series by sonifying different dimensions in non-overlapping frequency bands. We also place “earcons” at particular locations in space, which we play at speeds proportional to the velocity of the curve when it is in proximity to their locations. Finally, we use the “Viterbi Algorithm” to create a sequence of sound bytes whose shape in an audio feature space is similar to that of the data curve. We also show how to use a “sliding window embedding” to leverage these geometric approaches for 1D time series. We showcase our sonification tools on 1D time series of sunspot data, on splines from Usain Bolt’s record setting 100 meter race, and on COVID-19 data of cases/deaths.
Recommended Citation
Tecce, Michael, "Geometry Aided Sonification" (2020). Computer Science Summer Fellows. 4.
https://digitalcommons.ursinus.edu/comp_sum/4
Open Access
Available to all.
Comments
Presented during the 22nd Annual Summer Fellows Symposium, July 24, 2020 at Ursinus College.
A related presentation is available here.
The downloadable file is a zip file containing sonifications in mp4 format, audio wave forms, Sliding Window images, and a guide describing the files. The file is split into two folders, one for MACs and one for PCs.