Document Type

Paper

Publication Date

7-24-2020

Faculty Mentor

Christopher J. Tralie

Abstract

Sonification is the process of deriving an audio representation of a time series which conveys important information about that time series. Otology and vision science have established that humans process audio information more quickly than visual information, and sonification can convey data to the visually impaired. In our work, we implement pipelines using Python/Numpy, and we handle both ordinary 1D time series and multivariate time series. For 1D time series, we find that using data to modulate the pitch or timing of preselected sounds (such as sine waves) simply and effectively captures repeating patterns and anomalies/outliers within the data. To capture similar patterns in multivariate time series, we explore three geometric approaches in which we treat the time series as a curve shape. First, we generalize the pitch modulation to multivariate time series by sonifying different dimensions in non-overlapping frequency bands. We also place “earcons” at particular locations in space, which we play at speeds proportional to the velocity of the curve when it is in proximity to their locations. Finally, we use the “Viterbi Algorithm” to create a sequence of sound bytes whose shape in an audio feature space is similar to that of the data curve. We also show how to use a “sliding window embedding” to leverage these geometric approaches for 1D time series. We showcase our sonification tools on 1D time series of sunspot data, on splines from Usain Bolt’s record setting 100 meter race, and on politicians’ GPS data over time.

Comments

Presented during the 22nd Annual Summer Fellows Symposium, July 24, 2020 at Ursinus College.

The downloadable file is a slide show with audio commentary.

The final project is available here.

Download

Open Access

Available to all.

Share

COinS