This projects involves the design of a software environment for real-time, interactive performance with biosensors. Essential tools include beat-detection on an ECG waveform, as well as sonification and visualization of this data in a variety of ways. The software is intended to produce not only musical results, but to aid the musicians, dancers, and directors during the rehearsal and performance process, increasing understandings of the relationship between action and environment with internal bodily processes.
The design goals for the software come out of the following realizations:
- The project requires research, development, and production to be done concurrently, requiring an agile approach to software development, i.e., development must be able to take place on the fly without the fear of breaking software. One strategy we employ is the use of Aspect-Oriented Programming in Max/MSP.
- Software must work consistently if we are to build an intuition about the biological processes that form the basis of our creative work, therefore, testing and verification are critical.
- We understand that as we begin to scale the software and hardware up to support larger numbers of dancers and musicians, we will likely run into performance problems, but we do not currently know where, therefore, the software must be modular enough to be distributed across a number of (possibly heterogenous) computers.
- Robust timing and an understanding of the jitter present in the system are critical. We are interested not only in the heart rate of the dancers, but in its variability and wish to preserve that variability in performance, therefore, we must take steps to understand and minimize variability from other sources such as the Wi-Fi network (http://cnmat.berkeley.edu/system/files/attachments/OSC-time-ICMC2008-2.pdf).
The following software packages are currently being used for this project:
- Visual programming language and multimedia authoring environment
- Interactive visualization library for Python
- Open Music
- Lisp-based visual programming language for music representation
- Embeddable programming language (with bindings for Max/MSP, Python, Lisp, and Open Music) which uses Open Sound Control as its datatype
All software can be found at the github repository for the project: