The system described here is Jie Liu's the final project for ISE-575b, Computational Modeling of Expressive Performance, in Spring 2006.

Langer and Goebl proposed a tempo-loudness space for visualizing trajectories of, and analyzing, expressive performances in their 2002 paper, Visualizing Expressive Performance in Tempo-Loudness Space. They mentioned, at the end of the paper, that the method could be reversed, using it as an interactive control of music performance.

I have been working on the Expression Synthesis Project for several years. The goal of the project is to create a driving interface for synthesizing musical expressions. In ESP, the score is mapped to the road, but in this project, the square map is the tempo-loudness space. The tempo will be mapped to a suitable range, as well as the loudness. The user will still drive the car to control the tempo and loudness in real-time, but in a different space.

The project implementation is based on Modular Flow Scheduling Middleware, developed by Dr. Alexandre R.J. François, and the improv, developed by Dr. Craig Stuart Sapp.

Project Implementation

Space Construction
The tempo ranges from 0 BPM (beats per minute) to 150 BPM. Since I use MIDI files, the loudness value ranges from 0 to 127.

Control Points
An expressionless MIDI is processed by the ESP system, with tempo and loudness recorded in a file. An instruction file is made by selecting several rows from the file, which are control points for the map. These control points will navigate the direction and the velocity of the user's driving. The Hermite Spline rendering is employed to connect control points, and the following graph shows the final map.

Driving
Although a map is given, the user does not have to follow it. The system will indicate the driving direction to the user by rendering a green line from the car's current position to the upcoming control point. The system will also let the user know when to speed up or slow down, in order to encourage the user to come to the control point at the right time.

Experimental Results

The following graph shows the visual interface of the system.

Hungrian Dance No.5

Hungarian Dance No.3

The MIDI files generated by the system are shown below, with the target MIDI files (used to generate control points) and expressionless MIDI files.

Hungarian Dance No.3 | Output | Target | Expressionless

Hungarian Dance No.5 | Output | Target | Expressionless

Conclusions and Discussions

This system helps the user to create expressive performance in Langer's Tempo-Loudness space using a driving interface. Some control points are provided to guide the user's driving. By rendering signs and showing texts, the system gives suggestion about the driving direction and the driving speed.

On the other hand, since the tempo of the performance is related to the horizontal coordinate of the car's position, instead of the car's velocity, it is less intuitive than ESP, where tempo maps directly to car speed.

Hermite Spline curves were tested in this project, and some future work could focus on how to make use of them more efficiently in road design.