Searching for the Perfect Instrument 🔎

This project (REVOLVE) is, and extends upon, my master’s thesis, revolving around the use of user-centric genetic algorithms in virtual reality as a means of exploring musical instrument / interaction design space.

In simpler terms, this project / experience would randomly create mappings between a virtual reality object, and a simple subtractive synthesizer, forming different links between virtual reality interactions (such as velocity, elevation, rotation etc.) and the inputs to the synthesizer (ASDR envelope levels, pitch, volume etc.). Each object is represented as a genome, which forms the ‘core’ of the instrument-interaction. A user of the experience would then interact with the object, and indicate their preference for the mapping by either accepting or rejecting the object. After a certain number of accepted objects, a simple evolutionary algorithm is performed, essentially randomly merging the genomes of the objects, with some added mutations (random variations in numbers) to create a series of new objects – analogous to the real process. This experience keeps iterating, and the hypothesis being that the longer the time, user preference for the interaction increases.

The paper included a rigorous series of semi-structured interviews with participants, allowing for a series of suggested improvements to the technology, and revealing some insights around factors that increase telepresence in virtual reality, which can help inform further research into the area.

The paper was published and presented at NIME 2019. It can be found here.

The code for this project can be found on github here.