Searching for the Perfect Instrument 🔎

Searching for the Perfect Instrument 🔎

REVOLVE: Using user-centric genetic algorithms in virtual reality to explore musical instrument and interaction design space through evolutionary computing.

This project (REVOLVE) is, and extends upon, my master’s thesis, revolving around the use of user-centric genetic algorithms in virtual reality as a means of exploring musical instrument / interaction design space.

The Concept

In simpler terms, this project / experience would randomly create mappings between a virtual reality object, and a simple subtractive synthesizer, forming different links between virtual reality interactions (such as velocity, elevation, rotation etc.) and the inputs to the synthesizer (ASDR envelope levels, pitch, volume etc.).

Each object is represented as a genome, which forms the ‘core’ of the instrument-interaction. A user of the experience would then interact with the object, and indicate their preference for the mapping by either accepting or rejecting the object.

The Process

After a certain number of accepted objects, a simple evolutionary algorithm is performed, essentially randomly merging the genomes of the objects, with some added mutations (random variations in numbers) to create a series of new objects – analogous to the real process. This experience keeps iterating, and the hypothesis being that the longer the time, user preference for the interaction increases.

Research & Findings

The paper included a rigorous series of semi-structured interviews with participants, allowing for a series of suggested improvements to the technology, and revealing some insights around factors that increase telepresence in virtual reality, which can help inform further research into the area.

Publication & Code

The paper was published and presented at NIME 2019. It can be found here.

The code for this project can be found on GitHub.

Impact

This work explores the intersection of evolutionary computing, virtual reality, and musical interface design, contributing to our understanding of how users can collaboratively design with AI systems in immersive environments.