1. Principles and Practice


This project is an exploration in using senses, motion, and the human figure to personalise the learning experience. The starting point is to create a motion triggered piano that facilitates building new associations between sound and hand motion. The question to be answered in this project is: If the physicality and technical literacy of an instrument is eliminated, and the focus is set on building emotional connections with sound patterns as a starting point to learn, will it encourage users to pursue learning the piano further than initially intended?

What is it?

A hand motion triggered instrument that translates hand movements (input) to sound (output)


I wanted to use hand motion to trigger the sound to add a playful interactive element to the design. This is key to making it less intimidating for anyone using it. Another option I considered was to have a wall installation, because of the time supply for this project, I decided to go with the table top design as my prototype, then revisit the design if needed after my final presentation.

I will be using the C scale and two octaves (3 and 4) - this will give enough range and is a good starting point to test the prototype. Each row represents one octave, moving your hands left and right in one row triggers sound from one scale in one octave (C scale in either octave 3 or 4), and moving your hands up or down triggers the same key in a higher/lower octave.


The motion sensors I will be using are the essential part of making a working prototype.

My initial idea was to create a sensor grid of 30cm height and 60cm wide using an affordable short range motion sensor with low latency. My options were PIR or ultrasonic sensors; PIR sensors work on the principle of reflected light waves and are therefore sensitive to light. Ultrasonic sensors work using sound waves, they are more reliable because object detection is not affected by factors such light. For my project, I will be using an Ultrasonic Sensor HC-SR04.

Each piano key will have a sensor that triggers the respective sound as an output. The higher octave will have a high range than the lower octave (30cm and 15cm), this is to avoid triggering the lower octave keys when a hand is positioned on a higher octave key.

After doing more research on ultrasonic sensor, I found that a linear grid (70cm wide) would be the most suitable for this project.

Each octave will have a sensor, and each key will be programmed to play when the sensor detects an object (hand) within a set distance. For example, 10-20 cm will play keys E3 or E4 depending on if your hand is at the higher or lower octave.

The higher octaves are represented with a surface projecting outward, while the lower octave surface is projected inward. This will aid users in building association between high/low pitches and motion.