Final Project: Patchlet
Table of Contents
My final project was a wireless and wearable MIDI controller bracelet, called patchlet. It sends generic MIDI CC messages based on the pitch and roll. Those could then be mapped to control parameters in light software, but also synthesizers, visual art software, etc.. All design files and code can be downloaded at the bottom of this page to be able to reproduce it.

The Patchlet
An important part of music culture events is not only the music, but also visual art as well as the light scenery. My idea was to have wearable, wireless interface in form of a glove or a bracelet including several sensors (accelerometers, vector magnetometer, etc.). The device could then send control signals to a music or light component, e.g. via OSC or MIDI. This would enable light operators, musicians and visual artists to change parameters in their setup while moving and/or being away from their personal computer.


Alternative ideas for potential final projects are listed at the final project proposal page.
BOM
Item | Price/Unit (€) | Quantity | Where from |
---|---|---|---|
XIAO ESP32S3 | 6.09 | 2 | AliExpress |
ADAFRUIT 9-DOF ORIENTATION IMU F | 21.85 | 1 | Digikey |
Pin sockets SMD vertical, 1x20, (actually needed per piece: 1x7 (2x) + 1x6 (2x)) | 1.99 | 1.5 | Conrad |
Pin headers THT vertical, 1x40, (actually needed per piece: 2 1x7, 2 1x6) | 0.46 | 1.6 | Conrad |
DIN A3 Plywood, 3mm thick | 2.50 | 0.005 | Amazon |
Prusament PETG, Urban Grey | 27.99 | 0.055 | Prusa |
M2x8 mm wood screws (300 pack, actually needed: 5) | 5.99 | 0.1 | Amazon |
USB C to USB A cable | 1.22 | 1 | Amazon |
FR1 sheet | 0.53 | 0.5 | Amazon |
Velcro tape for sewing, 16mm x 3m (hooks + loops) | 6.99 | 0.2 | Amazon |
Polyester | 2.20 | 0.17 | Makema |
Sewing thread | 4.30 | less than one roll | Makema |
Jeans needle | 0.91 | 0.1 | Amazon |
The material costs for the final project therefore are 39.37€. The item with the largest cost, the Adafruit BNO085 board containing the gyroscope, can be substituted to directly use this item of 3.84€. This way, the cost per piece could be decreased to roughly 21€.
Besides that, soldering material (including soldering tin, flux, etc.) is needed.
Prior Work & Literature
Wearables
- Imogen Heap using MiMU gloves: https://youtu.be/3QtklTXbKUQ?feature=shared&t=554
- In the example, she uses them to modulate e.g. panning
- The inputs are
- pitch, yaw, roll, specific positions that can be reached,
- acceleration,
- position of the single fingers,
- and the position of all fingers (e.g. when making a fist).
- There is a software included with the controller so that different things can be mapped.
- She uses the glove in the following ways
- Fist: recording audio input (voice), can be looped.
- Another gesture: deleting the loop.
- Finger bending: each finger activates an effect, hand position changes parameter.
- The MiMU glove can also be used for air drumming. Different types of drumming movements are supported.
- wrist flick
- slap
- Modular body bend sensors, using velostat
- Hand muscle trainer
Other glove projects
- https://hackaday.io/project/8455-power-glove-universal-hid
- https://www.youtube.com/watch?v=NjTWAy4-lLI
Communication & MIDI
Fabrication
Programming
In the input-devices week (9), I set up a gyroscope for reading pitch and roll of the glove. In the networking-and-communication week (11), I set up the ESP32S3 to send MIDI signals to the laptop via USB-MIDI. Together with my fellow student Niclas, I set up two ESP32S3 boards to communicate with each other.
In the system integration week (15), I integrated the gyroscope with the MIDI communication. Initially, I developed entirely using the ESP-IDF. However, I faced reliability issues with the SPI interface for the BNO085 sensor using its library for the ESP-IDF. It was not possible to use the I2C interface in the BNO085 library for the ESP-IDF. Therefore, I switched to using the Arduino environment for the remote to enable communication with the sensor via I2C. The dongle I kept programming using the ESP-IDF for USB-MIDI support, which again was not available in the Arduino environment. ESP-NOW was used to transmit pitch and roll of the BNO085 sensor. Finally, I designed MIDI behavior where pitch controls CC messages (only when above a threshold) and roll triggers NOTE_ON/OFF messages. The dongle could then be registered as a MIDI controller and mapped in an arbitrary digital audio workstation for controlling filter cut-off, pitch, etc.
Electronics
In the electronics design week (6) I learned how to use KiCad for designing PCBs. In the electronics production week (8) I learned how to fabricate a PCB using a CNC mill for isolation milling. In the system integration week (15) I designed and fabricated PCBs for the remote and the dongle. Below one can see the final setup without housing.

Packaging
3D and 2D design was taught in multiple weeks. Most about CAD I learned during the computer-controlled cutting week (3), the computer-controlled machining week (7), and mechanical & machine design week (12). As a CAD software I used FreeCAD. With FreeCAD I designed the housings for the PCBs, as well as the wristbands to mount the remote onto the forearm. Details can be read on the page on the system integration week (15).



Assembly
During the design process of electronics and packaging much thought went into how to assemble everything. Details are documented on the page about the system integration week (15).

Evaluation
The product was evaluated by
- shaking and dropping it to check if any parts were loose,
- connecting the dongle to the PC and mapping the MIDI controls to a synthesizer plug-in.
- using the controller for performing visual art and re-iterating on where it did not feel good from the perspective of a user.
Implications
Implications were documented in week 18.
Answered Questions
In week 18 part of our assignment was to state questions we would need to answer to finish the final project. Below the questions and answers to them can be seen.
- On which inputs should the vibration motor be programmed to be feedback?
- Not answered, since the vibration motor was not included
- How is the cut pattern for cutting the glove look and how should it be attached to a hand?
- This was not answered, but it was found that a normal hand-like pattern from polyester is not elastic at all and therefore makes it impossible to function as a hand overlay for mounting bend sensors.
- How is the PCB integrated with the glove?
- The PCB would be integrated with the glove by mounting ribbon cables coming from the PCB using snap fasteners that are sewn onto the glove.
- How is the IMU integrated with the ESP-IDF?
- It is not. Instead of the ESP-IDF, the Arduino environment was used for programming the remote containing the Adafruit BNO085 board.
- How to built a sensor for detecting pressure at the fingertips using velostat? Is this even possible?
- Building pressure sensors was excluded from the scope of the project during development.
- How are the sensor data processed? (Acceleration peaks, noisy bend sensor signals)
- As of now, there is not much data processing implemented by me. I only use events and values provided by the Adafruit board.
- How are the bend sensors calibrated?
- No bend sensors were used so they did not need to be calibrated.
Further Work
There are many ideas to be implemented. Some of them are listed below.
- Second controller for the other hand.
- NOTE_ON message on acceleration peaks.
- Relative position of hands to each other as CC input.
- Glove-like overlay for the hand, including
- (velostat) bend sensors on the fingers,
- pressure sensors for drumming,
- WS2812B LEDs
- attached to the back of the hand to indicate pitch and roll
- attached to index, middle, ring fingers to indicate finger bends
- PCB on fabric: either embroidered or sewn, just like the Lilypad
- vibration motor for feedback
- Adding OSC mode to use controller as OSC controller. Alternatively, a converter can be created using e.g. puredata.
- Case for everything to carry around.
- Improve feel of wristband. It is not yet suitable for all wrists.
- Replace Adafruit IMU breakout board with the actual SMD part.
- Power management
- Add temperature monitoring.
- Add voltage monitoring.
- Research again about industry-grade battery management.
- Calculate position from acceleration values to have it as a control input.
- Laser TOF sensor to measure distance of palm from floor, send as CC message. Only turn on if palm is turned down.
Further links
- https://www.digikey.com/en/products/detail/gct/FFC3B11-04-T/10657221
- https://www.digikey.com/en/products/detail/molex/0982670701/3469873
- https://i.ytimg.com/vi/7ZRpyZp4gq8/maxresdefault.jpg
- https://duckduckgo.com/?q=lilypad+arduino
- https://fabricademy.fabcloud.io/handbook/
Bend Sensors
In the end, I did not include any bend sensors, but I started building them. For that I used velostat, a material that changes resistance when it is compressed. For making bend sensors using velostat and fabric, this tutorial could be considered. Additional resources include
- Post about making drum pads
- Playlist on making velostat-based sensors
Building a very basic bend sensor. The velostat is sandwitched by tape. The sensor velostat changes resistance when it is compressed by the surrounding tape.
For measuring the changing resistance one could use a wheatstone bridge or a voltage divider. I decided to use a voltage divider for the first iteration. The voltage above the variable resistance, according to the voltage devider rule, is \[V_{out} = V_{CC} \frac{R_{var}}{R_{fixed} + R_{var}}\] where \(R_{var}\) is the resistance of the velostat-strip and \(R_{fixed}\) is the value of the other resistor in the voltage divider. While the range of \(R_{var}\) is given, \(R_{fixed}\) can be chosen by the designer to maximize the range of \(V_{out}\). The reason for this is to use the full range of the ADC on the micro-controller so the impact of the quantization noise is minimized. Another point to consider was that the measured voltage should not be too close to the boundary of the ADC’s range. Otherwise, the ADC might saturate and the measured signal would be distorted. Following this objective, the voltage divider would need to be chosen so that the measured voltage is centered around the center of the ADC’s range.
The question now was how to choose \(R_{fixed}\). I chatted with a language model (GPT 4) to quickly gain an overview (the chat). The following I understood from that. The objective to maximize the voltage range could be expressed as \[\Delta V_{out} = V_{CC} \cdot \left( \frac{R_{max}}{R_{fixed} + R_{max}} - \frac{R_{min}}{R_{fixed} + R_{max}} \right).\] where \(R_{max}\) and \(R_{min}\) are the boundaries of \(R_{var}\). The term \[R_{fixed} = \sqrt{R_{max}\cdot R_{min}}\] is a rule of thumb for the estimated maximum. Due to time constraints, it was only verified graphically below. The \(R_{fixed}\) value to center \(V_{out}\) can be calculated by calculating the voltage divider factor to be \(0.5\), if \(R_{var}\) is centered. Setting \(R_{var}\) to the center of the resistance range, i.e. \(R_{var} = 0.5\Delta R + R_{min}\) with \(\Delta R = R_{max} - R_{min}\) leads to \[V_{out} = V_{CC} \frac{0.5\Delta R + R_{min}}{R_{fixed} + 0.5\Delta R + R_{min}}\]. Requiring the factor term of resistances to be \(0.5\) leads to \[R_{fixed} = 0.5(R_{max} + R_{min}).\] This presents two options to choose the fixed resistance. Below, these are visualized.

Since the exact resistance range of the velostat sensor is dependent on the size of the velostat strip, the exact position of the conductor contacts, etc. and can therefore vary from piece to piece. Therefore, I would choose to use the second proposed estimate, the arithmetic mean of the range boundaries.
Attach Cables to Fabric


WS2812 LEDs
In the output-devices week (10), I designed a PCB for integrating WS2812B LEDs and set them up to indicate pitch and roll from the gyroscope. The idea was to mount the LEDs onto the glove later on.
Vibration Motor
To have some haptic feedback, I researched on how to integrate a vibration motor in a circuit. Details can be read in the output-devices week (10) page.

Richard gave me the hint: there are different types of vibration motors, coin-style and linear vibration. The problem with coin-style vibration motors is that they have a longer step response. In the entertainment industry where ibration motors are used for feedback they use linear motors.
Digital Files
- Code
- code.zip The dongle sub-project is an ESP-IDF project and the remote sub-project is a PlatformIO project.
- PCB designs (KiCAD)
- CAD (FreeCAD),
.step
, and.dxf
files of all 2D and 3D designs (except circuit and PCB designs)