Week 17: Applications & Implications; Project Development

Table of Contents

This week, we were to officially propose our final project, develop an initial project plan and a bill of materials (BOM). The final project documentation together with the final BOM can be seen at the final project page.

This Week’s Tasks

Applications & Implications

Propose a final project masterpiece that integrates the range of units covered.
- Your project should incorporate:
  - 2D and 3D design
  - Additive and subtractive fabrication processes
  - Electronics design and production
  - Embedded microcontroller interfacing and programming
  - System integration and packaging.
- See Final Project Requirements for a complete list of requirements you must fulfill.
- Where possible, you should make rather than buy the parts of your project. Projects can be separate or joint, but need to show individual mastery of the skills, and be independently operable.
Create a BOM (Bill of Materials).
Prepare drafts of your final project summary slide & video, put them in your root directory, check that they are linked in the final presentation schedule
- (presentation.png, 1920x1080) and video clip
- (presentation.mp4, 1080p HTML5, < ~minute, < ~25 MB)

Project Development

Complete your final project tracking your progress.

Proposing the Project

My final project will be a glove where bend and pressure sensors are attached to. Those are read by an ESP32S3 to send MIDI signals to a personal computer via USB.

The final proposal and process documentation of the final project together with the BOM can be seen at the final project page. This page contains the first draft of a detailed project description and a BOM.

Links to the presentation slides and the video:

Explaining the Project

To develop an initial project plan, we have been given the following questions to answer. These are the questions as they have been answered in this week. The most recent list of questions is listed at the final project page

What will it do?
- It will read values of sensors attached to the fingers and send Note-On and, if possible, CC signals to a personal computer.
- Inputs:
  - pitch, roll (CC messages)
  - Acceleration peaks (Note-On)
  - Finger bend pinky, middle, ring (Note-On) (if possible)
  - If possible, finger drum pinky, middle, ring (Note-On) (if possible)
- Outputs:
  - LEDs on back of hand for pitch, roll
  - MIDI output
  - LEDs for each finger when a note-on is received (if possible)
  - Vibration Motor (if possible)
Who has done what beforehand?
- For a full list of prior projects, check the corresponding section at the final project page.
- The most important prior project are the MiMU Gloves used e.g. by the artist Imogen Heap. Essentially, my final project will be pretty much the same as this, but with less input options. The main difference between my final project and the MiMU Gloves is that the MiMU Gloves depend on a proprietary software called Glover that is to run on a personal computer. This software takes raw measurement values from the gloves, processes them and then sends MIDI or OSC signals to other processes. My final project, however, I plan to implement directly as a MIDI device, so there is no additional software needed other the one receiving the MIDI signals (e.g. the digital audio workstation).

What will you design?
- Electronics: one or multiple PCBs integrating velostat bend sensors as an input and LEDs, and motors as an output.
- Glove: the electronics are mounted onto a glove-like piece to attach them to a hand. I will design this piece. If possible, the PCB is embroidered onto that glove.
- Enclosing: I will design an enclosing for the IMU and the MCU to be mounted onto the wrist as a part of the glove.
- USB dongle Enclosing: To be able to operate the glove manually, one ESP32 is mounted onto the glove and one is plugged into the personal computer. They will communicate via ESP-NOW to transmit MIDI signals. For the MCU that is plugged into the computer, I want to make an enclosing. Adding the feature of wireless communication is optional.
- Packaging: I plan to design a packaging for the gloves to be able to transport them without damaging them.
What materials and components will be used?
- See BOM.
Where will they come from?
- See BOM.
How much will they cost?
- See BOM.
What parts and systems will be made?
- See “What will you design”.
What processes will be used?
- Laser cutting and sewing (maybe embroidery) for producing the glove piece.
- CNC milling and soldering for producing the PCBs
- 3D printing for making the enclosings. They are gonna be spray painted for a nice surface finish.
What questions need to be answered?
- On which inputs should the vibration motor be programmed to be feedback?
- How is the cut pattern for cutting the glove look and how should it be attached to a hand?
- How is the PCB integrated with the glove?
- How is the IMU integrated with the ESP-IDF?
- How to built a sensor for detecting pressure at the fingertips using velostat? Is this even possible?
- How are the sensor data processed? (Acceleration peaks, noisy bend sensor signals)
- How are the bend sensors calibrated?
How will it be evaluated?
- Sensor values (of any kind) are able to be sent to a personal computer as MIDI signals.
- The product is attachable to a wrist or a hand.
- Bonus:
  - The specific sensors to be attached are bend sensors, pressure sensors and an IMU.
  - Messages are sent wireless.
What tasks have been completed?
- Testing the gyroscope.
- Building bend sensor prototype & research on voltage dividers for resistance measurement.
- Built addressable LED matrix.
- Implemented first dummy program using ESP-NOW.
- Learned embroidery.
What tasks remain?
- Verifying, all sensors and actuators can be integrated in the code as desired.
- Make tests to transfer from FR1 PCBs to hybrid PCBs (sewn + FR1).
- Design and fabricate the glove.
- program everything.
- For a detailed list, see schedule.
What has worked? What hasn’t?
- As of now, the status of the project is not advanced enough to answer this question. Refer to the final project documentation for a more detailed description.
What will happen when?
- See schedule.
What have you learned?
- During assembling this list, I looked more into the MiMU glove. From that, I got inspiration for my own project.

Schedule

Week 18

Spiral 1

Write script for video (26.05.)
Design & mill PCB (26.05.)
- Hand PCB
  - ESP32, antenna
  - Accelerometer
  - 4 WS2812 LEDs
  - Battery
Program (26.05., 27.05., 29.05.)
- Read roll, pitch values from Gyro/Accelerometer
- Send values to dongle
- Dongle receives values & sends as MIDI CC message
- Make LEDs blink accordingly
Design & print enclosings (29.05., 30.05.):
- Hand PCB
  - Attachable to wrist using laser-cut polyester & klett
- Dongle

Spiral 2 (30.05., 31.05.)

Add bend sensors for index, middle, ring fingers
- Fabricate hand overlay where bend sensors are to be mounted.
- Add conductive thread + connect it to the Main PCB
- Update main pcb to have the connections + to have the voltage divider
- Program as buttons

Week 19

Spiral 3 (02.05.)

Add vibration motor

Spiral 4 (02.05.)

Accelerometer
- Implement note-on event on acceleration peak (regardless of direction)
- Calculate position and send as CC message
Bend sensors
- Implement as a variable resistor.

Spiral 5 (02.05., 03.05.)

Add one LED for each finger to light up according to activation of a finger.

Initial BOM

Please note that the followin BOM is just an initial version and lacks a lot of detail. For the final BOM, check the final project page.

Item	Quantity	Where from
Velcro tape
Elastic fabric		dm
Polyester sheets
Sewing thread
Conductive thread
Velostat
WS2812 LEDs
Resistors
Adafruit BNO08x board
ESP32 S3 incl. antenna	2
USB C to USB A cable

Use of Language Models

During writing this report, I did not use any language models.