Final Project: Patchlet

Table of Contents

This page contains the development process of the final project throughout the Fab Academy as well as the final result including the bill of materials.

Final Project Requirements

The final project is meant to show what I learned during the Fab Academy course. It must incorporate:

  • 2D and 3D design,
  • additive and subtractive fabrication processes,
  • electronics design and production,
  • embedded microcontroller design, interfacing and programming,
  • and system integration and packaging.

This page must contain answers to the following questions

  • What does it do?
  • Who’s done what beforehand?
  • What did you design?
  • What sources did you use?
  • What materials and components were used?
  • Where did they come from?
  • How much did they cost?
  • What parts and systems were made?
  • What processes were used?
  • What questions were answered?
  • What worked? What didn’t?
  • How was it evaluated?
  • What are the implications?

The Patchlet

My final idea was to have an interface in form of gloves that include several sensors (accelerometers, vector magnetometer, etc.) to send control signals to a music or light component, e.g. via OSC or MIDI. People using that might be musicians, performers, or light operators. Later I settled to have only one glove. I thought that would be sufficient for this project. That way, the operator would still have one hand free.

After a couple of weeks, I got a more detailed picture of which type of inputs I wanted. I thought of a gyroscope for detecting pitch and roll of the hand and pressure (plus maybe bend) sensors at some fingertips. That would enable the user to use finger drumming to send MIDI signals, thus playing music.

Alternative Ideas

Alternative ideas are listed at the final project proposal page.

BOM

Item Price/Unit Quantity Where from
Velcro tape - - -
Elastic fabric - - dm
Polyester sheets - - -
Sewing thread - - -
Conductive thread - - -
Velostat - - -
WS2812 LEDs - - -
Resistors - - -
Adafruit BNO08x board - - -
ESP32 S3 incl. antenna - 2 -
USB C to USB A cable - - -

Requirements of This Specific Project

Syntax: (m) “minimum” (s) “should” (c) “could”

circuitry:

  • (m) attachable to the right hand (e.g. by attaching each PCB to a glove, which would be bought)
  • (m) MCU: ESP32S3
  • (s) flexible, embroidered PCB

Inputs:

  • (m) gyroscope (BNO085): roll, pitch
  • (m) velostat bend sensors at fingers
  • (s) velostat pressure sensors at fingertips to detect “drums”
  • (c) acceleration peaks
  • (c) position (integrating acceleration) of each single hand
  • (c) relative position of hands
    • additional glove with magnet + vector magnetometer on the one with the pcb

Outputs:

  • (m) 4 ws2812b leds attached to back of hand to indicate pitch, roll
  • (s) 1 ws2812b led attached to index, middle, ring fingers to indicate drums
    • the leds are on separate pcbs and mounted to the glove using snaps as well as the main pcb
  • (s) vibration motor: vibrates with specific rotation of hand.
    • could be inside of some small 3d printed case at the inside of the hand to transfer the vibration correctly. the case could be wrapped in fabric for soft feel.
  • (c) oled led for… something
  • (c) optional direct audio ouput from on-board synthesizer

Networking:

  • (m) MIDI-USB interface for talking with computer
  • (s) wirless
    • espnow
  • (c) OSC mode: can be switched via switch. send osc messages to broadcast via wifi.

Enclosing/Integration (alternatives)

  • (m) PCBs sewn onto the glove
    • pattern embroidered on top of conductive thread for enclosing
    • main pcb in a (nice-looking) box on wrist
    • usb dongle in a nice-looking box as well.

Prior Work & Literature

Most relevant

Other cool stuff

Input & Output Devices

Gyroscope (BNO08x)

In week 9, I set up a gyroscope for reading pitch and roll of the glove.

Bend Sensors (Not Used)

building a voltage divider to measure the resistance: https://www.instructables.com/Voltage-Measurement-Using-Arduino/

  • llm dialog regarding measuring resistance and dimensioning a voltage divider: here. to verify it i checked the argumentation. was reasonable.
  • The voltage above the variable resistance would then be according to the voltage divider rule \[V_{out} = V_{CC} \frac{R_{var}}{R_{fixed} + R_{var}}\]
  • to maximize the range of voltage over the resistance of interest, find the \(R_{fixed}\) that maximizes the following term \[\Delta V_{out} = V_{CC} \cdot \left( \frac{R_{max}}{R_{fixed} + R_{max}} - \frac{R_{min}}{R_{fixed} + R_{max}} \right)\]
  • The following term is a rule of thumb for that estimate. For the sake of time, it only was verified grafically below. \[R_{fixed} = \sqrt{R_{max}\cdot R_{min}}\]
  • the \(R_{fixed}\) to center the range of \(V_{out}\) can be calculated by setting \(R_{var}\) to the center of the resistance range, i.e. \(R_{var} = 0.5\Delta R + R_{min}\): \[V_{out} = V_{CC} \frac{0.5\Delta R + R_{min}}{R_{fixed} + 0.5\Delta R + R_{min}} \] with \[\Delta R = R_{max} - R_{min}\]
  • Requiring that the term containing the resistances to be \(0.5\) leads to \[R_{fixed} = 0.5(R_{max} + R_{min})\].

Visualization of the above equations using GeoGebra.

since for now i do not really know the exact resistance, i will go for the option that maximizes range and have rough estimates of upper and lower resistance values.

WS2812 LEDs (Not Used)

In week 10, I designed a PCB for integrating WS2812B LEDs and set them up to indicate pitch and roll from the gyroscope. Furthermore, I researched on how to integrate a vibration motor in a circuit.

Vibration Motor (Not Used)

Schematic for integrating a DC motor.

Programming

In week 11, I set up the ESP32S3 to send MIDI signals to the laptop via USB-MIDI. Together with my fellow student Niclas, I set up two ESP32S3 boards to communicate with each other with the goal of having a wireless MIDI controller eventually.

PCB Fabrication

Dongle

…

Remote

Power

TP4056 power management board https://electronicshacks.com/how-to-use-the-tp4056/ https://wiki.seeedstudio.com/xiao_esp32s3_getting_started/#battery-usage

  • nicht unbeaufsichtigt laden!

Problems

Housing

System Integration

  • more than a box

  • PCB’s are mounted

  • wires are

    • soldered or connected with proper headers and/or connectors and
    • neatly laid out and every part has it’s dedicated place

  • surface finish

    • spray paint. then polish spreypaint. then put transparent spraypaint on it.
    • epoxyspachtel, grinding, epoxying, grinding

ideas for later

  • build sensors with fabric + sew them instead of tape
  • ring rigging mechanism would stay the same
  • connect things by sewing instead of pressing
  • build sensors by sewing yarn.
  • make diameter of rings adjustable with screws
  • attach neopixels to rings for indicating that sensor was pressed.
  • embroid complete pcb

Flexible PCBs

Attaching PCB

Rubber Band

Velcro Tape

https://www.youtube.com/watch?v=lAn-bxHsSt8

  • very robust needles

Rings

Snap Fastening

  • soldering snap buttons works
  • now create footprint for snap pins:
    • want to have direction (i.e. it woudl not be possible to attach diode in the wrong way) so i wanted to have 6mm snaps for one side and 7mm for the other one.
  • following this: https://www.pcbway.com/blog/PCB_Design_Tutorial/How_to_make_a_footprint_in_KiCad_.html
  • creating a new symbol (for schematics)
  • reference designator: j or p (bc we are plugging)
  • number of units per package: how many are in one pocket of the roll (relevant for toolchanger). 1. for me not relevant.
  • rest default
  • renamed to “…_male”
  • close editor. open footprint editor
  • new footprint
  • set name
  • create pad (the most important part)
  • the pad should be round of diameter of connector + via to connect to the other side.
  • choose largest via possible (easier to handle)
  • measure -> outer diameter of 1mm via -> 1.3mm
  • as hole choose 1.4mm

Evaluation

Implications

  • sell it or my services to build it or to teach building it in some form to professional dancers or art institutes working with multi-media art. they could use this for performances and for having immersive performances incorporating e.g. sound and visuals based on dances.

Quotes

(Jakob is beat boxing), Richard: “Look, you do not even need a glove to annoy people.”

video

3 mins

  • video
    • motivation
    • lustig sein, zeigen wies geht
    • zeigen welche techniken man eingesetzt hat und so
  • show 1 min video
    • thanks to
      • fablab
      • ilmkubator
      • tu ilmenau
      • stiftung innovation in der hochschullehre
      • examing
      • carl-zeiss
      • frank hirschvogel stiftung
  • 2 minutes left
    • lerneffekt ggüber eines normalen semesters

audience?

  • 130 gäste

  • leute von wirtschaftsförderer

  • oberbürgermeister

  • post lesen

  • was ich gelernt hab, hard skills

  • wie es im vergleich zu einem typischen semester war, schlechter besse

  • warum ich mir wünschen würde das hier nochmal zu machen

  • wie das trockene wissen in mir kreativität ausgelöst hat

  • Wie mir aus studium geholfen hat. Auf die seite der andere leute stellen

  • vorher: irgendwelche projekte neben dem studium hinimprovisiert, vielleicht git repo mit readme…

    • jetzt: website mit screenshots. wie man das effizient macht.

  • vorher: mit breadboards irgendwas was nicht funktioniert, wenig gefühl für elektronik

    • jetzt: schaltung designen in kicad, layouten, fräsen, löten.

  • 3d modellierung (hand, holzinstallation)

Iteration 1

code

components/esp32_BNO08x/etl/type_traits.h:72:1: error: unknown type name 'namespace'
  • say that you need cpp compiler
    • in root CMakeLists.txt: change project(espnow_example) project(espnow_example C CXX)
    • did not work
  • change every file ending to cpp and hpp
  • also change
idf_component_register(SRCS "main.c"
                INCLUDE_DIRS ".")

to

idf_component_register(SRCS "main.cpp"
                INCLUDE_DIRS ".")

in CMakeLists.txt in main/ because there is no main.cpp anymore, we renamed it.

  • compiled, worked.
  • using minicom to read serial out of the xiao revealed, it worked.
  • first send midi directly from wristband based on gyroscope values
  • afterwards separate and combine with dongle
  • combine accelerometer input from input devices week with espnow code developed in networking week to send stuff to dongle
  • setting CONFIG_TINYUSB_MIDI_COUNT=1 using
    • idf.py menuconfig -> component config -> tinyusb stack -> MIDI
  • i had a bug where the imu could not be configured properly anymore. had somethign to do with the gpios. i fddled aroun dfor one day and somehow it worked, i do not have an idea what id id
  • deciding for quaternion vs euler
  • bug continued to be there. some pins (6-11) cannot be used as they are reserved for a optional PSRAM module. switching to esp-wroom 32 i had lying around in some boxes.
  • referencing to this pinout
  • doesnt work. use this code for arduino ide. the arduino bno library has i2cc functionality. this way i have more pins left to talk to the sensors. https://developer.espressif.com/blog/arduino-esp-now-lib/
  • i2c with the sensor is said to be erratic. however, i did not have problems with it. also for the final project it should be enough.
  • using platformio.
  • for setting up timers using arduino using this reference https://github.com/cygig/TimerEvent/blob/master/examples/ThreeTimers/ThreeTimers.ino
  • event-based, because easier to keep track of everything than polling-based.
  • mac addresses
    • dongle: 64:e8:33:51:30:48
    • wrist: 64:e8:33:50:bc:70
  • using the one-way example from this and adapting it https://randomnerdtutorials.com/esp-now-esp32-arduino-ide/
  • transferred the receiver side to espnow
  • change espnow library for arduino to quickespnow. since it is one of two options for platformio and has 10 times more downloads than the alternative “easyespnow”.
  • take this as an example https://github.com/gmag11/QuickESPNow/blob/main/examples/basicespnow/basicespnow.cpp
  • learned about packed structs
  • define packet structure in separate header file that i symlinked between stuff
  • a bit hard to deal with since two knobs are turned at a time. makes it hard to map
  • for proof of concept i changed the roll to a switch: when sensor is turned down (palm is up), note-on signal. when turned back, note off.
  • insert code of midi handling
  • problem with the code: have to be fast enough. if you move slow, the change does not get detected. can be changed by simplifying the code.
  • i also introduced a threshold for the pitch value being sent at all: 30. this way i could turn it on and off so to say. this is relevant for mapping. eg when you want to map something you have to be able to move only one control at a time.
  • refer to here for mapping controllers in ardour https://manual.ardour.org/using-control-surfaces/generic-midi/midi-learn/

surface finish

  • härter zu spachtelmasse hinzupacken
  • auf druckteil schmieren
  • ne stunde warten und überhänge entfernen
  • 24 stunden warten
  • 120er schleifen (nicht zu grob, wie 80 oder so! sonst macht man sich rirtze rein die man nicht so aufbekommt.)
  • 500er schleifen
  • lackieren
  • über farbe selbst nochmal schleifen (500er)
  • nochmal lackieren, 3h warten
  • dann nochmal schleifen
  • nochmal lackieren

designed pcb

  • xiao on it
  • accel on it
  • leds on it
  • Problems & whatifound:
    • no voltage at 5v pin since battery-powered -> do leds still work?
      • yes, leds still work
    • soldering mistake: data line of leds was connected to ground.
    • by rigging the accelerometer not to the hand but to the arm, one could measure the arm movement which allows for much more expressive control without the need to calculate position in the code.

enclosing

  • first (ask ferdi how to design it so that it is possible to adjust it easily)
    • Hand PCB
      • Attachable to wrist using laser-cut polyester & klett
      • have: not only lid for end but also for top. this way one can add more slots for PCBs.
    • Dongle
      • separate pcb for second xiao for mounting it into a case… maybe even solder it…
    • whatilearned
      • freecad
        • split: cutting body using ebene, extruding form
        • dicke hinzufügen: objekt designen und dann kann man ne hülle darum bauen.
        • i learned: if you want to slice something in part, it is not parametric. rather just make a copy of a piece and a through pocket
      • with sliding mechanism one does not even need screws
      • a guide for the pcb is not necessary if te housing is chosen appropriately.
      • to improve the sliding mechanism, one can just apply dry soap to it.
      • when sliding something to cover a pcb, make sure it does not touch the pcb
      • tolerances (benedikt):
        • 0.25mm for slides
        • 0.25mm for sides of pcb
        • 0mm tolerance for height and radius
  • new design (1h) (19.00 - 20.00)
    • tolerances
    • radius to hold pcb in place
    • lichtkanten
      • halbe wandstärke dicke
      • ragt heraus 1mm
      • keine toleranz
    • something to put polyester band into
    • whatilearned
      • i wanted to create a completely hollow object without removing a face. i did this by cutting it in half, applying thickness, mirroring it.
      • during design: decision; just put pcb into box
      • light edges
      • if you want to apply the same sketch to somewhere else, use a shape binder
  • new new design
    • adding guides for top and bottom
    • add snapfit of 1mm durchmesser + hole at the other end.
    • decrease clearance of thickness of pcb half a millimeter
    • printed
    • tested by sliding on and off many times
    • whatilearned
      • print the top so that the positive of the snapfit is up. this way it does not need to be held by the support which might lead to a nicer finish.
      • the snapfit is not enough. the positive gets rubbed away. i will improve it by adding a clearance to the negative + by having velcro tape to fix the top and bottom together.
      • print from petg

surface finish

  • lack + überlack
  • alternatives from got
  • whatilearned
    • have nozzle 15 to 20 cm away from piece to not put too much lack in one place
    • wait 15 minutes for one layer to dry
    • multiple layers. be careful.
    • the 3d printed layers are still seen. what you would need to do: you coat, then grind, then coat, then grind etc. with applying bondo in between (spachtelmasse) it goes a bit faster.

velcro tape

  • adding with double sided klebeband bc epoxy does not hold on 3d prints and fast glue is not so nice for velcro tape
  • tried out different tapes. one worked
  • dongle
    • add copper plane to have same thickness clearance for guide as for other pcb

TODO

Spiral 2 (Tuesday)

  • design wristband + fixation band (Monday)
  • sew wristband
    • get sewing stuff from library
    • sew wristband
  • test lacks for surface finish + grinding + bondo
  • dongle
    • pcb design + mill
    • housing
  • update pcb
    • mount on/off switch
    • add power led to see if xiao is on. or if it needs to be charged.
    • add led that is on when initialization is finished.
    • remove neopixel leds
    • include connectors for finger bend sensors
    • include charger board (tp4056) for battery. it has
      • over discharge protection
      • overcurrent protection
      • overcharge protection
      • short circuit protection
  • make pcbs and housing more flat
    • for battery: wände hinzufügen um akku in place zu halten. wird unten abgeschlossen durch pcb selbst.

Evening

  • create dummy for video - 23.30
    • use footage + create storyboard

Spiral 3 (Wednesday)

  • housing
    • include slots for having finger bend sensors
    • think of how to connect finger bend sensors & pcb
  • make dehnungssensoren, design glove
  • laser & sew everything
  • Add bend sensors for index, middle, ring fingers
    • add slot for cables to come out at lid.
    • connect cables that come out of lid using snap fasteners that are mounted on sewn pcbs
    • check lillypad
    • Fabricate hand overlay where bend sensors are to be mounted.
    • Add conductive thread + connect it to the Main PCB
    • Update main pcb to have the connections + to have the voltage divider
    • Program as buttons
  • create bom
    • kicad
    • liste

spiral 4 (Thursday)

  • add (just the accelerometer) a second time. (document, but not in video)
  • check focussing more on docu from now

Spiral 3

  • Add one LED for each finger to light up according to activation of a finger.
  • add led matrix for back of hand showing values of gyroscope

Spiral 4

  • use pcb switch & have some 3d print to make switch accessible from outside.

during production

  • take videos & screen recordings of production process

Wrapping up

  • replace uuid at with chip number in code
  • morph notes into text/well-formulated bullet points at corresponding places

do later (never)

  • Add button to deactivate bend sensors in case they get wet.
  • replace imu with invensense icm 20948 9-dof (its cheaper).
  • Add vibration motor
  • glue some fabric below the thing to make it more soft
  • add z axis (yaw), make configurable with pressing a button
  • add functionality to easily turn on and off the controller input (persisting the state, not sending midi data or sending the last midi data offered.)
  • add “mitnahme”, when paused, have persistant value being sent until the finger gets to that value again.
  • battery
    • temperature monitoring
    • bms module (independent hardware protection. i did not understand what it does)
  • add config file for configuring which input sends what output.
    • maybe have a “config mode” where dongle and glove each run a webserver for configuration.
  • add functionality to be able to move only one dimension at once.
  • Accelerometer
    • Implement note-on event on acceleration peak (regardless of direction)
    • Calculate position and send as CC message
  • piezos for fingertips
  • optional laser tof sensor to measure distance of palm from boden. only turned on if hand is waagerecht. then this can also be used as a control thing.
  • open up osc stream to network via wifi

during FP presentations

  • add image of integrated system to system integration
  • check back on kostenrückerstattung von amazonbestellungen
  • check back on drawback group page, edit if necessary
  • tidy up structure assuming every page is the first page, one lands on.
    • move bootcamp into recitations
    • tidy up tips & tricks
    • final project
    • click through the site and refine everything that seems odd.
    • clean up ignore folders and archive
    • check that pcb fab settings are set correctly
    • add documentation of pcb production with
      • binarization script
      • vacuum pump (0.3mm depth clearance)
  • move existing documentation
    • to electronics design
      • from vector magnetometer:
        • do not block pins that are useful later
      • from led
        • add ground plane
        • design footprint
        • assign existing footprint to schematic
        • do not make pads too thin. they might tear off
    • to electronics production
      • milling flat pcb stock
      • vector magnetometer production
        • what can go wrong
        • how to older smd
  • refine orthography
    • fix typos & grammar mistakes
    • check that names are spelled correctly (Adriàn)
    • write e.g. and i.e. in italics
    • fix page-internal links linking to the “.md” file
    • words
      • tool paths
      • pin-out
    • check tense

not pressing (do after fabacademy)

  • cnc
    • cnc schleifen
    • lackieren
  • website for personal projects

later later

  • junky story aber mit fab. fabster der abhängig war von fabrication. immer wenn er nahe eines fablabs kommt wird er rückfällig “nur kurz ein pcb fräsen”. alle möglichen analogien zu glücksspiel, mafia etc. machen.

Names

  • patchlet
  • basslet
  • wavelet
  • GLOMM
  • Handler