Week 17: Applications and Implications: Project Development
Assignment
Plan a final project masterpiece that integrates the range of units covered, answering:
- What will it do?
- Who's done what beforehand?
- What sources will you use?
- What will you design?
- What materials and components will be used?
- Where will they come from?
- How much will they cost?
- What parts and systems will be made?
- What processes will be used?
- What questions need to be answered?
- How will it be evaluated?
Your project should incorporate:
- 2D and 3D design
- Additive and subtractive fabrication processes
- Electronics design and production
- Embedded microcontroller design, interfacing, and programming
- System integration and packaging
Where possible, you should make rather than buy the parts of your project.
Projects can be separate or joint, but need to show individual mastery of the skills and be independently operable.
Q & A
What will it do?
Vision Voice is a wearable glove that translates American Sign Language (ASL) gestures into audible speech and visible text in real time. The idea is simple: most people don't know sign language, so communicating with someone who is Deaf can be really difficult. Vision Voice bridges that gap by letting the user sign naturally with their hand, and the glove handles the translation on its own without needing a phone, laptop, or internet connection.
The glove uses five flex sensors (one on each finger) to detect the shape of the hand. These five values get compared against stored gesture templates using weighted Euclidean distance matching to identify which ASL gesture is being signed. Once recognized, the gesture is shown on the OLED display and spoken out loud through the DFPlayer Mini and speaker.
Originally I planned to use a kNN (k-Nearest Neighbors) machine learning algorithm trained on PC using Python and scikit-learn, then export the model and flash it onto the XIAO so the glove runs fully standalone. The plan was solid but time ran out before I could get it working properly, so I switched to on-device template matching instead which turned out to work really well. Maybe kNN is something I can explore in a future version. ^_^
The goal is to recognize 15 ASL gestures reliably in a standalone wearable device. :D
Who's done what beforehand?
A few similar projects have been done before and they really helped shape my thinking for this one.
SignAloud (University of Washington) is probably the most well known. They built gloves with flex sensors that won the Lemelson-MIT Student Prize, but they required a computer to process the data which is a big limitation for real-world use.
Sign-IO by Roy Allela is a smart glove that connects to a mobile app via Bluetooth. Really cool project but it still depends on your phone being nearby.
Inside Fab Academy, students have made data gloves for music and even a boxing glove. Those inspired me too.
What makes Vision Voice different from all of these is that everything runs on the wrist. The XIAO ESP32-C3 handles all the gesture recognition locally with no phone, no laptop, and no WiFi needed. You just wear the glove, capture your gesture templates directly on the device, and it works anywhere. ^_^
What sources will you use?
- Adafruit libraries (ADS1X15, SSD1306, GFX)
- DFRobotDFPlayerMini library
- Arduino IDE with ESP32 board support
- KiCad for PCB design
- Fusion 360 for enclosure design
- Prusa Slicer for 3D printing
- Roland SRM-20 and Mods CE for PCB milling
- Laser cutter for the enclosure lid
- Wokwi for early simulation and testing
What will you design?
I used Fusion 360 to design the wrist enclosure that holds all the electronics. The base is 3D printed and the lid was laser cut from acrylic. I also designed and milled a custom PCB in KiCad that connects the XIAO ESP32-C3 to all the modules. On top of that I sewed flex sensor slots directly into the glove fabric and made a Velcro adjustable wristband to hold the enclosure in place.
What materials and components will be used?
| Component | Function |
|---|---|
| XIAO ESP32-C3 | The brain. Runs the gesture matching and controls everything. |
| Flex Sensor 4.5" (x5) | One per finger. Detects how much each finger is bent. |
| ADS1115 ADC Module (x2) | Reads all 5 flex sensors over I2C (0x48 and 0x49). |
| OLED Display 0.96" SSD1306 | Shows the recognized gesture as text. |
| DFPlayer Mini | Plays the MP3 audio for the recognized gesture. |
| Speaker 3W 4Ω | Outputs the spoken word. |
| LiPo Battery 3.7V | Powers the whole system. |
| MicroSD Card | Stores the MP3 audio files for the DFPlayer. |
| Push Button (x2) | One to cycle letters in capture mode, one to save templates and trigger audio. |
| Slide Switch | On/Off switch. |
| Resistors | For DFPlayer RX protection and flex sensor voltage divider circuits. |
| 100nF Ceramic Capacitors (x2) | ADC noise filter on the onboard analog pins. |
| Velcro Strap | Makes the wristband adjustable. |
| Cloth Glove | The base that holds all the flex sensors. |
| PLA Filament | For 3D printing the enclosure base. |
| Acrylic Sheet | For laser cutting the enclosure lid. |
| Copper Clad Board | For milling the custom PCB. |
Where will they come from?
Most of the electronic components came from the Fab Lab inventory. Some parts like the ADS1115 modules and DFPlayer Mini were ordered online. The glove itself is a regular cloth glove from a local shop. The PLA filament, acrylic, and copper clad board all came from the lab.
How much will they cost?
| Component | Qty (Required) | Unit Price (Nu.) | Total (Nu.) | Purpose |
|---|---|---|---|---|
| XIAO ESP32-C3 | 1 | 534 | 534 | Main microcontroller. Controls all sensors, runs gesture recognition, drives the OLED and DFPlayer. LiPo charging built in. |
| Flex Sensor 4.5" | 5 | 600 | 3,000 | One per finger. Detects how much each finger bends. Core sensor for reading ASL hand shapes. |
| ADS1115 16-bit ADC Module | 1 | 250 | 250 | ADC extender since the ESP32-C3 only has 4 analog pins. Adds more channels via I2C with 16-bit resolution. |
| OLED Display 0.96" I2C | 1 | 150 | 150 | Shows the recognized ASL gesture as text. 128x64 px on the shared I2C bus. |
| DFPlayer Mini | 1 | 585 | 585 | Plays pre-recorded MP3 files from the MicroSD card. Voices the translated gesture aloud through the speaker. |
| Speaker 3W 4Ω (40mm) | 1 | 80 | 80 | Sound output. Wired directly to DFPlayer Mini SPK1 and SPK2 pins. |
| Tactile Push Button 6x6mm | 2 | 5 | 10 | Two buttons: one to cycle through letters in capture mode, one to save templates and trigger audio manually. |
| LiPo Battery 3.7V 1000mAh | 1 | 350 | 350 | Main power source. ~6 to 10 hours of use. Also powers DFPlayer directly. |
| Slide Switch SPDT | 1 | 20 | 20 | On/Off switch. Cuts all power when off to preserve battery. |
| Resistors (10kΩ x5, 1kΩ x1) | 1 pack | 12 | 12 | 10kΩ for flex sensor voltage dividers, 1kΩ for DFPlayer RX noise filter. |
| 100nF Ceramic Capacitors | 2 | 3 | 6 | ADC noise filter on the onboard analog pins. |
| MicroSD Card 8GB | 1 | 200 | 200 | Stores the MP3 audio files for all ASL gestures in /mp3 folder, FAT32 formatted. |
| Cloth Glove | 1 | 110 | 110 | Houses the flex sensors. Needs to be sewable, breathable, and tight fit. |
| Velcro Strap | 1 | 50 | 50 | Holds the electronics casing securely to the wrist. Allows easy on and off. |
| Total | Nu. 5,357 |
What parts and systems will be made?
Glove: The cloth glove with flex sensor slots sewn into each finger channel and a Velcro adjustable wristband attached.
PCB: A custom single-layer PCB designed in KiCad and milled on the Roland SRM-20. It connects the XIAO ESP32-C3 to the ADS1115 modules, OLED, DFPlayer, buttons, and slide switch in a compact layout.
Enclosure: A 3D printed base designed in Fusion 360 with screw mounts for the PCB, a slot for the USB-C cable, and a hole for the SD card. The lid is laser cut from acrylic with cutouts for the OLED and speaker.
Firmware: The gesture recognition system runs entirely on the XIAO ESP32-C3 with no PC or phone involved. My original plan was to train a kNN model in Python on my PC and export the weights onto the chip, but I ran out of time to implement that properly. So I switched to on-device template matching using weighted Euclidean distance, which honestly works really well for what I needed. For each gesture, the glove captures 8 templates directly on the device through a built-in capture mode. When recognizing, it reads all 5 flex sensors, normalizes each finger value using per-finger calibration, then finds the closest stored template using weighted distance. A stability vote filter (5 readings, needs 4 to agree) prevents false triggers. Once a gesture is confirmed, the OLED updates and the DFPlayer speaks the word.Everything is stored in flash so templates survive power cycles.
Audio Files: MP3 files for each of the 15 gestures stored on the SD card in the /mp3 folder, named 0001.mp3, 0002.mp3 and so on. Played through the DFPlayer when a gesture is recognized.
What processes will be used?
- 2D design (KiCad schematic, enclosure sketches)
- 3D CAD modelling (Fusion 360)
- 3D printing (Prusa Slicer + FDM printer)
- Laser cutting (enclosure lid in acrylic)
- PCB design (KiCad)
- PCB milling (Roland SRM-20 + Mods CE)
- Soldering (SMD components)
- Sewing (flex sensor slots and wristband)
- Embedded programming (Arduino IDE)
- Electronics testing and debugging
What questions need to be answered?
- How many templates per gesture gives the most reliable recognition?
- How sensitive is recognition when the glove is put on slightly differently each time?
- Can the system tell apart ASL letters that look very similar like A, E, and S?
- What is the right reject threshold so the glove doesn't misfire between gestures?
- How long does the LiPo battery last under normal use?
- How does the system handle hand positions that fall between two gestures?
How will it be evaluated?
The project will be considered successful if:
- The glove correctly recognizes at least 15 ASL gestures
- Recognition accuracy is consistently above 75% in real testing
- The OLED displays the correct gesture text after each sign
- The DFPlayer speaks the correct word for each gesture
- The full system runs on battery without USB
- The glove is comfortable to wear and the wristband stays secure
- Everything works end to end with no phone or computer involved
What tasks have been completed?
- Initial concept and planning
- Wokwi simulation
- Cardboard prototyping
- Single flex sensor testing with XIAO
- OLED integration and testing
- DFPlayer Mini + LiPo audio testing
- ADS1115 module testing with all 5 flex sensors
- Custom PCB design in KiCad
- PCB milling on Roland SRM-20
- PCB soldering
- Enclosure design in Fusion 360
- 3D printing the enclosure base
- Laser cutting the enclosure lid
- Sewing flex sensor slots into the glove
- Making the Velcro adjustable wristband
- First prototype assembly and testing
- Gesture template capture for 15 ASL signs
- Final firmware deployment (v5.1)
- Final assembly and testing
What tasks remain?
- Fine tuning gesture recognition accuracy
- Recording and organizing final MP3 audio files
- Final enclosure finishing
- Shooting the final project video
- Completing all documentation
What has worked? What hasn't?
Worked
- The ADS1115 modules read all 5 flex sensors cleanly over I2C with no conflicts
- The weighted Euclidean distance matching correctly identifies gestures when templates are captured cleanly
- DFPlayer plays audio reliably once wired directly to the LiPo positive (not the XIAO 5V pin)
- The OLED displays gesture text in real time with no flickering
- The stability vote filter prevents false triggers between gestures
- The Velcro wristband holds the enclosure firmly during signing
- Sewing the flex sensor slots into the glove keeps the sensors in the right position
Hasn't Worked
- The first PCB design had routing issues and had to be redesigned (I think I redesigned it like 3 times at this point 😅)
- DFPlayer VCC connected to the XIAO 5V pin caused it to stop responding on battery. It has to go directly to the LiPo positive
- The first enclosure lid was 3D printed but too thin and too loose, so I switched to laser cutting it from acrylic
- The first attempt at the wristband wasn't adjustable so I had to cut it, rethink the design, and sew it again with Velcro
What questions need to be resolved?
- How do I handle two gestures that produce very similar flex patterns?
- What is the best reject threshold so the glove doesn't misfire on in-between hand positions?
- How should the SD card MP3 files be organized for easy updating?
- Can the enclosure be made even more compact in a future version?
What have you learned?
I honestly still can't believe how far this project has come since Week 01 when it was just a rough idea on a Canva board. Building Vision Voice taught me so much, and I mean that in a way I didn't expect at the start of Fab Academy.
The biggest thing I learned is that a project like this is never really just about one skill. It's not just programming, or just PCB design, or just 3D modeling. All of it has to come together at the same time and actually work together as a single system. Getting that right was genuinely the hardest part, and also the most satisfying part when it finally clicked.
I also learned that failing a design is not a bad thing. I redesigned the PCB so many times, recut the enclosure lid, re-sewed the wristband, and spent a whole night debugging why the DFPlayer wouldn't respond on battery. Every single one of those setbacks taught me something I wouldn't have learned any other way.
If I had to rate this whole experience, I'd give it an easy 11/10. ^_^