Week 18 - Applications and Implications
Assignment
This week asks for a final project plan that covers the whole build: what it does, what I designed, what I bought, how much it costs, which Fab processes it uses, what still needs to be answered, and how I will judge the result. This page is the planning version for Brain Fog Insight Companion. The later build and packaging work is documented in Week 16 and the final project page.
What will it do?
Brain Fog Insight Companion is a desktop host plus a wearable band for people who experience brain fog: slow thinking, attention drop, short memory gaps, eye fatigue, and the feeling that work takes more effort than it should. The system watches simple signals that I can sense with maker hardware: blink state from a camera model, heart rate and SpO2 from a wearable sensor, and voice input from the user. It estimates whether the user may be in a foggy or fatigued state, then offers a small intervention such as 40 Hz audio, breathing guidance, or a voice suggestion.
I keep the claim narrow. The project does not diagnose disease. It is a personal feedback and intervention prototype for everyday cognitive fatigue.
How a session works
The user puts the desktop host on the desk and wears the band. After power-on, the camera tower watches eye state, the band sends heart-rate and SpO2 data, and the host screen shows the current state. The user does not need to open a laptop or a debug tool.
If the system sees a pattern that looks like fatigue or brain fog, it can suggest a short intervention. The simplest one is 40 Hz audio from the speaker. The user can also press the left button to start a voice conversation: the microphone records speech, the cloud voice path returns an answer, and the speaker plays it back. After the intervention, the same sensors keep running so the user can see whether their state changed.
| Phase | User sees | System does |
|---|---|---|
| Setup | Desktop host on, wearable nearby or worn. | Boot screen, camera model, BLE bridge, and wearable data path. |
| Monitoring | Screen shows state and measurements. | Counts blink state and receives HR / SpO2 from the band. |
| Prompt | User can press the left button for dialogue. | Microphone records speech and sends it through the voice pipeline. |
| Intervention | Audio, screen guidance, or spoken suggestion. | Runs 40 Hz audio or plays the AI response through the speaker. |
| Review | User checks whether they feel clearer. | Continues sensing so the before/after state can be compared. |
Who has done what beforehand?
I looked at four groups of prior work. HRV research explains why heart-rate timing can reflect stress and fatigue. Eye-blink research connects spontaneous blink rate with sustained attention and fatigue. 40 Hz gamma-stimulation papers gave me the idea for an audio intervention. I also compared consumer products: smart watches, meditation apps, white-noise apps, and neurofeedback headbands. They each cover one part of the problem, but they do not give me the desktop host + wearable + intervention loop I wanted to build.
The gap I am building around is the small closed loop: sense a few simple signals, decide whether the user may need help, intervene, then observe whether the state changes. It is a rough prototype, but it is more useful than a single sensor demo.
References used
I pulled the reference list from my principles and prior-art document, then kept only the sources that directly informed the Fab prototype. The full notes are available here: principles, references, and prior-art document.
| Reference | How I used it |
|---|---|
| Shaffer, F. & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Frontiers in Public Health. | Background for using heart-rate timing and HRV as a fatigue / stress-related signal. |
| Maffei, A. & Angrilli, A. (2018). Spontaneous eye blink rate: An index of dopaminergic component of sustained attention and fatigue. International Journal of Psychophysiology. | Reason to treat blink behavior as a useful attention and fatigue indicator. |
| Iaccarino, H. F., et al. (2016). Gamma frequency entrainment attenuates amyloid load and modifies microglia. Nature. | Background for the 40 Hz stimulation idea. I do not present the Fab prototype as therapy. |
| Martorell, A. J., et al. (2019). Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-Associated Pathology and Improves Cognition. Cell. | Reference for multi-sensory 40 Hz stimulation, which influenced the audio/visual direction. |
| Patent and product searches around brain fog, cognitive fatigue, HRV, blink detection, and 40 Hz stimulation. | Used to compare against single-purpose products such as watches, meditation apps, and neurofeedback headbands. |
What will I design?
I designed the desktop platform, the 3D printed host shell, the camera tower, the front control layout, the host PCB, the local eye-state model path, and the firmware that joins the boards. I also designed the way the wearable belongs to the desktop product: it has a reserved charging position instead of being a loose second object.
Materials, components, sources, and cost
Prices are approximate purchase costs in CNY (June 2026). Shipping and coupons are not included.
| Part | Use | Source | Price |
|---|---|---|---|
| ESP32-S3 WROOM host module | Main MCU, Wi-Fi, voice client, UI logic | Taobao / lab stock | ¥96 |
| 4.3″ touch screen + SPI adapter | Host display and touch input | Taobao | ¥56 |
| Seeed XIAO ESP32S3 Sense | Camera + local eye open/closed model | Seeed | ¥101 |
| Seeed XIAO ESP32-C3 | BLE bridge wearable → host UART | Seeed | ¥34 |
| DFRobot Beetle ESP32-C6 | Wearable controller + LiPo charging | DFRobot | ¥36 |
| DFRobot Gravity MAX30102 (SEN0518) | Wearable HR / SpO₂ | DFRobot | ¥159 |
| INMP441 I2S microphone module | Voice input | Taobao | ¥51 |
| MAX98357A I2S amplifier module | 40 Hz audio + TTS playback | Taobao | ¥43 |
| 8 Ω 3 W speaker | Audio output | Taobao | ¥12 |
| MicroSD module + 16 GB TF card | Audio files and logs | Taobao / lab stock | ¥18 |
| 3.7 V 500 mAh LiPo (JST) | Wearable battery | Taobao | ¥25 |
| Custom host PCB (5 pcs + shipping) | Host wiring carrier | JLCPCB | ¥58 |
| 100 µF electrolytic + passives | Beetle brownout fix, PCB decoupling | Component shop | ¥5 |
| Tact buttons (×2), master switch | Dialogue trigger, host power | Taobao | ¥8 |
| USB Type-C 5 V power module | Rear desk power input | Taobao | ¥10 |
| Charging magnets / contacts | Wearable dock on front platform | Taobao | ¥15 |
| Dupont wires, headers, ribbon | Bench bring-up and internal routing | Lab / Taobao | ¥22 |
| PLA filament (~200 g) | Host shell and fit tests | Chaihuo stock | ¥18 |
| Plywood / acrylic sheet | Laser-cut platform and camera tower | Chaihuo stock | ¥12 |
| Estimated total (prototype qty 1) | ~¥779 | ||
Project schedule
The project did not move in a straight line. I kept the schedule as a build sequence rather than a perfect calendar, because several parts had to be tested before they were worth closing into the platform.
| Stage | Work | Status |
|---|---|---|
| Concept and research | Define brain-fog use case, review HRV, blink-rate, and 40 Hz references. | Done before final build. |
| 2D / 3D design | Design the laser-cut platform, camera tower, and 3D printed host shell. | Done, with fit changes after early prints. |
| Electronics | Prototype on breadboard, then move the WROOM host wiring to the Brainfog PCB. | Done for the integrated prototype. |
| Embedded software | Bring up WROOM UI/audio/voice, XIAO S3 eye-state link, C3 BLE bridge, and Beetle wearable firmware. | Working in test builds. |
| System integration | Install electronics inside the platform, test power, buttons, screen, voice, camera, and wearable flow. | Documented in Week 16. |
| Final presentation | Polish documentation, update source links, then cut the final slide and one-minute video. | Draft files exist; final edit still open. |
Project progress
| Area | Completed | Still open |
|---|---|---|
| Mechanical design | Laser-cut platform, camera tower, 3D host shell, and wearable charging position. | Cleaner removable bottom and better cable strain relief. |
| Electronics | Host PCB, WROOM wiring, microphone, screen, SD, speaker path, XIAO S3 link, C3 bridge, and wearable circuit. | Future PCB revision could reduce jumpers and make service easier. |
| Firmware | WROOM UI and intervention logic, C3 BLE bridge, Beetle BLE advertisement, and XIAO S3 eye-state path. | Longer stability testing across lighting and BLE conditions. |
| AI / interaction | Local eye open / closed model and cloud voice interaction tested. | Advice wording should stay clearly non-medical. |
| Presentation | Draft slide and draft video are already in the website root. | Final edit should show more integrated product footage. |
What parts and systems will be made?
| System | What I made | Status |
|---|---|---|
| Desktop platform | Laser-cut base with internal space, camera tower, controls, Type-C power, and watch charging position. | Built and integrated. |
| Host shell | 3D printed enclosure for screen and speaker. | Built after fit changes. |
| Host PCB | KiCad board replacing the breadboard wiring for the main electronics. | Designed, fabricated, soldered. |
| Wearable band | Band prototype with Beetle ESP32-C6, battery, and heart-rate / SpO2 sensor. | Working prototype. |
| Local eye model | Simple AI model for eye open / closed classification on XIAO ESP32S3 Sense. | Trained and used for blink-state input. |
| Voice interaction | Microphone to cloud ASR / LLM / TTS path, with speaker playback from the host. | Working in breadboard test and planned for final host integration. |
Processes used
This project uses the Fab Academy processes as one build rather than as separate weekly exercises.
| Process | Use in the project |
|---|---|
| 2D design and laser cutting | Platform, camera tower, base panels, front control layout. |
| 3D design and printing | Host shell, screen and speaker fit, wearable-related holders. |
| Electronics design | Host PCB in KiCad to replace the fragile breadboard wiring. |
| Electronics production | JLCPCB prototype plus hand soldering and board bring-up. |
| Embedded programming | WROOM host firmware, C3 BLE bridge, Beetle wearable firmware, XIAO Sense model path. |
| Interface programming | Screen UI, voice interaction, serial/UART debugging, local model output. |
| System integration | Packaging, power routing, buttons, charging, camera position, and final assembly. |
Design files and wiring reference
The detailed WROOM pin map, wearable wiring, internal circuit photo, and KiCad source package are on Week 16 - System Integration. The PCB files are also packaged directly here for download: Brainfog KiCad source package. The 3D host shell is the same design family documented in Week 2 CAD and Week 5 3D printing.
AI parts
I used two different AI paths. The local model is small and boring on purpose: it only classifies eye open versus eye closed on the XIAO ESP32S3 Sense so I can count blinks. The cloud voice path is for conversation: speech in, AI answer, TTS out. The cloud agent is named Aquarius; the naming and wellness boundaries are on the final project page. Keeping those jobs separate makes the system easier to explain and debug.
Local eye model (Edge Impulse → SenseCraft)
I collected face images at my desk with the XIAO camera, then labeled each frame as
open or closed. The full set ended up at 2,270 samples. Edge Impulse
split them 81% train / 19% test (1,832 / 438). I tried to keep the two classes balanced so the
model would not bias toward one eye state.
Before training, I set the target device to Espressif ESP-EYE (ESP32, 240 MHz, 4 MB RAM). That
is close enough to the XIAO ESP32S3 Sense for memory and timing estimates. The impulse itself
is simple: 96×96 grayscale image in, transfer-learning classifier out, two labels
(closed, open).
In the image processing block I kept color depth at grayscale. That cut RAM use and matched what the camera tower actually needs: open versus closed, not skin tone. After saving DSP parameters, I generated features for the full training set. Edge Impulse reported about 15 ms processing time and 4 KB peak RAM for the DSP stage on device.
Transfer learning ran for 30 cycles at learning rate 0.0001, with a 20% validation split and int8 profiling enabled. Validation accuracy landed at 100% on the held-out split for both classes. That number is optimistic (same desk, same lighting), but it was enough to move on to export and on-board testing.
I did not stay inside Edge Impulse for the final flash step. I exported the trained model and
uploaded it in SenseCraft AI under the name Brainfog. The board preview confirmed
live inference. Default power-on mode is UART (GPIO), so the XIAO sends class strings to the
WROOM host over serial instead of keeping camera frames on the main chip. The local model only
reports eye state; it does not diagnose anything.
Cloud voice (Bailian multimodal kit)
The spoken path uses Alibaba Cloud Bailian, in the multimodal interaction development kit. The
cloud app is named Brainfog insight companion. Aquarius is the agent persona and
instruction set inside that app (see the final project page for
why I chose that name).
Under Voice interaction I enabled Fun-ASR for speech recognition and CosyVoice-v3-Flash for text-to-speech. Language is set to English because the host demo and my Fab documentation are in English. The Understanding and generation tab holds the Aquarius system prompt: short answers, wellness tone, no medical claims. On the WROOM, the left button starts a proactive session; the INMP441 captures audio, the cloud pipeline returns text, and the MAX98357A plays the reply.
Questions that need to be answered
- How stable is blink counting under different room light and camera angles?
- How comfortable is the wearable band after a longer use session?
- How often does BLE data drop when the desktop host, C3 bridge, and wearable are all running together?
- Does the 40 Hz audio feel acceptable to users, or is it annoying after the first minute?
- Can the system give advice without sounding like a medical device?
- How much can I simplify the wiring in the next PCB revision?
How will it be evaluated?
I will call the Fab prototype successful if it can run as one system. Separate demos are not enough. The minimum test is: power the desktop host, detect blink state from the XIAO ESP32S3 Sense, receive wearable data through the C3 bridge, show useful state on the screen, play an audio intervention, and complete a voice interaction without reconnecting wires.
| Criterion | Evidence |
|---|---|
| Integrated packaging | All main desktop electronics inside the laser-cut platform and 3D printed shell. |
| Wearable link | Band test video and BLE-to-UART bridge logs or screen state. |
| Local eye model | Open / closed eye state affects blink counting. |
| Voice loop | Microphone input, cloud reply, and speaker output demonstrated. |
| Safety of claims | Documentation states that the project is for wellness support, not diagnosis. |
Current risk list
The main risks are not single sensors. They are integration risks: power stability, wire strain, camera placement, and whether the user understands what the device is asking them to do. I would rather document those limits than pretend the first integrated build is already a consumer product.