5. MCU + peripherals
This page documents the distributed microcontroller architecture, firmware development workflows, and the complete integration of inputs and outputs that enable the interactive capabilities of the project.
1. Microcontroller (MCU) Selection & System Architecture
Instead of relying on a single processor, the project employs a multi-MCU distributed architecture. This decouples the human-machine interface, motion control, and AI processing, allowing for parallel development and improved system reliability.
| Functional Module | Model | Core Responsibilities |
|---|---|---|
| Horse Body (Main Actuator) | ESP32 DevKit V1 | Receives commands, drives motors, triggers MIDI music, and updates the OLED display. |
| Remote Controller (HMI) | ESP32 DevKit V1 | Scans physical button inputs and transmits control signals via BLE. |
| AI Voice Interaction | ESP32-S3 DevKit | Performs local voice capture, runs speech-to-intent recognition, and publishes commands via MQTT. |
Rationale for Selection:
-
ESP32 DevKit V1 (Body & Remote): This dual-core chip was chosen for its robust Bluetooth Low Energy (BLE) 4.2 stack, which ensures low-latency communication for the remote control. It also provides sufficient GPIOs to handle PWM for motors, UART for MIDI, and I2C for the display, all within a cost-effective and widely supported ecosystem.
-
ESP32-S3 (AI Voice): The ESP32-S3 was selected specifically for its vector acceleration capabilities (TensorFlow Lite Micro support) and larger PSRAM. These features are critical for running an on-device keyword spotting (KWS) or speech recognition model without external cloud latency.
2. Firmware Development Environments & Toolchains
To leverage the full potential of each chip, I utilized specialized development frameworks tailored to their performance profiles.
| Module | IDE / Platform | Language | Rationale |
|---|---|---|---|
| Body & Remote | VS Code + PlatformIO | C++ | PlatformIO provides excellent library dependency management (e.g., for BLE and MQTT) and allows for rapid debugging via serial monitors and breakpoints. |
| AI Voice (ESP32-S3) | ESP-IDF (v5.0+) | C | ESP-IDF is the official framework that provides the lowest-level access to the S3's AI vector instructions and audio peripherals (I2S), ensuring maximum performance for real-time audio processing. |
(Note: MicroBlocks was used during the initial prototyping phase for quick logic validation but was replaced by the above for the final production firmware.)
3. Core Logic & Operational Workflow
The system operates through two independent, parallel control channels. The central firmware on the Horse Body's ESP32 continuously polls these input sources.
Control Channel A: Remote (BLE)
-
User presses a physical button on the Remote ESP32.
-
The Remote firmware encodes the event and sends it via BLE GATT notifications.
-
The Horse Body ESP32 receives the BLE packet and parses the command.
Control Channel B: AI Voice (MQTT)
-
User speaks a command (e.g., "trot", "spin") into the microphone.
-
ESP32-S3 processes the audio via its AI model and outputs a structured intent string.
-
The S3 connects to the local Wi-Fi and publishes the command to a specific MQTT topic (e.g., horse/control).
-
The Horse Body ESP32 (subscribed to the same topic) receives the MQTT payload via Wi-Fi.
4. Inputs & Outputs Integration (Physical Layer)
The following table details the specific hardware connections, pin assignments, and communication protocols used to bridge the MCUs with their peripherals.
| Type | Device | Target MCU | Interface / Protocol | Specifics (Pins/Baud) |
|---|---|---|---|---|
| Input | Physical Buttons (x4) | Remote (ESP32) | GPIO (Digital Input) | Internal pull-up enabled; debounced in software. |
| Input | BLE Signal | Body (ESP32) | BLE (GATT Server) | Custom service UUID; Notify characteristic for commands. |
| Input | Microphone (INMP441) | AI (ESP32-S3) | I2S | BCK (GPIO 14), WS (GPIO 15), DOUT (GPIO 16). |
| Input | MQTT Messages | Body (ESP32) | Wi-Fi (2.4GHz) / MQTT | PubSubClient library; Broker IP assigned via DHCP. |
| Output | N20 Micro Metal Motors (x2) | Body (ESP32) | GPIO (PWM) | PWM Channels @ 5kHz; GPIO 18 & 19 (Left), 21 & 22 (Right). |
| Output | MIDI Sound Module | Body (ESP32) | UART | Baud Rate: 31250; GPIO 4 (TX) -> MIDI RX. |
| Output | OLED Display (SSD1306) | Body (ESP32) | I2C | Address: 0x3C; GPIO 21 (SDA), GPIO 22 (SCL). |
5. System Integration & Communication Protocols
The project employs a dual-protocol communication strategy to optimize both immediacy and network flexibility:
-
BLE (Bluetooth Low Energy) for Remote Control: Used for point-to-point, low-latency communication between the handheld remote and the horse. BLE minimizes power consumption on the remote side and ensures instant tactile feedback for button presses (latency < 30ms).
-
MQTT (Message Queuing Telemetry Transport) for AI Voice: Used over Wi-Fi between the ESP32-S3 and the Horse Body. MQTT utilizes a publish/subscribe model which decouples the voice recognition unit from the actuator. This means the AI module can be reset or replaced without interrupting the horse's operation, and it also allows potential future integration with home automation systems (e.g., Node-RED).
Summary of Data Flow:
User Action → Input Device → MCU Logic → Protocol → Main MCU (ESP32) Integration → Output Response (Motion, Sound, and Visuals).
Final Note:
This integrated system successfully merges physical interaction (buttons) and ambient interaction (voice) into a single, responsive robotic entity, fulfilling the core requirement of a digitally fabricated interactive machine.