16. System Integration

Robot Arm Lamp System Integration Plan

Project Title: Robot Arm Lamp inspired by Pixar Luxo Jr.

Integrated Build Evidence

Photos below are from the completed build documented on the final project page — wired boards, cable routing, mechanical assembly, and end-to-end gesture demo.

Week 8 custom PCB (populated and tested) and lamp head sub-assembly with I2C wiring.

I2C cable routed through the hollow base and arm (left); pre-assembly bench test with Serial Monitor (right).

End-to-end demo video (~1 min): fabrication, gesture recognition, and lighting response. Also on the final project page.

Final Project Timeline

This timeline shows the planned milestones for completing the final project, from system integration planning to fabrication, assembly, testing, and final documentation.

Final project timeline showing the planned milestones and deadlines.

System Architecture

The planned system uses a dual-MCU architecture: a standalone XIAO ESP32-S3 Sense in the base handles vision and WiFi; a custom PCB with ESP32-C3 handles NeoPixel lighting. The two communicate over I2C. Power comes from a 5 V USB-C supply.

Planned system block diagram for power, control, sensing, actuation, and interface.

                    +---------------------------+
                    |   User (hand gesture)     |
                    +-------------+-------------+
                                  |
                                  v
+------------------------------------------------------------------+
|  MASTER: XIAO ESP32-S3 Sense  (mounted in base)                  |
|  - OV2640 camera capture                                         |
|  - SenseCraft AI pretrained gesture model (Rock / Paper / Scissors)|
|  - I2C Master  SDA=GPIO8, SCL=GPIO9  ->  sends gesture code      |
|  - WiFi web UI (Week 15)                                         |
+----------------------------+-------------------------------------+
                             | I2C (400 kHz, 3-wire: GND, SDA, SCL)
                             v
+------------------------------------------------------------------+
|  SLAVE: ESP32-C3  (Custom NeoPixel PCB)                          |
|  - I2C Slave address 0x08                                         |
|  - Receives gesture ID -> selects NeoPixel color + pattern        |
|  - Drives WS2812B ring/strip in lamp head                        |
+------------------------------------------------------------------+

Gesture flow:
  Camera frame -> SenseCraft inference -> gesture label
       -> I2C write(gesture_id) -> NeoPixel color + pattern update

Subsystem Map

Gesture Interaction: Rock–Paper–Scissors

The lamp's primary interactive demo is a rock–paper–scissors game. The user shows a hand gesture in front of the camera; the lamp detects it and updates the NeoPixel light to express the result. This ties together vision and lighting in one integrated behavior.

Interaction Flow

Gesture → Light Response Mapping

SenseCraft AI Vision Pipeline

Instead of training a custom model from scratch (as in my Week 17 Edge Impulse facial expression work), the final project will use a pretrained hand-gesture model from SenseCraft AI on the XIAO ESP32-S3 Sense. SenseCraft provides ready-to-flash models for common ESP32 boards and handles camera init, inference, and serial output in one package.

SenseCraft AI deployment workflow (planned):

1. Open SenseCraft AI Model Assistant
2. Select board: XIAO ESP32-S3 Sense
3. Choose pretrained model: Hand Gesture Recognition (Rock / Paper / Scissors)
4. Flash model to board via USB
5. Verify Serial output shows gesture labels with confidence scores
6. Integrate inference results into JeLamp firmware:
     - Read gesture label from SenseCraft inference API / serial bridge
     - Map label -> GESTURE_ROCK (0x01), GESTURE_PAPER (0x02), GESTURE_SCISSORS (0x03)
     - Trigger I2C write on stable detection (confidence > 0.75)

I2C Communication: ESP32-S3 Master ↔ ESP32-C3 Slave

Lighting is handled on a separate custom PCB so NeoPixel timing (RMT / bit-banging) does not interfere with camera capture and AI inference on the ESP32-S3. The XIAO exposes I2C on GPIO8 (SDA) and GPIO9 (SCL); a 3-wire cable (GND, SDA, SCL) runs up through the hollow arm to the lamp head. Each board is powered locally — the XIAO via USB-C in the base, the ESP32-C3 from the 5 V NeoPixel rail on its own PCB.

Hardware Connection

XIAO ESP32-S3 Sense (base)                NeoPixel PCB (ESP32-C3)

GND  ------------------------------------>  GND
D4 / GPIO8 (SDA) ----------------------->  GPIO6 (SDA)
D5 / GPIO9 (SCL) ----------------------->  GPIO7 (SCL)

Cable: 3-wire silicone ribbon (GND, SDA, SCL), ~400 mm through lower + upper arm
Power: separate — XIAO via USB-C in base; ESP32-C3 from 5 V on NeoPixel PCB
Pull-ups: 4.7 kΩ on SDA and SCL to 3V3 (on NeoPixel PCB only)

I2C Protocol (planned)

Firmware Snippets (planned)

ESP32-S3 — I2C master (gesture sender):

#include <Wire.h>

#define I2C_SDA  8
#define I2C_SCL  9
#define NEO_SLAVE  0x08

#define GESTURE_ROCK      0x01
#define GESTURE_PAPER     0x02
#define GESTURE_SCISSORS  0x03
#define CMD_GESTURE       0x01

void sendGestureOverI2C(uint8_t gesture) {
  Wire.beginTransmission(NEO_SLAVE);
  Wire.write(CMD_GESTURE);
  Wire.write(gesture);
  Wire.endTransmission();
}

void onGestureDetected(const char* label) {
  if (strcmp(label, "rock") == 0) {
    sendGestureOverI2C(GESTURE_ROCK);
  } else if (strcmp(label, "paper") == 0) {
    sendGestureOverI2C(GESTURE_PAPER);
  } else if (strcmp(label, "scissors") == 0) {
    sendGestureOverI2C(GESTURE_SCISSORS);
  }
}

ESP32-C3 — I2C slave (NeoPixel driver):

#include <Wire.h>
#include <Adafruit_NeoPixel.h>

#define NEO_PIN     2
#define NUM_PIXELS  12
#define I2C_ADDR    0x08

Adafruit_NeoPixel pixels(NUM_PIXELS, NEO_PIN, NEO_GRB + NEO_KHZ800);

uint8_t rxBuf[2];
uint8_t rxLen = 0;

void receiveEvent(int len) {
  rxLen = 0;
  while (Wire.available()) rxBuf[rxLen++] = Wire.read();
}

void applyGesture(uint8_t gesture) {
  switch (gesture) {
    case 0x01: setPatternSolid(255, 140, 0);   break;  // rock
    case 0x02: setPatternBreathe(180, 220, 255); break;  // paper
    case 0x03: setPatternChase(255, 0, 80);    break;  // scissors
  }
}

void setup() {
  pixels.begin();
  pixels.setBrightness(180);
  Wire.begin(I2C_ADDR);
  Wire.onReceive(receiveEvent);
}

void loop() {
  if (rxLen >= 2 && rxBuf[0] == 0x01) applyGesture(rxBuf[1]);
  updatePattern();  // non-blocking animation tick
}

Custom ESP32-C3 NeoPixel PCB

This is the only custom PCB in the system — designed in KiCad (Week 6), milled and soldered (Week 8), then mounted in the lamp head as the I2C slave and NeoPixel driver. Full fabrication documentation is on the final project page.

Mechanical Integration

The lamp body uses a fixed giraffe-like arm structure. The base contains the XIAO ESP32-S3 Sense and ballast weight; the arm sections are hollow so the I2C and power cables can pass through to the lamp head.

Mechanical assembly — lamp head sub-assembly (left) and base with ESP32-S3 module mounted (right).

3D Printed Parts

Cable Routing

Cable routing was one of the most important integration risks in this design. A 3-wire I2C cable (GND, SDA, SCL) runs from the base through the hollow arm to the ESP32-C3 NeoPixel board in the lamp head. Each board is powered locally — the XIAO via USB-C in the base, the C3 from the 5 V rail on its PCB.

I2C cable (red / white / black) routed through the base lid opening with ~15 mm slack before entering the arm.

I2C wiring reference — SDA, SCL, and GND between the two XIAO modules.

Electronics Integration

The system uses one off-the-shelf controller and one custom PCB. The XIAO ESP32-S3 Sense mounts in the base with its built-in camera and USB-C port exposed for programming. The ESP32-C3 NeoPixel PCB mounts in the lamp head and drives the WS2812B ring. A 3-wire I2C cable (GND, SDA, SCL) runs through the hollow arm to connect them.

Populated C3 PCB (left) and pre-enclosure bench test showing SenseCraft inference over I2C (right).

Power Budget

The planned supply is a 5 V 2 A USB-C power adapter connected directly to the XIAO and NeoPixel PCB. A 470 µF capacitor near the NeoPixel connector absorbs current spikes during color changes.

Software Integration

The firmware is split across two MCUs and organized in layers on the ESP32-S3. Each subsystem will be tested independently — SenseCraft gesture detection alone, I2C ping alone, NeoPixel patterns alone — before the full rock–paper–scissors game is integrated.

ESP32-S3 (Master) — Layered Firmware

Layer 4: Interaction Modes
  Gesture game (rock/paper/scissors), web UI control, touch response, idle behavior

Layer 3: SenseCraft AI Bridge
  Camera capture -> pretrained gesture model -> debounced label output

Layer 2: Device Drivers
  SenseCraft inference, I2C master (Wire), WiFi WebServer, touch read

Layer 1: Hardware Abstraction
  I2C at 400 kHz, camera init (OV2640)


ESP32-C3 (Slave) — Lighting Firmware

  I2C slave handler -> gesture/pattern lookup table -> NeoPixel RMT driver
  Non-blocking animation engine: solid, breathe, chase, rainbow

Key Firmware Structure

#include <WiFi.h>
#include <WebServer.h>
#include <Wire.h>

#define I2C_SDA   8
#define I2C_SCL   9
#define TOUCH_PIN 6

#define NEO_SLAVE  0x08
#define GESTURE_ROCK      0x01
#define GESTURE_PAPER     0x02
#define GESTURE_SCISSORS  0x03
#define CMD_GESTURE       0x01

Integrated Behaviors

Web Interface Routes

Packaging

The packaging goal is to make the lamp look like a finished consumer product instead of a loose prototype. The planned design will hide wires, PCB, and solder joints while keeping the lamp serviceable.

Base Interior Layout

Top view, base cover removed

+--------------------------------------+
|                                      |
|   XIAO ESP32-S3 Sense module         |
|   USB-C accessible at rear           |
|             |                        |
|             | I2C + power cables up  |
|                                      |
|   Steel ballast plate, about 200 g   |
|                                      |
|   5 V USB-C power at rear            |
+--------------------------------------+

Lamp Head Detail

The lamp head contains the ESP32-C3 NeoPixel PCB, the WS2812B LED ring, a small aluminum plate for heat spreading, and a frosted acrylic diffuser press-fit into the shade rim. The I2C and 5 V power cable enters from the upper arm through a grommet. Keeping the NeoPixel controller close to the LEDs minimizes data-line length and improves timing reliability.

Planned Assembly Process

I will assemble the system in stages, testing each layer before hiding it inside the final enclosure. This should make it easier to identify whether a problem comes from electronics, wiring, mechanical constraints, or software calibration.

Planned Bill of Materials

This is my estimated material list for the integrated prototype. The final quantities and cost may change after fabrication and assembly.

Expected Challenges and Planned Solutions

Integration Notes Before Building

• Test each subsystem before final assembly: SenseCraft gestures alone, I2C ping alone, NeoPixel patterns alone.
• The dual-MCU split keeps NeoPixel RMT timing off the ESP32-S3 so camera capture and AI inference are not interrupted.
• Cable routing is part of mechanical design. The I2C cable must pass through the hollow arm with slack at the head joint.
• Mount the camera on the base or lower arm so it faces the user at desk height during the gesture game.

Files

Source document: Week 16 System Integration - Robot Arm Lamp

Planning Checklist

This checklist tracks the Fab Academy system integration requirements for the final project. Items marked "Planned" or "In progress" will be updated to "Done" with photos and video after the physical build is complete.

Hero Shots and Demo Video

The integrated build is documented with real photos and a one-minute demo video on this page and the final project page.

Integrated system test — web UI config alongside assembled lamp responding to stone gesture.

presentation.mp4 — fabrication steps, paper/stone gesture demo, and final operation.