An Autonomous Puppeteering System
A.k.a. Pitou would be proud π±π§΅
π‘ Project Idea
This project is an electronic puppeteering system inspired by Neferpitouβs Nen-based manipulation from Hunter x Hunter. It explores themes of control, autonomy, and possession through a controller entity that governs the movement of a skeletal puppet.
The system operates autonomously through pre-programmed motion sequences. I'm thinking movements like slow head tilt - pause - sudden jerk, arm lifts as if resisting then drops. Repetitive, clinical movements. Each sequence is loopable, time-based and/or triggered by presence or touch.
From here 2 things can be explored:
- The puppeteer is programmed to control the puppet.
- The puppet moves as if possessed, while the puppeteers digits/hand move out of sync which sell the whole eerie part and acts as a visual representation of invisible electronic control.
π§© Project Architecture Overview
This project consists of two primary entities:
- Controller Entity (Puppeteer Unit)
- Skeletal Puppet (Actuated Entity)
Rather than directly mirroring user input, the system filters and transforms signals through timing logic, motion constraints, and behavioural state switching. This creates movement that feels delayed, resistant, and semi-autonomous rather than fully obedient.
π Design Evolution
The project went through a bunch of iterations before landing on its final form. Each one changed how you interact with the puppet.
πΉ Trying out stuff
πΉ Figuring out the UI
πΉ Prototyping
π System Architecture
User (voice / buttons) β Web Interface β WiFi (WebSocket, port 81) β XIAO ESP32-S3 β I2C PCA9685 Servo Driver β 5 Γ SG90 Servos (head, wrists, knees) β Strings & Pulleys β Puppet Motion π
The ESP32 creates its own WiFi network. The browser connects to it, sends a command like {"command": "wave"}, and the ESP32 tells the PCA9685 which servos to move.
π οΈ Digital Fabrication
πΉ The Stage (Laser cutting)
πΉ The Puppet + Pulleys (3D printing)
Custom pulleys were designed and 3D printed to sit on the SG90 horns and wind the strings.
πΉ Servo Layout
In the final version the elbows were dropped, leaving 5 active servos: head, both wrists and both knees.
β‘ Electronics
Custom PCB designed in EasyEDA for the XIAO ESP32-S3 with a 4-pin header for the PCA9685 (SDA, SCL, VCC, GND).
π Things That Went Wrong
Failure 01: Mirrored PCB πͺ
Milled the whole board...only to find the design was mirrored. The XIAO had to sit upside down (β₯οΉβ₯)
Failure 02: Lifted Copper Trace
During soldering, a copper trace peeled off the board. RIP PCB #2.
Failure 03: Missing Antenna πΆ
Spent hours debugging WiFi before realising...the antenna wasn't plugged in. (ΰ²₯οΉΰ²₯)
π» Programming
Written in Arduino IDE. Libraries: Adafruit PWM Servo Driver, WebSockets (Markus Sattler), ArduinoJson.
Key parts of the program:- β’ ESP32 runs as a WiFi access point (
PuppetControl) - β’ WebSocket server on port 81 receives JSON commands
- β’ Each joint has 3 positions: limp (0Β°), neutral (90Β°), full (180Β°)
- β’
smoothMove()steps the servo 1Β° at a time so motion isn't jerky - β’ Combo poses: stand, limp, wave, bow, nod, dance
- β’ Several phrasings per command (e.g. "raise left wrist", "left wrist up", "lift left wrist") so voice input is more forgiving
πΉ Channel map (final version)
| Joint | PCA9685 channel |
|---|---|
| Right wrist | CH2 |
| Left wrist | CH13 |
| Left knee | CH15 |
| Right knee | CH0 |
| Head | CH5 |
void smoothMove(int ch, int target, int stepDelay = 10) {
int current = currentAngles[ch];
int step = (target > current) ? 1 : -1;
while (current != target) {
current += step;
setServo(ch, current);
delay(stepDelay);
}
}
π± Interface
The browser listens to your voice, converts it to text and sends it to the ESP32. Say "raise right wrist" and...it does π«‘
The full write-up of the interface (voice input, buttons, WebSocket messages) is in Week 14 Β· Interface & Application.
π§Ύ Bill of Materials
- β’ Seeed XIAO ESP32-S3 + antenna Γ 1
- β’ PCA9685 16-ch servo driver Γ 1
- β’ SG90 micro servos Γ 7
- β’ FR1 copper board (custom PCB) Γ 1
- β’ Acrylic sheet (stage)
- β’ PLA filament (puppet + pulleys)
- β’ Thread / strings
- β’ 5V external power supply
- β’ Fairy lights (stage lighting β¨)
π Reflection
β What worked
- β’ Voice + button control over WiFi
- β’ Smooth servo motion with the limp/neutral/full system
- β’ The stage turned out sooo pretty
β What didn't
- β’ PCB #1 mirrored, PCB #2 lost a trace
- β’ String tension needs constant re-tuning
π Future improvements
- β’ The autonomous "possessed" mode from my original idea π
- β’ Puppeteer hand moving out of sync with the puppet
- β’ Hiding the electronics inside the stage
π Design Files
Final Puppet Controller Code (.ino)Puppet Controller Interface (.html)
Puppet Controller v2 (.ino)
Puppet Controller v1 (.ino)
1 Servo Test (.ino)
2 Servo Test (.ino)
SG90 Pulley (.3mf)
PCB Gerber Files (.zip)
PCB Schematic, EasyEDA (.json)