Lumi; A Dancing TV

Inspired by the Disney Research team's bipedal animatronic characters that interact with crowds in real-time. I wanted to create a much smaller, much simpler desktop version of an interactive machine.

The inspiration behind the idea made by Disney

Since I did not have the time or technical knowledge (yet!) that Disney Imagineers have, I simplified the design to the shape of a simple TV that has legs. The legs will be based on the design used for Otto, which is an educational open-source robot that can be programmed to walk/shuffle and dance!

To better visualize how my final design would look like, I even made a 2D render of my concept using Photopea.

Table of Conents

The Idea
CAD Design
PCB Schematic
Voice Input
OLED Output
Servo Output
References

1. The Idea

Brainstorming

Many ideas were discussed including a mini companion bot and a robotic humanoid chatbot among others (see more in Week 1), but ultimately I choose the Lumi as my final project for the following reasons:

Sketching

Key Features

The Companion Bot has the following key features:

An LCD display on the robot's front shows three expressions—happy, sad, and neutral—with accompanying sound effects from a speaker.
The robot responds to voice commands for power control and can display time, weather, and temperature information, replacing its standard expressions.
The robot features bird-like legs that fold when powered down to save space. When activated, the legs extend to full height, enhancing its interactive presence.
Front-mounted sensors help the robot detect and avoid obstacles while preventing falls from the desktop.

2. CAD Design

This is model when rendered as a still image.

This is the model when rendered as a turntable.

Project FlowChart

Project Plan

I used the help of ChatGPT to be able to produce the the project plan given below. The ChatGPT prompts are given here

Spiral	Dates	Focus	Objectives	Risk Analysis	Deliverables
Spiral 1	April 12–19	Input Sensing & Cardboard Prototype	Build a cardboard prototype to finalize design, input (INMP 441), output (OLED), and proximity sensor	Cardboard structure durability Sensor range and accuracy OLED compatibility	Cardboard prototype with voice-controlled ON/OFF Head rotation via proximity sensor
Spiral 2	April 20–26	Voice Commands & Basic Emotion Display	Program voice input to toggle ON/OFF Display basic happy/sad faces on OLED	Voice recognition accuracy OLED display functionality	Reliable ON/OFF voice control Static happy/sad faces on OLED
Spiral 3	April 27–May 3	Head Rotation Based on Proximity	Integrate proximity sensor to rotate head toward user when ON	Smoothness of head rotation Sensor responsiveness	Servo-controlled head rotation via proximity sensor
Spiral 4	May 4–10	Finalizing Structure & Electronics	Finalize housing (cardboard or laser-cut/3D printed) Integrate servos, voice input, and OLED into structure	Fit and durability of housing parts Secure wiring and component placement	Final integrated prototype with head rotation, voice control, OLED
Spiral 5	May 11–17	System Integration & Testing	Integrate all components into one system Test full functionality: voice ON/OFF, head rotation, expression change	Component integration issues Power consumption and stability	Fully functional Lumi with core features
Spiral 6	May 18–24	Documentation & Final Presentation	Document all steps and create demo video Prepare final presentation (slides/website)	Clarity and completeness of documentation Technical reliability of demo	Final presentation materials 1‑minute demo video Complete project documentation

ChatGPT Prompt 1: Updated Lumi Project Plan Using the Spiral Model

Prompt 1: Updated Lumi Project Plan Using the Spiral Model

Here’s a structured spiral development plan for your project Lumi, the interactive animatronic robot with emotional expressions and sound:

Spiral Model Breakdown (April 12 – June 7)

Each spiral is planned weekly from Friday to the following Thursday.
Each spiral has a core objective and a tested outcome.

Spiral	Dates	Objective	Tasks	Deliverables
Spiral 1	April 12 – 18	Input + Expression	Test INMP441 mic with voice trigger for ON/OFF Display face on OLED based on power state Prototype folding leg mechanism in cardboard	Cardboard prototype + voice to OLED response
Spiral 2	April 19 – 25	Body + Head	Design OLED housing (head) Start body housing integration Refine face expressions	Laser-cut head, face reaction tests
Spiral 3	April 26 – May 2	Bluetooth + Emotions	Get input from Bluetooth module Show emotion based on command	Bluetooth to OLED emotional state control
Spiral 4	May 3 – 9	Leg Movement	Implement folding/standing motion Make leg housing stiffer	Moving legs on command, standing Lumi
Spiral 5	May 10 – 16	Interaction + Feedback	Make it react to proximity/human touch Sound effects based on state	Interactive Lumi with sensors + sound
Spiral 6	May 17 – 23	Final Prototype	Assemble final working Lumi with all elements	Final model assembled and working
Spiral 7	May 24 – 30	Testing + Polish	Polish movement & improve timing/behavior Power management & fail-safes	Refined Lumi ready for final presentation
Spiral 8	May 31 – June 6	Documentation	Create BOM sheet, exploded views Prepare final FabAcademy documentation	Project page + presentation ready

ChatGPT Prompt 2: Cardboard to A TV on Legs Plan

Prompt 2: Cardboard to A TV on Legs Plan

Here is a simplified version of your Lumi project goal and scope, framed as a week-by-week spiral development plan from cardboard prototype to the final model (“A TV on legs”):

Summary Objective

Lumi is a robotic pet with an expressive OLED face that responds to voice commands, rotates to face the user, and stands on foldable legs.

Phased Approach from Cardboard to Completion

Phase	Focus	Tools/Processes	Expected Outcome
Phase 1	Cardboard Prototype	Laser cutting cardboard parts OLED + INMP441 basic setup	Physical form + Input/Output basics tested
Phase 2	OLED Head + Emotion Display	FluxGarage eye graphics Facial expression coding	Robot with animated face display
Phase 3	Voice Input and Bluetooth	INMP441 for voice trigger BLE commands (mobile app or serial)	Lumi responds to voice/Bluetooth
Phase 4	Movement System	Design folding leg mechanism Test with servos or micro actuators	Folding/Standing motion works
Phase 5	Full Integration	Package OLED, sensors, body Final polish and bug fixing	“TV on legs” working robot pet!

Bill of Materials

This is the expected bill of materials for my project. Keep in mind that this will keep getting refined as I proceed with the final project

3. Possible PCB Schematic

Trying to create a PCB for my final project using the ESP32-WROOM-32E module

Here I have made a schematic for a PCB for my final project. The design has to be refined in the coming weeks, but theoretically, there is a voltage regulator circuit making sure that the MCU receives only 3.3V. Also I can communicate with my MCU using a USB A to C type connector. I will continue to refine this design as the weeks progress.

For more information, see Week 6 documentation.

4. Voice Input

In week 9, I made an an audio input sensor, which could potentially be able to recognize certain voice commands

If the sound input is loud enough to cross a threshhold, the LED lights up

To see more, check out my documentation for Week 9. In the future, I would like to make it possible for the device to recognize voice commands

5. OLED Output

In Output Devices Week I learnt to make a simple facial animation on the OLED display. To achive this, I used this tutorial that uses the U8g2 library, which is supported by Wokwi simulator. Following the tutorial I went to Lopaka.app, selected the U8g2 library, and created a new project


                    
    #include <U8g2lib.h>
    #include <Wire.h>
    
    long dt;
    long vert;
    
    U8G2_SSD1306_128X64_NONAME_F_HW_I2C u8g2(U8G2_R0, U8X8_PIN_NONE);
    
    void setup(){
        Serial.begin(115200);
        if (!u8g2.begin()){
        Serial.println("Display not initialized");
        for(;;);
        }
        delay(1000);
        Serial.println("Display initialized");
    }
    
    void mouth_open(){
        u8g2.clearBuffer();
        u8g2.setFontMode(1);
        u8g2.setBitmapMode(1);
        u8g2.drawFilledEllipse(83, 33+vert, 8, 13);
        u8g2.drawFilledEllipse(62, 54+vert, 8, 6);
        u8g2.drawFilledEllipse(41, 33+vert, 8, 13);
        u8g2.sendBuffer();
    }
    
    void mouth_half_open (){
        u8g2.clearBuffer();
        u8g2.setFontMode(1);
        u8g2.setBitmapMode(1);
        u8g2.drawFilledEllipse(83, 33+vert, 8, 13);
        u8g2.drawFilledEllipse(41, 33+vert, 8, 13);
        u8g2.drawFilledEllipse(62, 57+vert, 8, 3);
        u8g2.sendBuffer();
    } 
    
    void mouth_closed (){
        u8g2.clearBuffer();
        u8g2.setFontMode(1);
        u8g2.setBitmapMode(1);
        u8g2.drawFilledEllipse(83, 33+vert, 8, 13);
        u8g2.drawFilledEllipse(62, 59+vert, 8, 1);
        u8g2.drawFilledEllipse(41, 33+vert, 8, 13);
        u8g2.sendBuffer();
    }
    
    void loop(){
        vert = random(0,3);
        dt = random(0,150);
        
        mouth_closed();
        delay(dt);
        // Serial.println(dt);
    
        vert = random(0,3);
        dt = random(0,200);
        mouth_half_open();
        delay(dt);
        // Serial.println(dt);
    
        vert = random(0,3);
        dt = random(0,200);
        mouth_open();
        delay(dt);
        // Serial.println(dt);
    
        vert = random(0,3);
        dt = random(0,200);
        mouth_half_open();
        delay(dt);
        // Serial.println(dt);
    
        vert = random(0,3);
        dt = random(0,200);
        mouth_closed();
        delay(dt);
        // Serial.println(dt);
    }

Lumi; A Dancing TV

Table of Conents

1. The Idea

Brainstorming

Sketching

Key Features

2. CAD Design

Project FlowChart

Project Plan

Prompt 1: Updated Lumi Project Plan Using the Spiral Model

Spiral Model Breakdown (April 12 – June 7)

Prompt 2: Cardboard to A TV on Legs Plan

Summary Objective

Phased Approach from Cardboard to Completion

Prompt 3: Which features are essential and which are optional?

Essential Features (Core to Graduation Requirements)

Optional Features (Nice-to-Have Enhancements)

Recommendation

Prompt 4: What is the minimum I need to do to graduate?

✅ Minimum Graduation Criteria

📌 Summary

Prompt 5: Make my project simpler

🎯 Simplified Lumi Features

🛠 Technologies Used

Prompt 6: Why does this project exist?

Prompt 7: Modify this statement to increase clarity: "To create a machine that can interact in a human-like manner with empathy."

Prompt 8: A machine that can evoke human empathy (simplified)

Prompt 9: A robo pet for people who can’t keep real pets

Prompt 10: A TV on legs