My Final Project

From Listening to Stories to Creating a Storytelling Device

KOIPOI began with my interest in listening to stories. What stayed with me was the warmth of those moments—the expressions, changes in tone, pauses, and emotions that brought each story alive. Stories have a way of shaping how we imagine, how we understand experiences, and sometimes even how we perceive the world. That curiosity eventually led me to explore storytelling in different mediums.

While exploring traditional puppet shows and crankie theatre, I became interested in how movement, visuals, and narration work together to tell a story. I started wondering how these traditional storytelling forms could be combined with electronics and digital fabrication, and that question became the starting point for KOIPOI.

This page showcases the final project and how it evolved throughout my Fab Academy. It highlights the main features, design, and implementation, those linked weekly assignments provide the detailed documentation on conceptualization,development process, system integration, testing, and technical work.

System
Integration

Applications
AND
Implications

Project Development

Initial Concept

Concept idea: Week 1

The first concept was an interactive puppet theatre inspired by traditional storytelling. Stories would be played using story cards, while users could also create and narrate their own. The layered stage was designed to add depth and create a more immersive storytelling experience. This early concept helped define the direction that eventually evolved into final project.

3D Modeling: Week 2

In Week 2, I created a 3D model to visualize the initial concept. It was not the final design, but an early exploration to understand the form, layout, and overall idea of the project. Software used: Fusion 360, Inkscape, Blender.

How the Concept Evolved

The concept evolved from a layered puppet stage with IR-based story cards(week 9) to a crankie-inspired storytelling device with RFID-enabled story rolls using NTAG stickers.

KOIPOI- Storytelling Device

KOIPOI is an interactive storytelling device inspired by traditional puppet shows and crankie theatre. It combines illustrated story rolls, synchronized puppet movement, audio narration, and embedded electronics to create an engaging storytelling experience. The device encourages both experiencing stories and creating new ones, making storytelling an interactive and shared activity.

Concept Design: Sketch and CAD

Story Roll

Final Product Packaging and Assembly

System Integrating

BOM: Bill Of Material

Note: This Bill of Materials (BOM) includes only the electronic components and selected mechanical parts used in the project. It does not include the cost of fabricated parts, fasteners, enclosure materials, or consumables.

The estimated total cost is ₹3,286.06 (approximately USD $34.51, based on an exchange rate of 1 USD ≈ ₹95.22).

Firmware

              
#include <Arduino.h>
#include <SPI.h>
#include <MFRC522.h>
#include <Adafruit_NeoPixel.h>
#include "config.h"

// ── XY-V17B driver ──────────────────────────────────────────────────────────
// Protocol: 0xAA | CMD | LEN | DATA... | CHECKSUM
// CHECKSUM = lowest 8 bits of (sum of all preceding bytes)
// Baud: 9600 8N1   BUSY pin: LOW = playing, HIGH = idle

static void mp3SendCmd(uint8_t cmd, const uint8_t *data = nullptr, uint8_t len = 0) {
    uint16_t sum = 0xAA + cmd + len;
    MP3_SERIAL.write(0xAA);
    MP3_SERIAL.write(cmd);
    MP3_SERIAL.write(len);
    for (uint8_t i = 0; i < len; i++) {
        MP3_SERIAL.write(data[i]);
        sum += data[i];
    }
    MP3_SERIAL.write((uint8_t)(sum & 0xFF));
}

void mp3Play()  { mp3SendCmd(0x02); }
void mp3Pause() { mp3SendCmd(0x03); }
void mp3Stop()  { mp3SendCmd(0x04); }
void mp3Next()  { mp3SendCmd(0x06); }
void mp3Prev()  { mp3SendCmd(0x05); }

void mp3SetVolume(uint8_t vol) {         // 0–30
    mp3SendCmd(0x13, &vol, 1);
}

void mp3PlayTrack(uint16_t track) {      // 1-indexed
    uint8_t d[2] = { (uint8_t)(track >> 8), (uint8_t)(track & 0xFF) };
    mp3SendCmd(0x07, d, 2);
}

bool mp3IsPlaying() {
    return digitalRead(MP3_BUSY_PIN) == HIGH;  // HIGH = playing per datasheet
}

// ── Peripherals ──────────────────────────────────────────────────────────────
MFRC522 rfid(RFID_SS, RFID_RST);
Adafruit_NeoPixel pixel(NEOPIXEL_COUNT, NEOPIXEL_PIN, NEO_GRB + NEO_KHZ800);

// ── State machine ─────────────────────────────────────────────────────────────
enum State { IDLE, AUDIO_PLAYING, ADVANCING, REWINDING, FINISHED };
State state = IDLE;
int currentScene = 0;  // 0-indexed

// ── Pixel helper ─────────────────────────────────────────────────────────────
void setPixel(uint8_t r, uint8_t g, uint8_t b) {
    pixel.setPixelColor(0, pixel.Color(r, g, b));
    pixel.show();
}

// ── Motor helpers ─────────────────────────────────────────────────────────────
void enableMotor1() {
    digitalWrite(SM2_ENABLE, HIGH);  // disable SM2
    digitalWrite(SM1_ENABLE, LOW);   // enable SM1
}

void enableMotor2() {
    digitalWrite(SM1_ENABLE, HIGH);  // disable SM1
    digitalWrite(SM2_ENABLE, LOW);   // enable SM2
}

void disableAllMotors() {
    digitalWrite(SM1_ENABLE, HIGH);
    digitalWrite(SM2_ENABLE, HIGH);
}

void stepMotor1() {
    digitalWrite(SM1_STEP, HIGH); delayMicroseconds(STEP_DELAY_US / 2);
    digitalWrite(SM1_STEP, LOW);  delayMicroseconds(STEP_DELAY_US / 2);
}

void stepMotor2() {
    digitalWrite(SM2_STEP, HIGH); delayMicroseconds(STEP_DELAY_US / 2);
    digitalWrite(SM2_STEP, LOW);  delayMicroseconds(STEP_DELAY_US / 2);
}

// ── Scroll advance ────────────────────────────────────────────────────────────
// SM2 winds the take-up reel until the slot sensor finds the next frame edge.
bool advanceOneScene() {
    enableMotor2();
    digitalWrite(SM2_DIR, LOW);  // forward / wind direction

    unsigned long t0 = millis();

    // Step past the current slot if we're sitting on it.
    while (digitalRead(SLOT_SENSOR_PIN) == LOW) {
        stepMotor2();
        if (millis() - t0 > SCENE_ADVANCE_TIMEOUT_MS) { disableAllMotors(); return false; }
    }
    // Now advance until the next slot.
    while (digitalRead(SLOT_SENSOR_PIN) == HIGH) {
        stepMotor2();
        if (millis() - t0 > SCENE_ADVANCE_TIMEOUT_MS) { disableAllMotors(); return false; }
    }

    disableAllMotors();
    return true;
}

// ── Rewind ────────────────────────────────────────────────────────────────────
// SM1 runs (unwinds take-up reel) until we've crossed (currentScene) slots back.
void rewindToStart() {
    state = REWINDING;
    setPixel(100, 30, 0);
    Serial.println("Rewinding...");

    enableMotor1();
    digitalWrite(SM1_DIR, HIGH);  // reverse direction


    for (int i = 0; i < currentScene; i++) {
        unsigned long t0 = millis();
        while (digitalRead(SLOT_SENSOR_PIN) == HIGH) {
            stepMotor1();
            if (millis() - t0 > SCENE_ADVANCE_TIMEOUT_MS) break;
        }
        while (digitalRead(SLOT_SENSOR_PIN) == LOW) {
            stepMotor1();
            if (millis() - t0 > SCENE_ADVANCE_TIMEOUT_MS) break;
        }
    }

    disableAllMotors();
    currentScene = 0;
    state = IDLE;
    setPixel(0, 100, 0);
    Serial.println("Ready. Press PLAY.");
}

// ── Scene playback ────────────────────────────────────────────────────────────
void playCurrentScene() {
    mp3PlayTrack(currentScene + 1);  // tracks are 1-indexed on SD card
    delay(200);                       // give module time to start; BUSY goes LOW
    setPixel(0, 50, 100);
    state = AUDIO_PLAYING;
    Serial.printf("Scene %d: playing track %d\n", currentScene + 1, currentScene + 1);
}

// ── Setup ─────────────────────────────────────────────────────────────────────
void setup() {
    Serial.begin(115200);

    // Motor pins
    for (int p : {SM1_STEP, SM1_DIR, SM1_ENABLE, SM2_STEP, SM2_DIR, SM2_ENABLE})
        pinMode(p, OUTPUT);
    disableAllMotors();

    // Input pins
    pinMode(SLOT_SENSOR_PIN, INPUT_PULLUP);
    pinMode(BUTTON_PLAY,     INPUT_PULLUP);
    pinMode(BUTTON_REWIND,   INPUT_PULLUP);
    pinMode(MP3_BUSY_PIN,    INPUT);

    // NeoPixel
    pixel.begin();
    setPixel(0, 0, 0);

    // XY-V17B — hardware UART1 on GP12/GP11
    MP3_SERIAL.begin(9600);
    delay(500);
    mp3SetVolume(25);

    // RFID
    SPI.begin();
    rfid.PCD_Init();

    setPixel(0, 100, 0);
    Serial.println("Puppet show ready. Press PLAY to start.");
}

// ── Loop ──────────────────────────────────────────────────────────────────────
void loop() {
    bool playBtn   = digitalRead(BUTTON_PLAY)   == LOW;
    bool rewindBtn = digitalRead(BUTTON_REWIND) == LOW;

    switch (state) {

        case IDLE:
            if (playBtn) {
                delay(50);
                playCurrentScene();
            }
            if (rewindBtn && currentScene > 0) {
                delay(50);
                rewindToStart();
            }
            break;

        case AUDIO_PLAYING:
            if (!mp3IsPlaying()) {
                // Small debounce — BUSY can glitch at track start.
                delay(300);
                if (!mp3IsPlaying()) {
                    currentScene++;
                    if (currentScene >= TOTAL_SCENES) {
                        state = FINISHED;
                        setPixel(50, 0, 100);
                        Serial.println("Show complete!");
                    } else {
                        state = ADVANCING;
                        setPixel(100, 100, 0);
                        Serial.printf("Advancing to scene %d...\n", currentScene + 1);
                        if (advanceOneScene()) {
                            playCurrentScene();
                        } else {
                            Serial.println("ERROR: slot sensor timeout.");
                            setPixel(100, 0, 0);
                            state = IDLE;
                        }
                    }
                }
            }
            break;

        case ADVANCING:
            // Handled inline above (blocking call).
            break;

        case REWINDING:
            // Handled inside rewindToStart() (blocking).
            break;

        case FINISHED:
            if (rewindBtn) {
                delay(50);
                rewindToStart();
            }
            break;
    }

    // RFID: while idle, a card tag jumps to that scene.
    if (state == IDLE && rfid.PICC_IsNewCardPresent() && rfid.PICC_ReadCardSerial()) {
        uint8_t tag = rfid.uid.uidByte[0];
        Serial.printf("RFID tag 0x%02X → scene %d\n", tag, (tag % TOTAL_SCENES) + 1);
        currentScene = tag % TOTAL_SCENES;
        rfid.PICC_HaltA();
        playCurrentScene();
    }
}

View Development Log (PDF)

Future Scope:

The current prototype successfully synchronizes the story roll and audio narration. In the future, I plan to incorporate puppets as a separate modular system connected through I2C communication. The puppet will act as the narrator, with its movements, expressions, and actions synchronized with the story roll and narration for each scene. The modular design will also allow different puppet characters to be created and connected for different stories, expanding the storytelling experience.

Design Files

Koipoi Design: fusionfile
Main baord: Kicad-File
Breakout board: Kicad-File
Progam: codefiles