Weekly Assignments Final Project About me

Final Project

Talking Toy

After discussing with my instructor, I finalized the idea of creating a talking toy. The inspiration for this project came during a visit to my cousin's home, where I noticed her one-and-a-half-year-old daughter glued to a smartphone and unwilling to part with it. This made me wonder if there could be a toy that would engage her and encourage interaction. While researching, I couldn't find any toys that allowed customization of audio files—they all came preloaded with standard audio, which felt uninspiring.

As I was brainstorming ideas for my Fab Academy final project, I thought, why not create the kind of toy I had been searching for? I believe this would make the perfect gift for her. The toy will include an SD card slot, enabling us to play customized audio files such as stories or music. Additionally, it will have wheels, allowing it to follow the child while playing the audio, making it interactive and engaging.

This project is specifically designed for children below the age of five, providing them with an engaging and interactive experience. It allows young kids to play and interact with the toy in a way that is both entertaining and educational. Additionally, parents have the ability to play soothing music or captivating stories through the toy, creating a comforting environment for their children. I believe this toy offers an excellent solution for parents who are looking for healthier alternatives to screen time, helping their kids develop creativity and imagination while staying away from smartphones and other digital devices.

This toy features LED lights for the eyes, a display for the mouth to mimic lip movements, speakers for audio output, and ultrasonic sensors for measuring distance. It is equipped with four wheels for mobility, a DC power supply for battery charging, and an SD card slot for customizable audio playback.

CAD Design

I made a possible CAD design for my project in week 2 and it probably looks like this.


Voice Input

I tried to add microphone to my project so that I can interact with it using our voice as well. So during my Input devices week I made a board using INMP441 MEMS microphone. Tested and programmed it and it works well.

Schematics
Board Design

I included 3 LED's on the board to use it as an equiliser from the input recieved from the microphone. Hopefully it works fine and the video is included here,


Servomotor

As it is a talking toy, Speaker is the essential item in it. Also I am trying to impliment a head movement as well as you saw on the above concept video. So during Output devices week I made a board using XiaoESP32S3 to control speaker and Servomotor. As the microcontroller doesn't have I2S suport I needed to add an extra I2S converter IC PT8211 into the circuit.

I successfully control the servomotor but due to some issues with my board I think I can't able to play the audio.


Voice Recognition

As I needed to make the toy as a voice actuated one, our Instructor introduced me Grove- Offline Voice Recognition Module. Which contains a number of pre trained words so that we can assign functions for each or we can add our own trained words also. Upto 150 trained words can be stored in the module.

Image taken from amazon.

I followed Seed studio and hackster.io for getting started with the grove module. It uses UART communication protocol.

Connections


I tried to Turn ON and OFF the onboard LED on the ESP32 S3 when I say Cooler and Warmer. The wake up word for the module was "Hey Pudding". My fello batch mates were too irritated while I tested this thing.😁 😁

With the help of ChatGPT I wrote a code for it.


    
        #include 

#define SOFTWARE_RX 44   // RX of Grove Voice Module
#define SOFTWARE_TX 43   // TX of Grove Voice Module
#define LED_PIN 21       // User LED pin

SoftwareSerial Grove_Sensor(SOFTWARE_RX, SOFTWARE_TX); // RX, TX

const byte CMD_LENGTH = 5;
byte buffer[CMD_LENGTH];
byte index = 0;

void setup() 
{
  Serial.begin(9600);
  while (!Serial);

  Serial.println("USB Serial Port connection Established!");

  Grove_Sensor.begin(115200);

  pinMode(LED_PIN, OUTPUT);
  digitalWrite(LED_PIN, LOW); // Initially OFF
}

void loop() 
{
  while (Grove_Sensor.available()) 
  {
    byte incoming = Grove_Sensor.read();
    Serial.print(incoming, HEX);
    Serial.print(" ");

    // Store into buffer
    buffer[index++] = incoming;

    // If 5 bytes received
    if (index == CMD_LENGTH)
    {
      // Check for "warmer" command
      if (buffer[0] == 0x5A && buffer[1] == 0x20 && buffer[2] == 0x00 && buffer[3] == 0x00 && buffer[4] == 0x7A)
      {
        digitalWrite(LED_PIN, HIGH); // Turn LED ON
        Serial.println(" -> Warmer Command Recognized: LED ON");
      }

      // Check for "cooler" command
      else if (buffer[0] == 0x5A && buffer[1] == 0x21 && buffer[2] == 0x00 && buffer[3] == 0x00 && buffer[4] == 0x7B)
      {
        digitalWrite(LED_PIN, LOW); // Turn LED OFF
        Serial.println(" -> Cooler Command Recognized: LED OFF");
      }

      // Reset buffer index for next command
      index = 0;
    }
  }

  delay(10);
}

ChatGPT Promt: I have a groove voice recognition module and an ESP32s3. While i say warmer the user led on esp32s3 needed to be turn on and on cooler it needed to be turned off. led pin is 21


Then I tried to integrate Servo with this. While the light turn ON the servo needed to turn to 0 and while light turn OFF the servo needed to turn 180.

  
    #include <SoftwareSerial.h>
#include <ESP32Servo.h>

// Pin Definitions
#define VOICE_RX 44         // Grove module TX → ESP32 RX
#define VOICE_TX 43         // Grove module RX → ESP32 TX
#define LED_PIN 21
#define SERVO_PIN 2

// Command hex values received from Grove module
#define CMD_WARMER 0x6AFF
#define CMD_COOLER 0x6BFF

// Servo pulse range
#define PULSEMIN 500
#define PULSEMAX 2500

// Create instances
SoftwareSerial Grove_Sensor(VOICE_RX, VOICE_TX);
Servo myServo;

void setup() {
  Serial.begin(115200);
  Grove_Sensor.begin(115200);

  pinMode(LED_PIN, OUTPUT);
  digitalWrite(LED_PIN, LOW);  // Start with LED OFF

  // Attach servo with defined pulse range
  ESP32PWM::allocateTimer(0);  // Optional but good practice
  if (myServo.attach(SERVO_PIN, PULSEMIN, PULSEMAX)) {
    myServo.write(0);  // Initial position
    Serial.println("✅ Servo attached to pin 2.");
  } else {
    Serial.println("❌ Servo failed to attach!");
  }

  Serial.println("🟢 System initialized.");
}

void loop() {
  static uint8_t buffer[2];
  static uint8_t index = 0;

  // Read command bytes from Grove voice module
  while (Grove_Sensor.available()) {
    buffer[index++] = Grove_Sensor.read();

    if (index >= 2) {
      // Combine two bytes into one command
      uint16_t command = (buffer[0] <<< 8) | buffer[1];
      Serial.print("🎙️ Command Received: 0x");
      Serial.println(command, HEX);
      index = 0;

      // Match commands
      if (command == CMD_WARMER) {
        digitalWrite(LED_PIN, HIGH);
        myServo.write(180);
        Serial.println("✅ Warmer command: LED ON, Servo 180°");
      } else if (command == CMD_COOLER) {
        digitalWrite(LED_PIN, LOW);
        myServo.write(0);
        Serial.println("✅ Cooler command: LED OFF, Servo 0°");
      } else {
        Serial.println("⚠️ Unknown command received.");
      }
    }
  }

  delay(5);  // Small delay for stability
}

Block Diagram

As mentioned above, my project is a talking toy. So it needed a speaker. Audio is loaded using an SD card. Push buttons are needed for controls. For the head movement servomotors are needed. Microphones are for interacting it with voice and the lip movements are simulated by OLED display. A battery is needed for power power suppy.


User Flow Diagram

Bill Of Materials

These are the possible bill of materials for my project.


System Integration

I finalised my design in system integration week. I finalised each components fit where on my project.