Final Project Overall Summary

My proposed project, “Door Butler,” involves implementing a system where an ultrasonic sensor detects individuals near a door and sends alert notifications to the App on a phone. Simultaneously, a Xiao ESP32 camera module captures images. These images are then transmitted to an app, allowing me to verify who is at the door via the Blynk app and react accordingly by playing a sound through a speaker.

Final summary slide:

Final Project video:

Youtube Link for my final project video

FFmpeg command I used to compressed my video:

ffmpeg -i sangayfinal.mp4 -vcodec libx264 -crf 34 -preset veryslow -vf scale=-2:1080 -acodec libmp3lame -q:a 4 -ar 48000 -ac 2 presentation.mp4

Electronic Design & Production

For my final board, I took inspiration from Quentorres example and utilized KiCad for the PCB design. In this version, I integrated an Xiao_ESP32_S3 Sense MCU along with alongside various breakout pins.

Below you can see final version PCB schematic from KiCad:

The components I Used:

• 1X Seeed Studio XIAO ESP32S3.
• 1X CONN HEADER SMD R/A 6POS 2.54MM
• 1X Tactile Switch
• 3X LED BLUE CLEAR 1206 SMD/PTH
• 5X RES 1K OHM 1% 1/4W 1206
• RES 499 OHM 1% 1/4W 1206
• 2X CONN HDR 7POS 0.1 TIN SMD*
• 1X 1x7 Male connector
• 1X 1x4 Female connector

Here’s the final PCB layout in KiCad and finalboard:

Here’s video and image for the test done integrating Xiao Camera, Blynk App and Ultrasonic Sensor:

Below you can find integration test for all components

Final Demonstration Video:

For more details go to this Link

Packaging Design & Manufacturing

For the final version of my packaging design, after thoroughly calculating the required dimensions of the components that need to fit inside the box, I designed the following parts:

• Bottom Part: This will be 3D printed to hold the PCB securely.
• Middle Parts: These will be created using laser-cut 3mm plywood with finger joints for easy assembly and then painting to look it better.
• Top Part: This will also be 3D printed which includes designated spaces for mounting a speaker, ultrasonic sensor, LED, and camera.
• And finally, I pasted stickers where I got design from google source using vinyl cutting machine to looks better aesthetically.

Click here to learn more about my packaging desing and fabrication.

Final Programming

#include "esp_camera.h"
#include <WiFi.h>
#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM
#include "camera_pins.h"
#define BLYNK_PRINT Serial
/* Fill in information from Blynk Device Info here */
#define BLYNK_TEMPLATE_ID "TMPL3GrjQ3f1a"
#define BLYNK_TEMPLATE_NAME "SangayFA24"
#define BLYNK_AUTH_TOKEN "Sz2zq8TPagtaKguH3LhxaWlMGhrWVZR_"
#include<DFRobotDFPlayerMini.h>
#include<SoftwareSerial.h>

SoftwareSerial mySerial(3,4); // Initialize software serial on pins 3 (RX) and 4 (TX)
DFRobotDFPlayerMini myDFPlayer; // Initialize DFPlayer Mini object

#include <WiFi.h>
#include <WiFiClient.h>
#include <BlynkSimpleEsp32.h>

#define LED 1 // LED pin
#define BUTTON 2 // Button pin
#define trigPin 8 // TRIG pin for ultrasonic sensor
#define echoPin 7 // ECHO pin for ultrasonic sensor
#define BUTTON_Play 9 // Button pin for audio 1
#define BUTTON_Play2 6 // Button pin for audio 2
#define BUTTON_Play3 5 // Button pin for audio 3

int buttonPushCounter = 0; // Counter for button presses
int buttonState = 0; // Current state of the button
int lastButtonState = 0; // Previous state of the button

int buttonPushCounter_2 = 0; // Counter for button 2 presses
int buttonState_2 = 0; // Current state of button 2
int lastButtonState_2 = 0; // Previous state of button 2

int buttonPushCounter_3 = 0; // Counter for button 3 presses
int buttonState_3 = 0; // Current state of button 3
int lastButtonState_3 = 0; // Previous state of button 3

String my_Local_IP; // Store the local IP address

const char* ssid = "Fablab"; // WiFi SSID
const char* password = "JnwSf!2023"; // WiFi password
int count = 0; // Counter for something (not used in the code)
void startCameraServer(); // Function declaration to start the camera server
void setupLedFlash(int pin); // Function declaration to setup LED flash

BlynkTimer timer; // Initialize Blynk timer

boolean flag = true; // Flag for notification
float duration_us, distance_cm; // Variables for ultrasonic sensor readings

unsigned long previousMillis = 0; // Store last time LED was blinked
const long period = 10000; // Period at which to blink in ms

void notifyOnButtonPress()
{
  unsigned long currentMillis = millis(); // Store the current time
  if (currentMillis - previousMillis >= period || flag == true) { // Check if period passed or flag is true
    previousMillis = currentMillis; // Save the last time LED was blinked
    Serial.println("Alert! : Some at the Door");
    Blynk.logEvent("Intruder_Alert","Intruder Alert in Home"); // Log event to Blynk
    flag = false; // Reset flag
  }
}

void capture(){
  digitalWrite(LED,HIGH); // Turn on LED
  uint32_t number = random(40000000); // Generate a random number
  Serial.println("http://"+my_Local_IP+"/capture?_cb="+ (String)number); // Print capture URL
  Blynk.setProperty(V1, "urls", "http://"+my_Local_IP+"/capture?_cb="+(String)number); // Set Blynk property
  delay(1000); // Wait for 1 second
  digitalWrite(LED,LOW); // Turn off LED
}

bool flag_play = true; // Flag for playing audio 1
bool flag_play_2 = true; // Flag for playing audio 2
bool flag_play_3 = true; // Flag for playing audio 3

void playAudio(int x){
  Serial.println("Play: ");
  myDFPlayer.volume(30); // Set volume (0~30)
  if(x == 1 && flag_play_2 == true){
    Serial.println("Play 1 ");
    myDFPlayer.play(1); // Play the first mp3 file
    flag_play_2 = false; // Reset flag
  }
  if(x == 2 && flag_play == true){
    Serial.println("Play 2 ");
    myDFPlayer.play(2); // Play the second mp3 file
    flag_play = false; // Reset flag
  }
  if(x == 3 && flag_play_3 == true){
    Serial.println("Play 3 ");
    myDFPlayer.play(3); // Play the third mp3 file
    flag_play = false; // Reset flag
  }
}

void setup() {
  Serial.begin(115200); // Initialize serial communication
  pinMode(LED,OUTPUT); // Set LED pin as output
  pinMode(trigPin, OUTPUT); // Set TRIG pin as output
  pinMode(echoPin, INPUT); // Set ECHO pin as input
  pinMode(BUTTON_Play2, INPUT); // Set button 2 pin as input

  mySerial.begin(9600); // Initialize software serial communication
  if (!myDFPlayer.begin(mySerial)) { // Initialize DFPlayer
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1. Please recheck the connection!"));
    Serial.println(F("2. Please insert the SD card!"));
    while (true); // Stay in loop if DFPlayer initialization fails
  }

  Blynk.begin(BLYNK_AUTH_TOKEN, ssid, password); // Initialize Blynk

  Serial.setDebugOutput(true); // Enable debug output
  Serial.println();

  camera_config_t config; // Camera configuration
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sccb_sda = SIOD_GPIO_NUM;
  config.pin_sccb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.frame_size = FRAMESIZE_UXGA;
  config.pixel_format = PIXFORMAT_JPEG; // For streaming
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 12;
  config.fb_count = 1;

  if(config.pixel_format == PIXFORMAT_JPEG){
    if(psramFound()){ // If PSRAM is available
      config.jpeg_quality = 10; // Set higher quality
      config.fb_count = 2; // Set frame buffer count
      config.grab_mode = CAMERA_GRAB_LATEST;
    } else { // If PSRAM is not available
      config.frame_size = FRAMESIZE_SVGA; // Limit the frame size
      config.fb_location = CAMERA_FB_IN_DRAM;
    }
  } else {
    config.frame_size = FRAMESIZE_240X240; // Best option for face detection/recognition
#if CONFIG_IDF_TARGET_ESP32S3
    config.fb_count = 2;
#endif
  }

#if defined(CAMERA_MODEL_ESP_EYE)
  pinMode(13, INPUT_PULLUP); // Set pins as input with pull-up for ESP-EYE model
  pinMode(14, INPUT_PULLUP);
#endif

  // Initialize camera
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return; // Exit if camera initialization fails
  }

  sensor_t * s = esp_camera_sensor_get();
  if (s->id.PID == OV3660_PID) {
    s->set_vflip(s, 1); // Flip vertically
    s->set_brightness(s, 1); // Increase brightness
    s->set_saturation(s, -2); // Decrease saturation
  }
  if(config.pixel_format == PIXFORMAT_JPEG){
    s->set_framesize(s, FRAMESIZE_QVGA); // Set frame size to QVGA
  }

#if defined(CAMERA_MODEL_M5STACK_WIDE) || defined(CAMERA_MODEL_M5STACK_ESP32CAM)
  s->set_vflip(s, 1); // Flip vertically
  s->set_hmirror(s, 1); // Mirror horizontally
#endif

#if defined(CAMERA_MODEL_ESP32S3_EYE)
  s->set_vflip(s, 1); // Flip vertically for ESP32S3-EYE
#endif

#if defined(LED_GPIO_NUM)
  setupLedFlash(LED_GPIO_NUM); // Setup LED flash if defined
#endif

  WiFi.begin(ssid, password); // Connect to WiFi
  WiFi.setSleep(false); // Disable WiFi sleep

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print("."); // Print dots while waiting for WiFi connection
  }
  Serial.println("");
  Serial.println("WiFi connected"); // WiFi connected

  startCameraServer(); // Start camera server

  Serial.print("Camera Ready! Use 'http://");
  Serial.print(WiFi.localIP()); // Print local IP address
  my_Local_IP = WiFi.localIP().toString(); // Store local IP address
  Serial.println("' to connect");

  timer.setInterval(5000L,notifyOnButtonPress); // Set timer to call notifyOnButtonPress every 5 seconds
}

void pauseAudio(){
  myDFPlayer.disableLoopAll(); // Stop looping all mp3 files
  delay(1000); // Wait for 1 second
}

void loop() {
  Blynk.run(); // Run Blynk
  timer.run(); // Run timer

  buttonState = digitalRead(BUTTON_Play); // Read button state
  buttonState_2 = digitalRead(BUTTON_Play2); // Read button 2 state

  digitalWrite(trigPin, HIGH); // Send a pulse to the TRIG pin
  delayMicroseconds(10); // Wait for 10 microseconds
  digitalWrite(trigPin, LOW); // Stop the pulse

  duration_us = pulseIn(echoPin, HIGH); // Read the echo pin

  distance_cm = 0.017 * duration_us; // Calculate distance in cm

  Serial.print("distance: ");
  Serial.print(distance_cm); // Print distance
  Serial.println(" cm");

  if(digitalRead(BUTTON) == HIGH){
    capture(); // Capture an image if the button is pressed
  }

  // Compare the button state to its previous state
  if (buttonState != lastButtonState) {
    // If the state has changed, increment the counter
    if (buttonState == HIGH) {
      // If the current state is HIGH then the button went from off to on
      playAudio(2); // Play audio 2
    } else {
      // If the current state is LOW then the button went from on to off
      Serial.println("off");
      pauseAudio(); // Pause audio
      flag_play = true; // Reset flag
    }
    delay(50); // Avoid bouncing
  }

  lastButtonState = buttonState; // Save the current state as the last state

  // Compare the button state to its previous state
  if (buttonState_2 != lastButtonState_2) {
    // If the state has changed, increment the counter
    if (buttonState_2 == HIGH) {
      // If the current state is HIGH then the button went from off to on
      playAudio(1); // Play audio 1
    } else {
      // If the current state is LOW then the button went from on to off
      Serial.println("off");
      pauseAudio(); // Pause audio
      flag_play_2 = true; // Reset flag
    }
    delay(50); // Avoid bouncing
  }

  lastButtonState_3 = buttonState_3; // Save the current state as the last state

  // Compare the button state to its previous state
  if (buttonState_3 != lastButtonState_3) {
    // If the state has changed, increment the counter
    if (buttonState_3 == HIGH) {
      // If the current state is HIGH then the button went from off to on
      playAudio(3); // Play audio 3
    } else {
      // If the current state is LOW then the button went from on to off
      Serial.println("off");
      pauseAudio(); // Pause audio
      flag_play_3 = true; // Reset flag
    }
    delay(50); // Avoid bouncing
  }

  lastButtonState_3 = buttonState_3; // Save the current state as the last state

  if(distance_cm <= 100){
    notifyOnButtonPress(); // Notify if an object is within 100 cm
  }

  delay(1000); // Wait for 1 second
}

For more information, go to this link.

Core Concept & BOM

• Purpose: Enhance security and convenience by providing real-time alerts and door monitoring systems.
• Target Users: Individuals and communities in Bhutan seeking affordable and effective door monitoring solutions.
• Cost Estimate: Estimated cost of production around 29.5 USD, making it accessible to a wide range of users.

Component	Quantity	Price ($)	Total Price ($)	Source
Seeed XIAO ESP32S3 Sense	1	$7.49	$7.49	JNWSFL
Speaker 4 ohm 3W Rectangle	1	$4.4	$4.4	JNWSFL
Resistors 1K ohm 1206 SMD	5	$0.01	$0.05	JNWSFL
Resistor 0 ohm 1206 SMD	4	$0.01	$0.04	JNWSFL
Res 499 ohm 1206 SMD	1	$0.01	$0.01	JNWSFL
SWITCH TACTILE SPST-NO	1	$1	$1	JNWSFL
LED blue clear 1206 SMD	3	$0.35	$1.05	JNWSFL
Header pins 2.54mm 4 pos	5	$0.1	$0.5	JNWSFL
5V USB adapter	1	$2.5	$3.2	JNWSFl
HC-SR04 Ultrasonic sensor	1	$3.95	$3.95	JNWSFL
3D print PLA	1	$2.1	$2.1	JNWSFL
3mm plywood	1	$1	$1	JNWSFL
Vinyl sticker	1	$0.5	$0.5	JNWSFL
DFPlayer Mini	1	$4.99	$4.99	Amazon

Total Price: $29.5/- or Nu.2460/-

Questions and Answers

Go to this Link for Applications and Implications Q & A.

Inspiration and What was done before me

I found inspiration from projects like the “Smart Door Bell” by Techiesms and the “Contactless Doorbell” by Viral Science on YouTube. They used ESP development boards and cameras but lacked proper packaging.

For my final project, I took it a step further. I designed a proper casing with 3D-printed parts and a laser-cut middle section. Plus, I created a custom PCB board. Instead of the typical ESP boards, I used the Xiao ESP32 S3 as the microcontroller and the Xiao ESP32 S3 Sense as the camera. Taking hint from these projects, I wrote the code to ensure my Door-Butler was both functional and good-looking.

CC License

Creative Commons licenses give everyone a way to grant the public permission to use their creative work under copyright law. To learn more go to the Creative Common’s websites for Licenses, and read through the given options.

From the different type of CC Licenses I opted for ATTRIBUTION-NONCOMMERCIAL-SHAREALIKE

I chose this license because I want my work to be copy and redistribute the material in any medium or format, remix, transform, and build upon the material with following terms: ShareAlike, appropriate credit must be given and not allowed to use the material for commercial purposes.

ATTRIBUTION-NONCOMMERCIAL-SHAREALIKE Deed:

Here is the Legal Code of CC BY-NC-SA

Design Files

Enclosure SolidWork Files

Programming Arduino Files

PCB Finalboard KiCad Files