Final Project!¶

Week 1¶

Have some Inital Ideas for my final project that I wish to complete in this course of fabacademy

Initial Ideas¶

1. Problem Statement:

In the contemporary world, mental health has become a paramount concern. The fast-paced lifestyle, coupled with various stressors, often leads to challenges in maintaining optimal mental well-being. The lack of immediate and personalized interventions can exacerbate mental health issues. Recognizing the importance of positive interactions and emotional support, there is a need for innovative solutions that provide instant and tailored emotional upliftment to individuals.

Proposed Solution - Mood Mirror:¶

The proposed solution to address the pressing issue of mental health is the “Mood Mirror.” This interactive mirror incorporates a display interface and a camera sensor. When an individual stands in front of the mirror, the embedded machine learning algorithm analyzes facial expressions to determine the person’s emotional state—whether happy, sad, stressed, or any other sentiment. In instances where the mirror detects a less positive emotional state, it responds by displaying uplifting content. For instance, if the person appears sad, the mirror can showcase a funny or motivational quote directly on its surface. By providing instant and personalized emotional support, the Mood Mirror aims to contribute positively to an individual’s mental well-being, promoting moments of joy and consideration in their daily lives.

2. Problem Statement

In the hustle and bustle of today’s fast-paced life, people have become increasingly detached from nature. Extended hours in office spaces and urban environments have contributed to a significant loss of connection with the natural world. This disconnection can lead to stress, fatigue, and a diminished sense of well-being, highlighting the need for innovative solutions that reintroduce individuals to the tranquility and beauty of nature.

Proposed Solution - Art-tech Installation:¶

To address the aforementioned problem, the proposed solution is an “Art-tech Installation.” This immersive installation features suspended spheres or birds, each encapsulating a representation of the changing seasons. As individuals move through the installation, sensors detect their presence, triggering a harmonious symphony of nature-inspired sounds. The immersive auditory experience includes the gentle chirping of birds, the soothing flow of water from a stream, the melodic conversations of animals, and the serene silence of the jungle. By blending technology with art, this installation aims to transport individuals momentarily to the heart of nature, offering a sensory escape from the daily chaos and providing a therapeutic connection to the natural world.

Selected topic¶

Out of the two topics above after discussing with my mentors finally selected the first problem statement regarding the mental health, The solution being the “”Mood Mirror”

What the Mood Mirror is¶

“Mood Mirror” This interactive mirror incorporates a display interface and a camera sensor. When an individual stands in front of the mirror, the embedded machine learning algorithm analyzes facial expressions to determine the person’s emotional state—whether happy, sad, stressed, or any other sentiment. In instances where the mirror detects a less positive emotional state, it responds by displaying uplifting content. For instance, if the person appears sad, the mirror can showcase a funny or motivational quote directly on its surface. By providing instant and personalized emotional support, the Mood Mirror aims to contribute positively to an individual’s mental well-being, promoting moments of joy and consideration in their daily lives.

For Whom I am Making this Mood Mirror¶

The mood detection mirror I am creating serves a diverse range of users, each with unique needs and interests. At its core, this innovative mirror is designed to offer a captivating and interactive experience for individuals seeking to engage with technology in novel ways. For those at home, it provides a playful yet insightful tool for self-reflection and emotional awareness, enhancing their daily routines with a touch of modernity. In public spaces, the mirror becomes a focal point, offering passersby an engaging moment of self-discovery and perhaps sparking conversations around the intersection of technology and human emotion. Within therapeutic settings, mental health professionals can utilize the mirror as a valuable resource, empowering clients to track their moods and facilitating discussions about emotional well-being.

This is the basic sketch for hou my mirror whould look like.

Schedule for Completion of Final project¶

Below is the detailed schedule for the final project :

Task Descriptions¶

Design and Prototyping¶

Finalize Design: Ensure the design of the mirror frame and mounts are complete and detailed.
3D Models and Prototypes: Create 3D models of mounts and prototype them using 3D printing.
Component Fit Verification: Verify that all components fit within the frame and are securely held.

Hardware Assembly¶

Frame Assembly: Assemble the MDF frame and attach the mirror acrylic.
Component Installation: Secure the small LCD display, LED strip, ultrasonic sensor.
Connection Verification: Ensure all components are correctly positioned and connected.

Initial Software Development¶

ML Model Development: Develop the initial machine learning model on the Edge Impulse platform.
ESP32-CAM Programming: Program the ESP32-CAM with the ML model to detect emotions.
Firmware Development: Write firmware for the Seed Xiao RP2040 to handle sensor inputs and control the LED strip.
Basic Functionality Implementation: Implement basic functionalities and interactions with the push buttons.

ML Testing¶

Dataset Testing: Test the ML model with a variety of datasets to ensure accuracy.
Accuracy Analysis: Analyze the model’s accuracy in detecting different emotions.
Parameter Fine-tuning: Fine-tune the model parameters to improve performance.

ML Code Finalizing¶

Code Optimization: Optimize the ML code for efficient processing on the ESP32-CAM.
Latency Reduction: Ensure the model runs smoothly without significant latency.
Software Integration: Finalize integration of the ML code with the rest of the system’s software.

System Integration¶

Component Integration: Integrate the ESP32-CAM, Seed Xiao RP2040, and other components with the finalized software.
Communication Testing: Ensure all components communicate effectively and respond as expected.
Initial Testing: Perform initial tests to identify any integration issues.

Testing and Calibration¶

System Testing: Conduct comprehensive testing of the entire system in real-world scenarios.
Sensor Calibration: Calibrate the ultrasonic sensor for accurate presence detection.
Component Adjustment: Adjust the brightness of the LED strip and volume of the speaker.
User Feedback: Collect user feedback and refine the system based on the input.

Documentation and Presentation¶

Process Documentation: Document the development process, including design choices, challenges, and solutions.
Final Presentation: Prepare a final presentation that highlights the Mood Mirror’s features, functionality, and potential impact. Include visual aids such as photos, diagrams, and videos to enhance the presentation.

Meeting with Local Instructor¶

Regular Reviews: Schedule regular meetings with your local instructor to review progress.
Feedback Incorporation: Gather feedback on each completed task and make necessary adjustments.
Milestone Check-ins: Ensure all milestones are met according to the schedule and address any challenges promptly.

Bill Of Materials (BOM)¶

Estimated cost breakdown in INR:

ESP32-CAM: ₹472
Seed Xiao RP2040: ₹500
Ultrasonic Sensor: ₹100
LED Strip: ₹300
Small LCD Display: ₹200
Mirror Acrylic: ₹800
Frame Material (12 mm MDF): ₹500
Acrylic Enclosure: ₹600
3D Printing Filament: ₹1500
Relay Switch: ₹200
Miscellaneous: ₹500
Total Estimated Cost: ₹5672

Here is the BOM for my Project so that it is much more clear BOM for FP

Machine learning TinyML (EdgeImpulse)¶

Introduction to Edge Impulse¶

Edge Impulse is a user-friendly platform that makes it easy to create and deploy machine learning (ML) models on small, edge devices like microcontrollers and IoT gadgets. It provides tools to collect and process data, design and train models, and then run them directly on the device. This allows for real-time data analysis and applications like predictive maintenance and anomaly detection, bringing smart, AI-driven capabilities to even the most resource-limited devices in everyday products and industrial tools.belwo is the process of how made my first ever ML model.

Process of making ML model¶

This is the basic Interface of the Edge Impulse software where you make your own ML Model. On the left side there is a pannel which has steps to create your ML model.

The first step to make a a ML model is to add data of images you want for making your neural network. you can directly add file from your PC or you can add individual images. To make your work even much easier when you add the images there you can directly label the images as you want to differentiate between them . In my case I labelled the first set as happy as they were all happy images and the second set as sad as they were all sad images. (Note: Also if you want ready datasets for you ML model you can go to keggle a website where you can get lot of datasets.)

As you can see below I created two folders below named happy and sad.

After adding them Ijust labelled them and uploaded all the images.

Below you can see how the images are now labelled and uploaded in the dataset dashboard. you need to keep a split between training and testing so that the network can reflect itself after training the model to see if the trained model is good in recognising the two different datasets. the split that I had kept in my model to train and test was 82 and 18 respectively.

The next step is to add processing blocks and learning blocks for you Machine learning model and see if the output features matches with the datasets that you have uploaded.

The next step is to set the parameters for the parameters for the images that we have uploaded, As I am doing ML in a microcontroller I have set the images into grayscale as the size of images reduces and easier for the microcontroller to process.

The next step is to generate the featuures with the data set we have uploaded.

Once this step on the left side dashboard you will see that the under impulse design you have completed the two main steps of creating impulse and image.

Now the next step is to select the architecture for the neural network for our ML model. There are different architectures, Salman Faris who mentored me in my first ML project suggested me to select - (mobile netowrk V1 96x96 0.25) for the micro-controller to take the load of the ML model.

Below are the different architectures that are available for your neural network.

After this you hit the train button and this is the output I got. my model was 75% accurate in recognising happy and sad faces.

Once you are done with the training the panel on the left side turns green on all three headings and that tells that your model is ready to use now.

Now your model is ready! you just go in the devlopment section from the panel in your left and then select on the arduino library as shown in the image below (As I am using arduino library)

and then download the library! done you have your ML model now in a custom arduino library that you just created!!

Electronics¶

There were 2 circuits and PCB’s that were involved in this project one PCB was for the led strip,Relay swtich and the ultrasonic sensor while the other was for the LCD display and machine learning as the camera was in built in the micro-controller it got easier.

First circuit¶

The Micro-controller used for this circuit is Seed xiao rp2040 and the PCB that I have used for this is the one that I had produced in my electronic design week.

This is the PCB I used for ultrasonic, relay switch and the led strip.

The code for the relay, Ultrasonic and led strip

#define TRIG_PIN D2
#define ECHO_PIN D3
#define RELAY_PIN D4

void setup() {
  Serial.begin(9600);

  pinMode(TRIG_PIN, OUTPUT);
  pinMode(ECHO_PIN, INPUT);
  pinMode(RELAY_PIN, OUTPUT);

  digitalWrite(RELAY_PIN, LOW);
}

long getDistance() {
  digitalWrite(TRIG_PIN, LOW);
  delayMicroseconds(2);
  digitalWrite(TRIG_PIN, HIGH);
  delayMicroseconds(10);
  digitalWrite(TRIG_PIN, LOW);

  long duration = pulseIn(ECHO_PIN, HIGH);
  long distance = (duration / 2) / 29.1; // Convert to cm

  return distance;
}

void loop() {
  long distance = getDistance();
  Serial.print("Distance: ");
  Serial.print(distance);
  Serial.println(" cm");

  if (distance < 100) { // Adjust threshold distance as needed
    digitalWrite(RELAY_PIN, HIGH);
  } else {
    digitalWrite(RELAY_PIN, LOW);
  }

  delay(1000);
}

Testing¶

after the code was done I integrated the ultrasonic, LED and the relay switch with the frame of the mirror and here is how it came out to be

And it was working good!!!

Second circuit¶

The micro controller that I have used is a AI thinker ESP32-CAM board. As it is a devlopment board I have designed a break out board for the ESP32-CAM board. I first designed the PCB and then milled it with SRM-20. Let me show you the process of making the PCB.

PCB design¶

The software that I have used is EasyEDA

First I made the schematic of the PCB I wanted, added all the components in the PCB and then added connections to it.

Then I converted the schematic into PCB design and then placed all the components inside the rectangle that I had created as the outline of the PCB

The Instead I autorouting it I routed the whole PCB manually and it was done the PCB was now ready to be milled with the gerber file.

This is how the 3D file of the PCB looks like.

PCB milling¶

Next I downloaded the gerber file and then milled the PCB with SRM-20 this is how the ouput looks like!

After the PCB was milled I then soldered it with all the components and now the PCB was ready to be used!

Now the PCB is ready to use! I have not gone into very much detail of producing a PCB as I have already put the process in the Electronic design and electronic production week. Here is the link for both of the weeks

Electronic Production

Electronic Design

The next step was to see how the library that we have made for our Machine learning works or not with the camera of the microcontroller itself.

ML Code with ESP32-CAM¶

for this the library that we have made with edge impulse we have to first integrate it in the arduino IDE here are the steps of how you add a custom library to your arduino IDE

first step is to open the arduino IDE this is the interface of arduino IDE when you open it.

the next step is to go to sketch < include library < add zip library

next it will a tab where you can see documents of your PC select the zip ile that you have downloaded through the edge impulse.

now the zip file is added! next you go to the file and go in examples under examples there is a section of custom libraries, select the custom library of the ML and select esp32 and under that select esp32-CAM. as shown in the above image.

now you are done the ML code is up on the arduino IDE just select the board you are using and upload the code. I had many difficulties uploading this but at the end it did work!

ML Working on ESP32-CAM¶

ESP 32-CAM with LCD¶

Now that the ML was figured out I had to see how do we connect the ESP 32 CAM with the LCD to get the quotes on it for this I would like to thank Adrian who helped me in figuring out the SCL and SDA pins of the ESP32CAM. So the ESP 32 CAM has the SDA and SCL pins as GPIO 24 and 25 but they were not there on the board! I was really confused about what to do further and asked help from adrian, he told me that there is a I2C address that every component has I first had to get the address from the ESP 32 CAM and then I could just assign any pins as SDA and SCL in the code and the code would work. and it did work finally I was able to get the quote on the display.

sharing somelinks that really helped me for this task

I2C tutorial 1

I2C tutorial 2

Here is how my test output came along

this is how it looked when I integrated it with the mirror and the frame.

2D Designs in my Final Project (2D Designs)¶

In my Final Project below are the topics where I have used 2D Design : (ALL THE FILES ARE AT THE END OF THE PAGE)

Mirror Acrylic
Back cover
MDF rings (System Integration)

Mirror Acrylic¶

This was one of the main part of my mirror because this was the part in which my few components were there and it was the face of my mirror. In my MDF frame that held the mirror in the backside of the frame I had a slit that I had cut for the mirror to be placed in the slit.

In my mirror there were 3 components that had to pop out from the mirror so that my mirror works

Camera (For ML)
LCD Display (For the quote output)
Ultrasonic Sensor (detection for LED strip)

My first step to make the design was to take the dimensions of all the components, Once I took all the dimensions I placed it in such a way that a person could see his face and the quote comes below. I made the design on Adobe Illustrator Here is how it looks.

After this design was done I quickly exported it into dxf and did used the laser cutter to cut it this is how the mirror looked.

Back Cover¶

My mirror needed a enclosure from the back which can be removed if anything had to be repaired. and also there were some components that had to be mounted on the back cover as a part of my integration. The MDF rims that were cut in order to give depth to store electronics I had 4 holes drilled in it so that I can scre the back cover to the rings and it all looks like a sigle piece.

The components that were mounted on my back cover are :

PCB of the Seed xiao rp2040
Relay switch
Wall mount (on the other side)

I started with taking all the measurements of the components that were going on my back cover for the holes so that I could tighten it with the nut and bolt. This is how the design of my back cover looks like.

This is how it looked after I did the laser cutting and integrating the components

MDF Rings¶

These MDF rings were one of the most important part of my Integration.

These are the rings that I am talking about, The rings were added on the back side of the frame because when I saw the electronics that was going to go inside the mirror the frame did not have the required depth for my electronics. I took the measurement of the the component which had the maximum height out of all which was the relay swtich. I added 3 rings of MDF 5mm each that gave me a additional space of 15 mm in total to store my electronics. Also it helped me as the led strip got a proper base to stick on the rings.

This is how the 2D design looks like

This is how the backside of the mirror looks like with the components.

CAD Designs in my final Project (3D Designs)¶

In my final Project below are the topics where I have Used CAD modeling :(ALL THE FILES ARE AT THE END OF THE PAGE)

Frame for my mirror
PCB mounts
Wire Inserts (System Integration)
Mount for the Mirror

Frame for my mirror¶

For the frame of the mirror I wanted it to be asthetically apealing as it was the face of my product, hence went with selecting a very elegant design for my frame, Below is the design for the frame of the mirror.

This frame actually came out pretty nice after the model was done milled it in the CNC where it was my first time doing a 3D cut in the CNC.

Here is how the frame look like after the post processing with primer and paint.

PCB mounts¶

So as a part of my system integration I made mounts for my PCB that I used in my final project.

This is how my mount looked and then I 3D printed the mount and the PCB was installed very easily and looked neat! this is how the print came out.

Here is how the slot for data cable had a easy way in and out.

Wire Insterts¶

Okay let me just say this out loud, as small as this part is in my Product, It is the most useful and important part in my project as a part of my system integration as well. It held the wires on the rims of the mirrors so that there is no hassle of the wires. I didnt plan on doing this TBH but just thought of if it I could do this and it definetly worked for me.

Here is how it looks and let me show you how it works

This small thing really helped me a lot for not getting confused in my own project.

Mount for the mirror¶

So for my mirror for being able to be there on the wall I had 3D printed a wall mount which I had screwed to the back of my mirror. With this mount I can hang the mirror with just one nail anywhere needed.

This is how the wall mount works!

This is how it was integrated to my project

System Integration¶

How I went along¶

Well well, What all can I even say about system integration, To be honest I had not considered system integration in my final project for a long time and the finally when the whole project started to come toghether I eventually found a way to atleast for my product to look like a packaged single product. For this I had to keep in mind a lot of things, lets start with the electronics hardware that I had inside my mirror.

Hardware Used in the Mood Mirror Project¶

ESP32-CAM
Function: Captures the user’s facial expressions and processes them using a machine learning model to detect emotions.
Seed Xiao RP2040
Function: Controls the LED strip and ultrasonic sensor, managing the power supply to the LED strip via a relay switch.
Ultrasonic Sensor
Function: Detects the presence of a person in front of the mirror and sends a signal to the Seed Xiao RP2040 to activate the LED strip.
LED Strip
Function: Provides ambient lighting that enhances the user experience, controlled by the Seed Xiao RP2040 via a relay switch to light up when the ultrasonic sensor detects someone.
Small LCD Display
Function: Displays funny or motivational quotes based on the detected emotion, receiving data from the ESP32-CAM which processes the emotion detection.
Mirror Acrylic
Function: Serves as the reflective surface of the mirror, laser-cut to fit perfectly within the frame.
12 mm MDF
Function: Material for the frame of the mirror, providing structural support and housing all components securely.
Acrylic Enclosure
Function: Conceals the internal components from behind, providing a clean and aesthetically pleasing look.
3D Printing Filament
Function: Used for printing mounts and other components to secure the hardware within the frame.
Relay Switch
- Function: Controls the LED strip, allowing it to turn on and off based on signals from the Seed Xiao RP2040.
Miscellaneous Components
- Function: Includes wiring, connectors, and soldering materials necessary for assembling the hardware components.

Summary¶

The Mood Mirror project integrates various hardware components, including microcontrollers, sensors, displays, and structural materials. Each component plays a critical role in ensuring the mirror’s functionality, from detecting emotions to providing visual feedback and maintaining an aesthetically pleasing design.

MDF Rings¶

Fisrt I saw what all components are going to go Inide of my Mirror and accordingly I took MDF and got some rings to go behind the mirror like this

Let me tell you the reason that went behind cutting out the rings, After I saw all the electronics That I was going to use in my Final project I tried to place it on the back of the mirror but then I realised that the back side does not have the amount of depth that I need to have for my electronics and hence I got the rings cut out by taking the height of the relay module as it was the biggest of all the other components that I had which were going inside the mirror. I have also made hole in the mirror for the data cable to go in and power the Microcontroller.

The MDF rings also helped me for the led strips the led strips got a base to stick properly. Also as we can see in the image there is a hole in the rings so that the data cables can pass through.

Mounts used for all the components¶

PCB mounts

So as a part of my system integration I made mounts for my PCB that I used in my final project.

This is how my mount looked and then I 3D printed the mount and the PCB was installed very easily and looked neat! this is how the print came out.

Here is how the slot for data cable had a easy way in and out.

Back Cover Mounts

I also have added mounts on the back cover for the components this is how it looks!

Wire Inserts

It held the wires on the rims of the mirrors so that there is no hassle of the wires. I didnt plan on doing this TBH but just thought of if it I could do this and it definetly worked for me.

Here is how it looks and let me show you how it works

This small thing really helped me a lot for not getting confused in my own project.

Project Development¶

The project devlopment was all about tracking the progress of our Final project, There was a schedule that I had created for the completion of my final project during the mid term break.

This is the schedule I had decided for the completion and it pretty much went in this direction but somethings took way much longer than expected. I tracked the dates in a excel sheet that I had created of the same schedule where I would just overlay green when the task was done.

This is how I would mark each task to be complete or pending.Tracking sheet

what’s working? what’s not?¶

This is a really important question and a very valid question so the Machine learning model that I created was working really fine on the phone and on the laptop but when you try to embed it into a micro controller things really go out of your hands. eventually everything came together but for a long time what wasnt working was the ML on the micro-controller. Before the ESP32 CAM I was working with seed xiao esp32S3 Sense. Uploading the ML library to this micro-controller was failing everytime when I uploaded it even Salman faris my mentor for ML could not resolve about what was actually happening. Below are the screenshots of the errors I was reciving.

Eventually what worked for me was the decision to change the microcontroller, I changed the microcontroller from the SEED XIAO ESP32S3 Sense to AI thinker ESP 32 CAM and it worked on the first go with the same code I had. I was so relived when it worked cant express in words. Below is the screenshot of the ML working

The time that I made to order a new micro-controller with the lead time of 2 days to reach home was really cut to cut. If I had not made the decision of switching the micro-controller 2 weeks before the presentation date I think my final project would be incomplete.

what have you learned?¶

What I learnt, can really sum up everything in this simple question. I learnt a lot! Fabacademy has really taught me a lot of things. devloping the skillsets that I have surely but a lot more than that. Fabacademy taught me descipline in my work , It taught me the importance of documenting the work you do. It taught me how important peer to peer network is and most importantly Fabacademy taught how to be better and never stop.