Final Project
Final Presentation
Live Demos
Initial Final Project Idea
The Problem
Approximately 30% of Australians rent.
Most are not allowed to have pets or have difficulty getting approval from landlords to have a pet.
This means many renters miss out on the mental and physical health benefits of owning and caring for a pet.
Final Project Idea
To create an artificial pet that can provide comfort and motivation to its owner.
The pet will need to be able to:
- Provide opportunities for its owner to care for it
- Encourage its owner to exercise
- Know when it is being pat
- Be huggable
Sketch
Owner will be able to take the separate device on walks/runs if they do not want to take the whole pet with them
Pet will provide an alert when it:
- Needs charging
- Has been neglected for too long (hugs/pats required)
- When the owner has not taken it for a walk for too long
Extra ideas
- Being able to play games with the pet
- Adding an RFID reader into the pet and making different food and toys which when touched to the pet will result in different sounds and screen outputs
- Adding LED lights to the pet
Who is this for?
This will appeal to people who cannot own a pet themselves. It will be appropriate for both children and adults, although with its cuteness factor it will appeal to the young and young at heart.
Idea Name: Kokoro Friend
The Japanese Concept of Kokoro
Kokoro is a Japanese word that is often translated as “heart” or “spirit.” However, the concept of kokoro encompasses what a person thinks, feels, and would like to express. It is made of a person’s thoughts and will that drive them forward, the knowledge and experience they have acquired through time, and all their feelings.
Yet, kokoro is more than the sum of all these elements: it is all these elements at the same time. Each person’s features give their kokoro its own “color” and “shape,” making every human a distinct individual with a unique inner richness.
A person’s kokoro cannot be seen or touched:it resides in the area of the heart. Hidden in this invisible kokoro, lies a complex patchwork of ideas, concepts, and feelings that cannot be fully accessed from the outside.
One’s kokoro lives in everything he or she creates. The creation can be a material or conceptual.
Source: https://kokoro-jp.com/what-is-kokoro/
Inspiration
An example of bringing huggability and electronics together is Scream Body by Kelly Dobson https://web.media.mit.edu/~monster/screambody/
Fluf Bot by a Shinobu Ishimura Fab Academy Student https://fabacademy.org/2021/labs/kannai/students/shinobu-ishimura/projects/final-project/
Qoobo - commercial product out now https://qoobo.info/index-en/
Paro Therapeutic Robot - commercial product out now http://www.parorobots.com/
Furby - Inspiration from my childhood https://en.wikipedia.org/wiki/Furby
Tamagotchi - Inspiration from my childhood https://en.wikipedia.org/wiki/Tamagotchi
Render of possible device for interaction
From Week 2
Possible Electronics
Possible electronics input/output components (manufactured locally in Newcastle, Australia by Core Electronics) to go with Xiao RP2040 and custom designed board.
Visit to Piicodev Manufacturing Line February 2023
*Video from Core Electronics
Mid Term Review
Plan of items to include in Final Project toy and remote/console
Item | Toy | Console |
---|---|---|
PCB | Xiao RP2040, custom PCB | Xiao RP2040, custom PCB |
2D Design | Pattern for toy | |
3D Design | Nose and electronics case | Case |
Additive | Nose and case 3D print, casting eyes | 3D print case |
Subtractive | Mold for eyes, laser cut fabric | |
Input | Capacitive touch | Buttons, Accelerometer |
Output | Lights, sound | OLED |
Network | Transceiver | Transceiver |
Pakaging | Toy and internal electronic case | Case |
The plan will be to focus on completing the toy and to complete as much of the console as possible. The toy contains all the elements to pass the final project without the console, so this is extra.
Stuffed Toy
Eyes for Toy
I made a mold to make eyes out of UV resin for Kokoro. From Week 12, see that page for instructions.
I recast the eyes again, putting in the LEDs at the end to allow for me to be able to better hot glue them in place. I also added a little glitter into the purple resin to make it more sparkly. The eyes will be hot glued to the toy.
3D Printed Nose
I designed the nose in Fusion 360 and printed it on at Ultimaker S5. I created the design based on the experience I have gained over the course of Fab Academy using Fusion 360.
The nose will be hot glued to the toy and booping it will cause a response from the toy.
Laser Cutting and Engraving Fabric
During Wildcard Week I designed the pattern for Kokoro and cut and engraved sewing lines on the faux fur fabric using the laser cutter.
Final set of pieces
Sewing
During Wildcard Week I sewed Kokoro together including adding conducive steel thread to the ears to make them be able to be picked up using a capacitive touch sensor.
See Wildcard Week for detailed sewing instructions.
Electronics Selection
I originally thought about actuating the ears on the soft toy, but after some testing they were too heavy and the noise of the servos took away from the serenity of patting/hugging Kokoro.
The final list of electronics to be incorporated into the toy are:
- Xiao RP2040
- PiicoDev Capacitive Touch Sensor connected to conductive thread in ears, conducive fabric square and 3D printed nose
- Speaker and amplifier
- 2x addressable LEDs with a 1k resistor on 5V power and a 470 resistor on signal
- Flex sensor with 10k resistor
- PiicoDev Transceiver
The idea is that when you touch one of the capacitive touch elements (ears, back, boopable nose) that the eyes will light up and Kokoro will make a sound.
I also wanted to be able to detect hugs, after a bit of thinking and discussions with friends with more electronics experience we came up with the idea to incorporate a flex sensor into the belly, that will flex and therefore change resistance when the toy is hugged. This will also make the eyes light up and Kokoro will make a sound.
I have included the transceiver so that when a touch or a hug is detected this can be sent to the console. Also if in the future I want to make something happen with the toy using the console, I will be able to do this as the transceiver can work both ways.
The electronics will be powered by a powerbank that is easy to remove and recharge. In the future I want to explore replacing this with alkaline batteries which are safer to have in a toy like this.
Toy PCB
I designed a new PCB in Kicad, made gcode using Mods and milled it using the SRM-20. I was able to do this based on the experience I had with electronics design and production during Fab Academy.
3D Printed Case
I designed a case to hold the electronics in Fusion 360 and printed it on the Ultimaker S5. I created the design based on the experience I have gained over the course of Fab Academy using Fusion 360.
The case is able to be sealed using M3 screws and the PCB can be secured in the case using M3 screws. I included holes for the wires to run into the box and an opening for the cable to run to the powerbank.
Sound Recording
I recorded myself making the noises for Kokoro and edited them in Audacity. I then saved the files as 16bit WAV so they would work with the speaker program.
hugs.wav
ears.wav
pats.wav
boop.wav
Programming
The program includes:
- Detect when ears are touched
- Detect when back is patted
- Detect when nose is booped
- Detect when hugged
- Make LEDs turn on/off based on detection
- Make sounds based on detection
- Send transmission to console based on detection
References used to create this code:
- Capacitive touch Piicodev Guide by Core Electronics
- Transceiver PiicoDev Guide by Core Electronics
- Speaker code adapted from PicoAudionPWM by Daniel Perron
- Addressable LED code exmaple written by ChatGPT 4.0 May 24 Version
Question to Chat GPT: write example micropython code to make 2 addressable rgb leds, connected to Pin 1 on a xiao rp2040, turn red
Answer:
To control addressable RGB LEDs (like WS2812B or similar), you can use the neopixel
module in MicroPython. Here's an example of how you might make two LEDs connected to Pin 1 on a Seeed Xiao RP2040 turn red:
from machine import Pin
import neopixel
# Define the pin that the NeoPixels are connected to
pin = Pin(1)
# Define the number of NeoPixels
num_pixels = 2
# Create a NeoPixel object
pixels = neopixel.NeoPixel(pin, num_pixels)
# Set all pixels to red
for i in range(num_pixels):
pixels[i] = (255, 0, 0)
# Update the pixels to show the new colors
pixels.write()
This script creates a NeoPixel
object for two LEDs connected to Pin 1. It then sets the color of each LED to red (255, 0, 0) and updates the LEDs to show the new colors.
Please note that the neopixel
module uses GRB order for colors, not RGB. So, to set a pixel to red, you use (255, 0, 0), not (0, 255, 0).
- Flex sensor code example was written by ChatGPT 4.0 May 24 Version
Question to Chat GPT: write example micropython code to read a 10k ohm flex sensor that is connected to a seed xiao rp2040 on pin A0 (micropython Pin26)
Answer: Sure, here's an example of how you might read a 10k ohm flex sensor connected to a Seeed Xiao RP2040 on pin A0 using MicroPython:
from machine import ADC, Pin
# Define the analog pin
flex_pin = ADC(Pin(26))
def read_flex_sensor():
# Read the analog value from the flex sensor
analog_value = flex_pin.read_u16()
# Convert the 16-bit value to a voltage (0-3.3V)
voltage = analog_value * 3.3 / 65535
# Calculate the resistance of the flex sensor
# Assuming a 10k ohm pull-down resistor is used
resistance = 10000 * (3.3 / voltage - 1)
return resistance
while True:
flex_resistance = read_flex_sensor()
print("Flex sensor resistance: {} ohms".format(flex_resistance))
This script will continuously read the resistance of the flex sensor and print it to the console. The read_flex_sensor
function reads the analog value from the flex sensor, converts it to a voltage, and then calculates the resistance of the flex sensor based on the voltage.
Please note that the calculation assumes that you're using a 10k ohm pull-down resistor in your circuit. If you're using a different value, you'll need to adjust the calculation accordingly.
Final Code
# Code by Claire Chaikin-Bryan Kokoro Friend (c) 2023
# Modified from Core Electronics PiicoDev (c) 2023, Daniel Perron PicoAudioPWM (c) 2021 and ChatGPT 4.0 May 24 Version
from machine import Pin, I2C
from PiicoDev_Transceiver import PiicoDev_Transceiver
from PiicoDev_CAP1203 import PiicoDev_CAP1203
from PiicoDev_Unified import sleep_ms # cross-platform-compatible sleep
from machine import ADC, Pin
import os as uos
from wavePlayer import wavePlayer
import neopixel
# LED setup
# Define the pin that the NeoPixels are connected to
pinpix = Pin(27)
# Define the number of NeoPixels
num_pixels = 2
# Create a NeoPixel object
pixels = neopixel.NeoPixel(pinpix, num_pixels)
# Function to turn on all pixels
def turn_on_pixels():
for i in range(num_pixels):
pixels[i] = (43,138,226) # Set color to purple
pixels.write() # Update the pixels
# Function to turn off all pixels
def turn_off_pixels():
for i in range(num_pixels):
pixels[i] = (0, 0, 0) # Set color to black (off)
pixels.write() # Update the pixels
# Setup Transceiver
radio = PiicoDev_Transceiver(1, scl=Pin(7), sda=Pin(6), freq=200000, group=1, radio_address=2)
# Setup Flex Sensor
# Define the analog pin
flex_pin = ADC(Pin(26))
def read_flex_sensor():
# Read the analog value from the flex sensor
analog_value = flex_pin.read_u16()
# Convert the 16-bit value to a voltage (0-3.3V)
voltage = analog_value * 3.3 / 65535
# Calculate the resistance of the flex sensor
# Assuming a 10k ohm pull-down resistor is used
resistance = 10000 * (3.3 / voltage - 1)
return resistance
# Capacitative touch sensor setup
touchSensor = PiicoDev_CAP1203(1, scl=Pin(7), sda=Pin(6), freq=200000, sensitivity=5)
# Sound setup
player = wavePlayer()
waveFolder = "/waveFolder"
boop = waveFolder+"/boop.wav"
hugs = waveFolder+"/hugs.wav"
ears = waveFolder+"/ears.wav"
pats = waveFolder+"/pats.wav"
while True:
# reset luv score and LEDs
luv = 0
turn_off_pixels()
#Touch sensor
status = touchSensor.read() # read the state of all the touch pads
# Play a note depending on which pad is pressed
if status[1] == 1:
print("1")
turn_on_pixels()
player.play(pats)
luv = 1
if status[2] == 1:
print("2")
turn_on_pixels()
player.play(boop)
turn_off_pixels()
sleep_ms(100)
turn_on_pixels()
sleep_ms(200)
turn_off_pixels()
sleep_ms(200)
turn_on_pixels()
sleep_ms(200)
turn_off_pixels()
sleep_ms(200)
luv = 2
sleep_ms(3000)
if status[3] == 1:
print("3")
turn_on_pixels()
player.play(ears)
luv = 3
# Flex sensor
flex_resistance = read_flex_sensor()
if flex_resistance >15000:
turn_on_pixels()
player.play(hugs)
luv = 4
print(flex_resistance)
# Send to console luv score tells us what was detected
if luv > 0:
radio.send(luv)
print("love detected")
sleep_ms(1000)
Electronics Assembly
I soldered headers onto the PCB and the resistors for the LEDs and the flex sensor.
Note: I ended up removing the headers for the Xiao RP2040 as it was too tall for the case, and soldered the Xiao directly to the board.
I soldered the LED legs to wires and covered the connections with heat shrink, refer to Week 4 around LED wiring. I then hot glued the LEDs into the eyes.
I soldered wires to the flex sensor and covered the connections with heat shrink. I then sewed the flex sensor on the inside of the toy, behind where the pocket is. Note: Reference for wiring https://learn.sparkfun.com/tutorials/flex-sensor-hookup-guide/all
I soldered wires onto the ends of the conductive thread for the ears. I did this by looping the wire over, applying solder to secure that loop, tying the thread through that loop and applying solder to fill the loop and secure the thread.
I attached the conductive fabric to the inside the top back of the toy using hot glue. I then sewed some conductive thread into the fabric and soldered a wire onto the thread. I did this by looping the wire over, applying solder to secure that loop, tying the thread through that loop and applying solder to fill the loop and secure the thread.
I used solder to attach the bare ends of a wire to the inside of the 3D printed nose. I also filled the holes in the nose with solder (connected to the wire) both to help with conductivity, but also for aesthetics. Note: Through sensitivity testing with the capacitive touch sensor I found that having the solder through the holes helped.
I then soldered all the wires to the board and the speaker. I screwed the board to the inside of the case and then screwed the case shut. The transceiver is connected by a cable to the capacitive touch sensor and is stuck to the case.
Note: I had the speaker pull off and remove some copper, so I soldered one of the wires directly to the amplifier.
Final Toy Result
Console
Electronics Selection
For the console I wanted to be able to keep a record of the amount of love Kokoro receives and be able to measure user activity levels (a rough equivalent to number of steps).
The idea is that if you do not give Kokoro enough love or do enough activity the console will remind you. This is part of making Kokoro not just a cute companion to hang out with, but something the user feels responsible for, to look after it and to also be encouraged to look after themselves by exercising.
The final list of electronics for the console are:
- Xiao RP2040
- PiicoDev OLED
- PiicoDev Accelerometer
- PiicoDev Real Time Clock
- PiicoDev Buzzer
- Tactile buttons x3
- SD card adapter
- Lipo battery and charger
- PiicoDev Transceiver
The OLED displays the information and the buttons can be used to navigate to different menus/functions.
The accelerometer will be used to detect activity and is able to be used to sense the rough amount of steps being taken.
The buzzer is there to alert the user when Kokoro is not feeling loved or they have not been doing enough activity.
The SD card adapter allows for long term data storage and the real time clock will allow the data to be stored against a consistent time even if the console runs out of battery and need recharging.
The transceiver receives transmissions from Kokoro when touches and hugs are detected.
The electronics will be powered by a rechargeable Lipo battery.
A key thing for me was that Kokoro was not going to be connected to the internet or a mobile app, this is both for online safety for child users, but also allows users to detach from being connected to the internet for a while.
Console PCB
I designed a new PCB in Kicad, made gcode using Mods and milled it using the SRM-20. I was able to do this based on the experience I had with electronics design and production during Fab Academy.
I also made a breakout PCB for the buttons, this is to allow for the buttons to be at the level with the OLED in the case.
Main PCB
Note: I ended up needing to add a couple of jumper resistors to get all the traces in place.
There was a slight error in the milling where a couple of traces were not separated, I believe this was because my bit was too blunt. I was able to fix this by removing copper and soldering on a small piece of wire.
Button Breakout PCB
3D Printed Case
I designed a case in Fusion 360 and printed it on the Ultimaker S5. I created the design based on the experience I have gained over the course of Fab Academy using Fusion 360. The case needed to be small enough to carry around and also fit inside the pocket on the toy.
The case is able to be sealed using M3 screws. I included holes for the OLED and the buttons.
Note: I made the bottom hole a little close to the edge of the print, so it didn't print completely properly, just needs a minor tweak and re-print. I also printed a version without the charging cable hole so that I can assemble and then work out where it needs to be and tweak the design to match on re-print.
Programming
The program includes:
- Receiving a transmission from the toy for touch/hug detected and recording to SD card
- Time keeping with real time clock
- Displaying results on the OLED
- Recording of activity using accelerometer to SD card
- Use buttons to switch screens
- When activity or hug/touch drops low alert user using buzzer
I am still working on finalising this code, in the meantime I have made a place holder which displays on the OLED a heart animation and says "Love Detected!, it also picks up the touch/hug detection from the toy via the transceiver.
References used so far:
- Transceiver PiicoDev Guide by Core Electronics
- OLED PiicoDev Guide by Core Electronics
- Pixel graphics Guide by Marvin Fitzpatrick
# Code by Claire Chaikin-Bryan Kokoro Friend (c) 2023
# Modified from Core Electronics PiicoDev (c) 2023, Marvin Fitzpatrick (c) 2023
from ssd1306 import SSD1306_I2C
from PiicoDev_SSD1306 import *
from machine import Pin, I2C
from PiicoDev_Transceiver import PiicoDev_Transceiver
from PiicoDev_Unified import sleep_ms
from time import sleep
import urandom
i2c = I2C(1, scl=Pin(7), sda=Pin(6), freq=200000)
oled = SSD1306_I2C(128, 64, i2c)
radio = PiicoDev_Transceiver(1, scl=Pin(7), sda=Pin(6), freq=200000, group=1, radio_address=1)
ICON = [
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 1, 1, 0, 0, 0, 1, 1, 0],
[ 1, 1, 1, 1, 0, 1, 1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1],
[ 0, 1, 1, 1, 1, 1, 1, 1, 0],
[ 0, 0, 1, 1, 1, 1, 1, 0, 0],
[ 0, 0, 0, 1, 1, 1, 0, 0, 0],
[ 0, 0, 0, 0, 1, 0, 0, 0, 0],
]
def random_heart():
xofs = urandom.getrandbits(8)
yofs = urandom.getrandbits(5)
for y, row in enumerate(ICON):
for x, c in enumerate(row):
oled.pixel(x + xofs, y + yofs, c)
luv_count = 0
while True:
oled.fill(0)
if radio.receive():
message = radio.message
luv_count = int(message) + luv_count
print(luv_count)
for n in range(100):
random_heart()
luv_string = str(luv_count)
oled.text("Love Detected!",0,45)
oled.text("Score: " + luv_string, 0, 55)
oled.show()
Electronics Assembly
Most of the electronics are being soldered to the board as it will be moving around a bit, also given the accelerometer, users will want to shake it.
I am still completing this work, but have soldered the Xiao and the OLED to the board. The transceiver is connected by a cable to the OLED and is stuck to the case.
I found that I had to trim down the PCB to fit in the right spot in the case. The button PCB also is a bit large and needs to be made smaller, will do a re-design and mill before finalising updates to the case re-print.
Current State
Further Work for Kokoro to be ready for Bhutan
Remaining work is mainly in regards to the console, but will be completed shortly as it is a part of enhancing the interactivity with Kokoro.
- Finish soldering all the components for the console
- Adjust holes and reprint console case
- Finish programming the console
Bill of Materials
Item | Where to Get | Cost Estimate |
---|---|---|
Faux fur fabric and matching sewing thread | Spotlight | $20/m and thread $5/spool |
PP Cotton Stuffing | Amazon | $89.99 (repurpose dog bed, will have extra left over) |
3D printing filament | Fab Lab | Estimate less than $15 |
PCB copper milling board | Fab Lab | $114.95 (pack of 25) |
Silicone for mold making | Amazon | $44.99 (1kg amount) |
Wax mold making | Fab Lab | $45.39 |
UV Resin | Amazon | $49.99(kit with UV light and pigments) |
Silicone Electronics wire | Core Electronics | $10.90 |
Stainless Steel conductive thread | Core Electronics | $12.80 |
Conductive knit fabric patch | Core Electronics | $19.55 |
Xiao RP2040 (x2) | Core Electronics | $10.00 each |
Flex Sensor | Core Electronics | $34.30 |
Capacitive touch sensor | Core Electronics | $5.20 |
Speaker and amplifier | Core Electronics | $8.80 and $4.65 |
Transceiver (x2) | Core Electronics | $21.70 each |
Addressable LEDs (x2) | Core Electronics | $9.15 (pack of 10) |
OLED | Core Electronics | $13.60 |
Accelerometer | Core Electronics | $7.20 |
Buttons (x3) | Fab Lab | $0.76 each |
Resistors | Fab Lab | $8.30 (variety pack of 500) |
Buzzer | Core Electronics | $4.49 |
Real Time Clock | Core Electronics | $16.65 |
SD card and adapter | Core Electronics | $19.30 and $4.65 |
Lipo battery and charger | Core Electronics | $15.35 battery $5.10 charger |
Powerbank (replace in future with batteries) | Officeworks | $75 |
Note: Amounts in AUD
License
Claire Chaikin-Bryan Kokoro Friend (c) 2023
This work may be reproduced, modified, distributed, performed, and displayed for any purpose, but must acknowledge "Kokoro Friend by Claire Chaikin-Bryan". Copyright is retained and must be preserved. The work is provided as is; no warranty is provided, and users accept all liability.
Final Project Summary
Check out Week 18 which provides a summary of:
- what tasks have been completed?
- what tasks remain?
- what has worked? what hasn't?
- what questions need to be resolved?
- what will happen when?
- what have you learned?
Parts and systems that were made:
- Soft toy will be designed, laser cut fabric and sewn
- 3D printed cases for electronics
- PCBs
- 3D printed nose
- UV Resin cast eyes
Processes that used:
- 3D design using Fusion 350
- 2D design using Illustrator
- 3D printing
- Laser cutting and engraving fabric
- PCB Milling
- Wax milling
- Silicone and UV resin casting
- Soldering
- Programming
Questions that were answered:
Q. How will hugs be measured?
A. Using a flex sensor
Q. Will the 3D printed nose be able to be boop-able? A. Yes it was able to be made boop-able using by soldering a wire to the back of the nose and also filling the holes in the nose with solder and joing that solder to the wire.
Q. Will there be enough time to get both the toy and console/remote done? Focus should be on the toy?
A. There wasn't enough time, focus was put on the toy as it met all the requirments for a Fab Academy Final Project on its own, the console is extra.
How it was evaluated:
Key criteria:
- Is it cute?
- Is it huggable?
- Does it encourage me to look after it?
Answer is YES to all of the above, it met the critrea.
Files
Toy
Laser Cut and Engrave Fabric
Pattern
Eyes
3D model
Fusion File
Nose
3D model
Fusion File
Case
3D model
Fusion File
Audio Files
hugs.wav
ears.wav
pats.wav
boop.wav
PCB
Kokoro_toy.kicad_pro
Kokoro_toy.kicad_sch
Kokoro_toy.kicad_pcb
Kokoro_toy-F_Cu 1.png
Kokoro_toy-B_Cu 1.png
Kokoro_toy-Edge_Cuts 1.png
Kokoro_toy-F_Cu 1.png.rml
Kokoro_toy-B_Cu 1.png.rml
Kokoro_toy-Edge_Cuts 1.png.rml
Code
PiicoDev_Unified.py
PiicoDev_Transceiver.py
PiicoDev_CAP1203.py
SpeakerCode.zip (Unzip to use)
main_toy.py (rename to main.py to auto run)
Console
Case
3D model
Fusion File
Main PCB
Kokoro_console.kicad_pro
Kokoro_console.kicad_sch
Kokoro_console.kicad_pcb
Kokoro_console-F_Cu.png
Kokoro_console-B_Cu.png
Kokoro_console-Edge_Cuts.png
Kokoro_console-F_Cu.png.rml
Kokoro_console-B_Cu.png.rml
Kokoro_console-Edge_Cuts.png.rml
Button PCB
button.kicad_pro
button.kicad_sch
button.kicad_pcb
button-F_Cu.png
button-B_Cu.png
button-Edge_Cuts.png
button-F_Cu.png.rml
button-B_Cu.png.rml
button-Edge_Cuts.png.rml
Code
PiicoDev_Unified.py
PiicoDev_Transceiver.py
ssd1306.py
PiicoDev_LIS3DH.py
main_console.py (rename to main.py to auto run)