Final Project

Final Presentation

Live Demos

Initial Final Project Idea

The Problem

Approximately 30% of Australians rent.
Most are not allowed to have pets or have difficulty getting approval from landlords to have a pet.

This means many renters miss out on the mental and physical health benefits of owning and caring for a pet.

Final Project Idea

To create an artificial pet that can provide comfort and motivation to its owner.

The pet will need to be able to:
- Provide opportunities for its owner to care for it
- Encourage its owner to exercise
- Know when it is being pat
- Be huggable

Sketch

Owner will be able to take the separate device on walks/runs if they do not want to take the whole pet with them
Pet will provide an alert when it:
- Needs charging
- Has been neglected for too long (hugs/pats required)
- When the owner has not taken it for a walk for too long

Extra ideas
- Being able to play games with the pet
- Adding an RFID reader into the pet and making different food and toys which when touched to the pet will result in different sounds and screen outputs
- Adding LED lights to the pet

Who is this for?

This will appeal to people who cannot own a pet themselves. It will be appropriate for both children and adults, although with its cuteness factor it will appeal to the young and young at heart.

Idea Name: Kokoro Friend

The Japanese Concept of Kokoro

Kokoro is a Japanese word that is often translated as “heart” or “spirit.” However, the concept of kokoro encompasses what a person thinks, feels, and would like to express. It is made of a person’s thoughts and will that drive them forward, the knowledge and experience they have acquired through time, and all their feelings.
Yet, kokoro is more than the sum of all these elements: it is all these elements at the same time. Each person’s features give their kokoro its own “color” and “shape,” making every human a distinct individual with a unique inner richness.

A person’s kokoro cannot be seen or touched:it resides in the area of the heart. Hidden in this invisible kokoro, lies a complex patchwork of ideas, concepts, and feelings that cannot be fully accessed from the outside.

One’s kokoro lives in everything he or she creates. The creation can be a material or conceptual.
Source: https://kokoro-jp.com/what-is-kokoro/

Inspiration

An example of bringing huggability and electronics together is Scream Body by Kelly Dobson https://web.media.mit.edu/~monster/screambody/

Fluf Bot by a Shinobu Ishimura Fab Academy Student https://fabacademy.org/2021/labs/kannai/students/shinobu-ishimura/projects/final-project/

Qoobo - commercial product out now https://qoobo.info/index-en/

Paro Therapeutic Robot - commercial product out now http://www.parorobots.com/

Furby - Inspiration from my childhood https://en.wikipedia.org/wiki/Furby

Tamagotchi - Inspiration from my childhood https://en.wikipedia.org/wiki/Tamagotchi

Render of possible device for interaction

From Week 2

Possible Electronics

Possible electronics input/output components (manufactured locally in Newcastle, Australia by Core Electronics) to go with Xiao RP2040 and custom designed board.

Visit to Piicodev Manufacturing Line February 2023

*Video from Core Electronics

Mid Term Review

Plan of items to include in Final Project toy and remote/console

Item	Toy	Console
PCB	Xiao RP2040, custom PCB	Xiao RP2040, custom PCB
2D Design	Pattern for toy
3D Design	Nose and electronics case	Case
Additive	Nose and case 3D print, casting eyes	3D print case
Subtractive	Mold for eyes, laser cut fabric
Input	Capacitive touch	Buttons, Accelerometer
Output	Lights, sound	OLED
Network	Transceiver	Transceiver
Pakaging	Toy and internal electronic case	Case

The plan will be to focus on completing the toy and to complete as much of the console as possible. The toy contains all the elements to pass the final project without the console, so this is extra.

Stuffed Toy

Eyes for Toy

I made a mold to make eyes out of UV resin for Kokoro. From Week 12, see that page for instructions.

I recast the eyes again, putting in the LEDs at the end to allow for me to be able to better hot glue them in place. I also added a little glitter into the purple resin to make it more sparkly. The eyes will be hot glued to the toy.

3D Printed Nose

I designed the nose in Fusion 360 and printed it on at Ultimaker S5. I created the design based on the experience I have gained over the course of Fab Academy using Fusion 360.

The nose will be hot glued to the toy and booping it will cause a response from the toy.

Laser Cutting and Engraving Fabric

During Wildcard Week I designed the pattern for Kokoro and cut and engraved sewing lines on the faux fur fabric using the laser cutter.

Final set of pieces

Sewing

During Wildcard Week I sewed Kokoro together including adding conducive steel thread to the ears to make them be able to be picked up using a capacitive touch sensor.

See Wildcard Week for detailed sewing instructions.

Electronics Selection

I originally thought about actuating the ears on the soft toy, but after some testing they were too heavy and the noise of the servos took away from the serenity of patting/hugging Kokoro.

The final list of electronics to be incorporated into the toy are:
- Xiao RP2040
- PiicoDev Capacitive Touch Sensor connected to conductive thread in ears, conducive fabric square and 3D printed nose
- Speaker and amplifier
- 2x addressable LEDs with a 1k resistor on 5V power and a 470 resistor on signal
- Flex sensor with 10k resistor - PiicoDev Transceiver

The idea is that when you touch one of the capacitive touch elements (ears, back, boopable nose) that the eyes will light up and Kokoro will make a sound.

I also wanted to be able to detect hugs, after a bit of thinking and discussions with friends with more electronics experience we came up with the idea to incorporate a flex sensor into the belly, that will flex and therefore change resistance when the toy is hugged. This will also make the eyes light up and Kokoro will make a sound.

I have included the transceiver so that when a touch or a hug is detected this can be sent to the console. Also if in the future I want to make something happen with the toy using the console, I will be able to do this as the transceiver can work both ways.

The electronics will be powered by a powerbank that is easy to remove and recharge. In the future I want to explore replacing this with alkaline batteries which are safer to have in a toy like this.

Toy PCB

I designed a new PCB in Kicad, made gcode using Mods and milled it using the SRM-20. I was able to do this based on the experience I had with electronics design and production during Fab Academy.

3D Printed Case

I designed a case to hold the electronics in Fusion 360 and printed it on the Ultimaker S5. I created the design based on the experience I have gained over the course of Fab Academy using Fusion 360.

The case is able to be sealed using M3 screws and the PCB can be secured in the case using M3 screws. I included holes for the wires to run into the box and an opening for the cable to run to the powerbank.

Sound Recording

I recorded myself making the noises for Kokoro and edited them in Audacity. I then saved the files as 16bit WAV so they would work with the speaker program.

hugs.wav

ears.wav

pats.wav

boop.wav

Programming

The program includes:
- Detect when ears are touched
- Detect when back is patted
- Detect when nose is booped
- Detect when hugged
- Make LEDs turn on/off based on detection
- Make sounds based on detection
- Send transmission to console based on detection

References used to create this code:
- Capacitive touch Piicodev Guide by Core Electronics
- Transceiver PiicoDev Guide by Core Electronics - Speaker code adapted from PicoAudionPWM by Daniel Perron
- Addressable LED code exmaple written by ChatGPT 4.0 May 24 Version

Question to Chat GPT: write example micropython code to make 2 addressable rgb leds, connected to Pin 1 on a xiao rp2040, turn red

Answer:
To control addressable RGB LEDs (like WS2812B or similar), you can use the neopixel module in MicroPython. Here's an example of how you might make two LEDs connected to Pin 1 on a Seeed Xiao RP2040 turn red:

from machine import Pin
import neopixel

# Define the pin that the NeoPixels are connected to
pin = Pin(1)

# Define the number of NeoPixels
num_pixels = 2

# Create a NeoPixel object
pixels = neopixel.NeoPixel(pin, num_pixels)

# Set all pixels to red
for i in range(num_pixels):
    pixels[i] = (255, 0, 0)

# Update the pixels to show the new colors
pixels.write()

This script creates a NeoPixel object for two LEDs connected to Pin 1. It then sets the color of each LED to red (255, 0, 0) and updates the LEDs to show the new colors.

Please note that the neopixel module uses GRB order for colors, not RGB. So, to set a pixel to red, you use (255, 0, 0), not (0, 255, 0).

• Flex sensor code example was written by ChatGPT 4.0 May 24 Version

Question to Chat GPT: write example micropython code to read a 10k ohm flex sensor that is connected to a seed xiao rp2040 on pin A0 (micropython Pin26)

Answer: Sure, here's an example of how you might read a 10k ohm flex sensor connected to a Seeed Xiao RP2040 on pin A0 using MicroPython:

from machine import ADC, Pin

# Define the analog pin
flex_pin = ADC(Pin(26))

def read_flex_sensor():
    # Read the analog value from the flex sensor
    analog_value = flex_pin.read_u16()

    # Convert the 16-bit value to a voltage (0-3.3V)
    voltage = analog_value * 3.3 / 65535

    # Calculate the resistance of the flex sensor
    # Assuming a 10k ohm pull-down resistor is used
    resistance = 10000 * (3.3 / voltage - 1)

    return resistance

while True:
    flex_resistance = read_flex_sensor()
    print("Flex sensor resistance: {} ohms".format(flex_resistance))

This script will continuously read the resistance of the flex sensor and print it to the console. The read_flex_sensor function reads the analog value from the flex sensor, converts it to a voltage, and then calculates the resistance of the flex sensor based on the voltage.

Please note that the calculation assumes that you're using a 10k ohm pull-down resistor in your circuit. If you're using a different value, you'll need to adjust the calculation accordingly.

Final Code

# Code by Claire Chaikin-Bryan Kokoro Friend (c) 2023
# Modified from Core Electronics PiicoDev (c) 2023, Daniel Perron PicoAudioPWM (c) 2021 and ChatGPT 4.0 May 24 Version  

from machine import Pin, I2C
from PiicoDev_Transceiver import PiicoDev_Transceiver
from PiicoDev_CAP1203 import PiicoDev_CAP1203
from PiicoDev_Unified import sleep_ms # cross-platform-compatible sleep
from machine import ADC, Pin
import os as uos
from wavePlayer import wavePlayer
import neopixel

# LED setup
# Define the pin that the NeoPixels are connected to
pinpix = Pin(27)

# Define the number of NeoPixels
num_pixels = 2

# Create a NeoPixel object
pixels = neopixel.NeoPixel(pinpix, num_pixels)

# Function to turn on all pixels
def turn_on_pixels():
    for i in range(num_pixels):
        pixels[i] = (43,138,226)  # Set color to purple
    pixels.write()  # Update the pixels

# Function to turn off all pixels
def turn_off_pixels():
    for i in range(num_pixels):
        pixels[i] = (0, 0, 0)  # Set color to black (off)
    pixels.write()  # Update the pixels


# Setup Transceiver
radio = PiicoDev_Transceiver(1, scl=Pin(7), sda=Pin(6), freq=200000, group=1, radio_address=2)


# Setup Flex Sensor
# Define the analog pin
flex_pin = ADC(Pin(26))

def read_flex_sensor():
    # Read the analog value from the flex sensor
    analog_value = flex_pin.read_u16()

    # Convert the 16-bit value to a voltage (0-3.3V)
    voltage = analog_value * 3.3 / 65535

    # Calculate the resistance of the flex sensor
    # Assuming a 10k ohm pull-down resistor is used
    resistance = 10000 * (3.3 / voltage - 1)

    return resistance

# Capacitative touch sensor setup
touchSensor = PiicoDev_CAP1203(1, scl=Pin(7), sda=Pin(6), freq=200000, sensitivity=5)

# Sound setup
player = wavePlayer()
waveFolder = "/waveFolder"
boop = waveFolder+"/boop.wav"
hugs = waveFolder+"/hugs.wav"
ears = waveFolder+"/ears.wav"
pats = waveFolder+"/pats.wav"


while True:
# reset luv score and LEDs
    luv = 0
    turn_off_pixels()

#Touch sensor
    status = touchSensor.read() # read the state of all the touch pads
    # Play a note depending on which pad is pressed
    if status[1] == 1:
        print("1")
        turn_on_pixels()
        player.play(pats)
        luv = 1
    if status[2] == 1:
        print("2")
        turn_on_pixels()
        player.play(boop)
        turn_off_pixels()
        sleep_ms(100) 
        turn_on_pixels()
        sleep_ms(200)
        turn_off_pixels()
        sleep_ms(200) 
        turn_on_pixels()
        sleep_ms(200)
        turn_off_pixels()
        sleep_ms(200)
        luv = 2
        sleep_ms(3000)
    if status[3] == 1:
        print("3")
        turn_on_pixels()
        player.play(ears)
        luv = 3

# Flex sensor        
    flex_resistance = read_flex_sensor()
    if flex_resistance >15000:
        turn_on_pixels()
        player.play(hugs)  
        luv = 4
        print(flex_resistance)

# Send to console luv score tells us what was detected
    if luv > 0:
        radio.send(luv)
        print("love detected")

    sleep_ms(1000)

Electronics Assembly

I soldered headers onto the PCB and the resistors for the LEDs and the flex sensor.
Note: I ended up removing the headers for the Xiao RP2040 as it was too tall for the case, and soldered the Xiao directly to the board.

I soldered the LED legs to wires and covered the connections with heat shrink, refer to Week 4 around LED wiring. I then hot glued the LEDs into the eyes.

I soldered wires to the flex sensor and covered the connections with heat shrink. I then sewed the flex sensor on the inside of the toy, behind where the pocket is. Note: Reference for wiring https://learn.sparkfun.com/tutorials/flex-sensor-hookup-guide/all

I soldered wires onto the ends of the conductive thread for the ears. I did this by looping the wire over, applying solder to secure that loop, tying the thread through that loop and applying solder to fill the loop and secure the thread.

I attached the conductive fabric to the inside the top back of the toy using hot glue. I then sewed some conductive thread into the fabric and soldered a wire onto the thread. I did this by looping the wire over, applying solder to secure that loop, tying the thread through that loop and applying solder to fill the loop and secure the thread.

I used solder to attach the bare ends of a wire to the inside of the 3D printed nose. I also filled the holes in the nose with solder (connected to the wire) both to help with conductivity, but also for aesthetics. Note: Through sensitivity testing with the capacitive touch sensor I found that having the solder through the holes helped.

I then soldered all the wires to the board and the speaker. I screwed the board to the inside of the case and then screwed the case shut. The transceiver is connected by a cable to the capacitive touch sensor and is stuck to the case.
Note: I had the speaker pull off and remove some copper, so I soldered one of the wires directly to the amplifier.

Final Toy Result

Console

Electronics Selection

For the console I wanted to be able to keep a record of the amount of love Kokoro receives and be able to measure user activity levels (a rough equivalent to number of steps).

The idea is that if you do not give Kokoro enough love or do enough activity the console will remind you. This is part of making Kokoro not just a cute companion to hang out with, but something the user feels responsible for, to look after it and to also be encouraged to look after themselves by exercising.

The final list of electronics for the console are:
- Xiao RP2040
- PiicoDev OLED
- PiicoDev Accelerometer
- PiicoDev Real Time Clock
- PiicoDev Buzzer
- Tactile buttons x3
- SD card adapter
- Lipo battery and charger
- PiicoDev Transceiver

The OLED displays the information and the buttons can be used to navigate to different menus/functions.

The accelerometer will be used to detect activity and is able to be used to sense the rough amount of steps being taken.

The buzzer is there to alert the user when Kokoro is not feeling loved or they have not been doing enough activity.

The SD card adapter allows for long term data storage and the real time clock will allow the data to be stored against a consistent time even if the console runs out of battery and need recharging.

The transceiver receives transmissions from Kokoro when touches and hugs are detected.

The electronics will be powered by a rechargeable Lipo battery.

A key thing for me was that Kokoro was not going to be connected to the internet or a mobile app, this is both for online safety for child users, but also allows users to detach from being connected to the internet for a while.

Console PCB

I designed a new PCB in Kicad, made gcode using Mods and milled it using the SRM-20. I was able to do this based on the experience I had with electronics design and production during Fab Academy.

I also made a breakout PCB for the buttons, this is to allow for the buttons to be at the level with the OLED in the case.

Main PCB
Note: I ended up needing to add a couple of jumper resistors to get all the traces in place.

There was a slight error in the milling where a couple of traces were not separated, I believe this was because my bit was too blunt. I was able to fix this by removing copper and soldering on a small piece of wire.

Button Breakout PCB

3D Printed Case

I designed a case in Fusion 360 and printed it on the Ultimaker S5. I created the design based on the experience I have gained over the course of Fab Academy using Fusion 360. The case needed to be small enough to carry around and also fit inside the pocket on the toy.

The case is able to be sealed using M3 screws. I included holes for the OLED and the buttons.

Note: I made the bottom hole a little close to the edge of the print, so it didn't print completely properly, just needs a minor tweak and re-print. I also printed a version without the charging cable hole so that I can assemble and then work out where it needs to be and tweak the design to match on re-print.

Programming

The program includes:
- Receiving a transmission from the toy for touch/hug detected and recording to SD card
- Time keeping with real time clock
- Displaying results on the OLED
- Recording of activity using accelerometer to SD card
- Use buttons to switch screens
- When activity or hug/touch drops low alert user using buzzer

I am still working on finalising this code, in the meantime I have made a place holder which displays on the OLED a heart animation and says "Love Detected!, it also picks up the touch/hug detection from the toy via the transceiver.

References used so far:
- Transceiver PiicoDev Guide by Core Electronics
- OLED PiicoDev Guide by Core Electronics
- Pixel graphics Guide by Marvin Fitzpatrick

# Code by Claire Chaikin-Bryan Kokoro Friend (c) 2023
# Modified from Core Electronics PiicoDev (c) 2023, Marvin Fitzpatrick (c) 2023
from ssd1306 import SSD1306_I2C
from PiicoDev_SSD1306 import *
from machine import Pin, I2C
from PiicoDev_Transceiver import PiicoDev_Transceiver
from PiicoDev_Unified import sleep_ms
from time import sleep
import urandom

i2c = I2C(1, scl=Pin(7), sda=Pin(6), freq=200000)
oled = SSD1306_I2C(128, 64, i2c)

radio = PiicoDev_Transceiver(1, scl=Pin(7), sda=Pin(6), freq=200000, group=1, radio_address=1)

ICON = [
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 1, 1, 0, 0, 0, 1, 1, 0],
[ 1, 1, 1, 1, 0, 1, 1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1],
[ 0, 1, 1, 1, 1, 1, 1, 1, 0],
[ 0, 0, 1, 1, 1, 1, 1, 0, 0],
[ 0, 0, 0, 1, 1, 1, 0, 0, 0],
[ 0, 0, 0, 0, 1, 0, 0, 0, 0],
]


def random_heart():
    xofs = urandom.getrandbits(8)
    yofs = urandom.getrandbits(5)
    for y, row in enumerate(ICON):
        for x, c in enumerate(row):
            oled.pixel(x + xofs, y + yofs, c)

luv_count = 0

while True:
    oled.fill(0)
    if radio.receive():
        message = radio.message
        luv_count = int(message) + luv_count
        print(luv_count)
    for n in range(100):
        random_heart()
    luv_string = str(luv_count)
    oled.text("Love Detected!",0,45)
    oled.text("Score: " + luv_string, 0, 55)
    oled.show()

Electronics Assembly

Most of the electronics are being soldered to the board as it will be moving around a bit, also given the accelerometer, users will want to shake it.

I am still completing this work, but have soldered the Xiao and the OLED to the board. The transceiver is connected by a cable to the OLED and is stuck to the case.

I found that I had to trim down the PCB to fit in the right spot in the case. The button PCB also is a bit large and needs to be made smaller, will do a re-design and mill before finalising updates to the case re-print.

Current State

Further Work for Kokoro to be ready for Bhutan

Remaining work is mainly in regards to the console, but will be completed shortly as it is a part of enhancing the interactivity with Kokoro.
- Finish soldering all the components for the console
- Adjust holes and reprint console case
- Finish programming the console

Bill of Materials

Item	Where to Get	Cost Estimate
Faux fur fabric and matching sewing thread	Spotlight	$20/m and thread $5/spool
PP Cotton Stuffing	Amazon	$89.99 (repurpose dog bed, will have extra left over)
3D printing filament	Fab Lab	Estimate less than $15
PCB copper milling board	Fab Lab	$114.95 (pack of 25)
Silicone for mold making	Amazon	$44.99 (1kg amount)
Wax mold making	Fab Lab	$45.39
UV Resin	Amazon	$49.99(kit with UV light and pigments)
Silicone Electronics wire	Core Electronics	$10.90
Stainless Steel conductive thread	Core Electronics	$12.80
Conductive knit fabric patch	Core Electronics	$19.55
Xiao RP2040 (x2)	Core Electronics	$10.00 each
Flex Sensor	Core Electronics	$34.30
Capacitive touch sensor	Core Electronics	$5.20
Speaker and amplifier	Core Electronics	$8.80 and $4.65
Transceiver (x2)	Core Electronics	$21.70 each
Addressable LEDs (x2)	Core Electronics	$9.15 (pack of 10)
OLED	Core Electronics	$13.60
Accelerometer	Core Electronics	$7.20
Buttons (x3)	Fab Lab	$0.76 each
Resistors	Fab Lab	$8.30 (variety pack of 500)
Buzzer	Core Electronics	$4.49
Real Time Clock	Core Electronics	$16.65
SD card and adapter	Core Electronics	$19.30 and $4.65
Lipo battery and charger	Core Electronics	$15.35 battery $5.10 charger
Powerbank (replace in future with batteries)	Officeworks	$75

Note: Amounts in AUD

License

Claire Chaikin-Bryan Kokoro Friend (c) 2023

This work may be reproduced, modified, distributed, performed, and displayed for any purpose, but must acknowledge "Kokoro Friend by Claire Chaikin-Bryan". Copyright is retained and must be preserved. The work is provided as is; no warranty is provided, and users accept all liability.

Final Project Summary

Check out Week 18 which provides a summary of:
- what tasks have been completed?
- what tasks remain?
- what has worked? what hasn't?
- what questions need to be resolved?
- what will happen when?
- what have you learned?

Parts and systems that were made:
- Soft toy will be designed, laser cut fabric and sewn - 3D printed cases for electronics - PCBs - 3D printed nose - UV Resin cast eyes

Processes that used:
- 3D design using Fusion 350 - 2D design using Illustrator - 3D printing - Laser cutting and engraving fabric - PCB Milling - Wax milling - Silicone and UV resin casting - Soldering
- Programming

Questions that were answered:
Q. How will hugs be measured?
A. Using a flex sensor

Q. Will the 3D printed nose be able to be boop-able? A. Yes it was able to be made boop-able using by soldering a wire to the back of the nose and also filling the holes in the nose with solder and joing that solder to the wire.

Q. Will there be enough time to get both the toy and console/remote done? Focus should be on the toy?
A. There wasn't enough time, focus was put on the toy as it met all the requirments for a Fab Academy Final Project on its own, the console is extra.

How it was evaluated:
Key criteria:
- Is it cute?
- Is it huggable?
- Does it encourage me to look after it?

Answer is YES to all of the above, it met the critrea.

Files

Toy

Laser Cut and Engrave Fabric
Pattern

Eyes
3D model
Fusion File

Nose
3D model
Fusion File

Case
3D model
Fusion File

Audio Files
hugs.wav
ears.wav
pats.wav
boop.wav

PCB
Kokoro_toy.kicad_pro
Kokoro_toy.kicad_sch
Kokoro_toy.kicad_pcb Kokoro_toy-F_Cu 1.png
Kokoro_toy-B_Cu 1.png
Kokoro_toy-Edge_Cuts 1.png
Kokoro_toy-F_Cu 1.png.rml
Kokoro_toy-B_Cu 1.png.rml
Kokoro_toy-Edge_Cuts 1.png.rml

Code
PiicoDev_Unified.py
PiicoDev_Transceiver.py
PiicoDev_CAP1203.py
SpeakerCode.zip (Unzip to use)
main_toy.py (rename to main.py to auto run)

Console

Case
3D model
Fusion File

Main PCB
Kokoro_console.kicad_pro
Kokoro_console.kicad_sch
Kokoro_console.kicad_pcb
Kokoro_console-F_Cu.png
Kokoro_console-B_Cu.png
Kokoro_console-Edge_Cuts.png
Kokoro_console-F_Cu.png.rml
Kokoro_console-B_Cu.png.rml
Kokoro_console-Edge_Cuts.png.rml

Button PCB
button.kicad_pro
button.kicad_sch
button.kicad_pcb
button-F_Cu.png
button-B_Cu.png
button-Edge_Cuts.png
button-F_Cu.png.rml
button-B_Cu.png.rml
button-Edge_Cuts.png.rml

Code
PiicoDev_Unified.py
PiicoDev_Transceiver.py
ssd1306.py
PiicoDev_LIS3DH.py
main_console.py (rename to main.py to auto run)