This week we will focus on the integration of the project systems
The assignment was:
Design and document the system integration for your final project
The "Finger Score Toy" is a Toy for shooting hoops and scoring points with your fingers
This toy combines a sensor, an actuator and a microcontroller integrated into a PCB designed to enhance the system's integration and functionality. This PCB connects via serial communication to a computer to interact with an AI-based hand recognition system and an interactive user interface.
Infrared Sensor FC-51:
- Detects when the ball passes through the hoop.
- Sends a signal to the XIAO RP2040 microcontroller.
Test Infrared Sensor FC-51:
- When the ball passes through the hoop, the green LED light on the sensor turns on.
XIAO RP2040 Microcontroller:
- Receives the signal from the infrared sensor FC-51.
- Processes the information and activates the buzzer for auditory feedback.
- Sends data to the computer via serial communication.
Designed PCB:
- Designed in kicad 8.0
- Integrate the system
Test subsystem:
- Xiao RP2040 working.
- IR sensor calibrated and buzzer working.
- Transmits data from the XIAO RP2040 microcontroller to the computer.
Code subsystem:
int buzzerPin = D5; int sensorPin = D8; int sensorState = 0; void setup() { pinMode(buzzerPin, OUTPUT); pinMode(sensorPin, INPUT); Serial.begin(115200); } void loop() { sensorState = digitalRead(sensorPin); if (sensorState == LOW) { // Detects movement digitalWrite(buzzerPin, HIGH); delay(1000); // Buzzer sounds for 1 second digitalWrite(buzzerPin, LOW); Serial.println('h'); // Send 'h' via serial port delay(1000); // Wait 1 second to avoid multiple signals } }
Case subsystem:
Hands Detection:
The MediaPipe Gesture Recognizer task lets you recognize hand gestures in real time, and provides the recognized hand gesture results along with the landmarks of the detected hands. You can use this task to recognize specific hand gestures from a user, and invoke application features that correspond to those gestures.
More information visit mediapipe website
User Interface:
Tkinter Python interface is used in the "Finger Score Toy" to provide a real-time score display, facilitating user interaction.
- Displays the score in real-time.
- Tracks the points scored.
More information visit Tkinter website
code subsystem:
import cv2 import mediapipe as mp import tkinter as tk from threading import Thread import serial import time class BasketballScoreboard(tk.Tk): def __init__(self): super().__init__() self.title("Finger Score") # Score initialization self.score = 0 # Creating labels to display the score self.label_team = tk.Label(self, text="Score", font=("Arial", 100)) self.label_team.grid(row=0, column=0, padx=20, pady=10) self.label_score = tk.Label(self, text=self.score, font=("Arial", 300)) self.label_score.grid(row=1, column=0, padx=20, pady=10) # Serial port initialization self.serial_port = 'COM3' # Change this to the correct port self.baud_rate = 115200 self.ser = serial.Serial(self.serial_port, self.baud_rate) # Start the hand detection and serial listening thread self.hand_detection_thread = Thread(target=self.detect_hands) self.hand_detection_thread.daemon = True self.hand_detection_thread.start() self.serial_thread = Thread(target=self.listen_serial) self.serial_thread.daemon = True self.serial_thread.start() self.update_score = False def increment_score(self, points): for _ in range(points): self.score += 1 self.label_score.config(text=self.score) time.sleep(0.1) # Increase slowly def listen_serial(self): while True: if self.ser.in_waiting > 0: signal = self.ser.readline().decode().strip() if signal == 'h': self.update_score = True time.sleep(0.1) # Short pause to avoid multiple readings def detect_hands(self): mp_hands = mp.solutions.hands mp_drawing = mp.solutions.drawing_utils hands = mp_hands.Hands() cap = cv2.VideoCapture(0) while cap.isOpened(): ret, frame = cap.read() if not ret: break frame = cv2.cvtColor(cv2.flip(frame, 1), cv2.COLOR_BGR2RGB) results = hands.process(frame) if results.multi_hand_landmarks: hand_landmarks = results.multi_hand_landmarks[0] # Draw reference points and connections mp_drawing.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS) # Count raised fingers and update score if 'h' signal is received if self.update_score: fingers = self.count_fingers(hand_landmarks) if fingers > 0: self.increment_score(fingers) self.update_score = False cv2.imshow('Hand Detection', cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)) if cv2.waitKey(5) & 0xFF == 27: break cap.release() cv2.destroyAllWindows() def count_fingers(self, hand_landmarks): # Count raised fingers finger_tips = [4, 8, 12, 16, 20] count = 0 if hand_landmarks.landmark[finger_tips[0]].x < hand_landmarks.landmark[finger_tips[0] - 1].x: count += 1 # count thumb for tip in finger_tips[1:]: if hand_landmarks.landmark[tip].y < hand_landmarks.landmark[tip - 2].y: count += 1 return min(count, 3) # Maximum 3 points if __name__ == "__main__": app = BasketballScoreboard() app.mainloop()
Test:
- Finger gestures (1, 2, or 3 fingers raised) are translated into points (1, 2, or 3).
- These points are sent to the user interface (UI) developed with Tkinter.
conclusions:
- Developing this toy allowed me to integrate different technologies into a single system.
- Designing a PCB improved my understanding of electronics.
- Implementing artificial intelligence and a user interface gave me some experience in programming using python, I still find it difficult.
- The toy shows some complexity in real-time data processing and the need for efficient communication between hardware and software.
- The importance of testing to refine functionality is highlighted.
- Creating a product from start to finish gave me immense personal satisfaction.
what I learned?:
- To integrate different technologies in a project.
- To customize a PCB to control the toy.
- To solve some problems and challenges both in the code and in the physical part.
- A Perform multiple tests of each of the subsystems.
- That technology can be used to create fun and educational tools.
Here we can find and download the design original files