Final Project - Bio-signal Data Collection & Analyzation Tool Kit for Motion Sickness Prediction

In this webpage I present the detail work in my final project - How to build a Bio-signal Data Collection & Analyzation Tool Kit for Motion Sickness Prediction

Final Result

Motion sickness has long been recognized as an important factor affecting a person's car riding experience. In this project, I developed a bio-signal data collection and analysis tool kit that can be further developed for predict the occurrence of motion sickness based on human physiological signals. This device will be used in a car riding environment, anyone who is likely to encounter car-sickness will be a potential user.

Summary Slide:

Video of final system:

Hero shots

The hero shot and video below provide an overview of the system.

The system components before assemble.

Idea Sketch

The figure below has shown the original idea design of this tool set.

This detection tool consists of 5 parts:

A set of sensors for collecting human physiological signals includes PPG, SKT etc.
A data processor module based on Raspberry Pi with Wifi connection.
A trained AI algorithm for data processing which can be deployed on Raspberry Pi.
A screen to show motion sickness dose value.
A control panel for function selection.

After weeks of learning I have adjusted my final project design and the idea is demonstrated as below. In this new design the Tool kit is divided into two parts: A Raspberry Pi 4B Data Processing Module and A Seeed Studio Xiao ESP32-C3 Biosignal Data Collection Module.

Computer Aided Design

Creating the housing design for the data processing & detection module using Fusion CAD, with a focus on ergonomics and durability.

The 3D design process is document as video below.

You can download my 3D housing design by link below:

Computer Controlled Cutting

Design and cutting the logo sticker with a vinal cutter.

The vinal cutting process is shown as video below.

Find the vinyl cutting design file at link below:

https://gitlab.fabcloud.org/academany/fabacademy/2023/labs/ningbo/students/yaorun-zhang/-/tree/main/docs/assignments/week3/design-file?ref_type=heads

3D Printing

3D print the data processing & detection module housing.

I printed my design using both an FDM and an SLA printer. The SLA printer's curing platform seems to have issues, so the FDM printer produced better results.

Input & Output device testing

Testing the input sensors and output screen.

Here is a video shows me testing different sensors.

Electronic Design & Production

Design the PCB for detection module with Fusion 360.

Manufacture the PCB with Roland and soldering it.

The video below shown the PCB design and manufacture process.

You can download my PCB design by link below:

https://a360.co/4fv4euf

Embedded Programming for sensors and network communication

Programming and testing the bio-signal detection module, setting up wifi communication between the Raspberry Pi and bio-signal detection module.

#include <Arduino.h>
#include <WiFi.h>

// Create a WiFi server instance on port 1122
WiFiServer server(1122);
WiFiClient client = server.available();

// Define variables to hold sensor data
short grs = 1024;
short ibi = 996;
short bpm = 60;
float temperature = 30.80;
float humid = 20.90;

// Define character arrays to store the string representations of sensor data
char grs_str[10];
char ibi_str[10];
char bpm_str[10];
char temperature_str[10];
char humid_str[10];
int chang = 0;

// Function to initialize WiFi as a SoftAP and start the server
void Wifi_TCP_Init() {
  WiFi.softAP("ESP32-hong"); // Set the access point name
  server.begin(); // Start the server
  Serial.begin(115200); // Initialize serial communication at 115200 baud
  Serial.println("TCP server start"); // Print message to serial monitor
}

// Function to send sensor data over TCP
void Wifi_TCP_Send() {  
  WiFiClient client = server.available(); // Check if a client is connected
  if (client) {
    // Convert sensor data to string
    dtostrf(grs, 4, 0, grs_str);
    dtostrf(ibi, 4, 0, ibi_str);
    dtostrf(bpm, 3, 0, bpm_str);
    dtostrf(temperature, 5, 2, temperature_str);
    dtostrf(humid, 5, 2, humid_str);
    Serial.println("connected"); // Print message to serial monitor
    Serial.println("Sending message"); // Print message to serial monitor
    for (int i = 0; i < 3; i++) {
      client.print('G');
      chang = sizeof(grs_str);
      for (short i = 0; i < chang; i++) {
        Serial.println(grs_str[i]); // Print data to serial monitor
        client.print(grs_str[i]); // Send data to the client
      }

      client.print('I');
      chang = sizeof(ibi_str);
      for (short i = 0; i < chang; i++) {
        Serial.println(ibi_str[i]); // Print data to serial monitor
        client.print(ibi_str[i]); // Send data to the client
      }
      
      client.print('B');
      chang = sizeof(bpm_str);
      for (short i = 0; i < chang; i++) {
        Serial.println(bpm_str[i]); // Print data to serial monitor
        client.print(bpm_str[i]); // Send data to the client
      }

      client.print('T');
      chang = sizeof(temperature_str);
      for (short i = 0; i < chang; i++) { 
        Serial.println(temperature_str[i]); // Print data to serial monitor
        client.print(temperature_str[i]); // Send data to the client
      } 

      client.print('H');
      chang = sizeof(humid_str);
      for (short i = 0; i < chang; i++) { 
        Serial.println(humid_str[i]); // Print data to serial monitor
        client.print(humid_str[i]); // Send data to the client
      } 
    }
    Serial.println("sending end"); // Print message to serial monitor
  } else {
    Serial.println("no connection"); // Print message to serial monitor
    client.stop(); // Stop the client
    delay(500); // Wait for 500 milliseconds
  }
}

void setup() {
  Wifi_TCP_Init(); // Initialize WiFi and start the server
  Serial.println("TCP server start"); // Print message to serial monitor
}

void loop() {
  while (1) {
    Wifi_TCP_Send(); // Continuously send sensor data over TCP
  }
}

Interface and application programming

Developing the data processing program and deploy it on the Raspberry Pi. Then build a user interface to display analyzed bio-signal data, making results accessible for end users.

import socket
import time

if __name__ == '__main__':
    Read_message = []  # List to store the received messages
    tcp_client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # Create a TCP/IP socket
    tcp_client_socket.connect(("192.168.4.1", 1122))  # Connect to the server at IP address 192.168.4.1 on port 1122
    time.sleep(1)  # Wait for 1 second

    recv_data = tcp_client_socket.recv(1024)  # Receive data from the server (up to 1024 bytes)
    recv_content = recv_data.decode("gbk")  # Decode the received data using GBK encoding
    print("Received data from the server: ", end='')  # Print message "Received data from the server: " without a newline

    # Loop through the received content to parse and print the sensor data
    for i in range(0, 100):
        if recv_content[i] == 'G':  # Check if the current character is 'G' (indicating GSR data)
            Read_message.append('Gsr:')  # Append 'Gsr:' to the list
            if recv_content[i + 1] != ' ':
                Read_message.append(recv_content[i + 1] + recv_content[i + 2] + recv_content[i + 3] + recv_content[i + 4])
            if recv_content[i + 1] == ' ':
                Read_message.append(recv_content[i + 2] + recv_content[i + 3] + recv_content[i + 4])
            if recv_content[i + 2] == ' ':
                Read_message.append(recv_content[i + 3] + recv_content[i + 4])
            if recv_content[i + 3] == ' ':
                Read_message.append(recv_content[i + 4])
            print(Read_message[0], Read_message[1], end=' ')  # Print the GSR data

        if recv_content[i] == 'I':  # Check if the current character is 'I' (indicating IBI data)
            Read_message.append('Ibi:')  # Append 'Ibi:' to the list
            if recv_content[i + 1] != ' ':
                Read_message.append(recv_content[i + 1] + recv_content[i + 2] + recv_content[i + 3] + recv_content[i + 4])
            if recv_content[i + 1] == ' ':
                Read_message.append(recv_content[i + 2] + recv_content[i + 3] + recv_content[i + 4])
            print(Read_message[2], Read_message[3], end=' ')  # Print the IBI data

        if recv_content[i] == 'B':  # Check if the current character is 'B' (indicating BPM data)
            Read_message.append('Bpm:')  # Append 'Bpm:' to the list
            if recv_content[i + 1] != ' ':
                Read_message.append(recv_content[i + 1] + recv_content[i + 2] + recv_content[i + 3])
            if recv_content[i + 1] == ' ':
                Read_message.append(recv_content[i + 2] + recv_content[i + 3])
            print(Read_message[4], Read_message[5], end=' ')  # Print the BPM data

        if recv_content[i] == 'T':  # Check if the current character is 'T' (indicating Temperature data)
            Read_message.append('Temperature:')  # Append 'Temperature:' to the list
            Read_message.append(recv_content[i + 1] + recv_content[i + 2] + recv_content[i + 3] + recv_content[i + 4] + recv_content[i + 5])
            print(Read_message[6], Read_message[7])  # Print the Temperature data

        if recv_content[i] == 'H':  # Check if the current character is 'H' (indicating Humidity data)
            Read_message.append('Humidity:')  # Append 'Humidity:' to the list
            Read_message.append(recv_content[i + 1] + recv_content[i + 2] + recv_content[i + 3] + recv_content[i + 4] + recv_content[i + 5])
            print(Read_message[8], Read_message[9])  # Print the Humidity data
            break  # Exit the loop

Assemble & test.

See assemble and test result.

Material BOM List

The purchase list involved in this project is shown as table below.

Questions Answered

What does it do?
The project is a motion sickness detection toolkit that collects and analyzes human physiological signals (such as motion, PPG, and skin temperature) to help predict the onset of motion sickness. It is intended for use in a car riding environment to assist individuals prone to motion sickness.
Who’s done what beforehand?
Previous research has focused on understanding motion sickness and detecting it through biosignal analysis. There are some existing tools and devices for general health monitoring, but they are often not specialized for real-time motion sickness prediction, especially in vehicle settings.
What did you design?
The project involved designing:

A sensor system to collect relevant physiological signals.
A data processing module based on a Raspberry Pi to manage and process the collected data.
A user interface with a screen to display the motion sickness dose value.
A housing to securely contain the biosignal detection module, with branding elements like a logo sticker.

What materials and components were used?
Materials and components included:

Sensors: for measuring motion, PPG (photoplethysmography), and skin temperature.
Raspberry Pi: for data processing and Wifi communication.
Screen: to display motion sickness values.
Housing materials: for the 3D-printed casing.
Miscellaneous electronics: for sensor connections and PCB production.

Where did they come from?
These components were sourced from various electronics suppliers, such as online electronic stores (e.g., Adafruit, SparkFun), 3D printing services or local labs, and general hardware suppliers for materials like acrylic sheets for cutting.
How much did they cost?
The costs is approximately 3000 CNY including materials and fabrication cost.
What parts and systems were made?

The sensor system for data collection.
The data processing and communication system on the Raspberry Pi.
The display interface showing motion sickness values.
The housing and logo sticker for branding and protection.

What processes were used?
Key processes included:

Embedded programming for sensor data processing.
Electronic design and assembly for integrating sensors and the Raspberry Pi.
3D printing for the housing.
Computer-controlled cutting for the logo sticker.
Interface programming to display processed data on the screen.

What questions were answered?
The project answered:

Can physiological signals reliably predict motion sickness?
Is it feasible to build a compact, portable toolkit for real-time use in a car?
What combination of sensors and processing techniques provides the most accurate detection?

What worked? What didn’t?
Worked: Data collection from sensors, basic data processing, Wifi communication, and initial testing in a controlled environment.
Didn’t work: The original plan was to use AI models deployed on raspberry-pi to predict the motion sickness, this was not realized since the time is limited.
How was it evaluated?
The toolkit was evaluated through:

Controlled testing of sensor accuracy and data processing.
User feedback on the display interface and usability.
Performance tests in a car environment to assess real-world reliability.

What are the implications?
This toolkit could advance real-time motion sickness detection technology in automotive settings, making it useful for both passengers and automotive companies. It opens up possibilities for developing more personalized, adaptive vehicle environments and for future studies on motion sickness mitigation techniques.

Final Project - Bio-signal Data Collection & Analyzation Tool Kit for Motion Sickness Prediction ​

Final Result ​

Hero shots ​

Idea Sketch ​

Computer Aided Design ​

Computer Controlled Cutting ​

3D Printing ​

Input & Output device testing ​

Electronic Design & Production ​

Embedded Programming for sensors and network communication ​

Interface and application programming ​

Assemble & test. ​

Material BOM List ​

Questions Answered ​