14. Interface and application programming¶
Trying out Unity Game Engine¶
Setting up the Unity Project¶
First, we started a new Unity project using the simple 3D template:
In order for the serial connection to work with Unity, we need to change the App Compatibility Level to .NET 4.x:
We used a free model from turbosquid:
We can then just drag and drop the obj file into our Assets and it is automatically imported into Unity:
To create the environment, we just used cubes that we scaled in different dimensions to make the ground and some buildings.
We then created a new Material to give some colors to our world:
And here is the result:
We added a Box Collider and a RigidBody to our plane to allow for collisions. But we had to turn off gravity in order for our plane to stay airborne:
Scripting the Behavior¶
We then added a script to our plane and had to program it. The harder part was reading the serial port and processing the input. In order to understand how to handle the serial port, we used this former group assignment. To process the input, we used deepseek.
Prompt for deepseek R1:
ok in unity i am reading from serial like:
if (read > 0)
{
var str = System.Text.Encoding.Default.GetString(buff);
Debug.Log(str);
}
and this str contains text for rotation coorinates, in this format
x.xx;y.yy;z.zz/
x.xx;y.yy;z.zz/
x.xx;y.yy;z.zz/
but the problem is that its not sure that its complete all the time. (this thing is in update). so maybe i would need to store some part of the string to get all the coord. I want to have at the end 3 floats one for each coord
It gave us some functions that enabled us to get 2 functions that parse the buffer input. And the, we just had to use these rotation rates to rotate the plane in game.
Final C# Code:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.IO.Ports;
using System.IO; // Requires .NET 4 in Project Settings
using System;
public class PlaneController : MonoBehaviour
{
[SerializeField] private float speed = 5f;
private string dataBuffer = "";
private float xRotationRate = 0;
private float yRotationRate = 0;
private float zRotationRate = 0;
SerialPort _serial;
// Start is called before the first frame update
void Start()
{
//Serial initialization
string the_com = "";
/*foreach (string mysps in SerialPort.GetPortNames())
{
print(mysps);
if (mysps != "COM1") { the_com = mysps; break; }
}*/
the_com = "/dev/cu.usbmodem14101";
//sp = new SerialPort("\\\\.\\" + the_com, 9600);
_serial = new SerialPort(the_com, 115200);
if (!_serial.IsOpen)
{
print("Opening " + the_com + ", baud 115200");
_serial.ReadTimeout = 100;
_serial.Handshake = Handshake.None;
_serial.Open();
if (_serial.IsOpen) { print("Open"); }
}
}
// Update is called once per frame
void Update()
{
transform.Translate(Vector3.forward * speed * Time.deltaTime);
xRotationRate = 0;
yRotationRate = 0;
zRotationRate = 0;
// Make sure we have a serial port and that the connection is open
if (_serial != null) {
if (_serial.IsOpen) {
if (_serial.BytesToRead > 0) {
byte[] buff = new byte[_serial.BytesToRead];
int read = _serial.Read(buff, 0, buff.Length);
if (read > 0)
{
// Append new data to buffer
dataBuffer += System.Text.Encoding.ASCII.GetString(buff);
// Process complete coordinates
ProcessBuffer();
}
}
} else {
Debug.LogWarning("serial Closed");
}
} else {
Debug.LogWarning("serial NULL");
}
transform.Rotate(Vector3.up, 70 * xRotationRate * Time.deltaTime);
transform.Rotate(Vector3.right, 70 * zRotationRate * Time.deltaTime);
transform.Rotate(Vector3.forward, 70 * yRotationRate * Time.deltaTime);
}
void ProcessBuffer()
{
// Split buffer into complete messages (ending with '/') and remaining data
int lastSplitIndex = dataBuffer.LastIndexOf('/');
if (lastSplitIndex >= 0)
{
// Extract complete messages
string completeData = dataBuffer.Substring(0, lastSplitIndex);
string[] coordinates = completeData.Split(new[] { '/' }, StringSplitOptions.RemoveEmptyEntries);
// Keep remaining incomplete data
dataBuffer = dataBuffer.Substring(lastSplitIndex + 1);
// Process all complete coordinates
foreach (string coord in coordinates)
{
ParseCoordinate(coord.Trim());
}
}
}
void ParseCoordinate(string coordinate)
{
string[] components = coordinate.Split(';');
if (components.Length == 3)
{
float x, y, z;
if (float.TryParse(components[0], out x) &&
float.TryParse(components[1], out y) &&
float.TryParse(components[2], out z))
{
// Use the parsed coordinates
//Debug.Log($"Parsed coordinates: X={x}, Y={y}, Z={z}");
xRotationRate = x;
yRotationRate = y;
zRotationRate = z;
// transform.rotation = Quaternion.Euler(x, y, z);
}
else
{
Debug.LogWarning("Failed to parse coordinate: " + coordinate);
}
}
else
{
Debug.LogWarning("Invalid coordinate format: " + coordinate);
}
}
}
The key lines are:
//moves the plane forward:
transform.Translate(Vector3.forward * speed * Time.deltaTime);
...
//rotates the planes:
transform.Rotate(Vector3.up, 70 * xRotationRate * Time.deltaTime);
transform.Rotate(Vector3.right, 70 * zRotationRate * Time.deltaTime);
transform.Rotate(Vector3.forward, 70 * yRotationRate * Time.deltaTime);
Result¶
The result is a not very playable game, but we were able to somewhat control the plane using our IMU and Arduino: