15. Interface & App Programming

---

Week assignment

• We have to compare as many tool options as possible (in group)
• I have to write an application that interfaces a user with an input or output device that I’ve made (individually)

---

Comparing tool options

I wanted to compare different types of interfaces. So, I picked Scratch which uses a no-code UI, Processing which is pretty similar to Arduino IDE but focused on electronic arts, new media art, and visual design, and then I used Unity 3D and Adruino IDE for my individual assignment.

Scratch is by far the easiest tool to use. It uses block-based visual programming language and it’s aimed at kids. You just need to pick a character and then drag and drop blocks to animate it. I choose to animate a bat by creating a loop. The bat just moved, rotate, changed it’s size and displayed a thinking bubble. I decided to add a background and found that you can customize your character and add sounds. There are multiple tutorials and it seems you can get to a pretty complex thing.

When I finished my first sketch I added a donut and make it follow the bat while rotating and displaying a text bubble. Overall, I found this platform enjoyable and simple to use.

Processing is quite a friendly tool too, although not as friendly as Scratch. They define the tool as “a flexible software sketchbook and a language for learning how to code within the context of the visual arts.” So, as I wanted to experiment I decided to make a simple animated 2D shape. I created this character: Dynamic Bob and write the following code. Bob follows your mouse cursor while printing “Hi! I’m Bob and I will follow you forever :)”

To acomplish this, I followed some of the basic tutorials they have on their website. I found it easy and intuitive, probably because I’ve already have some experience using Arduino IDE. However, I felt that Processing is easier as it’s logic is similar to the way I think. When using Arduino I have to learn how things work or to look for examples every time I need to create something new.

void setup() {
  size(250,250); 

}

void draw() {
  background(250);
// Bob's body
noStroke();
fill(0); 
ellipseMode(CENTER);
ellipse(mouseX,mouseY,110,100);

// Bob's head
ellipse(mouseX,mouseY-70,70,70);
fill(255); 
ellipse(mouseX-10,mouseY-70,5,5);
ellipse(mouseX+10,mouseY-70,5,5);
stroke(255);
strokeWeight(4); 
line(mouseX-10,mouseY-60,mouseX+10,mouseY-60); 

// Bob's hands
fill(0); 
noStroke();
rect(mouseX-60,mouseY-30,20,20,30);
rect(mouseX+40,mouseY-30,20,20,30);

// Bob's legs
rect(mouseX-25,mouseY+40,15,25,10);
rect(mouseX+10,mouseY+40,15,25,10);

// BoB Talking

println("Hi! I'm Bob and I will follow you forever :)"); 

}

After Dynamic Bob, I change a little bit the code and created Fullfilling Bob.

void setup() {
  size(250,250); 
  background(250);
}

void draw() {

// Bob's body
noStroke();
fill(0); 
ellipseMode(CENTER);
ellipse(mouseX,mouseY,110,100);

// Bob's head
ellipse(mouseX,mouseY-70,70,70);
fill(255); 
ellipse(mouseX-10,mouseY-70,5,5);
ellipse(mouseX+10,mouseY-70,5,5);
stroke(255);
strokeWeight(4); 
line(mouseX-10,mouseY-60,mouseX+10,mouseY-60); 

// Bob's hands
fill(0); 
noStroke();
rect(mouseX-60,mouseY-30,20,20,30);
rect(mouseX+40,mouseY-30,20,20,30);

// Bob's legs
rect(mouseX-25,mouseY+40,15,25,10);
rect(mouseX+10,mouseY+40,15,25,10);

// BoB Talking

println("Hi! I'm Bob and I will fill you forever :)"); 

}

I noticed that when they were running at the same time they printed a line with ‘fill’ and next one with ‘follow’.

I used Unity 3D to develop the interface for my final project. It is a tool that mixes code and a no-code interface. While the platform’s UI is confusing, once you get used to it, it’s relatively straightforward. But doing advanced things can be very complex. At least for someone with basic knowledge like me.

This platform allows you to combine scripts written in C# with a UI from where you can also modify variables defined in the code.

In the next section of this assignment I explained how I made the graphic interface for the OLED using Arduino IDE and how I developed the first part of the game for my final project using Unity 3D.

Interface Programming

For this week I wanted to continue with the progress for my final project. My project has 2 interactive interfaces. The first involves a pedometer, which I have not yet programmed, and that when walking changes the faces in an oled. The second is a 2D game made in unity 3D that can be controlled using the gyroscope of the MPU6050 module on my board.

I decided to start with the game since I still have not solved the pedometer.

Using the bluetooth BLE and the gyro

My board has an Attiny3216, an MPU605, a 128x64 I2C OLED and a bluetooth BLE HC-10. I knew there was a library for HC-10 in Unity. Therefore I decided to acquire it to make my life easier. The other decision I made was to download a demo of a game since the time is very short and I will not be able to develop something decent from scratch. My idea is to use something pre-existing and create my custom sprites.

1. Library HC-10
2. Demo game Unity

I had the code for the gyroscope I used for the OLED output week. Therefore, the first change I made was to modify the code so that in addition to sending the X it sends the Y on the Arduino. I used the same code but didn’t connect the OLED. It didn’t work. Later, I realized that if the OLED is not connected the code does not work. For this assignment I just wanted to try the bluetootha and gyro with Unity 3D. So, I removed all the parts related to the OLED.

This is the code that I used:

#include <Wire.h>
#include <MPU6050_light.h>
MPU6050 mpu(Wire);

#include <SoftwareSerial.h>

SoftwareSerial bluetooth(6, 7); // RX, TX


void setup() {
 // set the data rate for the SoftwareSerial port
  bluetooth.begin(9600);

  Serial.begin(9600);
   Wire.begin();
 byte status = mpu.begin();
   Serial.print(F("MPU6050 status: "));
   Serial.println(status);
   while (status != 0) { } // stop everything if could not connect to MPU6050
 Serial.println(F("Calculating offsets, do not move MPU6050"));
   delay(1000);
   mpu.calcOffsets(); // gyro and accelero
   Serial.println("Done!\n");
}

void loop() {

  mpu.update();


  delay(50);
  float angley = mpu.getAngleY(); // declare the angle Y of the gyro on the variable angle
  float anglex = mpu.getAngleX(); // declare the angle X of the gyro on the variable angle

  Serial.print(angley);
  Serial.print(":");
  Serial.println(anglex);

}

Interfacing the board with Unity 3D

1. First, I created a new project in Unity
2. Then, imported the game that I downloaded
3. Finally, imported the library for HC-10

How to get bluetooth to work

First, I used the example of the bluetooth BLE file to understand how it worked. It did not work 0_0

Therefore I had to modify it minimally to scan the device. Just change the device name in the script and in the Unity editor. The name of the device is “BT05”. I already knew the name from the iOS app.

I changed the name of the button to Roberto because I didn’t understand if my changes were saved. Then, it started showing data on the console. It was easy to make it work but a little bit tricky to understand.

Getting the bluetooth into the game

Once I managed to connect the bluetooth to Unity, I moved on to the next stage. I did the following steps:

1. Copy the file from ex. BTLE and moved it to the game project. PlayerBluetoothConnect. Just the script.
2. Make changes to the script to save and calculate what the bluetooth sends
3. Add bluetooth name
4. Rename the public class so that it is not equal to the previous one and does not collide

public class PlayerBluetoothConnect : MonoBehaviour
{
    public string DeviceName = "BT05";
    public string ServiceUUID = "FFE0";
    public string Characteristic = "FFE1";

    public static float AngleY { get; set; } //define angle Y and make it public so the game can access and use the value
    public static float AngleX { get; set; } //define angle X and make it public so the game can access and use the value

Then I had to:

• Change the values of variables X and Y
• Use gyroscope values

string data = Encoding.UTF8.GetString (bytes); //getting the bluetooth data and turning it into text to be able to process it (data = y : x)

                    string[] values = data.Split(':'); //values to recieve and tranform into an array (split the x and the y into separate values --> values = ["10.0","20.0"])
                    AngleY = float.Parse (values[0]); // variable Y: transform the text for Y into a number
                    AngleX = float.Parse (values[1]); //Variable X: transform the text for X into a number
                    });

Last but not least, I had to:

• Add in main camera the script so that it is initialized when the game starts.

• Change playerMoving.cs to use bluetooth’s X and Y values to move the player. It takes the current values of X and Y and adds or subtracts the x and y of the bluetooth. By making these changes the character moves with the gyroscope using the values calculated in the script

transform.position = Vector3.MoveTowards(transform.position, new Vector3(transform.position.x + PlayerBluetoothConnect.AngleX,transform.position.x + PlayerBluetoothConnect.AngleY), 30 * Time.deltaTime);

• Disable the collaider so they don’t kill the ship while it’s looking for the bluetooth. (To be improve)

I wanted to try some ideas for the game so I made some changes to the assest on the template I was using. Not the final ones but wanted to experiment some options and to get an idea of how much time was going to take me to create and update all sprites.

Printing faces on the OLED

This was the hardest part of the week. Probably because I was expecting to be easier. How hard it can be to draw 2 basic faces on the OLED? Well, it can get hard for someone that does not know a lot and is using an Attiny, which has limited memory. I changed my board from 1614 to 3216, which means a 100% memory upgrade. But it’s still not enough to use Adafuit Libraries. So, I needed to find some workaround. One of the best things I found this week was this website. Not only you can try your code to check if it’s going to work (without going through all the wiring to flash the memory). But also, it’a great for collaboration. I was able to share my board to get some help on how to make it work.

Sencond most important finding of the week, this website to turn your bitmaps into code. So, I used this to create the code for my faces. First I made 2 .png files for the sad and happy faces.

• Happy face code

• Sad face code

I also used the https://github.com/datacute/Tiny4kOLED/tree/master/examples

To understand how to use bitmaps with the OLED and the Attiny I used this tutorial and also Tiny4KOled examples. I have to ask for additional help because for some reason I was not able to make it work. The reason was that I was using the wrong size. So simple and so confusing 0_0

So, after a couple of hours I was able to change the faces from the sad face to the happy face. I used a 1000ms delay between one face and the other.

So, for the OLED I used the following code:

#include <Tiny4kOLED.h>
#include "SolomonSystech.h"

void setup() {

oled.begin(128, 64, sizeof(tiny4koled_init_128x64r), tiny4koled_init_128x64r);

  // To save space, the bitmap is cropped left and right,
  // intended to be drawn onto a clear screen
  oled.clear();

  // Now that the display is all setup, turn on the display
  oled.on();

  Serial.begin(9600);
   Wire.begin();

}

void loop() {

  oled.bitmap(0, 0, 128 , 8, sadface_bitmap);
  delay (1000);

  oled.bitmap(0, 0, 128 , 8, happyface_bitmap);
  delay (1000);

}

This was just the first part, now I needed to figure out how the pedometer works. Then show a sad face if the accelerometer is not changing and use a happy face if it is. I had no idea how the pedometer works. So, I started by looking up how a pedometer works.

How the pedometer works

Note

A pedometer calculates the total number of steps taken by a person using the three components of motion that are forward, vertical, and side. The pedometer system uses an accelerometer to get these values. Accelerometer continuously updates the maximum and minimum values of this 3-axis acceleration after every defined number of samples. The average value of these 3-axis (Max + Min)/2, is called the dynamic threshold level, and this threshold value is used to decide whether the step is taken or not.

An approach to how the code should work:

1. The pedometer starts the calibration as soon as it gets powered.
2. Then in the void loop function, it continuously gets the data from X, Y, and Z-axis.
3. After that, it calculates the total acceleration vector from the starting point.
4. The acceleration vector is the square root (x^2+y^2+z^2) of the X, Y, and Z-axis values.
5. Then, it compares the average acceleration values with the threshold values to count the step number.
6. If the acceleration vector crosses the threshold value, then it increases the step count; otherwise, it discards the invalid vibrations.

So, then I followed this tutorial

Used the code in the comments of that video and errased all the parts related to the OLED.

#include <MPU6050_tockn.h>
#include <Wire.h>

MPU6050 mpu6050(Wire);
long timer = 0;
int steps=0;
  float distanceinonestep=71; //change it according to your distance between your legs in cm
  float distance;

void setup() {
  Serial.begin(9600);

Wire.begin();
  mpu6050.begin();
  mpu6050.calcGyroOffsets(true);
}

void loop() {
  mpu6050.update();

  if(millis() - timer > 1000){


    Serial.print("\taccY : ");Serial.println(mpu6050.getAccY());   
  }
if(mpu6050.getAccY()>1)
{
  steps+=1;
  delay(350);
}
   distance = steps*distanceinonestep/100;
    Serial.println(distance);
}

How to change the OLED faces using the pedometer

Finally, I was able to merge both codes and I also decided to include the one I was using for the gyroscope with the bluetooth. I did this to check that everything was working fine with the Attiny3216.

This was the final code I used:

//accelerometer and gyro libraries
#include <Wire.h>
#include <MPU6050_tockn.h>
MPU6050 mpu(Wire);

//bluethooth
#include <SoftwareSerial.h>
SoftwareSerial bluetooth(6, 7); // RX, TX

//OLED and faces
#include <Tiny4kOLED.h>
#include "SolomonSystech.h"

//pedometer 
float vectorprevious; //previous movement
float vector; //current movement
float totalvector; //difference between vector and previous vector
int steps = 0; //steps 
int prevSteps = 0; //saves previous steps

void setup() {

// set the data rate for the SoftwareSerial port
  bluetooth.begin(9600);

// put your setup code here, to run once:
oled.begin(128, 64, sizeof(tiny4koled_init_128x64r), tiny4koled_init_128x64r);

  // To save space, the bitmap is cropped left and right,
  // intended to be drawn onto a clear screen
  oled.clear();

  // Now that the display is all setup, turn on the display
  oled.on();

   Serial.begin(9600);
   Wire.begin();
   mpu.begin();

   Serial.println(F("Calculating offsets, do not move MPU6050"));
   delay(1000);
   mpu.calcGyroOffsets(true); // gyro and accelero
   Serial.println("Done!\n");
}

void loop() {
mpu.update();

  delay(50);
  float angley = mpu.getAngleY(); // declare the angle Y of the gyro on the variable angle
  float anglex = mpu.getAngleX(); // declare the angle X of the gyro on the variable angle

  calculatesteps(); //calls the acction defined as calculatesteps

  Serial.print(angley);
  Serial.print(":");
  Serial.print(anglex);
  Serial.print(":");
  Serial.println(steps);//sending the amount of steps

  //If the user walks then change the face 
  if (steps - prevSteps > 0) { 
    oled.bitmap(0, 0, 128 , 8, happyface_bitmap); //happy face
  } else {
    oled.bitmap(0, 0, 128 , 8, sadface_bitmap); //sad face
  }

  prevSteps = steps;
  delay(500); 

}

void calculatesteps(){

  float accelX = mpu.getAccX();
  float accelY =   mpu.getAccY();
  float accelZ =   mpu.getAccZ();
  Serial.print(accelX);
  Serial.print(",");
  Serial.print(accelY);
  Serial.print(",");
  Serial.println(accelZ);

  vector = sqrt( (accelX * accelX) + (accelY * accelY) + (accelZ * accelZ) ); //Calculates a uniform acceleration vector
  totalvector = vector - vectorprevious; //Difference between vectors 

  if (accelY > 1){
    steps++;
  } 
// Uses the value 6 to define the difference between 2 moments and understand if there was a 'step'

  vectorprevious = vector; //saves the last value to be able to compare it with the next value 
} 

Find the final files here:

• gyro Y X + bluetooth
• Unity game bluetooth script
• Unity game player movement script
• Sad and Happy faces code
• Sad and Happy bitmap
• faces + pedometer
• faces.h
• Bob dynamic
• Bob full

---

Last update: July 7, 2021