Skip to content

15. Interfacing and Application

The goal of this week was to program a interface and apply it to one’s individual project. Since I was working with the BNO-055, I wanted to use the different values to try and program multiple small interfaces. I used micropython, u8g2, and tried Adafruit’s 3D viewer over the span of this week.

Assignment

Individual Assignment:

  • Write an application that interfaces a user with an input &/or output device that you made.

Group Assignment:

  • Compare as many tool options as possible.

BNO-055 3D Viewer

The first task that I wanted to accomplish was to use Adafruit’s Web Serial 3D Visualizer to display the orientation values of the BNO055, so I went to this tutorial for the visualizer in hopes of being able to connect it and properly, well, visualize the values. The first major error that I faced was that the calibration method that the tutorial specified did not operate properly. Later on, it turned out that I had the wrong tutorial pulled up, as I had a page that was still under Adafruit but for some other way of doing the tutorial. When running the separate tutorial’s “sensor_calibration_read” example code, I recieved dozens of errors, and the message shown below was very similar to many of them. 

Multiple libraries were found for "SdFat.h"
  Used: C:\Users\lando\OneDrive\Documents\Arduino\libraries\SdFat_-_Adafruit_Fork
  Not used: C:\Users\lando\AppData\Local\Arduino15\packages\rp2040\hardware\rp2040\3.8.0\libraries\ESP8266SdFat
Multiple libraries were found for "SD.h"
  Used: C:\Users\lando\AppData\Local\Arduino15\packages\rp2040\hardware\rp2040\3.8.0\libraries\SD
  Not used: C:\Users\lando\AppData\Local\Arduino15\libraries\SD

However, one of my fellow students said that this was not the actual error, and there was something else going on that was preventing the code from working. Since these messages had mentioned the RP2040, as in the Seeed Xiao RP2040, I decided to try switching boards to my ATtiny1614. Even after I did so, it still did not operate correctly, with more errors stating that I did not have access or the port wasn’t identified properly. Ensure of what these meant and upset with how long it was taking, I switched to a different interfacing method.

Orientation Visualizer

For this next interface, I wanted to try to use Thonny and micropython to create a graph that displays the orientation values. I installed Thonny from here and used it to read the Serial data from the Arduino Monitor. I had GPT 4.0 assist me in a significant amount of the coding, as I believed that finding a specific example or tutorial for my goal would be more time consuming than spending the effort to attempt to understand and use the code that GPT presents. I moved the Thonny folder to an accessible location in my computer to make it easier to install things, and I installed some libraries for the program. I typed in pip install _ for the different libraries, inlcuding serial, matplotlib, and numpy. Then, I opened the program in Thonny and pasted the code into it. At first, I recieved an error since I had not installed the matplotlib library correctly, so I went back and ensured that it was correct. The code that I initially used is the following:

import serial
import time
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

// Setup Serial
ser = serial.Serial('COM12', 9600) 

// Setup matplotlib for real-time updating in 3D
plt.ion()
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
arrow = ax.quiver(0, 0, 0, 0, 0, 1)  # Initial arrow pointing along z-axis

ax.set_xlim(-2, 2)
ax.set_ylim(-2, 2)
ax.set_zlim(-2, 2)

def read_orientation():
    if ser.in_waiting > 0:
        line = ser.readline().decode('utf-8').strip()
        if line.startswith('Orientation'):
            _, values = line.split(':')
            x, y, z = map(float, values.split(','))
            return x, y, z
    return None

def update_arrow(x, y, z):
    global arrow
    arrow.remove()
    arrow = ax.quiver(0, 0, 0, x, y, z, color='blue')
    plt.draw()
    plt.pause(0.01)

while True:
    orientation = read_orientation()
    if orientation:
        x, y, z = orientation
        update_arrow(x, y, z)
        print("X: {:.2f}, Y: {:.2f}, Z: {:.2f}".format(x, y, z))
    time.sleep(1)

I also updated the Arduino IDE sensor code to greatly simplify it so that it would only display the necessary information in the Serial monitor, which could then be translated into the correct values in Thonny.

#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BNO055.h>
#include <utility/imumaths.h>

Adafruit_BNO055 bno = Adafruit_BNO055(55, 0x28, &Wire);

void setup(void) {
  Serial.begin(9600);
  while (!Serial);  // Wait for serial port to connect

  if (!bno.begin()) {
    Serial.println("No BNO055 detected ... Check your wiring or I2C ADDR!");
    while (1);
  }
  delay(1000);  // Give sensor time to stabilize
}

void loop(void) {
  sensors_event_t orientationData;
  bno.getEvent(&orientationData, Adafruit_BNO055::VECTOR_EULER);
  if (orientationData.type == SENSOR_TYPE_ORIENTATION) {
    Serial.print("Orientation: ");
    Serial.print(orientationData.orientation.x);
    Serial.print(", ");
    Serial.print(orientationData.orientation.y);
    Serial.print(", ");
    Serial.println(orientationData.orientation.z);
  }
  delay(100);  // Sampling rate delay
}

Since it was not updating, I went into the Thonny code to try and adjust the limit values, as I saw on the code that these were very low and thought that they needed to be increased to the maximum values the each of the orientation values could go up to. Additionally, I found the FuncAnimation function of matplotlib, which I found could be used to display the 3D and updatable orientation data. I used ChatGPT once again to adjust the Thonny file to be the following:

import serial
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from mpl_toolkits.mplot3d import Axes3D

// Setup Serial
ser = serial.Serial('COM12', 9600, timeout=1)

// Setup matplotlib for real-time updating in 3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.set_xlim(-1, 361)  # X goes from 0 to 360
ax.set_ylim(-91, 91)  # Y goes from -90 to 90
ax.set_zlim(-181, 181)  # Z goes from -180 to 180

def read_orientation():
    if ser.in_waiting > 0:
        line = ser.readline().decode('utf-8').strip()
        if line.startswith('Orientation'):
            _, values = line.split(': ')
            x, y, z = map(float, values.split(', '))
            return x, y, z
    return None

def update(frame):
    orientation = read_orientation()
    if orientation:
        x, y, z = orientation
        ax.cla()  # Clear the previous arrow
        ax.quiver(0, 0, 0, x, y, z, color='blue', length=1, normalize=True)
        ax.set_xlim(-1, 361)
        ax.set_ylim(-91, 91)
        ax.set_zlim(-181, 181)
        ax.figure.canvas.draw()

ani = FuncAnimation(fig, update, interval=100)  # Update the plot every 100 ms
plt.show()

While this did produce a decent result, it did not work fully as intended, and it would not adjust the x value properly, instead randomly defaulting to just pointing forward. I went back to GPT, as I had virtually no experience in coding with micropython, much less with the libraries and relation of Arduino IDE to Thonny. Even though I had believed that creating these boundaries would allow the program to properly display the correct information, it ended up making the graph into a large 3D space where the arrow wouldn’t turn around and represent the motion sensor, like I intended, and instead it had made the arrow point towards the x value on this long graph. I decided to revert to similar value to what GPT gave me, and from there I changed the code to calculate, using the numpy functions, the direction the arrow should be pointing. The z value ended up not being used to determine the direction the arrow need to face, as this value would have rotated the arrow around itself and not the center point of the graph. 

import serial
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from mpl_toolkits.mplot3d import Axes3D
import numpy as np  # Make sure numpy is imported

# Setup Serial
ser = serial.Serial('COM12', 9600, timeout=1)

# Setup matplotlib for real-time updating in 3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.set_xlim(-1, 1)
ax.set_ylim(-1, 1)
ax.set_zlim(-1, 1)

def read_orientation():
    if ser.in_waiting > 0:
        line = ser.readline().decode('utf-8').strip()
        if line.startswith('Orientation'):
            _, values = line.split(': ')
            x, y, z = map(float, values.split(', '))
            return x, y, z
    return None

def update(frame):
    orientation = read_orientation()
    if orientation:
        x, y, z = orientation
        # Convert degrees to radians
        x_rad = np.radians(x)  # Heading
        y_rad = np.radians(y)  # Pitch
        z_rad = np.radians(z)  # Roll

        # Compute direction vector in 3D space
        dx = np.cos(z_rad) * np.cos(x_rad)
        dy = np.cos(z_rad) * np.sin(x_rad)
        dz = np.sin(z_rad)

        ax.cla()  # Clear the previous arrow
        ax.quiver(0, 0, 0, dx, dy, dz, color='blue', length=1, normalize=True)
        ax.set_xlim(-1, 1)
        ax.set_ylim(-1, 1)
        ax.set_zlim(-1, 1)
        ax.figure.canvas.draw()

ani = FuncAnimation(fig, update, interval=100, cache_frame_data=False)  # Update the plot every 100 ms
plt.show()

This ended up producing the intended result, with the arrow pointing around in a 360 degree circle, as well as being able to rotate up and down if the sensor is moved.

Now that I had this, Mr. Dubick suggested that I use a website for OLEDs that he found, so I decided to look into it as well.

OLED Graphic Interface

To program the OLED in a effecient method and not spend hours testing over and over to recieve the intended result, I used the Lopaka Graphics Editor. This allowed me to design what I would put on the OLED screen ahead of time, resulting in a better made display with far more customizability. At this point in my project, I had been using the ATtiny 1614 for a long time, and as a result I had to use the SSD1306 Ascii library, which did not take up signficant storage but correspondingly did not have many capabilites. To fix this and be able to use this library, Mr. Dubick told me to just switch over to a larger chip. For now, I have decided to just use the Xiao RP2040, which has a large amount of storage and is our lab’s semi-standard for larger chips. In doing so, I was no able to do far more with my programming, as I had originally been limited to just the aformentioned library. Something I realized in this process that I had been worried about before was the storage concerns of using both u8g2 and the BNO055 libraries, but this ended up not being a problem until I tried to add a second bitmap.

Similar to the idea I had for the interface of the sensor with Thonny, I wanted to create a circular graph that would display the orientation of the x value for the sensor. I went back to my code from Week 9 and grabbed the u8g2 code that I had before, which included the proper constructor line for the SSD1306 OLEDs that our lab has.

U8G2_SSD1306_128X64_NONAME_F_HW_I2C u8g2(U8G2_R0, U8X8_PIN_NONE);

This part allowed me to specifically interact with the SSD1306 instead so that the library could adjust to the proper settings and address. I then used the aforementioned graphics design website to create a line within a circle and attempted to match the position of the line to the sensor’s orientation x value. Then, since I still had the setup with the RP2040 from the previous project, I tried to upload the following code to the OLED, with the sensor also being connected.

#include <Arduino.h>
#include <U8g2lib.h>
#include <SPI.h>
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BNO055.h>
#include <utility/imumaths.h>

static const unsigned char image_Vol_up_hvr_bits[] U8X8_PROGMEM = {
  0xf8,0xff,0x3f,0x00,0xfc,0xff,0x7f,0x00,0xfe,0xff,0xff,0x00,0xff,0xff,0xff,0x01,0xff,0xff,0xff,0x01,0xff,0xff,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0x3f,0x00,0xf8,0x01,0x3f,0x00,0xf8,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xef,0xff,0x01,0xff,0xff,0xff,0x01,0xff,0xff,0xff,0x01,0xff,0xff,0xff,0x01,0xff,0xff,0xff,0x01,0xfe,0xff,0xff,0x00,0xfc,0xff,0x7f,0x00,0xf8,0xff,0x3f,0x00
};

Adafruit_BNO055 bno = Adafruit_BNO055(55, 0x28, &Wire);

U8G2_SSD1306_128X64_NONAME_F_HW_I2C u8g2(U8G2_R0, /* reset=*/ U8X8_PIN_NONE);

int radius = 28;
int x_offset = 32;
int y_offset = 39;

double x;


void setup(void) {
  u8g2.begin();
  Serial.begin(9600);
  if (!bno.begin()) {
    Serial.println("No BNO055 detected ... Check your wiring or I2C ADDR!");
    while (1);
  }
  delay(1000);  // Give sensor time to stabilize

}

void loop(void) {
  delay(100);
  sensors_event_t orientationData;
  bno.getEvent(&orientationData, Adafruit_BNO055::VECTOR_EULER);
  if (orientationData.type == SENSOR_TYPE_ORIENTATION) {
    x = orientationData.orientation.x;
    Serial.print("Orientation: ");
    Serial.print(orientationData.orientation.x);
    Serial.print(", ");
    Serial.print(orientationData.orientation.y);
    Serial.print(", ");
    Serial.println(orientationData.orientation.z);
  }

  double x_val = cos(radians(x)) * radius;
  double y_val = sin(radians(x)) * radius; 

  int x_real = x_val + x_offset;
  int y_real = y_val + y_offset;

  u8g2.firstPage();
  u8g2.clearBuffer();
  u8g2.setFontMode(1);
  u8g2.setBitmapMode(1);
  u8g2.drawFrame(5, 6, 59, 11);
  u8g2.drawEllipse(31, 39, 20, 20);
  u8g2.drawEllipse(31, 39, 21, 21);
  u8g2.drawXBM(96, 24, 25, 27, image_Vol_up_hvr_bits);
  u8g2.drawLine(x_real, y_real, 32, 39);
  u8g2.sendBuffer();

}

Unfortunately, this did not work, so I tried troubleshooting for a while until I changed two things - switching to my ATtiny1614 board through the QuenTorres board and changing the OLED from the one with the manually adjusted address to the standard one.

While not sure which one of these two changes made it work, it still ended up turning on and displaying the orientation along the circle. Some of the values were still off, meaning that the line was not aligned to the circle, but it was still good progress. Additionally, the bitmaps/images were not displaying, so I planned to remove those. Lastly, since the current value it was displaying was meant to be the orientation in the x-direction, I needed to adjust the circle to an ellipse to make it look better in perspective for what it was representing.

After I had this result and knew the changes I wanted to make, I went back into the Lopaka website and adjusted my code, before modifying it to fit within my current code. Below is the code used.

#include <Arduino.h>
#include <U8g2lib.h>
#include <SPI.h>
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BNO055.h>
#include <utility/imumaths.h>


Adafruit_BNO055 bno = Adafruit_BNO055(55, 0x28, &Wire);

U8G2_SSD1306_128X64_NONAME_F_HW_I2C u8g2(U8G2_R0, /* reset=*/ U8X8_PIN_NONE);

int x_radius = 29;
int y_radius = 13;
int x_offset = 32;
int y_offset = 39;

double x;


void setup(void) {
  u8g2.begin();
  Serial.begin(9600);
  if (!bno.begin()) {
    Serial.println("No BNO055 detected ... Check your wiring or I2C ADDR!");
    while (1);
  }
  delay(1000);  // Give sensor time to stabilize

}

void loop(void) {
  delay(100);
  sensors_event_t orientationData;
  bno.getEvent(&orientationData, Adafruit_BNO055::VECTOR_EULER);
  if (orientationData.type == SENSOR_TYPE_ORIENTATION) {
    Serial.print("Orientation: ");
    Serial.print(orientationData.orientation.x);
    Serial.print(", ");
    Serial.print(orientationData.orientation.y);
    Serial.print(", ");
    Serial.println(orientationData.orientation.z);
  }

  double angle = radians(orientationData.orientation.x);
  double x_val = cos(angle) * x_radius;
  double y_val = sin(angle) * y_radius; 

  int x_real = x_val + x_offset;
  int y_real = y_val + y_offset;

u8g2.clearBuffer();
u8g2.setFontMode(1);
u8g2.setBitmapMode(1);
u8g2.drawFrame(5, 6, 69, 11);
u8g2.drawEllipse(x_offset, y_offset, x_radius, y_radius);
u8g2.drawLine(x_real, y_real, x_offset, y_offset);
// u8g2.drawEllipse(31, 39, 21, 21);
// u8g2.drawXBM(56, 35, 30, 23, image_Warning_bits);
u8g2.setFont(u8g2_font_6x10_tr);
u8g2.drawStr(7, 15, "Orientation");
// u8g2.drawXBM(93, 6, 29, 29, image_Pin_attention_dpad_bits);
u8g2.sendBuffer();
}

After I got this to work, I wanted to be able to program two OLED screens with I2C, for which there was not an immediately available solution, such as changing an address of something of the sort. Before I tried to do this, I wanted to be able to read the accelerometer data, and after some testing I figured out that the data was impacted by gravity whenever I attempted to turn it. I had the header pins for my sensor on the bottom, so I wanted to change them to be soldered onto the top, which required a very lengthy process. However, when I finished soldering, as I suspected from the immense amount of heat I had put into the board, the chip had broken.

:(

This was a very devastating issue, and it meant that I would have to wait a couple days before I could resume my work.

Once I got my sensor back once again, I soldered on the pieces and made sure to test out the sensor using the “read_all_data” sample code under the BNO-055 library. I found that it worked like normal, and got back to trying to program the dual OLEDs through U8G2

I searched up how to program two OLEDS through U8G2 and found this forum that detailed the use of the setI2CAddress() function in the u8g2 library, which allowed me to set two different I2C addresses for the two different OLEDs. Before this specific forum, I had looked across several other forums, all of which provided options that did not solve the issue. The solution I obtained from the linked forum worked, and I was now effectively done with the week.

#include <Arduino.h>
#include <U8g2lib.h>
#include <SPI.h>
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BNO055.h>
#include <utility/imumaths.h>


Adafruit_BNO055 bno = Adafruit_BNO055(55, 0x28, &Wire);

U8G2_SSD1306_128X64_NONAME_F_HW_I2C u8g1(U8G2_R0, U8X8_PIN_NONE);
U8G2_SSD1306_128X64_NONAME_F_HW_I2C u8g2(U8G2_R0, U8X8_PIN_NONE);

int I2C_SLA;



void setup(void) {
  u8g1.setI2CAddress(0x3C * 2);        
  u8g2.setI2CAddress(0x3D * 2);        

  u8g1.begin();
  u8g2.begin();
  delay(1000);  
}

void loop(void) {
  u8g1.clearBuffer();
  u8g1.setFontMode(1);
  u8g1.drawLine(40,10, 15, 30);
  u8g1.sendBuffer();

  u8g2.clearBuffer();
  u8g2.setFontMode(1);
  u8g2.drawFrame(5, 6, 69, 11);
  u8g2.drawLine(5,10,15,20);
  u8g2.sendBuffer();

  delay(100);
}

Now that I had accomplished this, my interfacing week was done.

Group Project

The goal of this week’s group project was to detail each process used by the members of the group. This was a simple goal that just required writing, with no physical work being necessary. The link to the group page can be found here.

Individual Contribution

My contribution to the week was writing about Matplotlib, a library for graphing data in various ways in Python, and writing about Lopaka and U8G2, the softwares I used for programming the OLED screens in my project.

Reflection

This was a useful week for my final project that helped me increase the visual appeal and neatness of my OLED screens. It incorporated many of the previous aspects of my Fab experience while also letting me make significant progress on my final project. The primary struggle I encountered was the loss of my BNO-055 sensor, so the delay of getting a second one slowed down my progress a fair bit. However, I am glad that I was able to accomplish what I did in this week, and the overall process was fun.