Interface and Application Programming

Group Assignment:

Compare as many tool options as possible.

Document your work on the group work page and reflect on your individual page what you learned.

Individual Assignment:

Write an application for the embedded board that you made. that interfaces a user with an input and/or output device(s)

Have you answered these questions?

Linked to the group assignment page ✅

Documented your process.✅

Explained the UI that you made and how you did it.✅

Explained how your application communicates with your embedded microcontroller board.✅

Explained any problems you encountered and how you fixed them.✅

Included original source code (or a screenshot of the app code if that's not possible).✅

Included a ‘hero shot’ of your application running & communicating with your board.✅

Group assignment

Compare as many tool options as possible.

Document your work on the group work page and reflect on your individual page what you learned.

I met with my colleague Jhonatan for a collaborative session focused on the analysis and experimentation of various technical processes. During this meeting, we worked with five specific tools, which allowed us to explore their functionality in depth, identify their main advantages, and understand their limitations within different usage contexts. This experience not only strengthened our practical knowledge but also facilitated critical discussion about the applicability of these tools in real-world scenarios, promoting a more meaningful and reflective learning process.

Here is the link to learn more about the group project.

Reflections

This week, I had the opportunity to work with various tools for creating control interfaces, which allowed me to explore how different platforms and technologies can interact with devices like the ESP32. Each of the tools had unique features and applications, allowing me to evaluate how they fit different needs and scenarios. Below are my reflections on the tools I used:

MIT App Inventor (Android) turned out to be an excellent tool for quickly developing mobile applications. Its block-based approach is ideal for those with little to no programming experience, as it allows you to build applications visually. In this case, we used the app to send "ON"/"OFF" signals via Bluetooth Classic or BLE to the ESP32, making it easy to control devices simply. However, its limitation is that it only supports Classic Bluetooth or BLE, which restricts some functionalities depending on the type of device.

NodePyQt5 (Desktop) combined the best of both worlds: a graphical user interface (GUI) with Python. This tool allowed me to create desktop applications with interactive buttons and controls to turn LEDs on and off via serial communication. What I liked most was the flexibility of Python, which offers a vast array of libraries, and PyQt5, which provided a clean and professional user interface. Although the installation and setup can be more complex than other platforms, the total control over the design and functionality was a great benefit.

Kodular (Android) Similar to MIT App Inventor, Kodular also allows you to create mobile applications for Android without writing code, but with a stronger focus on creating interfaces. The tool offered greater flexibility in controlling devices over Wi-Fi, such as servos and LEDs, using sliders and switches. However, the challenge I faced was managing the Wi-Fi network, as it requires more precise configuration to ensure stable communication. Despite this, the intuitive interface made it easy to quickly create functional mobile applications.

Using Unity with Meta to create virtual reality (VR) experiences was one of the most exciting parts. The ability to interact with a virtual environment and control physical devices, such as triggering a buzzer on the ESP32 via a virtual switch, is fascinating. Although creating a VR environment is more complex and requires more resources (like C# programming and network setup), the potential for interaction in an immersive setting is very appealing, especially for advanced applications like simulators or interactive experiences.

Processing Processing is a tool that allows you to create interactive visualizations with ease. Although it wasn’t used as much as the other tools this week, its ability to generate graphics and animations is invaluable, especially when aiming to create visually appealing interfaces to represent data or control systems in real-time. Programming in Processing is quite accessible, and its integration with hardware, like the ESP32, can be a powerful complement when trying to display information visually.

Individual Assignment:

Write an application for the embedded board that you made. that interfaces a user with an input and/or output device(s)

The objective of this assignment, aside from having fun, is to: Develop a graphical interface that interacts with an electronic board (XIAO ESP32C3). In this case, I will create two examples—one with LEDs and another with a servomotor—using serial communication between the PC and the hardware. This task combines software and electronics.

Download the Processing Program

To start this assignment, the first step is to download the program. Click here to access the program.

The software is very easy to get—just go to the official website and click the download button. This program is open source, so it’s free and available to everyone.

Next, the website automatically redirects you to a page that detects your computer’s operating system.In my case, it detected Windows version 4.4.1.

All you have to do is click the green-highlighted button to start the download.

During the installation process, a window appears with three setup options.

In my case, I selected the Typical option, as it is intended for most users.

Since I’m just getting started in the world of interfaces, I preferred this option for its simplicity and default configuration.

The interface of the Processing development environment is composed of four main sections:

Main Menu
Located at the top, it provides access to functions such as opening, saving, importing libraries, and other general settings.
Code Editor
This is the central area where you write the sketch code, including functions like setup(), draw(), and interaction logic.
Toolbar
Found just above the code editor, it includes key buttons:
- Run (▶️): Runs the sketch.
- Stop (⏹): Stops the sketch execution.
Console
Located at the bottom, it displays system messages, error logs, and output from commands like println().

1. NeoPixel LEDs

To control the power and color of a NeoPixel LED from a user interface created in Processing, communicating with the Xiao ESP32-C3.

I used Processing to create an interface with virtual buttons (e.g., red, green, blue, off).

Each button in the Processing window is linked to a specific character or command sent via serial communication to the Xiao ESP32-C3.

The ESP32-C3 receives the character and uses the Adafruit_NeoPixel library to light the NeoPixel with the corresponding color.

When I click a button in the Processing interface, the NeoPixel changes color immediately. I also added an “off” button to turn off the LED.

The goal of this project is to create a user interface in Processing that allows the user to control a NeoPixel LED connected to a Xiao ESP32-C3 microcontroller. To achieve this, a program is written and uploaded to the microcontroller using the Arduino IDE. This program listens for serial commands. Then, the Processing interface is run, and it sends commands through the USB port based on the user’s interaction with the virtual buttons. These commands trigger functions in the microcontroller that control the LED color, allowing it to light up in red, blue, yellow, or a random color.

1.1. Tools and Materials Used

Hardware:

ESP32-C3 microcontroller
NeoPixel LED strip or ring (WS2812B), 8 LEDs
Jumper wires
Breadboard (optional)
330–470Ω resistor and 1000 µF capacitor (recommended)
5 V power supply (if using multiple LEDs)

Software:

Arduino IDE
ESP32 board support for Arduino
Adafruit NeoPixel library

1.2. Programming Process

1.2.1. Arduino Code – LED Control

#include <Adafruit_NeoPixel.h>

    #define PIN        D3     // Change this pin to the one you're using for DIN
    #define NUMPIXELS 8       // Change this based on how many LEDs you have
    
    Adafruit_NeoPixel pixels(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);
    
    void setup() {
      Serial.begin(9600);
      pixels.begin();
      pixels.clear();  // Clear any previous data
      pixels.show();   // Initialize all pixels to 'off'
    }
    
    void loop() {
      if (Serial.available()) {
        char c = Serial.read();
        uint32_t color;
    
        switch (c) {
          case 'r':
            color = pixels.Color(255, 0, 0); // red
            break;
          case 'b':
            color = pixels.Color(0, 0, 255); // blue
            break;
          case 'y':
            color = pixels.Color(255, 255, 0); // yellow
            break;
          case 'x': // random color
            color = pixels.Color(random(0, 255), random(0, 255), random(0, 255));
            break;
          default:
            return; // Ignore unknown characters
        }
    
        for (int i = 0; i < NUMPIXELS; i++) {
          pixels.setPixelColor(i, color);
        }
        pixels.show();
      }
    }

This program allows you to control an addressable RGB LED strip (such as NeoPixel WS2812) connected to an ESP32-C3 microcontroller. The LEDs can be turned on with specific colors—red, blue, yellow—or a randomly generated color, based on commands sent through the serial port.

The following code was generated using OpenAI's ChatGPT language model, based on the following prompt:

Prompt used:

"Generate a code in ChatGPT with the following prompt: turn on red, blue, yellow, and random LEDs in an Arduino IDE program using an ESP32-C3 microcontroller."

Using this instruction, a functional sketch was generated that allows LED color control via commands sent over the serial monitor. The logic was implemented using the Adafruit_NeoPixel library, which is compatible with both the Arduino environment and the ESP32-C3 microcontroller—widely used in IoT and embedded electronics applications.

With the code ready, I upload it to the Arduino IDE to flash it onto the microcontroller.

1.2.2. Processing Code – GUI


import processing.serial.*; // Import the serial library to communicate with microcontrollers

Serial port; // Declare the serial port object

// Coordinates for each "eye" and their corresponding positions
int[][] eyes = {
  {70, 75},   // Red eye
  {170, 75},  // Blue eye
  {270, 75},  // Yellow eye
  {370, 75}   // Random eye
};

void setup() {
  size(460, 150); // Set the size of the window
  println(Serial.list()); // Print the list of available serial ports to the console
  port = new Serial(this, Serial.list()[0], 9600);  // Open the first available serial port at 9600 baud rate
  // Note: You may need to change the index [0] depending on your system
}

void draw() {
  background(240); // Light grey background
  textAlign(CENTER); // Center-align text
  textSize(12); // Set font size

  // Draw each eye with a specific color and label
  drawEye(eyes[0][0], eyes[0][1], color(255, 0, 0), "Red");
  drawEye(eyes[1][0], eyes[1][1], color(0, 0, 255), "Blue");
  drawEye(eyes[2][0], eyes[2][1], color(255, 255, 0), "Yellow");
  drawEye(eyes[3][0], eyes[3][1], color(100), "Random");
}

// Function to draw an eye with a moving pupil
void drawEye(int x, int y, color irisColor, String label) {
  float dx = mouseX - x;
  float dy = mouseY - y;
  float angle = atan2(dy, dx); // Calculate angle to mouse position
  float distance = min(dist(mouseX, mouseY, x, y), 10); // Limit how far the pupil can move
  float px = x + cos(angle) * distance; // X position of pupil
  float py = y + sin(angle) * distance; // Y position of pupil

  // Draw white part of the eye
  fill(255);
  stroke(0);
  ellipse(x, y, 60, 60);

  // Draw iris (colored part)
  fill(irisColor);
  noStroke();
  ellipse(x, y, 30, 30);

  // Draw pupil (black part)
  fill(0);
  ellipse(px, py, 10, 10);

  // Draw label below the eye
  fill(0);
  text(label, x, y + 45);
}

// Function triggered when the mouse is pressed
void mousePressed() {
  for (int i = 0; i < eyes.length; i++) {
    float d = dist(mouseX, mouseY, eyes[i][0], eyes[i][1]);
    if (d < 30) { // If the mouse click is within the eye
      // Send a specific character to the ESP32-C3 via serial depending on the eye clicked
      switch (i) {
        case 0: port.write('r'); break; // Red eye clicked
        case 1: port.write('b'); break; // Blue eye clicked
        case 2: port.write('y'); break; // Yellow eye clicked
        case 3: port.write('x'); break; // Random eye clicked
      }
    }
  }
}

Now, I open the Processing program and load the following code (generated with the prompt: “create an interface with colored eyes to select red, blue, yellow, and random”):

This code generates an interactive visual interface with colored eyes, each representing a command to turn on red, blue, yellow, or a random color on the LEDs. When you click on one of the eyes, a character is sent through the serial port to the microcontroller, which then activates the corresponding LED color.

This interface was created with help from ChatGPT, where I specifically requested colored eyes to make each option visually distinct and intuitive.

Here we can see the interface I will be using: animated eyes that move and are colored to match the NeoPixel LED that will light up.

1.3. Results

Now, the results of the interaction between the interface and the LED can be seen. By clicking on the colored eyes in the interface, the corresponding LED lights up according to the selected color. Everything worked correctly.

2. SG90 - Micro Gear Servo Motor

To move a servo motor to different angles using a graphical slider in the user interface made with Processing.

I designed a slider in Processing ranging from 0 to 180 degrees.

When I move the slider, the program generates a numerical value and sends it via serial communication to the Xiao ESP32-C3.

On the ESP32-C3 side, I use the Servo.h library to move the servo to the received angle.

The interface allows precise movement of the servo motor, updating the angle in real time as I move the slider.

For this interface to work properly, it is connected to a servo motor. The procedure is the same as previously used, with the same goal: when the user clicks on the interface, the servo motor will be activated and will move according to the user's interaction.

2.1. Tools and Materials Used

Hardware:

Servo Motor: Used to rotate based on commands received via serial communication.
XIAO RP2040: Controls the servo motor's movement.
USB Cable: For connecting the Arduino to the computer.
Jumper Wires: For connecting the servo motor to the Arduino.

Software:

Arduino IDE: Used for programming the Arduino to control the servo based on serial commands.
Processing IDE: Used to create a graphical interface that interacts with the Arduino through serial communication.

2.2. Programming Process

2.2.1. Arduino Code – Servo Motor

#include <Servo.h>

Servo myservo;  // Create the Servo object
int pos = 0;    // Initial position of the servo
bool moving = false;

void setup() {
  Serial.begin(9600);  // Initialize serial communication
  myservo.attach(29);  // Connect the servo to pin 29 (change if using a different pin)
}

void loop() {
  if (Serial.available()) {
    char command = Serial.read();  // Read the command from the serial port

    switch (command) {
      case 'L':  // Turn left
        pos = (pos + 1) % 180;  // Increase the servo position (max 180 degrees)
        moving = true;
        break;
      case 'R':  // Turn right
        pos = (pos - 1 + 180) % 180;  // Decrease the servo position (min 0 degrees)
        moving = true;
        break;
      case 'S':  // Stop movement
        moving = false;
        break;
    }
  }

  if (moving) {
    myservo.write(pos);  // Move the servo to the specified position
    delay(15);  // Delay to allow smooth movement of the servo
  }
}

To generate this code, I used the following prompt in ChatGPT:

"Write a code in Arduino IDE that controls a servo motor connected to digital pin 29. The servo should respond to commands sent via the serial port: 'L' to rotate left (increase angle), 'R' to rotate right (decrease angle), and 'S' to stop movement. Be sure to include the Servo.h library and make the motion smooth using delay. The code should be organized and well-commented for easy understanding."

I encountered a few minor issues: I had to adjust the pin number and make sure the correct serial port was selected. After making those changes, the code worked correctly.

Once the code was ready, I uploaded it to the XIAO RP2040 microcontroller using the Arduino IDE.

2.2.2. Processing Code – GUI

import processing.serial.*; // Import the Serial library for communication with microcontrollers

Serial myPort;              // Declare the serial port
float angle = 0;            // Angle for rotating the doughnut
int outerRadius = 150;      // Outer radius of the doughnut shape
int innerRadius = 80;       // Inner radius (the "hole") of the doughnut
boolean rotating = false;   // Tracks whether the doughnut is currently rotating
char direction = ' ';       // Direction of rotation: 'L' for left, 'R' for right
boolean clicked = false;    // Tracks if the doughnut was clicked

void setup() {
  size(400, 400); // Set the window size

  // List and print all available serial ports
  String[] portList = Serial.list();
  println("Available serial ports: ");
  for (int i = 0; i < portList.length; i++) {
    println(portList[i]);
  }

  // Open the first available port (you may need to change the index)
  if (portList.length > 0) {
    String portName = portList[0]; // Choose the first port
    myPort = new Serial(this, portName, 9600); // Initialize serial communication
  } else {
    println("Error: No serial ports found."); // Handle the case where no ports are available
  }

  ellipseMode(RADIUS); // Set ellipse mode to use radius rather than diameter
}

void draw() {
  background(0); // Set background to black

  translate(width / 2, height / 2); // Move drawing origin to the center of the canvas

  // Change doughnut color based on click status
  if (clicked) {
    fill(0, 255, 0); // Green when clicked
  } else {
    fill(255, 0, 255); // Magenta when not clicked
  }

  noStroke();      // Remove borders
  rotate(angle);   // Rotate the doughnut by the current angle

  // Draw the outer circle (doughnut body)
  ellipse(0, 0, outerRadius, outerRadius);

  // Draw the inner circle (doughnut hole)
  fill(0);         // Black hole
  ellipse(0, 0, innerRadius, innerRadius);

  // Control the rotation and send commands via serial
  if (rotating) {
    if (direction == 'L') {
      angle += 1;          // Rotate counter-clockwise
      myPort.write('L');   // Send 'L' to Arduino/microcontroller
    } else if (direction == 'R') {
      angle -= 1;          // Rotate clockwise
      myPort.write('R');   // Send 'R' to Arduino/microcontroller
    }
    angle = (angle + 360) % 360; // Keep angle within 0–360 degrees
  }
}

void mousePressed() {
  // Calculate distance from mouse to center of doughnut
  float d = dist(mouseX, mouseY, width / 2, height / 2);

  // If the click is within the doughnut area (not too close or too far)
  if (d > innerRadius && d < outerRadius) {
    direction = mouseX < width / 2 ? 'L' : 'R'; // Left or right depending on click position
    rotating = true;       // Start rotation
    clicked = true;        // Mark as clicked
    myPort.write('C');     // Send 'C' to Arduino to indicate click
  }
}

void mouseReleased() {
  rotating = false;        // Stop rotation
  direction = ' ';         // Reset direction
  clicked = false;         // Reset clicked status
  myPort.write('S');       // Send 'S' to Arduino to stop all motion
}

Prompt used:

"Write a code in Processing that creates a doughnut-shaped visual interface. When I click inside the doughnut, it should detect whether the click was on the left or right side and send 'L' or 'R' via serial accordingly. Each click should rotate the doughnut exactly 90 degrees in that direction. The doughnut should change color when clicked. When I release the mouse, send 'S' to stop the movement. Make sure to handle serial ports correctly."

Did I have any complications?

Yes, I did. The issue was with Processing not working properly. There was an extra parameter, degrees, that was causing the problem. Once I identified it, I was able to fix the issue.

And it's done! The code is now ready for the exercise.

2.3. Results

And this is the result: Every time I click on the doughnut, it turns green, and the servo motor starts rotating by 90°. When I release the click, it turns magenta and stops moving.

Reflections

I learned to work with two independent systems: the servo motor and NeoPixels. The servo was controlled through interaction with a rotating donut on the interface, while the NeoPixels were controlled with a "eye" system that changed colors. This helped me understand how each component can be managed separately and how their functions can be handled independently.

I used Processing to create interactive graphical interfaces, such as the donut for the servo, and the "eyes" for the NeoPixels. In both cases, I learned how to connect the physical world (LEDs and servos) to the virtual world through a serial port, which was key to making both systems respond to user interaction.

During the process, I had to troubleshoot several issues. The main one was when Processing didn’t recognize certain parameters, such as degrees in the code. After removing this issue, I learned how to quickly identify and fix problems, which is crucial in programming.

The satisfaction of seeing the NeoPixels change colors and the servo motor move when I clicked on the interface was very rewarding. This kind of project shows the power of programming in creating interactive systems, where physical and digital technologies come together to produce tangible results.

Access to files

fabcloud/fabacademy/2025/labs/lima/students/evelyn-cuadrado