Skip to content

15. Interface and Application Programming

group assignment:

group assignment:compare as many tool options as possible.

For this week’s task, I used Processing/Blink and Arduino IDE (XIAO-ESP32-C3) to communicate, I utilized the AI tool Deepseek, which helped me understand the Processing code and provided me with some suggestions when the program encountered problems.

Processing

An open-source programming language specifically designed for visual arts and interactive design. It enables seamless integration with Arduino via serial communication/networking and allows developers to achieve stunning visual effects with just dozens of lines of code.Click here to download Processing
processing_web.jpg)

1. Getting started with processing

1.1 Program Structure

  • setup() Function,Runs only once when the program starts,Used for initialization settings (window size, resource loading, etc.)

  • Draw() function,Continuously looping at a frequency of approximately 60 frames per second for creating animations and interactive effects. runs repeatedly until stopped.

void setup(){
  (insert some code here...);
  }
void draw(){
   (insert some code here...
  }
  • Common Usage Examples: Create a 400px by 400px square white canvas>>Size (width, height), background (255).
void setup(){
size(400, 400);  // Set window size 400px*400px
}
void draw(){
background(255); // Set white background;
}

Program_Structure.jpg

1.2 Basic Shape Drawing in Processing

Coordinate System Note:Origin (0,0) at top-left corner,X increases to the right,Y increases downward. (I reviewed some documents from other classmates. Besides, I also referred to Deepseek)

  • Rectangle
// Syntax: rect(x, y, width, height)
rect(50, 50, 100, 80); // Draws rectangle at (50,50) with 100px width and 80px height**

rect.jpg

  • Ellipse/Circle
// Syntax: ellipse(x, y, width, height)
ellipse(200, 200, 120, 120); // Perfect circle (equal width/height)
ellipse(300, 150, 80, 40);   // Oval

circle.jpg
ellipse.jpg

  • Line
// Syntax: line(x1, y1, x2, y2)
line(10, 10, 400, 400); // Diagonal line from top-left to bottom-right  

Line.jpg

  • Triangle
// Syntax: triangle(x1, y1, x2, y2, x3, y3)
triangle(50, 300, 150, 200, 250, 300); // Three vertex points  

Triangle.jpg

1.3 Style Modifiers

  • Style modifiers are used before drawing shapes:
fill(255, 0, 0);    // Red fill color
stroke(0, 0, 255);  // Blue outline
strokeWeight(3);     // 3px thick outline
noFill();            // Transparent fill
noStroke();          // No outline
  • Design a square button
void setup() {
  // Set canvas size to 1000x400 pixels
  size(1000, 400);
 }

void draw() {
  // Set white background
  background(255);

  // Set fill color to yellow (RGB: 255,255,0)
  fill(255, 255, 0);

  // Set stroke color to blue (RGB: 0,0,255)
  stroke(0, 0, 255);

  // Set stroke weight to 2 pixels
  strokeWeight(2);

  // Draw rectangle at position (200,150) with 100px width/height
  rect(200, 150, 100, 100);
 } 

button1.jpg

  • Write Button1 inside the block
void setup() {
  // Set canvas size to 1000x400 pixels
  size(1000, 400);
}

void draw() {
  // Set white background
  background(255);

  // Set fill color to yellow (RGB: 255,255,0)
  fill(255, 255, 0);

  // Set stroke color to blue (RGB: 0,0,255)
  stroke(0, 0, 255);

  // Set stroke weight to 2 pixels
  strokeWeight(2);

  // Draw rectangle at position (200,150) with 100px width/height
  rect(200, 150, 100, 100);

  // Add centered text
  textAlign(CENTER, CENTER);  // Set text alignment
  textSize(25);               // Set font size
  fill(0, 0, 255);           // Set text color to bule
  text("Button1", 250, 200); // Draw text at rectangle center
}

button1_001.jpg

1.4 Mouse-Following Circle Program

Based on the original switch, create a circular ball that follows the mouse pointer. Wherever the mouse moves, the ball is there. The ball is blue and has a diameter of 30px.

void setup() {
  // Set canvas size to 1000x400 pixels
  size(1000, 400);
}

void draw() {
  // Clear canvas with white background
  background(255);

  // ------ Button Drawing ------
  // Set button styles
  fill(255, 255, 0);  // Yellow fill
  stroke(0, 0, 255);   // Blue border
  strokeWeight(2);     // 2px border

  // Draw button rectangle
  rect(200, 150, 100, 100);

  // Draw button text
  textAlign(CENTER, CENTER);
  textSize(25);
  fill(0, 0, 255);     // Blue text color
  text("Button1", 250, 200);

  // ------ Mouse Follower ------
  noStroke();          // Remove border for circle
  fill(0, 0, 255);     // Blue fill color
  ellipse(mouseX, mouseY, 30, 30); // Draw following circle
}  

The processing program has the following effects:


indiviual assignment

indiviual assignment:write an application that interfaces a user with an input &/or output device that you made.

In this week’s assignment, I plan to use Processing to create buttons to control the LED lights on D1-D3 of my Xiao-ESP32-C3. When the mouse clicks on buttons 1/2/3, the button status changes to on, and at the same time, the LED of D1/2/3 lights up. Click the button again, the button will turn off, and the LEDs of D1/2/3 will turn off.

1. Communication between processing and Arduino

The core principle of communication between the Xiao ESP32-C3 and Processing is through serial communication (Serial Communication), using the USB/UART to serial protocol to transfer data between hardware and software.

  • Physical connection:The Xiao ESP32-C3 is directly connected to the computer via the USB interface, and the computer recognizes the ESP32-C3 as a virtual serial port (such as COM3 or /dev/ttyACM0).

  • Communication Protocol: Both parties communicate via the serial port protocol (UART), and the baud rate must be consistent (e.g., 115200).

  • Processing → ESP32-C3 data flow: Click the button → Processing sends “1\n” or “0\n” → Transmitted to ESP32-C3 via USB serial port.

  • ESP32-C3 response data flow: Receive string → Parse instruction → Control GPIO output high/low level → LED on/off.

  • The ESP32-C3 uses Serial.println() to send data (with a newline character \n at the end) to facilitate the use of bufferUntil(‘\n’) in Processing for splitting the data and ensuring its integrity.

2.1 Processing controls XIAO-ESP32-C3 to light up. (1 button & 1 LED)

I plan to start with the simplest function, designing a button to control an LED.This is the Processing code(This program referenced Deepseek’s help):

  • Processing program code (GUI control terminal)
import processing.serial.*;

Serial xiaoPort;  
boolean ledState = false;
int buttonX = 200, buttonY = 150;
int buttonW = 100, buttonH = 100;

void setup() {
  size(1000, 400);
  surface.setTitle("XIAO LED Controller - Classic Style");

  // Print available serial ports list
  printArray(Serial.list());
  xiaoPort = new Serial(this, "COM12", 9600); // Modify to actual port

  textAlign(CENTER, CENTER);
  textSize(25);
  cursor(CROSS); // Set cross cursor
}

void draw() {
  // Classic white-blue style background
  background(255);

  // ===== Button Drawing =====
  // Button style
  if (ledState) {
    fill(100, 255, 100); // Active state light green
  } else {
    fill(255, 255, 0);   // Classic yellow fill
  }
  stroke(0, 0, 255);     // Blue border
  strokeWeight(2);
  rect(buttonX, buttonY, buttonW, buttonH, 5); // Slight rounded corners

  // Button text
  fill(0, 0, 255);       // Blue text
  text("Button1\n" + (ledState ? "ON" : "OFF"), 
       buttonX + buttonW/2, 
       buttonY + buttonH/2);

  // ===== Mouse Follower Effect =====
  noStroke();
  fill(0, 0, 255, 150);  // Semi-transparent blue
  ellipse(mouseX, mouseY, 30, 30);
}

void mousePressed() {
  // Detect button click area
  if (mouseX > buttonX && mouseX < buttonX + buttonW &&
      mouseY > buttonY && mouseY < buttonY + buttonH) {

    ledState = !ledState;
    xiaoPort.write(ledState ? '1' : '0');

    // Add click animation
    fill(255, 0, 0, 100);
    rect(buttonX, buttonY, buttonW, buttonH);
  }
}

processing_button1.jpg

  • At the same time, in the Arduino IDE, it is necessary to upload the following program code to the XIAO-ESP32-C3 program:
#include <HardwareSerial.h>
#define LED_PIN D1 // Use D1 pin of XIAO

void setup() {
  Serial.begin(9600);
  pinMode(LED_PIN, OUTPUT);
  digitalWrite(LED_PIN, LOW); // Initial state: LED off
}

void loop() {
  if(Serial.available() > 0){
    char cmd = Serial.read();

    if(cmd == '1'){
      digitalWrite(LED_PIN, HIGH);
      Serial.println("LED is turned on"); // Optional feedback
    }
    else if(cmd == '0'){
      digitalWrite(LED_PIN, LOW);
      Serial.println("LED is turned off"); // Optional feedback
    }

    Serial.flush(); // Clear serial buffer
  }
}

Arduino_LED1.jpg

Reminder:
1. When the processing upload program fails, I closed Arduino and restarted the Processing program, and it worked fine. It’s strange, later on during testing, they were able to run both programs simultaneously.
2. When I use XIAO-ESP32-C3, every time I upload the program, it encounters this problem: “Failed uploading: uploading error: exit status 1”. However, in reality, the program has already been successfully uploaded. I have tried booting, but it cannot be resolved. This reminder is not a problem, so it can be ignored.

2.2 Processing controls XIAO-ESP32-C3 to light up. (3 buttons&3 LEDs)

I used Deepseek to help me design code, and I sent it the following prompt:
Refer to the first processing program and make some changes. Previously, there was only one button in the program, but now two buttons have been added with the same function and color. The buttons 1/2/3 are used to control the LED lights on D1, D2, and D3 of XIAO-ESP32-C3 respectively. But the positions of the three buttons are evenly distributed on the canvas.

Procesing_3 buttons

import processing.serial.*;

Serial xiaoPort;  
boolean[] ledStates = {false, false, false};
int buttonCount = 3;
int buttonW = 100;
int buttonH = 100;
int[] buttonX;
int buttonY;

void setup() {
  size(1000, 400);
  surface.setTitle("XIAO LED Controller - Three Buttons");

  // Calculate button positions
  buttonX = new int[buttonCount];
  int spacing = (width - buttonCount * buttonW) / (buttonCount + 1);
  buttonY = height/2 - buttonH/2;

  for(int i = 0; i < buttonCount; i++) {
    buttonX[i] = spacing + i * (buttonW + spacing);
  }

  // Initialize serial port
  printArray(Serial.list());
  xiaoPort = new Serial(this, "COM12", 9600);

  textAlign(CENTER, CENTER);
  textSize(20);
  cursor(CROSS);
}

void draw() {
  background(255);

  // Draw buttons
  for(int i = 0; i < buttonCount; i++) {
    drawButton(i);
  }

  // Draw mouse follower
  drawMouseFollower();
}

void drawButton(int index) {
  if (ledStates[index]) {
    fill(100, 255, 100);
  } else {
    fill(255, 255, 0);
  }

  stroke(0, 0, 255);
  strokeWeight(2);
  rect(buttonX[index], buttonY, buttonW, buttonH, 5);

  fill(0, 0, 255);
  text("Button " + (index+1) + "\n" + (ledStates[index] ? "ON" : "OFF"), 
       buttonX[index] + buttonW/2, 
       buttonY + buttonH/2);
}

void drawMouseFollower() {
  noStroke();
  fill(0, 0, 255, 150);
  ellipse(mouseX, mouseY, 30, 30);
}

void mousePressed() {
  for(int i = 0; i < buttonCount; i++) {
    if(mouseX > buttonX[i] && mouseX < buttonX[i] + buttonW &&
       mouseY > buttonY && mouseY < buttonY + buttonH) {

      ledStates[i] = !ledStates[i];
      sendCommand(i);
      showClickEffect(i);
    }
  }
}

void sendCommand(int index) {
  // Fixed command mapping: 
  // Button1: 0/OFF=48, 1/ON=49
  // Button2: 2/OFF=50, 3/ON=51 
  // Button3: 4/OFF=52, 5/ON=53
  char command = (ledStates[index]) ? 
                char(49 + index*2) :  // ON commands: 49,51,53 ('1','3','5')
                char(48 + index*2);   // OFF commands: 48,50,52 ('0','2','4')
  xiaoPort.write(command);
  println("Sent: " + command + " (Button" + (index+1) + ")");
}

void showClickEffect(int index) {
  fill(255, 0, 0, 100);
  rect(buttonX[index], buttonY, buttonW, buttonH);
}

button3_processing.jpg

Arduino Code_3 LEDS The three LED lights of XIAO-ESP32-C3 are D1/D2/D3, and the Arduino IDE code is as follows:

#include <HardwareSerial.h>
const int LED_PINS[] = {D1, D2, D3}; // D1-D3 pins

void setup() {
  Serial.begin(9600);
  for(int i = 0; i < 3; i++) {
    pinMode(LED_PINS[i], OUTPUT);
    digitalWrite(LED_PINS[i], LOW);
  }
}

void loop() {
  if(Serial.available() > 0){
    char cmd = Serial.read();

    // Process command: A/a → D1, B/b → D2, C/c → D3
    for(int i = 0; i < 3; i++) {
      if(cmd == 'A'+i || cmd == 'a'+i) {
        digitalWrite(LED_PINS[i], (cmd < 'a') ? HIGH : LOW);
        break;
      }
    }
  }
}     

2. Communication between processing and Blynk

Next, I will attempt to use Blynk to create three buttons to control the on and off states of the D1/D2/D3 lights on the XIAO-esp32.

Blynk is an IoT development platform that focuses on quickly connecting hardware with mobile and web interfaces. By dragging and dropping controls such as buttons and charts, users can build interactive interfaces without complex coding to remotely control hardware (such as ESP32 and Arduino) or monitor sensor data.

2.1 Download and install Blynk

  • Open the Blynk.io website and register an account using your email address.
    download_Blynk.jpg
  • Download the Blynk app for your mobile device. In China, you may need a VPN to use it normally.
    blynk_telephone.jpg

2.2 Set up Blynk on the computer end

  • Create a new template
    new_template.jpg
  • Next, fill in the template information. Name (xiao led test) - HARDWARE (ESP32) - CONNECTION TYPE (WIFI) - Done.
    template_name.jpg
  • Open this template - Datastreams - Edit - New Datastream - Virtual Pin.
    Datastreams.jpg
  • Virtual Pin Datastream ——name(LED)——PIN(V1)——DATA TYPE(Integer)——Create. virtual_pin.jpg
  • Set up Virtual Pins (V2 and V3) in the same way.The second picture shows the final result.
    Virtual_pin_2_3.jpg
    Virtual_pin_1_2_3.jpg
  • Next I clicked on the mobile dashboard and clicked on save option. Mobile_dashboard.jp
  • I then clicked on the device, added new device from the template.
    new_device.jpg
  • After that I had selected my project name and clicked on create. new_device_name.jpg
  • Template ID and Auth Token are two crucial pieces of data that will be used in the subsequent program. Save these two pieces of data in Devices.
    auto_token.jpg
    template_ID.jpg

2.2 Set up BLYNK on the mobile phone

  • Search for Blynk in the app store and download it, or scan the QR code provided earlier to download the app. Install Blynk and log in with your email. On the home page, you can see the projects you have created. Open the project named “xiao led test”.
    APP_Setting_1.jpg
  • Add three switches according to the following operation.
    APP_Setting_2.jpg
  • Click the switch icon to change the DATE of each switch. The DATESTREAMs of the three switches are V1/V2/V3 respectively.
    APP_Setting_3.jpg

2.3 Upload the program to XIAO-ESP32-C3

  • By referring to the technical documentation of the xiao-esp32-C3, it is learned that the D1/D2/D3 of the xiao-C3 are GPIO3/GPIO4/GPIO5 respectively.
    C3_pin_map.png
  • Design the program and modify some data in the program: TEMPLATE_ID, AUTH_TOKEN,TEMPLATE_NAME,WIFI.
    program_arduino.jpg
#define BLYNK_TEMPLATE_ID "TMPL6qQheKl7c"     
#define BLYNK_TEMPLATE_NAME "xiao led test"  
#define BLYNK_AUTH_TOKEN "zAY6vxwItDJRbJ6tA6Iu-tZ38fU2BkrU"
#include <WiFi.h>
#include <BlynkSimpleEsp32.h>
// WiFi configuration
char ssid[] = "ChinaNet-c3Y7";
char pass[] = "2hk3f3ey";
// Define LED pins
const int ledPin_V1 = 3;  // V1 controls GPIO3
const int ledPin_V2 = 4;  // V2 controls GPIO4
const int ledPin_V3 = 5;  // V3 controls GPIO5
// V1 controls GPIO3
BLYNK_WRITE(V1) {
  int state = param.asInt();
  digitalWrite(ledPin_V1, state);
}
// V2 controls GPIO4
BLYNK_WRITE(V2) {
  int state = param.asInt();
  digitalWrite(ledPin_V2, state);
}
// V3 controls GPIO5
BLYNK_WRITE(V3) {
  int state = param.asInt();
  digitalWrite(ledPin_V3, state);
}
void setup() {
  Serial.begin(115200);
  // Initialize all pins to output mode
  pinMode(ledPin_V1, OUTPUT);
  pinMode(ledPin_V2, OUTPUT);
  pinMode(ledPin_V3, OUTPUT);
  // Start Blynk
  Blynk.begin(BLYNK_AUTH_TOKEN, ssid, pass);
}
void loop() {
  Blynk.run();
} 
  • Note: Do not change the data at this position.
    note.jpg

2.4 The outcome of this project

  • This project was a huge success! The three switches on Blynk can wirelessly control three lights!!! However, its response is a bit slow. Anyway, I made it work. Thanks to roshan-rai’s documentation support~~~

Summary:

This week was very interesting. It interacted with hardware in new ways, and this is also a very practical technology. In daily products, it is very common to control devices with apps. In the future, I want to continue to improve my project and design a mini program on the phone to connect to my project.

Besides, I’m quite sorry that I’m not very proficient in programming. So most of the time, I relied on AI tools. By asking it questions, I understood the meanings of different codes.