15. Interface and Application Programming¶
group assignment:¶
group assignment:compare as many tool options as possible.
For this week’s task, I used Processing/Blink and Arduino IDE (XIAO-ESP32-C3) to communicate, I utilized the AI tool Deepseek, which helped me understand the Processing code and provided me with some suggestions when the program encountered problems.
Processing¶
An open-source programming language specifically designed for visual arts and interactive design. It enables seamless integration with Arduino via serial communication/networking and allows developers to achieve stunning visual effects with just dozens of lines of code.Click here to download Processing
1. Getting started with processing¶
1.1 Program Structure¶
-
setup() Function,Runs only once when the program starts,Used for initialization settings (window size, resource loading, etc.)
-
Draw() function,Continuously looping at a frequency of approximately 60 frames per second for creating animations and interactive effects. runs repeatedly until stopped.
void setup(){
(insert some code here...);
}
void draw(){
(insert some code here...
}
- Common Usage Examples: Create a 400px by 400px square white canvas>>Size (width, height), background (255).
void setup(){
size(400, 400); // Set window size 400px*400px
}
void draw(){
background(255); // Set white background;
}
1.2 Basic Shape Drawing in Processing¶
Coordinate System Note:Origin (0,0) at top-left corner,X increases to the right,Y increases downward. (I reviewed some documents from other classmates. Besides, I also referred to Deepseek)
- Rectangle
// Syntax: rect(x, y, width, height)
rect(50, 50, 100, 80); // Draws rectangle at (50,50) with 100px width and 80px height**
- Ellipse/Circle
// Syntax: ellipse(x, y, width, height)
ellipse(200, 200, 120, 120); // Perfect circle (equal width/height)
ellipse(300, 150, 80, 40); // Oval
- Line
// Syntax: line(x1, y1, x2, y2)
line(10, 10, 400, 400); // Diagonal line from top-left to bottom-right
- Triangle
// Syntax: triangle(x1, y1, x2, y2, x3, y3)
triangle(50, 300, 150, 200, 250, 300); // Three vertex points
1.3 Style Modifiers¶
- Style modifiers are used before drawing shapes:
fill(255, 0, 0); // Red fill color
stroke(0, 0, 255); // Blue outline
strokeWeight(3); // 3px thick outline
noFill(); // Transparent fill
noStroke(); // No outline
- Design a square button
void setup() {
// Set canvas size to 1000x400 pixels
size(1000, 400);
}
void draw() {
// Set white background
background(255);
// Set fill color to yellow (RGB: 255,255,0)
fill(255, 255, 0);
// Set stroke color to blue (RGB: 0,0,255)
stroke(0, 0, 255);
// Set stroke weight to 2 pixels
strokeWeight(2);
// Draw rectangle at position (200,150) with 100px width/height
rect(200, 150, 100, 100);
}
- Write Button1 inside the block
void setup() {
// Set canvas size to 1000x400 pixels
size(1000, 400);
}
void draw() {
// Set white background
background(255);
// Set fill color to yellow (RGB: 255,255,0)
fill(255, 255, 0);
// Set stroke color to blue (RGB: 0,0,255)
stroke(0, 0, 255);
// Set stroke weight to 2 pixels
strokeWeight(2);
// Draw rectangle at position (200,150) with 100px width/height
rect(200, 150, 100, 100);
// Add centered text
textAlign(CENTER, CENTER); // Set text alignment
textSize(25); // Set font size
fill(0, 0, 255); // Set text color to bule
text("Button1", 250, 200); // Draw text at rectangle center
}
1.4 Mouse-Following Circle Program¶
Based on the original switch, create a circular ball that follows the mouse pointer. Wherever the mouse moves, the ball is there. The ball is blue and has a diameter of 30px.
void setup() {
// Set canvas size to 1000x400 pixels
size(1000, 400);
}
void draw() {
// Clear canvas with white background
background(255);
// ------ Button Drawing ------
// Set button styles
fill(255, 255, 0); // Yellow fill
stroke(0, 0, 255); // Blue border
strokeWeight(2); // 2px border
// Draw button rectangle
rect(200, 150, 100, 100);
// Draw button text
textAlign(CENTER, CENTER);
textSize(25);
fill(0, 0, 255); // Blue text color
text("Button1", 250, 200);
// ------ Mouse Follower ------
noStroke(); // Remove border for circle
fill(0, 0, 255); // Blue fill color
ellipse(mouseX, mouseY, 30, 30); // Draw following circle
}
The processing program has the following effects:
indiviual assignment¶
indiviual assignment:write an application that interfaces a user with an input &/or output device that you made.
In this week’s assignment, I plan to use Processing to create buttons to control the LED lights on D1-D3 of my Xiao-ESP32-C3. When the mouse clicks on buttons 1/2/3, the button status changes to on, and at the same time, the LED of D1/2/3 lights up. Click the button again, the button will turn off, and the LEDs of D1/2/3 will turn off.
1. Communication between processing and Arduino¶
The core principle of communication between the Xiao ESP32-C3 and Processing is through serial communication (Serial Communication), using the USB/UART to serial protocol to transfer data between hardware and software.
-
Physical connection:The Xiao ESP32-C3 is directly connected to the computer via the USB interface, and the computer recognizes the ESP32-C3 as a virtual serial port (such as COM3 or /dev/ttyACM0).
-
Communication Protocol: Both parties communicate via the serial port protocol (UART), and the baud rate must be consistent (e.g., 115200).
-
Processing → ESP32-C3 data flow: Click the button → Processing sends “1\n” or “0\n” → Transmitted to ESP32-C3 via USB serial port.
-
ESP32-C3 response data flow: Receive string → Parse instruction → Control GPIO output high/low level → LED on/off.
-
The ESP32-C3 uses Serial.println() to send data (with a newline character \n at the end) to facilitate the use of bufferUntil(‘\n’) in Processing for splitting the data and ensuring its integrity.
2.1 Processing controls XIAO-ESP32-C3 to light up. (1 button & 1 LED)¶
I plan to start with the simplest function, designing a button to control an LED.This is the Processing code(This program referenced Deepseek’s help):
- Processing program code (GUI control terminal)
import processing.serial.*;
Serial xiaoPort;
boolean ledState = false;
int buttonX = 200, buttonY = 150;
int buttonW = 100, buttonH = 100;
void setup() {
size(1000, 400);
surface.setTitle("XIAO LED Controller - Classic Style");
// Print available serial ports list
printArray(Serial.list());
xiaoPort = new Serial(this, "COM12", 9600); // Modify to actual port
textAlign(CENTER, CENTER);
textSize(25);
cursor(CROSS); // Set cross cursor
}
void draw() {
// Classic white-blue style background
background(255);
// ===== Button Drawing =====
// Button style
if (ledState) {
fill(100, 255, 100); // Active state light green
} else {
fill(255, 255, 0); // Classic yellow fill
}
stroke(0, 0, 255); // Blue border
strokeWeight(2);
rect(buttonX, buttonY, buttonW, buttonH, 5); // Slight rounded corners
// Button text
fill(0, 0, 255); // Blue text
text("Button1\n" + (ledState ? "ON" : "OFF"),
buttonX + buttonW/2,
buttonY + buttonH/2);
// ===== Mouse Follower Effect =====
noStroke();
fill(0, 0, 255, 150); // Semi-transparent blue
ellipse(mouseX, mouseY, 30, 30);
}
void mousePressed() {
// Detect button click area
if (mouseX > buttonX && mouseX < buttonX + buttonW &&
mouseY > buttonY && mouseY < buttonY + buttonH) {
ledState = !ledState;
xiaoPort.write(ledState ? '1' : '0');
// Add click animation
fill(255, 0, 0, 100);
rect(buttonX, buttonY, buttonW, buttonH);
}
}
- At the same time, in the Arduino IDE, it is necessary to upload the following program code to the XIAO-ESP32-C3 program:
#include <HardwareSerial.h>
#define LED_PIN D1 // Use D1 pin of XIAO
void setup() {
Serial.begin(9600);
pinMode(LED_PIN, OUTPUT);
digitalWrite(LED_PIN, LOW); // Initial state: LED off
}
void loop() {
if(Serial.available() > 0){
char cmd = Serial.read();
if(cmd == '1'){
digitalWrite(LED_PIN, HIGH);
Serial.println("LED is turned on"); // Optional feedback
}
else if(cmd == '0'){
digitalWrite(LED_PIN, LOW);
Serial.println("LED is turned off"); // Optional feedback
}
Serial.flush(); // Clear serial buffer
}
}
Reminder:
1. When the processing upload program fails, I closed Arduino and restarted the Processing program, and it worked fine. It’s strange, later on during testing, they were able to run both programs simultaneously.
2. When I use XIAO-ESP32-C3, every time I upload the program, it encounters this problem: “Failed uploading: uploading error: exit status 1”. However, in reality, the program has already been successfully uploaded. I have tried booting, but it cannot be resolved. This reminder is not a problem, so it can be ignored.
2.2 Processing controls XIAO-ESP32-C3 to light up. (3 buttons&3 LEDs)¶
I used Deepseek to help me design code, and I sent it the following prompt:
Refer to the first processing program and make some changes. Previously, there was only one button in the program, but now two buttons have been added with the same function and color. The buttons 1/2/3 are used to control the LED lights on D1, D2, and D3 of XIAO-ESP32-C3 respectively. But the positions of the three buttons are evenly distributed on the canvas.
Procesing_3 buttons
import processing.serial.*;
Serial xiaoPort;
boolean[] ledStates = {false, false, false};
int buttonCount = 3;
int buttonW = 100;
int buttonH = 100;
int[] buttonX;
int buttonY;
void setup() {
size(1000, 400);
surface.setTitle("XIAO LED Controller - Three Buttons");
// Calculate button positions
buttonX = new int[buttonCount];
int spacing = (width - buttonCount * buttonW) / (buttonCount + 1);
buttonY = height/2 - buttonH/2;
for(int i = 0; i < buttonCount; i++) {
buttonX[i] = spacing + i * (buttonW + spacing);
}
// Initialize serial port
printArray(Serial.list());
xiaoPort = new Serial(this, "COM12", 9600);
textAlign(CENTER, CENTER);
textSize(20);
cursor(CROSS);
}
void draw() {
background(255);
// Draw buttons
for(int i = 0; i < buttonCount; i++) {
drawButton(i);
}
// Draw mouse follower
drawMouseFollower();
}
void drawButton(int index) {
if (ledStates[index]) {
fill(100, 255, 100);
} else {
fill(255, 255, 0);
}
stroke(0, 0, 255);
strokeWeight(2);
rect(buttonX[index], buttonY, buttonW, buttonH, 5);
fill(0, 0, 255);
text("Button " + (index+1) + "\n" + (ledStates[index] ? "ON" : "OFF"),
buttonX[index] + buttonW/2,
buttonY + buttonH/2);
}
void drawMouseFollower() {
noStroke();
fill(0, 0, 255, 150);
ellipse(mouseX, mouseY, 30, 30);
}
void mousePressed() {
for(int i = 0; i < buttonCount; i++) {
if(mouseX > buttonX[i] && mouseX < buttonX[i] + buttonW &&
mouseY > buttonY && mouseY < buttonY + buttonH) {
ledStates[i] = !ledStates[i];
sendCommand(i);
showClickEffect(i);
}
}
}
void sendCommand(int index) {
// Fixed command mapping:
// Button1: 0/OFF=48, 1/ON=49
// Button2: 2/OFF=50, 3/ON=51
// Button3: 4/OFF=52, 5/ON=53
char command = (ledStates[index]) ?
char(49 + index*2) : // ON commands: 49,51,53 ('1','3','5')
char(48 + index*2); // OFF commands: 48,50,52 ('0','2','4')
xiaoPort.write(command);
println("Sent: " + command + " (Button" + (index+1) + ")");
}
void showClickEffect(int index) {
fill(255, 0, 0, 100);
rect(buttonX[index], buttonY, buttonW, buttonH);
}
Arduino Code_3 LEDS The three LED lights of XIAO-ESP32-C3 are D1/D2/D3, and the Arduino IDE code is as follows:
#include <HardwareSerial.h>
const int LED_PINS[] = {D1, D2, D3}; // D1-D3 pins
void setup() {
Serial.begin(9600);
for(int i = 0; i < 3; i++) {
pinMode(LED_PINS[i], OUTPUT);
digitalWrite(LED_PINS[i], LOW);
}
}
void loop() {
if(Serial.available() > 0){
char cmd = Serial.read();
// Process command: A/a → D1, B/b → D2, C/c → D3
for(int i = 0; i < 3; i++) {
if(cmd == 'A'+i || cmd == 'a'+i) {
digitalWrite(LED_PINS[i], (cmd < 'a') ? HIGH : LOW);
break;
}
}
}
}
2. Communication between processing and Blynk¶
Next, I will attempt to use Blynk to create three buttons to control the on and off states of the D1/D2/D3 lights on the XIAO-esp32.
Blynk is an IoT development platform that focuses on quickly connecting hardware with mobile and web interfaces. By dragging and dropping controls such as buttons and charts, users can build interactive interfaces without complex coding to remotely control hardware (such as ESP32 and Arduino) or monitor sensor data.
2.1 Download and install Blynk¶
- Open the Blynk.io website and register an account using your email address.
- Download the Blynk app for your mobile device. In China, you may need a VPN to use it normally.
2.2 Set up Blynk on the computer end¶
- Create a new template
- Next, fill in the template information. Name (xiao led test) - HARDWARE (ESP32) - CONNECTION TYPE (WIFI) - Done.
- Open this template - Datastreams - Edit - New Datastream - Virtual Pin.
- Virtual Pin Datastream ——name(LED)——PIN(V1)——DATA TYPE(Integer)——Create.
- Set up Virtual Pins (V2 and V3) in the same way.The second picture shows the final result.
- Next I clicked on the mobile dashboard and clicked on save option.
- I then clicked on the device, added new device from the template.
- After that I had selected my project name and clicked on create.
- Template ID and Auth Token are two crucial pieces of data that will be used in the subsequent program. Save these two pieces of data in Devices.
2.2 Set up BLYNK on the mobile phone¶
- Search for Blynk in the app store and download it, or scan the QR code provided earlier to download the app. Install Blynk and log in with your email. On the home page, you can see the projects you have created. Open the project named “xiao led test”.
- Add three switches according to the following operation.
- Click the switch icon to change the DATE of each switch. The DATESTREAMs of the three switches are V1/V2/V3 respectively.
2.3 Upload the program to XIAO-ESP32-C3¶
- By referring to the technical documentation of the xiao-esp32-C3, it is learned that the D1/D2/D3 of the xiao-C3 are GPIO3/GPIO4/GPIO5 respectively.
- Design the program and modify some data in the program: TEMPLATE_ID, AUTH_TOKEN,TEMPLATE_NAME,WIFI.
#define BLYNK_TEMPLATE_ID "TMPL6qQheKl7c"
#define BLYNK_TEMPLATE_NAME "xiao led test"
#define BLYNK_AUTH_TOKEN "zAY6vxwItDJRbJ6tA6Iu-tZ38fU2BkrU"
#include <WiFi.h>
#include <BlynkSimpleEsp32.h>
// WiFi configuration
char ssid[] = "ChinaNet-c3Y7";
char pass[] = "2hk3f3ey";
// Define LED pins
const int ledPin_V1 = 3; // V1 controls GPIO3
const int ledPin_V2 = 4; // V2 controls GPIO4
const int ledPin_V3 = 5; // V3 controls GPIO5
// V1 controls GPIO3
BLYNK_WRITE(V1) {
int state = param.asInt();
digitalWrite(ledPin_V1, state);
}
// V2 controls GPIO4
BLYNK_WRITE(V2) {
int state = param.asInt();
digitalWrite(ledPin_V2, state);
}
// V3 controls GPIO5
BLYNK_WRITE(V3) {
int state = param.asInt();
digitalWrite(ledPin_V3, state);
}
void setup() {
Serial.begin(115200);
// Initialize all pins to output mode
pinMode(ledPin_V1, OUTPUT);
pinMode(ledPin_V2, OUTPUT);
pinMode(ledPin_V3, OUTPUT);
// Start Blynk
Blynk.begin(BLYNK_AUTH_TOKEN, ssid, pass);
}
void loop() {
Blynk.run();
}
- Note: Do not change the data at this position.
2.4 The outcome of this project¶
- This project was a huge success! The three switches on Blynk can wirelessly control three lights!!! However, its response is a bit slow. Anyway, I made it work. Thanks to roshan-rai’s documentation support~~~
Summary:¶
This week was very interesting. It interacted with hardware in new ways, and this is also a very practical technology. In daily products, it is very common to control devices with apps. In the future, I want to continue to improve my project and design a mini program on the phone to connect to my project.
Besides, I’m quite sorry that I’m not very proficient in programming. So most of the time, I relied on AI tools. By asking it questions, I understood the meanings of different codes.