15 Interface and application programming
Reflection
This week's learning was a process of in-depth communication and collaboration with AI programming. I have an achievable idea and goal, along with the path to achieve it. Based on the information I provided, the AI agent assisted me in making a more detailed plan, which was then implemented. Some problems that may arise in the middle stage will also be addressed by continuously conducting checks to optimize the program.
Group Assignment
- [x] Compare as many tool options as possible
AI
Cursor is an AI agent that helps me code, understand code, and fix bugs. In a limited amount of time, it can help me handle very complex work. Its most useful function is its ability to use the CLI to assist me with localhost tasks.
Although AI tools can help me do almost anything through a computer, they still require very specific content and path directions. This means we can't simply let them run freely now—we need to set a plan and control the potential future. Theories, working methods, and techniques are more crucial than ever. The only thing we can save is time; the cost of trial and error has decreased.
I used Cursor to design a website for a Real-time visual effects testing platform with display test modules (BFI, uniformity, color space), kinetic typography, motion blur testing, and audio visualization.
Graphic
Built with React
Three.js in Individual Assignment
Audio-Visual-Studio Website
This main screen test is mainly designed to demonstrate the left and right channel sound effect tests.
This Grid mainly checks for any defective pixels on the screen, and it can also be turned into a simple sound synthesizer.
The Scroll test is testing the smoothness of rolling.
Big text is tesing the the variation of type weight, font size, stretching, etc.
Rain and photon are testing the render of particles.
Device Interfaces
MQTT (Message Queuing Telemetry Transport: 消息队列遥测传输协议) is an OASIS (Organization for the Advancement of Structured Information Standards) standard messaging protocol for the Internet of Things (IoT). It is designed as an extremely lightweight publish/subscribe messaging transport that is ideal for connecting remote devices with a small code footprint and minimal network bandwidth.
MQTT broker
Individual Assignment
The Web Serial is a JavaScript interface provided by modern browsers, enabling web pages to directly communicate with users' serial port devices.
Hello world
Programming funtion : Keep print "Hello from RP2040" from Xiao Seeed RP2040.
Then Upload the code to Xiao Seeed RP2040.
Check baudRate (9600) // need to be the same baudrate as the webpage.
void setup() {
Serial.begin(9600);
delay(500);
Serial.println("Hello from RP2040");
}
void loop() {
delay(5000);
Serial.println("Hello from RP2040");
}
Webpage: Displays the received string after connection.
There is a button to active the port.
After connection, "Hello from RP2040" will keep print on the screen.
I use AI to generate the code and webpage. However, it generated too many rich text formatting elements, making it difficult to clearly see the key code that is functioning properly. So I tried to remove the unnecessary parts and keep only the most concise ones.
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<title>hello</title>
</head>
<body>
<button id="btn" type="button">connect</button>
<!-- pre keeps newlines; we append each line so you see a growing log from the board. -->
<pre id="out"></pre>
<script>
let port;
let lineBuf = '';
document.getElementById('btn').onclick = async () => {
port = await navigator.serial.requestPort(); // Prompt user to select any serial port.
await port.open({ baudRate: 9600 }); // Wait for the serial port to open.
lineBuf = '';
// The <pre id="out"> element: we append each received line here as a scrollable log.
const out = document.getElementById('out');
out.textContent = '';
const dec = new TextDecoder();
const reader = port.readable.getReader();
try {
// Keep reading until the port is closed (then done === true).
// Listen to data coming from the serial device.
while (true) {
const { value, done } = await reader.read();
if (done) break;
if (value && value.byteLength) {
lineBuf += dec.decode(value, { stream: true });
}
let i;
while ((i = lineBuf.indexOf('\n')) >= 0) {
const line = lineBuf.slice(0, i).replace(/\r$/, '').trim();
lineBuf = lineBuf.slice(i + 1);
if (line) {
// Append each line so repeated prints still grow the log (overwrite hides repeats).
out.textContent += line + '\n';
if (out.textContent.length > 12000) {
out.textContent = out.textContent.slice(-8000);
}
out.scrollTop = out.scrollHeight;
}
}
}
} finally {
reader.releaseLock();
}
};
</script>
</body>
</html>
Servo rotate Control
Because the final project will using servo to develop the product. Then I try to explore how can I use servo. This time I use MG90s to test.
Trouble and Solution
- There was no response after connecting to the serial port. (Close the Arduino IDE and all other web pages that have previously connected to devices. They are sharing the same serial port.)
Servo Digital Twin
Ask Deepseek to generate the detaile of prompt
"I need to create a web and device end for real-time interaction. For the web end, I will use three.js to model the servo motor. I will use a 180-degree adjustable-angle servo motor. When I swing the angle on the page, the device will adjust to the corresponding angle position. I will use cursor to implement this project. I will write an HTML page control and code to pass to the device end. You will write the prompt. The development board I am using is Xiao RP2040, and the servo motor is connected to D5. Please refer to the link and explain the technical path and methods. https://fabacademy.org/2026/classes/interface_application_programming/index.html"
AI Generated Prompt for Cursor:
"I want to build a real-time servo motor control project:
- Development board: Seeed XIAO RP2040, servo signal wire connected to D5 (GPIO 5).
- Device-side code: Arduino, using the Servo library, serial baud rate 9600, reads commands ending with \n in the format "ANGLE:90", parses the angle (0–180), and controls the servo with servo.write().
-
Web interface: A single HTML file containing:
-
A Three.js 3D scene, building a servo motor model from basic geometries (base, rotating part, horn/arm).
- Place the rotatable parts inside a Group, with initial angle of 90°.
- Model orientation: motor shaft vertical, rotation around the Y axis, corresponding to real-world 0–180° (assume 0° points the arm to the left, 180° to the right).
- An HTML range slider (0 to 180). When the slider is dragged:
Update the 3D model’s rotation Group's Y-axis rotation, mapping the slider value to an appropriate radian range (e.g. 0° → -π/2, 180° → π/2, or adjusted for visual effect).
Simultaneously send "ANGLE:
\n" to the connected device via Web Serial. - Also support direct drag rotation in the 3D view: add an invisible helper object (e.g. a small sphere) on the rotating part, use TransformControls constrained to Y-axis rotation only. While dragging, read the angle in real time, map it to 0–180, update the slider and the servo angle.
- Web Serial section:
- A "Connect Serial" button. When clicked, request a serial port and let the user select the device, baud rate 9600.
- Display connection status after success, obtain the writer.
- Write the command whenever the angle changes.
- Optional read loop: print device confirmation messages to the console or at the bottom of the page.
-
Layout: left side a 3D view (70% width), right side a control panel (slider, connect button, status text).
-
All libraries (Three.js, OrbitControls, TransformControls, Web Serial polyfill if needed) are loaded from a CDN, using importmap or ES module imports (e.g. from unpkg or jsdelivr, Three.js version r160+).
-
Code requirements:
-
Clear comments explaining key steps.
-
Consider error handling (serial connection failure, device not selected, etc.).
-
Use TextEncoder when sending commands.
-
-
Final output: a complete index.html file and an arduino.ino file.
Please generate the code based on the above requirements."
Trouble and solution
It works, but has some bugs:
-
The connection is not smooth, and the page control is not being transmitted in real time to the device.(The data transmission will be automatically retried, which reduces the data transmission speed. Therefore, the data retry function has been disabled.)
-
It stops working after running for a period of time.(Data transmission may have packet loss, so data flushing is performed, and reconnection is carried out after disconnection.)
After funcion works and bug fixed, I found some picture of MG90s servo motor to update the model in pages.
Servo Watering game
Maybe for a fun way, not just a 3d servo model. I am thinking about a watering machine that our group discuss in in the machine week. It is the time to make it.
I found water gun pictures and talk with Cursor about change the model into the watergun and make a game to watering flower. It is important to ask put the flower on the parabolic radius, otherwise the water bubble can not hit the flower.
Combine with the control pad, I did serval weeks ago. I can get the reaction from the pad to move the watergun left or right.
Ha, A birth of a watering game ~
