Week 14: Interface and Application Programming¶
Group Assignment¶
Individual Assignment¶
Not long ago, I had ordered a XIAO RGB LED matrix, a very small one – 6x10 pixels. It was a very cool component I qanted to experiment with, and so I had tried giving it cool light effects and all, but I wanted to make it work through an interface.
This idea came to me from my wireless speaker – JBL Pulse series. It features an RBG matrix covered by a diffuser and you can control the animations using either a dedicated button, or the phone app for more complex adjustments [brightness, patterns, colors, etc].
The LED Matrix¶
Next, I soldered the matrix to a XIAO ESP32 S3 Plus. Its identical footprint makes it compatible across the XIAO lineup. I then used Claude to build a macOS app in Swift. We began with basic features based on my initial guidelines, gradually extending the application’s functionality as the foundation proved stable.
I first uploaded the .ino code onto the XIAO, and then proceeded to running the interface. As Claude’s credit usage is very high, I decided to skip an app icon completely. That did not seem like a priority, so instead, once I had changed to the correct directpry I ran the following code in Terminal:
*Prompt14.2
swift run
The app has port connectivity with a dedicated dropdown menu to select the device, and a button to connect, or disconnect to it.
Screen Saver Mode¶
In the center, there’s a button which boots the screen up, and acts like a screen saver or puts the screen on a standby mode. Basically the screen oscillates from bright to dim resembling a sinusoid graph. You can pick your desired color, or change from a color wheel where you can also change the brightness.
Teleprompter¶
I found teleprompter to be a cool usecase for this component, especially considering the challange of such minimal pixel height and width. For readability purposes, I considered having a monospaced font, meaning the height and the width of all the letters should be the same, e.g. 5x5 px.
You type in some text, and the matrix outputs it by sliding the text from right to left. You can change the color of text, and also adjust the wpm [words per minute] speed.
Finally, the classic snake game. A green array of pixels should chase a red solo pixel. Both the screen buttons and keyboard are funtional.
Results in the video below.
How do they talk?¶
The XIAO ESP32 S3 has a built-in USB controller that presents itself to macOS as a CDC serial device [Communications Device Class] — essentially a virtual serial port at /dev/cu.usbmodemXXXXX.
The Mac app opens that path with raw POSIX calls [open, tcsetattr, read/write], no drivers needed. Both sides send plain newline-terminated text at 115200 baud — the Mac writes commands like ON or TEXT:HELLO, the board writes back lines like STATE:ON or SCORE:3. That’s the entire protocol.
The key Arduino setting that makes it work: USB CDC On Boot → Enabled in the board config, which tells the ESP32 S3 to expose its native USB port as a serial interface rather than just a flashing target.
Problems¶
I have not yet figured out how this array works, but I to try to understand it, I had written a simple code which would first tested if all the pixels worked correctly.
How does a pixel behave?
As a single RGB pixel is made up of 3 small LEDs – Red, Green, and Blue - each pixel is supposed to fully function in each of those individual colors. By mixing these red, green, and blues you get different colors, and once they all work at equalt levels they produce the white color. So that simple test itereated over each pixel, and finally lit the whole matrix in red, green, then blue.
C++
#include <Adafruit_NeoPixel.h>
// Define the digital pin connected to the LED matrix data input
#define PIN D1
// Total number of pixels in the XIAO matrix (typically a 6x10 grid)
#define NUMPIXELS 60
// Initialize the NeoPixel library
Adafruit_NeoPixel matrix(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);
// Delay times in milliseconds
int pixelDelay = 50; // Speed of the individual pixel chase
int colorDelay = 1000; // How long full colors stay on screen
void setup() {
matrix.begin(); // Initialize the matrix
matrix.setBrightness(30); // Set brightness low (0-255) to protect eyes and save power
}
void loop() {
// --- PART 1: Individual Pixel Test ---
// Iterates through every pixel one by one in white
for(int i=0; i<NUMPIXELS; i++) {
matrix.clear(); // Turn off previous pixel
matrix.setPixelColor(i, matrix.Color(255, 255, 255)); // White
matrix.show();
delay(pixelDelay);
}
matrix.clear();
matrix.show();
delay(200);
// --- PART 2: Full Matrix Red Test ---
fillMatrix(255, 0, 0); // Red
delay(colorDelay);
// --- PART 3: Full Matrix Green Test ---
fillMatrix(0, 255, 0); // Green
delay(colorDelay);
// --- PART 4: Full Matrix Blue Test ---
fillMatrix(0, 0, 255); // Blue
delay(colorDelay);
// Clear at the end of the loop cycle
matrix.clear();
matrix.show();
delay(1000);
}
// Helper function to light up the whole matrix at once
void fillMatrix(uint8_t r, uint8_t g, uint8_t b) {
for(int i=0; i<NUMPIXELS; i++) {
matrix.setPixelColor(i, matrix.Color(r, g, b));
}
matrix.show();
}
*Prompt14.3
Conclusion¶
Back in the “old” days, before the rise of LLMs I had hand coded a couple of games using Java’s GUI framworks – Swing and JavaFX. Java’s GUI was not the easies, but it was fun!
I cannot say that “vibe coding” is not fun, but it does not feel rewarding. At least not in these small projects. One thing that I loved about this week, and the assistance from AIs, is that I can program in any language, or for any OS I desire. Just like using Swift, which I have never ever used before…
Resources¶
Prompts¶
Prompt14.1 Text
Prompt14.2 You built a macOS app that controls a physical LED matrix over USB, covering hardware, firmware, and software end-to-end.
Hardware: A Seeed Studio XIAO ESP32S3 microcontroller stacked onto a 6×10 RGB matrix of 60 WS2812 LEDs, powered entirely over a single USB-C cable from a MacBook.
Firmware (Arduino/C++ on the ESP32S3): A multi-mode state machine that receives plain-text serial commands and drives the LEDs. Modes include ambient breathing (with a runtime-adjustable color), a teleprompter that scrolls pixel-font text across the matrix, and a playable Snake game. Includes a hard brightness safety ceiling to prevent overheating, a watchdog that auto-shuts the matrix off if the Mac app stops responding, and a calibration command to verify the physical pixel wiring order.
Mac App (SwiftUI + POSIX serial, no external dependencies): A native macOS app that opens the board by its USB bus address, sends mode/color/text/game-input commands, and receives live state back (power state, score, game over). Snake captures keyboard input via a custom NSViewRepresentable that sits in the responder chain. The app builds into a proper double-clickable .app bundle with a generated icon.
The system satisfies the assignment requirement: wired node (XIAO ESP32S3) addressed by USB bus path, local input (MacBook keyboard), local output (LED matrix), with bidirectional communication over the serial bus.Prompt14.3 A simple testing script for a XIAO RGB LED matrix connected to a XIAO ESP32-S3 microcontroller to check for dead or damaged pixels. The code needs to first iterate through every single pixel one by one by illuminating them in white, and then transition to lighting up the entire matrix simultaneously in three solid color phases: first red, then green, and finally blue.