14. Interface and Application Programming¶
Individual assignment:¶
- Write an application that interfaces a user with an input &/or output device that you made.
Group assignment:¶
- Compare as many tool options as possible.
From the group assignment on Interface and Application Programming, I learned the following:
- Processing:
- Built graphical interfaces using code to create colorful canvases and 2D shapes (e.g., rectangles, circles).
- Mastered core programming concepts like setup() and draw() loops for initialization and continuous execution.
-
Implemented interactive elements such as text display and custom pointers using mouseX and mouseY for user input handling.
-
Blynk:
- Understood how to connect microcontrollers (e.g., Arduino, my custom board) to the internet for IoT applications.
- Explored Blynk’s components: app builder for creating widget-based apps, server for communication, and libraries for hardware integration.
-
Learned about connectivity options (Wi-Fi, Bluetooth, Ethernet, Cellular, Serial) and the flexibility of using Blynk Cloud or local servers.
-
ESP32 Web Server:
- I learned how I can transform an ESP32 board into a WiFi-controlled smart switch by hosting a local web server.
- Developed a webpage with ON/OFF buttons to control GPIO pin 10, toggling an LED via HTTP GET requests.
- Gained knowledge of networking concepts, WiFi configuration, and real-time hardware control with Serial Monitor feedback.
Here is the link to the group assignment.
Individual assignment:¶
I download Processing form here for my individual assignment.
According to Processing official page, Processing is a flexible software sketchbook and a language for learning how to code. Since 2001, Processing has promoted software literacy within the visual arts and visual literacy within technology. There are tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning and prototyping.
Work Flow between Processing Program and Embedded Programming
The Processing program communicates with the Arduino program through serial communication, enabling a two-way interface for controlling and visualizing data. Processing, running on a PC, establishes a serial connection with the Arduino microcontroller using a specified port (e.g., COM4) and baud rate (e.g., 9600).
In the input scenario, Processing acts as a sender, transmitting commands to the Arduino. For example, a graphical user interface (GUI) created with the ControlP5 library in Processing allows users to click buttons (e.g., labeled A, B, C, or “alloff”), sending specific characters (‘1’, ‘2’, ‘3’, or ‘0’) over the serial port. The Arduino, programmed to listen for these characters, interprets them to control output devices, such as turning LEDs on or off based on the received value.
In the output scenario, the Arduino acts as a sender, capturing data (e.g., random values or sensor readings) and transmitting it over the serial port. Processing, as the receiver, imports the serial library, reads the incoming data, and visualizes it on a canvas, such as displaying a rectangle whose height corresponds to the received value.
This bidirectional communication allows Processing to serve as both an interactive control interface and a data visualization platform, while the Arduino handles physical inputs and outputs, creating a seamless workflow for user-driven applications.
After having a basic understanding of work flow, I try to create a interface of my Nappod where user can control the music and choose a specific types of music. I wrote the program where I can control music and music type via Blynk using Wifi communication. I will change this code and create a interface using Processing.
Here is the Blynk version of interface to control music and choose Music Type. Go to this link for more details on Blynk Interface I created in Networking and Communications week.
- Here is the Program which interface my mp3 with Blynk.
/* Fill-in information from Blynk Device Info here */
#define BLYNK_TEMPLATE_ID "TMPL3RChFD034"
#define BLYNK_TEMPLATE_NAME "NAP POD"
#define BLYNK_AUTH_TOKEN "wBKDIwuXVMXQLEJSb_QJpJKnXZ_sDJcK"
#define BLYNK_PRINT Serial
#include <DFRobotDFPlayerMini.h>
#include <SoftwareSerial.h>
#include <WiFi.h>
#include <WiFiClient.h>
#include <BlynkSimpleEsp32.h>
// Initialize software serial on pins D0 (RX) and D1 (TX)
SoftwareSerial mySerial(D4, D5);
// Create DFPlayer object
DFRobotDFPlayerMini myDFPlayer;
// Your WiFi credentials.
// Set password to "" for open networks.
char ssid[] = "esp32";
char pass[] = "esp32369369";
BlynkTimer timer;
bool NapPodActivated = false;
// Volume control from Blynk slider on V1
BLYNK_WRITE(V1) {
int vol = constrain(param.asInt(), 0, 30);
myDFPlayer.volume(vol);
Serial.print(F("Volume set to: "));
Serial.println(vol);
}
bool status = 1;
// This function is called every time the Virtual Pin 0 state changes
BLYNK_WRITE(V0) {
if(NapPodActivated){
// Called when the datastream virtual pin V0 is updated by Blynk.Console, Blynk.App, or HTTP API.
String value = param.asStr();
if (value == "play") {
if (status){
Serial.println("'play' button pressed");
myDFPlayer.play(3); //Play the first mp3
delay(1000);
status = !status; // Toggle pause state
}else if(!status) {
myDFPlayer.start(); //start the mp3 from the pause
delay(1000);
}
} else if (value == "stop") {
Serial.println("'stop' button pressed");
myDFPlayer.pause(); //pause the mp3
} else if (value == "prev") {
Serial.println("'prev' button pressed");
myDFPlayer.previous(); //Play previous mp3
delay(1000);
} else if (value == "next") {
Serial.println("'next' button pressed");
myDFPlayer.next(); //Play next mp3
delay(1000);
} else {
Serial.print("V0 = '");
Serial.print(value);
Serial.println("'");
}
}
}
void setup(){
// Debug console
Serial.begin(115200);
mySerial.begin(9600);
Blynk.begin(BLYNK_AUTH_TOKEN, ssid, pass);
// Initialize DFPlayer
if (!myDFPlayer.begin(mySerial)) {
Serial.println(F("DFPlayer initialization failed!"));
Serial.println(F("1. Check RX/TX connections (must be crossed)"));
Serial.println(F("2. Insert SD card with MP3 files"));
while(true); // Halt if initialization fails
}
}
BLYNK_WRITE(V2){
int pinValueV2 = param.asInt(); // Get the value from the app
if (pinValueV2 && !NapPodActivated) {
myDFPlayer.play(1); //Play the first mp3
Serial.println(F("Activating nappod"));
delay(1000);
NapPodActivated = true;
}else if(!pinValueV2 && NapPodActivated ){
myDFPlayer.play(2); //Play the first mp3
Serial.println(F("Deactivating nappod"));
delay(1000);
NapPodActivated = false;
}
}
//-----------------------------------------------------------------------
// Music Therapy(nature and meditation music) (0003 - 0005)
BLYNK_WRITE(V3){
int pinValueV3 = param.asInt(); // Get the value from the app
if (pinValueV3 && NapPodActivated) {
Serial.println(F("Button V3 pressed"));
myDFPlayer.play(3); //Play the first mp3
delay(1000);
}
}
// Soothing Bhutanese Music (000 - 0008)
BLYNK_WRITE(V4){
int pinValueV4 = param.asInt(); // Get the value from the app
if (pinValueV4 && NapPodActivated) {
Serial.println(F("Button V4 pressed"));
myDFPlayer.play(6); //Play the first mp3
delay(1000);
}
}
// Soothing Western Music (0009 - 00011)
BLYNK_WRITE(V5){
int pinValueV5 = param.asInt(); // Get the value from the app
if (pinValueV5 && NapPodActivated) {
Serial.println(F("Button V5 pressed"));
myDFPlayer.play(9); //Play the first mp3
delay(1000);
}
}
// Soothing Indian Music (00012 - 00014)
BLYNK_WRITE(V6){
int pinValueV6 = param.asInt(); // Get the value from the app
if (pinValueV6 && NapPodActivated) {
Serial.println(F("Button V6 pressed"));
myDFPlayer.play(12); //Play the first mp3
delay(1000);
}
}
//-----------------------------------------------------------------------------
void loop(){
Blynk.run();
timer.run();
}
-
I will try to create a similar Interface using Processing. Following is the simple music control user interface I created using processing for my NapPod.
-
Here is the circuit connection.
-
This is the image of interacting with my Xiao RP2040 board via User Interface I programmed.
-
Here is my program for the interactive user interface I created. This program communicate with my embedded program via COM PORT 8. As I interact with interface, it will write certain character on port 8 and any character written on the port will be picked up by my embedded program and send relevant command to speaker connected to my custom XIAO board.
// Nap Pod Controller GUI
import processing.serial.*;
Serial port;
class MusicPlayer {
float x, y;
float width, height;
boolean isPlaying;
int currentTrack = 1;
int totalTracks = 14;
float volume = 1;
String currentPlaylist = "None";
MusicPlayer(float x, float y, float w, float h) {
this.x = x;
this.y = y;
this.width = w;
this.height = h;
this.isPlaying = false;
}
void display() {
// Player background
fill(30);
rect(x, y, width, height, 10);
// Display area
fill(50);
rect(x + 20, y + 20, width - 40, height - 100, 5);
// Track info
fill(200);
textSize(16);
textAlign(LEFT, CENTER);
text("Track: " + currentTrack + "/" + totalTracks, x + 30, y + 40);
text("Playlist: " + currentPlaylist, x + 30, y + 70);
text("Status: " + (isPlaying ? "Playing" : "Stopped"), x + 30, y + 100);
// Volume display
fill(100, 150, 255);
rect(x + 30, y + 130, (width - 60) * volume, 20, 5);
fill(200);
text("Volume: " + int(volume * 100) + "%", x + 30, y + 160);
// Control buttons
drawButton(x + width/2 - 80, y + height - 50, 40, 40, "|<", color(100)); // Previous
drawButton(x + width/2 - 30, y + height - 50, 40, 40, isPlaying ? ">" : "||", color(100, 200, 100)); // Play/Pause
drawButton(x + width/2 + 20, y + height - 50, 40, 40, ">|", color(100)); // Next
}
void drawButton(float bx, float by, float bw, float bh, String label, color btnColor) {
fill(btnColor);
rect(bx, by, bw, bh, 5);
fill(255);
textAlign(CENTER, CENTER);
textSize(16);
text(label, bx + bw/2, by + bh/2);
}
boolean isPlayButtonClicked(float mx, float my) {
return (mx > x + width/2 - 30 && mx < x + width/2 + 10 &&
my > y + height - 50 && my < y + height - 10);
}
boolean isPrevButtonClicked(float mx, float my) {
return (mx > x + width/2 - 80 && mx < x + width/2 - 40 &&
my > y + height - 50 && my < y + height - 10);
}
boolean isNextButtonClicked(float mx, float my) {
return (mx > x + width/2 + 20 && mx < x + width/2 + 60 &&
my > y + height - 50 && my < y + height - 10);
}
boolean isVolumeClicked(float mx, float my) {
return (mx > x + 30 && mx < x + width - 30 &&
my > y + 130 && my < y + 150);
}
void setVolume(float mx) {
volume = constrain((mx - (x + 30)) / (width - 60), 0, 1);
}
}
class PlaylistButton {
float x, y;
float width, height;
String label;
color btnColor;
boolean active;
PlaylistButton(float x, float y, float w, float h, String label, color btnColor) {
this.x = x;
this.y = y;
this.width = w;
this.height = h;
this.label = label;
this.btnColor = btnColor;
this.active = false;
}
void display() {
fill(active ? color(red(btnColor), green(btnColor), blue(btnColor), 200) : btnColor);
rect(x, y, width, height, 5);
fill(255);
textAlign(CENTER, CENTER);
textSize(14);
text(label, x + width/2, y + height/2);
}
boolean isClicked(float mx, float my) {
return (mx > x && mx < x + width && my > y && my < y + height);
}
}
MusicPlayer player;
PlaylistButton[] playlistButtons;
boolean NapPodActivated = false;
color activeColor = color(23, 241, 115);
color inactiveColor = color(100);
void setup() {
size(800, 700);
smooth();
port = new Serial(this, "COM8", 9600);
// Create music player
player = new MusicPlayer(50, 50, width - 100, 300);
// Create playlist buttons
playlistButtons = new PlaylistButton[4];
playlistButtons[0] = new PlaylistButton(50, 400, width - 100, 50, "Music Therapy (Nature Sounds)", color(123, 25, 116));
playlistButtons[1] = new PlaylistButton(50, 470, width - 100, 50, "Bhutanese Music", color(206, 69, 100));
playlistButtons[2] = new PlaylistButton(50, 540, width - 100, 50, "Western Music", color(143, 14, 190));
playlistButtons[3] = new PlaylistButton(50, 610, width - 100, 50, "Indian Music", color(226, 147, 80));
// Set text properties
rectMode(CORNER);
textAlign(CENTER, CENTER);
textSize(12);
}
void draw() {
background(20);
// Draw title
fill(255);
textSize(24);
text("Nap Pod Controller", width/2.8, 40);
// Draw activation button
fill(NapPodActivated ? activeColor : inactiveColor);
rect(width - 150, 10, 120, 30, 5);
fill(255);
textSize(16);
text(NapPodActivated ? "ACTIVATED" : "ACTIVATE", width - 95, 30);
// Draw music player
player.display();
// Draw playlist buttons
for (PlaylistButton btn : playlistButtons) {
btn.display();
}
// Draw instructions
fill(255);
textSize(15);
textAlign(CENTER);
text("Click on volume bar to adjust volume", 400, 250);
}
void mousePressed() {
// Check Nap Pod activation
if (mouseX > width - 150 && mouseX < width - 30 &&
mouseY > 10 && mouseY < 40) {
NapPodActivated = !NapPodActivated;
if (NapPodActivated) {
println("Nap Pod Activated");
port.write('A');
player.isPlaying = true;
player.currentTrack = 1;
player.currentPlaylist = "Startup Sound";
} else {
println("Nap Pod Deactivated");
port.write('D');
player.isPlaying = false;
player.currentPlaylist = "None";
// Deactivate all playlist buttons
for (PlaylistButton btn : playlistButtons) {
btn.active = false;
}
}
return;
}
// Only respond to music controls if Nap Pod is active
if (!NapPodActivated) return;
// Check music player controls
if (player.isPlayButtonClicked(mouseX, mouseY)) {
println(player.isPlaying ? "Play" : "Pause");
if (player.isPlaying) {
port.write('x');
player.isPlaying = !player.isPlaying;
} else {
player.isPlaying = !player.isPlaying;
port.write('y');
}
} else if (player.isPrevButtonClicked(mouseX, mouseY)) {
player.currentTrack = max(1, player.currentTrack - 1);
println("Previous Track");
if (player.isPlaying){
player.isPlaying = !player.isPlaying;
}else{
player.isPlaying = player.isPlaying;
}
port.write('z');
} else if (player.isNextButtonClicked(mouseX, mouseY)) {
player.currentTrack = min(player.totalTracks, player.currentTrack + 1);
println("Next Track");
if (player.isPlaying){
player.isPlaying = !player.isPlaying;
}else{
player.isPlaying = player.isPlaying;
}
port.write('l');
} else if (player.isVolumeClicked(mouseX, mouseY)) {
player.setVolume(mouseX);
int volumePercent = int(player.volume * 100);
int scaledVolume = (int)map(volumePercent, 0, 100, 0, 9);
port.write(scaledVolume);
println("Volume set to " + (scaledVolume));
}
// Check playlist buttons
for (int i = 0; i < playlistButtons.length; i++) {
if (playlistButtons[i].isClicked(mouseX, mouseY)) {
// Deactivate all other buttons
for (PlaylistButton btn : playlistButtons) {
btn.active = false;
}
// Activate this button
playlistButtons[i].active = true;
player.isPlaying = true;
// Set playlist info
switch(i) {
case 0:
player.currentPlaylist = "Music Therapy";
player.currentTrack = 3;
println("Playing Music Therapy playlist");
if (player.isPlaying){
player.isPlaying = !player.isPlaying;
}else{
player.isPlaying = player.isPlaying;
}
port.write('M');
break;
case 1:
player.currentPlaylist = "Bhutanese Music";
player.currentTrack = 6;
println("Playing Bhutanese Music playlist");
if (player.isPlaying){
player.isPlaying = !player.isPlaying;
}else{
player.isPlaying = player.isPlaying;
}
port.write('B');
break;
case 2:
player.currentPlaylist = "Western Music";
player.currentTrack = 9;
println("Playing Western Music playlist");
if (player.isPlaying){
player.isPlaying = !player.isPlaying;
}else{
player.isPlaying = player.isPlaying;
}
port.write('W');
break;
case 3:
player.currentPlaylist = "Indian Music";
player.currentTrack = 12;
println("Playing Indian Music playlist");
if (player.isPlaying){
player.isPlaying = !player.isPlaying;
}else{
player.isPlaying = player.isPlaying;
}
port.write('I');
break;
}
}
}
}
- Here is my embedded program which listen to my Processing program.
#include <DFRobotDFPlayerMini.h>
#include <SoftwareSerial.h>
// Constants ------------------------
#define LED_PIN D0
#define SOFTWARE_SERIAL_RX D4
#define SOFTWARE_SERIAL_TX D5
#define INITIAL_VOLUME 15
#define RESPONSE_DELAY_MS 1000
// Track numbers-----------------------------------------
enum {
TRACK_ACTIVATE = 1,
TRACK_DEACTIVATE = 2,
TRACK_MEDITATION_START = 3, // Default starting track
TRACK_BHUTANESE_START = 6,
TRACK_WESTERN_START = 9,
TRACK_INDIAN_START = 12
};
// Control commands--------------------------------------
enum Command {
CMD_ACTIVATE = 'A', // Activate the NapPod
CMD_DEACTIVATE = 'D', // Deactivate the NapPod
CMD_MEDITATION = 'M', // Musics for Meditation
CMD_BHUTANESE = 'B', // Bhutanese Music
CMD_WESTERN = 'W', // Western Music
CMD_INDIAN = 'I', // Indian Music
CMD_PLAY = 'P', // Play track
CMD_PAUSE_TOGGLE = 'x', // Toggle
CMD_PAUSE = 'y', // Pause current track
CMD_PREVIOUS = 'z', // Play Previous track
CMD_NEXT = 'l' // Play Next track
};
// Global Variables -----------------------------------------------------------
SoftwareSerial softwareSerial(SOFTWARE_SERIAL_RX, SOFTWARE_SERIAL_TX);
DFRobotDFPlayerMini dfPlayer;
bool napPodActive = false;
bool playbackPaused = true;
bool firstPlayAfterActivation = true; // flag to track first play
// Forward Declarations -------------------------------------------------------
void activateNapPod();
void playDefaultTrack();
void playMeditation();
void playBhutanese();
void playWestern();
void playIndian();
void handleVolumeControl(char value);
void handleTransportControls(char command);
// Core Functions -------------------------------------------------------------
void setup() {
Serial.begin(9600);
softwareSerial.begin(9600);
//pinMode(LED_PIN, OUTPUT); used this led for debugging. It was really helpful!!!
if (!dfPlayer.begin(softwareSerial)) {
while (true); // Hardware freeze on failure
}
dfPlayer.volume(INITIAL_VOLUME);
}
void loop() {
if (!Serial.available()) return;
const char command = Serial.read();
switch (command) {
case CMD_ACTIVATE:
case CMD_DEACTIVATE:
activateNapPod();
break;
case CMD_PLAY: // New play command
playDefaultTrack();
break;
case CMD_MEDITATION:
playMeditation();
break;
case CMD_BHUTANESE:
playBhutanese();
break;
case CMD_WESTERN:
playWestern();
break;
case CMD_INDIAN:
playIndian();
break;
case CMD_PAUSE_TOGGLE:
case CMD_PAUSE:
case CMD_PREVIOUS:
case CMD_NEXT:
handleTransportControls(command);
break;
default:
if (command >= 0 && command <= 9) {
handleVolumeControl(command);
}
break;
}
}
//Control Implementations ---------------------------------------------
void activateNapPod() {
dfPlayer.volume(30); // setting volume to max whenever user activates or deactivates the NapPod
dfPlayer.play(napPodActive ? TRACK_DEACTIVATE : TRACK_ACTIVATE);
napPodActive = !napPodActive;
firstPlayAfterActivation = true; // Reset flag on activation
delay(RESPONSE_DELAY_MS);
}
void playMeditation() {
if (napPodActive) dfPlayer.play(TRACK_MEDITATION_START);
delay(RESPONSE_DELAY_MS);
}
void playBhutanese() {
if (napPodActive) dfPlayer.play(TRACK_BHUTANESE_START);
delay(RESPONSE_DELAY_MS);
}
void playWestern() {
if (napPodActive) dfPlayer.play(TRACK_WESTERN_START);
delay(RESPONSE_DELAY_MS);
}
void playIndian() {
if (napPodActive) dfPlayer.play(TRACK_INDIAN_START);
delay(RESPONSE_DELAY_MS);
}
void handleVolumeControl(char value) {
int mappedValue = map(value, 0, 9, 0, 30);
dfPlayer.volume(mappedValue);
}
void playDefaultTrack() {
if (napPodActive) {
if (firstPlayAfterActivation) {
dfPlayer.play(3);
firstPlayAfterActivation = false;
} else {
dfPlayer.start(); // Resume playback from current track
}
}
}
void handleTransportControls(char command) {
switch (command) {
case CMD_PAUSE_TOGGLE:
playDefaultTrack();
playbackPaused ? dfPlayer.start() : dfPlayer.pause();
playbackPaused = !playbackPaused;
break;
case CMD_PAUSE:
dfPlayer.pause();
playbackPaused = true;
break;
case CMD_PREVIOUS:
dfPlayer.previous();
break;
case CMD_NEXT:
dfPlayer.next();
break;
}
delay(RESPONSE_DELAY_MS);
}
Processing GUI and EmbeddedProgram Interaction Overview
1. Processing GUI (Interface) The Processing code creates a graphical user interface (GUI) for controlling the NapPod system. Key components include:
- Activation Toggle Button
- Top-right button to activate/deactivate the NapPod (
A/Dcommands sent to Arduino). -
Changes color to indicate status (green = active, gray = inactive).
-
Music Player Panel
- Displays current track, playlist, and playback status (playing/paused).
- Interactive volume slider (sends values
0–9mapped to 0–30% volume). -
Transport controls:
- Play/Pause (
xcommand to toggle,yto pause). - Previous (
zcommand). - Next (
lcommand).
- Play/Pause (
-
Playlist Selection Buttons
- Four playlists (Meditation, Bhutanese, Western, Indian).
- Sends corresponding commands (
M,B,W,I) to start playback from predefined track numbers.
2. Arduino-Processing Interaction
- Serial Communication
- Processing sends single-character commands via COM23 (e.g., A for activate, M for Meditation playlist).
- Embedded program reads these commands and executes actions (play/pause, change track, adjust volume).
- Volume Control
- Processing maps GUI slider (0–100%) to
0–30range by sending values0–9. -
Embedded program scales these to
0–30usingmap(). -
Playback Control
- The GUI’s play/pause buttons toggle playback state, while playlist buttons reset the track number (e.g., Meditation starts at track
3). -
Embedded program handles track changes (
previous()/next()) and pauses/resumes playback. -
Activation Logic
- On activation (
A), Embedded program plays a startup sound (TRACK_ACTIVATE) and enables playback. -
Deactivation (
D) stops playback and plays a shutdown sound (TRACK_DEACTIVATE). -
Here is the video of me trying to control My nappod via the interface I programmed using processing.