Week 15 - Interface and Application Programming¶
Tool Comparison¶
p5.js¶
Description: Renders creative visuals in a browser; hardware data fed via WebSocket or HTTP.
Complexity: Low - JS-based, beginner-friendly, minimal setup.
Performance: Moderate (browser canvas; fine for 2D, limited for heavy 3D).
Data Transfer: WebSocket, HTTP fetch, or p5.serialport (Serial via browser bridge).
Required Hardware: Any MCU with Wi-Fi or Serial; client with browser.
Capabilities: 2D/3D sketches, animations, audio-reactive visuals, interactive controls.
Use Cases: Sensor dashboards, generative art, beat-reactive displays.
Pros: Beginner-friendly, huge community, runs in any browser, no install.
Cons: Real-time Serial needs a browser bridge; limited for complex 3D.
Free / Open Source / Account needed? Yes - free and open source (LGPL).
Processing¶
Description: Java-based creative coding environment that runs as a standalone desktop app. Reads data from a microcontroller over Serial or network to drive visuals.
Complexity: Low to Moderate - Java-like syntax but beginner-friendly; Serial communication is straightforward with the built-in Serial library.
Performance: High - runs natively on the desktop CPU/GPU; handles smooth animations and fast Serial polling well.
Data Transfer: Serial (USB) via Processing’s built-in Serial library, or network via OSC, UDP, or TCP sockets.
Required Hardware: Microcontroller connected via USB Serial (or Wi-Fi for network mode); computer to run the Processing sketch.
Capabilities: 2D/3D graphics, animations, generative art, real-time data visualization, audio/video, interactive sketches.
Use Cases: Sensor data displays, interactive installations, generative visuals driven by physical input.
Pros: Huge community and example library; very fast to prototype; no browser needed; runs offline; great for physical computing projects.
Cons: Requires a laptop/desktop to run - not standalone on the microcontroller; less suitable for web-based or remote access interfaces.
Free / Open Source / Account needed? No account needed. Download directly from processing.org.
Arduino IDE Serial Plotter / Monitor¶
Description: Plotter graphs data / Monitor displays text output.
Complexity: Low - built into Arduino IDE.
Performance: Real-time at baud rate; no stored data.
Data Transfer: USB serial (UART), text-based using Serial.print() / Serial.println(); Serial Monitor can also send data back to the board.
Required Hardware: Microcontroller through USB.
Capabilities: Monitor displays text in real time and sends typed commands to device. Plotter plots numerical data.
Use Cases: Debugging code, monitoring sensor readings, visualizing sensor trends, basic input/output testing.
Pros: No extra software; visualize sensor data; can send commands back to board.
Cons: Basic; no data saving or complex charts; plots numerical data only.
Free / Open Source / Account needed? Free, open source.
Notes: Available on Mac, Windows, Linux. Plotter is a separate panel. Works with any board that has a serial connection.
Node.js + Express¶
Description: Node allows you to run JavaScript from your computer (as opposed to the browser); Express is an add-on that makes it easier to create a web server.
Complexity: Medium - familiar if you’ve written JavaScript before. More setup than Processing, but very flexible once running.
Performance: Fine for most projects. Not as powerful as Processing for heavy graphics, but great for dashboards, charts, and real-time updates in a browser using WebSockets.
Data Transfer: USB Serial or Wi-Fi. An ESP32 in AP mode fits the Wi-Fi pattern well.
Required Hardware: A microcontroller and a computer to run the server. Anyone on the same network can open the interface in their browser.
Capabilities:
Use Cases: Browser-based dashboards, control panels, real-time charts, data loggers, and command interfaces.
Pros: No app to install - anyone with a browser can use it. Huge library of free packages via npm. Works naturally with Chart.js.
Cons: More setup than Processing. You’re essentially building a mini web app. Still requires a computer to run the server.
Free / Open Source / Account needed? Free at nodejs.org. Express installs with one terminal command.
HTML/CSS/JS (ESP32)¶
Description: The ESP32 hosts a web server that serves a plain HTML/CSS/JS page directly from the microcontroller. Any device on the same network can open a browser and interact with it - no external server needed.
Complexity: Low to Moderate - requires writing both the ESP32 firmware (web server + data endpoints) and the front-end HTML/JS. Both pieces live in the same Arduino sketch.
Performance: Moderate - limited by the ESP32’s processing power and Wi-Fi stack; fine for simple dashboards and controls but not suited for heavy real-time streaming.
Data Transfer: HTTP (GET/POST) for simple requests; WebSockets for real-time bidirectional communication - both served directly from the ESP32.
Required Hardware: ESP32 (or similar Wi-Fi-capable MCU); any device with a browser on the same network. No external server or cloud needed.
Capabilities: Sensor dashboards, motor/LED controls, parameter tuning sliders, real-time data display via fetch or WebSocket polling.
Use Cases: Standalone device UIs, configuration pages, quick prototypes that need a phone/laptop interface without any infrastructure.
Pros: Completely self-contained - no server, no cloud, no account; works offline on a local network or AP mode.
Cons: Limited by ESP32 memory - large HTML/JS files must be stored in SPIFFS/LittleFS or inlined as strings; no persistent data storage without an external database.
Free / Open Source / Account needed? Yes - HTML/CSS/JS are open standards; ESP32 web server libraries (ESPAsyncWebServer, Arduino WebServer) are free and open source. No account needed.
React / Vue¶
Description: Component-based UI frameworks running in browser; hardware data fed via WebSocket, REST, or MQTT. A step up from vanilla.
Complexity: Medium - requires Node.js toolchain, component thinking, and state management.
Performance: High - virtual DOM, efficient re-renders, handles rapid real-time data well.
Data Transfer: WebSocket, HTTP fetch, MQTT, REST API.
Required Hardware: Any MCU with Wi-Fi (ESP32, Raspberry Pi); client with browser.
Capabilities: Real-time dashboards, data visualization, interactive controls, multi-panel UIs, charting via libraries like Recharts or Chart.js.
Use Cases: Sensor monitoring dashboards, machine control panels, IoT device interfaces, data logging displays. Best for complex multi-panel dashboards or team projects with lots of components.
Pros: Highly scalable, massive ecosystem, excellent for complex multi-component UIs, reusable components, strong community.
Cons: Overkill for simple projects; requires build toolchain (Vite/webpack); steeper learning curve than p5.js.
Free / Open Source / Account needed? Yes - both free and open source (MIT). No account needed. Recharts, Chart.js, etc. also free/open source.
TouchDesigner / Max/MSP¶
Description: TouchDesigner is a node-based programming language used to create interactive audio, visual, and graphic content.
Complexity: Moderate to High - node-based systems can be difficult for beginners to understand.
Performance: High; real-time performance optimized for GPU acceleration.
Data Transfer: Send and receive real-time data. MIDI, OSC, TCP/IP, UDP. Capable of integrating with external hardware and software.
Required Hardware: Multi-core CPU, dedicated GPU, 8-16 GB of RAM.
Capabilities: Real-time interactive visuals, generative art, and multimedia experiences.
Use Cases: Interactive art, live visual performances, projection mapping, immersive experiences, real-time data visualization.
Pros: Works well with external hardware and software; highly customizable through Python scripting.
Cons: Steep learning curve; requires high-powered computer hardware; higher-tier version licenses are expensive.
Free / Open Source / Account needed? Partially free - has a non-commercial version, a paid professional version, and a paid commercial version. Not open source. Account required: Yes.