Assignments
Our tasks for this week are:
- Group assignment: Compare as many tool options as possible
- Individual assignment: Write an application that interfaces a user with an input &/or output device that you made
Group Assignment
The group assignment is linked here.
In the group assignment we summarized several of the user interfacing tools used by members of our class including PyQt5, React, MediaPipe, Processing, and Flutter. A general overview of each package, as well as its strengths, challenges, and a tutorial and/or review based on personal experience was included for each tool.
Individual Assignment
Selected UI
I used Processing to interface with the Arduino IDE and a potentiometer input device. I based my assignment off of this instructables tutorial by BhattaG which I first discovered during Inputs week when I was testing detecting heart rate via a piezoelectrode sensor.
I used the base board I developed and produced for my Final Project. The microcontroller is the Xiao ESP32c3.
In my arduino code I read the serial value from the analog input device (in this case a potentiometer, but it could also be something like a piezoelectic input device). In the processing sketch I listen for the serial values from COM11 (the port my board is connected to) and make a visualization based off those values.
Code
Arduino IDE
Code for the Arduino IDE can be downloaded here or copied and pasted from below.
You’ll have to update the pin name to reflect the correct one of the input device you have attached to your board.
// code from https://www.instructables.com/Heart-Rate-Monitor/, BhattaG
int POT_PIN = D0; // analog output pin for the potentiometer
void setup() {
Serial.begin(9600);
}
void loop() {
Serial.println(analogRead(POT_PIN));
delay(500);
}
Processing sketch
The Processing sketch I used can be downloaded here or copied and pasted from below.
N.B.: you’ll have to update the myPort = new Serial(this, "COM11", 9600); line to whatever port you’re using on your computer. In mine it was COM11.
// processing code based on https://www.instructables.com/Heart-Rate-Monitor/, BhattaG
// processing code modified with the help of chatGPT
import processing.serial.*;
Serial myPort;
int currentBPM = 60; // Current BPM value
int pulseState = 0; // Pulse state (on/off)
int lastPulseTime = 0; // Time when the last pulse occurred
void setup () {
size(600, 400);
frameRate(25);
println(Serial.list());
myPort = new Serial(this, "COM11", 9600); // UPDATE WITH THE PORT ON YOUR COMPUTER
background(0);
}
void draw () {
background(0);
textSize(32);
fill(255); // White color for text
text("BPM: " + currentBPM, 10, height - 50);
int currentTime = millis();
int pulseDuration = 60000 / currentBPM / 2; // Duration of each pulse state (on/off)
// Control the red light pulsing
if (currentTime - lastPulseTime >= pulseDuration) {
pulseState = 1 - pulseState; // Toggle pulse state
lastPulseTime = currentTime;
}
if (pulseState == 1) {
fill(255, 0, 0); // Red color for light
} else {
fill(0); // Turn off the light
}
ellipse(width / 2, height / 2, 100, 100); // Draw the red light
}
void serialEvent (Serial myPort) {
String inString = myPort.readStringUntil('\n');
if (inString != null) {
inString = trim(inString);
println(inString);
int analogValue = int(inString);
// Map the analog value (0-4095) to BPM (30-180)
currentBPM = int(map(analogValue, 0, 4095, 30, 180));
}
}
Challenges
A common beginner challenge to working with Processing and Arduino is trying to have the serial monitor open in both. In order for Processing to read the serial values, the Arduino serial monitor must be closed. If Processing is still not working/able to access the port, try reseting the board via the Reset button onboard.
Hero Shots
Reflections
I had a few hiccups due to trying to do this assignment outside of the lab (not having an FTDI cable, or little connector wires, with me) but I figured out how to do it with a the base board I had with me since the Xiao allow me to both program and read serial from the same port.
Notes from the week below
Notes:
IFttt OSC (wireless protocol) vs serial PureData Pixel sorting effect for OSC projects, just buy a router for it ($35) for AR, use home assistant on Raspberry Pi 5 or 4
Processing
- open source, free
- like arduino but for designers
Touch Designer
- some functionality free, some not
- node based, like grasshopper
- a lot of people use just for visuals
- info from phone –> physical output or create visuals
PureData
- Max MSP - for audio/song creation, is the commercial version of PureData Link
- more “rustic” interface, but open source
- Maria Arnal, Catalan singer
MaxMSP
Martin Messier - drops of water
Max
Max Air Guitar
Zappar
AR interactions through phone
Hands on Demo
Processing
info transmitted between Arduino <–> Processing
OSC seems more robust (as long as have own router) Serial is better for if using a cable, direct wire
Touch Designer
CHOPs DMX
Max MSP
Link connect to phone gyroscope/accelerometer
Zigzam
Really useful app for interfacing with phone
Including but not limited to: image detection NFC reader (read NFC tag, produce some output) Apple Pencil data reading Acceleration Gravity (if throw phone) gyroscope quaternation compass (orientation from phone) pressure (how many finger pressing) GPS - location
TouchOSC
“DJ” interface app need to connect with Processing, Touch Designer, etc. that uses OSC Will always need an intermediate program
AR kit
kit apple has for face sensors ex: turn a light on by blinking an eye, open mouth image detection
demo w/ AR on phone: Zappar / Zapworks
Tree and fox example Web hook - application connected to home assistant
Home assistant demo w/ Raspberry Pi
Raspberry pi 5 - it’s a tiny computer in itself will need an SD card to burn any program to a Raspberry pi, need an SD card & Raspberry pi image burner can be used for 3D printing operations also Home Assistant connect raspberry pi to router via ethernet connect RPi to power
- important to use google chrome browser, only browser which supports these interactions
Ideas
interact with heart in 3D, hospital installation interact with heart in OR
visualize flow through heart/tube visualize heart valve function visualize valve support structures