16. Interface and application programming¶
My interface utilizes Prcessing and Arduino. I like making GUI with these two programs because there are a ton of examples online to rift from. I am basing my progrm off the examples listed below.
Examples¶
Serial Connection between Processing and Arduino
How it Works¶
I have two codes, below, one is written in Arduino controlling the Fabduino I made connected to a Adafruit motor shield. The other one is written in Processing for the GUI. The two seperate programs are connected via Serial. Basically, I created buttons for each note that processing displays on the screen. Each button is waiting for a mouse click within a specified area. Once the mouse cursor is moved within the location of the button labeled “A” and the mouse is clicked Processing sends the character A out over serial. All this time Arduino has been watching for any characters to come over serial. Once it detects the “A” arduino runs a command that moves the mallet over the A tine and plays the note.
It is interesting creating a GUI because you realize every computer program is only as smart as the person creating it and GUIs are really just an illusion. When the user clicks A they are really just clicking in space, the correct space, and I, the designer, have placed a visual cue to click there. I could move the image and technically the button would still work but the GUI would be bad because the program and the visual would not match.
Arduino Code¶
I created this code to test the music machine from our group project . To test the group project I intially tested the mechanism with an arduino and since I moved to my Fab Lab made Fabduino for this project I changed with output pins for the motors to pin 7 and pin 2 since those are the pins I can connect to the motorshield.
/* Sweep
by BARRAGAN <http://barraganstudio.com>
This example code is in the public domain.
modified 10 June 2019
by Brent Richardson
*/
#include <Servo.h>
Servo servo1; // create servo object to control a servo
// twelve servo objects can be created on most boards
Servo servo2;
int c1 = 108;
int d = 95;
int e = 80;
int f = 65;
int g = 50;
int a = 40;
int b = 25;
int c2 = 12;
void setup() {
servo1.attach(7); // attaches the servo on pin 7 to the servo object
servo2.attach(2); // attaches the servo on pin 2 to the servo object
Serial.begin(9600); //start serial communication @9600 bps
}
//39-c, 46-B, 54-A, 60-G, 68-F, 74e, 82d, c93,
void loop() {
if(Serial.available()){ //id data is available to read
char val = Serial.read();
if(val == 'a'){ //if a received
servo2.write(a); // tell servo to go to position in variable 'a'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'b'){
servo2.write(b); // tell servo to go to position in variable 'b'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'h'){ //if a received
servo2.write(c2); // tell servo to go to position in variable 'c2'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'c'){ //if a received
servo2.write(c1); // tell servo to go to position in variable 'c'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'd'){
servo2.write(d); // tell servo to go to position in variable 'd'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'e'){ //if a received
servo2.write(e); // tell servo to go to position in variable 'e'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'f'){
servo2.write(f); // tell servo to go to position in variable 'f'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
if(val == 'g'){
servo2.write(g); // tell servo to go to position in variable 'g'
hammer();
delay(10); // waits 15ms for the servo to reach the position
}
}
}
void hammer(){
servo1.write(138); // tell servo to go to position in variable 'pos'
delay(50);
servo1.write(150); // tell servo to go to position in variable 'pos'
delay(65);
servo1.write(138); // tell servo to go to position in variable 'pos'
delay(50);
}
Processing Code¶
Thanks to the processing library controlP5 making a GUI is quick. Each button has only a few variable to define.
The graphic part written to the screen is in the setup. I’m just telling the computer where to place the top left corner of each box, then I tell it how big to make each box, and I assign a font to each box for the text. Then in the draw loop I just call the libraries button watch function.
So, where is says
void C(){ port.write(‘c’);
That calling another bunch of code in the library that says if the mouse click is over the button and the mouse is clicked do something. The something is write the CHAR “c” to the port I defined in the sketch. Meanwhile the fabduino is reading the port and when it sees the CAR “c” it runs a command that moves the hammer over the C tine and hammers.
import controlP5.*; //import ControlP5 library
import processing.serial.*;
Serial port;
ControlP5 cp5; //create ControlP5 object
PFont font;
void setup(){ //same as arduino program
size(600, 450); //window size, (width, height)
printArray(Serial.list()); //prints all available serial ports
port = new Serial(this, "COM9", 9600); //i have connected arduino to com3, it would be different in linux and mac os
//lets add buton to empty window
cp5 = new ControlP5(this);
font = createFont("arial", 20); // custom fonts for buttons and title
cp5.addButton("C") //"red" is the name of button
.setPosition(100, 50) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("D") //"yellow" is the name of button
.setPosition(100, 150) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("E") //"blue" is the name of button
.setPosition(100, 250) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("F") //"alloff" is the name of button
.setPosition(100, 350) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("G") //"red" is the name of button
.setPosition(300, 50) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("A") //"yellow" is the name of button
.setPosition(300, 150) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("B") //"blue" is the name of button
.setPosition(300, 250) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
cp5.addButton("H") //"alloff" is the name of button
.setPosition(300, 350) //x and y coordinates of upper left corner of button
.setSize(120, 70) //(width, height)
.setFont(font)
;
}
void draw(){ //same as loop in arduino
background(150, 0 , 150); // background color of window (r, g, b) or (0 to 255)
//lets give title to our window
fill(0, 255, 0); //text color (r, g, b)
textFont(font);
text("Music Machine Control", 80, 30); // ("text", x coordinate, y coordinat)
}
//lets add some functions to our buttons
//so whe you press any button, it sends perticular char over serial port
void C(){
port.write('c');
}
void D(){
port.write('d');
}
void E(){
port.write('e');
}
void F(){
port.write('f');
}
void G(){
port.write('g');
}
void A(){
port.write('a');
}
void B(){
port.write('b');
}
void H(){
port.write('h');
}
Problem and Solution¶
I was doing this on my Surface computer and when I used the track pad to click on the buttons the worked great. When I tried to use the touch screen it didn’t work. Then I realized it was delayed. So, I would tap on a button and nothing would happen, but if I tapped a second time it would work. I found this forum that discusses the problem. The problem is in the library and there are other libraries that support touch screen. This library used the mousePressed() function which has problems with the touch screen. It looks like the 4GP library fixes the issue.