14. Interface and Application Programming¶
Members: Hajime Itoh / Koji Yamada
Group Assignment Overview¶
This is the group assignment of the week.
- Compare as many tool options as possible
1. MIT App Inventor¶
We first tried this app because one of our team members has a final project that incurs the usage of a mobile app. Here is the exerpt from About us.
MIT App Inventor is an intuitive, visual programming environment that allows everyone – even children – to build fully functional apps for Android phones, iPhones, and Android/iOS tablets.
As it describes itself, we can operate it almost intuitively. Its blocks-based programming environment facilitates the creation of complex, high-impact apps in significantly less time than traditional ones. We could follow the tutorials to learn how to operate. This is the output created by one of our team members who tried the first tutorial “HelloPurr”.
Data: APK File
For more exploration, please check Koji’s Individual Assignment page.
2. Processing with Serial Data¶
Processing is a programming language based on Java. It is open source and downloadable free of charge. It is characterized by its ease of use for drawing and interactive animation. At the same time, it is suitable for beginners to learn programming because it is easy to write programs with a syntax similar to Java and C, which are more complex programming languages.
Here is the code we wrote:
int count = 30; // number of balls
int[] xs = new int[count]; // Center of each ball on X-coordinate
int[] ys = new int[count]; // Center of each ball on Y-coordinate
int d = 50; // Diameter of the ball
int[] xSteps = new int[count]; // Distance each ball moves along the X-axis
int[] ySteps = new int[count]; // Distance each ball moves along the Y-axis
color[] ballCols = new color[count]; // Ball color
color bgCol = color(0, 0, 0); // Background color
void setup(){
size(800, 600);
for(int i = 0; i< count; i++){
xs[i] = int(random(width));
ys[i] = int(random(height));
xSteps[i] = int(random(3, 10));
ySteps[i] = int(random(3, 10));
ballCols[i] = color(random(255), random(255), random(255));
}
}
void draw(){
background(bgCol);
for(int i = 0; i < count; i++){
fill(ballCols[i]);
ellipse(xs[i], ys[i], d, d);
if(xs[i] < 0 || xs[i] > width){
xSteps[i] *= -1;
}
if(ys[i] < 0 || ys[i] > height){
ySteps[i] *= -1;
}
xs[i] += xSteps[i];
ys[i] += ySteps[i];
}
}
2-1. Processing x Arduino Serial Communication¶
Based on the above warm-up, we tried to convert the serial data to animation.
First, we set up the serial communication with Ardino IDE. Here we used our board in hand, ESP32S3. We wrote the following sample code on the Arduino IDE and uploaded it on our board.
void setup(){
Serial.begin(9600);
}
void loop(){
Serial.write(100);// 1 byte = 0-255
delay(1000);
Serial.write(200);// 256 goes to 0
delay(1000);
}
Once the board started sending out the serial data, we closed the Arduino IDE and opened Processing 4.3. There we wrote the following sample code. In our case, the serial port was COM23 and we wrote it accordingly.
import processing.serial.*;
Serial myPort;
int x;
void setup(){
size(540, 540);// canvas size
myPort = new Serial(this, "COM23", 9600); //port
}
void draw(){
background(540);
ellipse(x+100,270,200,200); //(center_x,center_y,width,height)
}
void serialEvent(Serial p){
x = p.read(); // read value from serial
println(x);
}
2-2. Combine “2D Distance” Sketch and Serial Communication¶
For further exploration, we added the serial communication to the basic sample sketch 2D Distance available on the Processing webpage.
////////////////////////////////////////////////////////add
import processing.serial.*;
Serial myPort;
int x;
//////////////////////////////////////////////////////added
float max_distance;
void setup() {
size(640, 360);
noStroke();
max_distance = dist(0, 0, width, height);
myPort = new Serial(this, "COM23", 9600); //port (Added)
}
void draw() {
background(0);
for(int i = 0; i <= width; i += 20) {
for(int j = 0; j <= height; j += 20) {
float size = dist(x, mouseY, i, j);//////// changed mouseX to X
size = size/max_distance * 66;
ellipse(i, j, size, size);
}
}
}
void serialEvent(Serial p){ //Added
x = p.read(); // read value from serial
println(x);
}
Once we succeed in running this code, we could add image transition according not only to mouse movement, but also the x position set by the on/off of the serial data.
2-3. Combine “Storing Input” Sketch and Serial Communication¶
As the third exercise, we used the basic sample sketch Storing Input, which changes the position of the circle according to the mouse movement. If the sketch is just simply copied and pasted, we could get the animation that the circle moves according to the mouse.
We added the serial on/off data and attached it to the X-position of the circle. We added a few lines to the basic sample code that confines the X-position to just two options. Also, we had to make sure that there should be one slight modification of the original sample code.
/**
* Storing Input.
*
* Move the mouse across the screen to change the position
* of the circles. The positions of the mouse are recorded
* into an array and played back every frame. Between each
* frame, the newest value are added to the end of each array
* and the oldest value is deleted.
*/
import processing.serial.*; // Added
Serial myPort; // Added
int x; // Added
int num = 60;
float mx[] = new float[num];
float my[] = new float[num];
void setup() {
size(640, 360);
noStroke();
fill(255, 153);
myPort = new Serial(this, "COM23", 9600); //port Added
}
void draw() {
background(51);
// Cycle through the array, using a different entry on each frame.
// Using modulo (%) like this is faster than moving all the values over.
int which = frameCount % num;
mx[which] = x; // Changed from mouseX to X
my[which] = mouseY;
for (int i = 0; i < num; i++) {
// which+1 is the smallest (the oldest in the array)
int index = (which+1 + i) % num;
ellipse(mx[index], my[index], i, i);
}
}
void serialEvent(Serial p){ // Added
x = p.read(); // read value from serial
println(x);
}
As the integration of the two programs on Processing platform, we could get the following animation.
2-4. Exporting the Application¶
Once we made programs, the next step is to export the application according to our needs. On the Processing platform, we should first save the file and then go to File > Expoert Application. Then the Export Option dialogue window will be popped up.
With these default settings, we were supposed to get the app file in the sketch folder. However, we failed a few times to get the app file for both programs described in the 2-2 and 2-3 above. So far, we have not been successful in rectifying the problem.
3. Processing and p5*js¶
No matter whether we design the UI for PC or mobile phone, we may wish to create a graphic animation and embed it in the interface. Since we had hard time exporting the app data created with the Processing program as described in the above section, we also tried to approach this issue in a different way.
For this, we learned p5*js, another free and open-source program available for artists, designers, educators and creators. While p5*js also has rich libraries for beginners to refer to, we can also make use of the Processing to p5*js Converter, with which we could easily convert the codes made with Processing to the ones for p5. We could copy the code on the sketch panel on Processing and past it on the Processing Input panel on the Converter. Once we click the “Convert and run!!” button under the Input Panel, the source code is automatically translated to the code for p5.
What we should do next is to copy the code on the Output Panel and paste it on the Input Panel on the p5*js. Once we could confirm that it’s running as we expect, we could proceed to download the files compiled in the zip file.
Note: We tried to convert a few sample source codes available in the Examples on the Processing website and found that for some examples the converted codes shown in the Output Panel, do not run on p5*js as we expect.
4. p5*js¶
p5*js alone is another powerful tool to write creative codes for animations that could be embedded in the websites and mobile app. Also, after converting the animation made with Processing to p5 data, we would probably wish to use the animation for enhancing our UI.
In order to further explore the potential of the p5*js, we have signed up on the website and created our own accounts. Then we referred to the sample sketches available in the “Examples” tab and picked up one attractive example. After slightly modifying the original source code, we ran it.
After confirming that it worked, we first saved the file so that we could open it any time. We found that it consists of three different files. On the top left corner of the sketch panel, there is an inequality sign. By clicking it, we could expand and see those three files: index.html, sketch.js, and style.css. In order to embed the program on the website, we have to integrate these files and write it in markdown language as one single code.
index.html¶
<!DOCTYPE html>
<html lang="en">
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.9.2/p5.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.9.2/addons/p5.sound.min.js"></script>
<link rel="stylesheet" type="text/css" href="style.css">
<meta charset="utf-8" />
</head>
<body>
<main>
</main>
<script src="sketch.js"></script>
</body>
</html>
sketch.js¶
function setup() {
createCanvas(windowWidth, windowHeight, WEBGL);
}
function draw() {
background(0, 0, 0);
for (var y = 0; y <= 1000; y = y + 500) {
for (var x = 0; x <= 1000; x = x + 500) {
noFill();
stroke(255, 147, 206);
rotateX(frameCount * 0.01);
rotateY(frameCount * 0.01);
box(200, 200, 200);
}
}
}
style.css¶
html, body {
margin: 0;
padding: 0;
}
canvas {
display: block;
}
Integrated Code¶
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.9.2/p5.js"></script> // From index.html
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.9.2/addons/p5.sound.min.js"></script> // From index.html
<link rel="stylesheet" type="text/css" href="style.css"> // From index.html
<meta charset="utf-8" /> // From index.html
<script> // New
function setup() { // From sketch.js
createCanvas(windowWidth, windowHeight, WEBGL);
}
function draw() { // From sketch.js
background(0, 0, 0);
for (var y = 0; y <= 1000; y = y + 500) {
for (var x = 0; x <= 1000; x = x + 500) {
noFill();
stroke(255, 147, 206);
rotateX(frameCount * 0.01);
rotateY(frameCount * 0.01);
box(200, 200, 200);
}
}
}
</script> // New
As a result of the above integration, we could getthe following animation embedded in our website.