Interface and Application Programming

WEEK 14

(05/05/2021 18:28 p.m.) Wednesday afternoon. This week has been a bit monotonous. I am still at home confined and sick from COVID-19 πŸ˜ͺ🏠. Little by little, I am feeling better! I still have a lot of snot but at least the discomfort and fever have subsided. We are on the good way! πŸ’ͺπŸ’ͺ

(06/05/2021 11:25 a.m.) Despite the monotony, I have been very animated and accompanied from the distance thanks to my friends, family and especially, my crewmates and instructors from Fab Lab Leon, who have been concerned and have offered me help and provided entertainment during these days. They are the best!! πŸ˜πŸ˜˜πŸ’šπŸ’š

(05/05/2021 18:37 p.m.) After Neil's class of Interface and Application , compared to Networking week, I am looking forward to seeing what I am do, despite my possible clumsiness with electronics and programming. πŸ˜…πŸ™ƒ

(06/05/2021 10:18 a.m.). Fortunately, on this week assignment, it is not necessary to manufacture anything at the fab lab, and the work that we have to do we can do it at home. So, COVID has not altered much the continuity of my assignments.

Little by little, the end is getting closer. And I have to give a push to the final project. So these days I have to organize me and do things for the project. Despite feeling better, I still am not 100% πŸ˜…, and I am working slower than I would like. However, to regain strength you have to listen to music that increases the spirit πŸ’ͺ. So this week's song is I like it like that by Pete RodrΓ­guez .

Wow, am I feeling good, man?

Let me say this now

Organizing tasks

Like previous week, we have two types of assignments, one's group and other individual. So as usual, I organize my schedule and here the following Evaluation Criteria that we have to approach to complete the Assignment:

  • βœ“ Compare as many tool options as possible
  • βœ“ Document our work and progress

  • βœ“ Write an application that interfaces a user with an input and/or output device that we made
  • βœ“ Documented our process and what we have learned from application programming
  • βœ“ Explained problems and how you fixed them
  • βœ“ Included our design files and codes

    Extra credit

  • βœ“ Include a "hero shot" of our application runnning with our boards

Group Assignment

  • MIT APP INVENTOR

(10/05/2021 09:10 a.m.) For the Group Assignment, since I'm at home and I don't have access to the fab lab πŸ˜…, I have bought online some Bluetooh and Wifi Modules to test other types of programs and programming languages.

As I wanted to try some application developer, and I discovered MIT App Inventor had a BETA version to use on IOS. So, I downloaded the App Inventor 2 Version 3 emulator and the MIT App Inventor app on my iPad.

To begin, I go to Create Apps, and I register with an email. Once registered, you enter an initial page where all our projects will be saved. First, I click on Start New Project.

Once inside, following this tutorial (for beginners), the first thing I check is the connection between the online emulator and my iPad application. From the computer I go to Connnect > Al Companion, and a QR code will appear, which we will have to scan on my iPad.

Once scanned, to iPad, I verify the home screen app from the computer appears. At the moment, everything its okey! So, lets beginning to design the interface!. πŸ€—

In MIT App Inventor, the Design tab appears first. Here I see some windows to the right and left, with which we are adding images, animations, elements, sensors that will interact, etc ... and how to modify and adjust them for our app.

In my case, to do something funny and using the stickers that I designed for Week 03 in the Laser Cuttinberg manual, I am going to create a detector of Neil phrases, which connected with a Wemos R1 Mini module and a Relay, will turn on a red light when the answer you say is "not good", or a green light will turn on if the answer is "good". To do it, I followed this tutorial and AdriΓ‘nΒ΄s steps in his assignment for turn the lights on an house model.

So, the first thing I do is design the pages and the interface. When I have squared the images, buttons and interactions that I am going to make graphically, we go on to programm each one of the elements we have arranged.

This is done from the Blocks option. The type of programming is very similar to micro:bit, very visual and using conection blocks.

Once, we have added all the options and windows, here you see the final design of each pages that can be navigated through the application.

This is a video of how to navigate using the iPad simulator. 😊

Once I have finished designing the app, I am going to export it to iPad. From Build > I select App (provide QR code for .apk), and a QR code appears that I have to scan from the MIT inventor app on my iPad.

From iPad, I scan the code in Scan QR code, and then select Connect with Code .

After several attempts to connect my iPad with the scanned QR code, I see that the device does not react, no matter how many times I press the Connect with Code button. πŸ˜•

I try another way to export my app, from Build > I select App (save .apk to my computer). Once I download the app on my computer, I export the app to my iPad from AirDrop. But when I do this, my iPad informs me that the app is not valid for my country or region. 😰😰

Seeing this issues, I start to investigate what this problem is due to. After several searches I see the MIT app inventor application for IOS is still in BETA version, and informs users that there are still problems exporting the app to the devices. 😭😭

I hope this issue is resolved as soon as possible to test with it. πŸ˜“

  • TRY TO CONNECT ESP8266 MODULE + RELAY WITH MIT APP INVENTOR

In parallel to the MIT App Inventor application I have worked on Neil's Phrase detector circuit using a derivative similar to the Wemos R1 Mini with a built-in ESP8266, which controls a Relay.

The first thing I do is familiarize myself with the ESP8266, its documentation and the uses that I can give it. Investigating, I discover the ESP8266 is a module that allows microcontrollers to connect to a Wi-Fi network and make simple IP (Internet Protocol) connections.

On the other hand, I become familiar with the operation of a Relay. The relay is an electromagnetic device. It functions as a switch controlled by an electrical circuit in which, by means of a coil and an electromagnet, a set of one or more contacts are actuated that allow other independent electrical circuits to be opened or closed.

Thanks to this page, I begin to understand how the relays work and the meaning of each of their connections.

Once I understand, (more or less), how it would be the operation of the components that I am going to use, I draw by hand the schematic of my circuit. Also, the connections I must make to the red led turns on when the answer is "bad" and the green led lights up when the answer is "good".

After drawing the circuit, I prepare the connections using a small breadboard that I have at home.

Now comes the truth moment! Program the circuit. The first thing I do is configure the toolchains for the ESP8266 using the following library.

Once the library is installed and downloaded, I select my Wemos D1 R1 board, as well as the output port to program it.

But at the moment I load it, something stranger happens. Suddenly, my Arduino IDE informs me that the ESP8266, Software Serial and Ethernet libraries are desapear from my Arduino folder. 😳😱

It is strange, because when I go to libraries from Arduino IDE, the three folders are located, as well as inside my Arduino folder in My Documents. 🀨🀨

I try to uninstall all the folders, and install and reinstall Arduino (twice), and keep getting the same problem over and over again. What happens?? 😭😭

(12/05/2021 10:17 a.m.) During Wednesday morning, AdriΓ‘n realized the problem I had with the ESP8266 and also I explained him the added problem with the MIT App Inventor application. Finally, we both came to the conclusion that my experiment (on the one hand or the other) has been an absolute failure. πŸ™ƒπŸ™ƒπŸ˜‘πŸ˜‘

I hope to be able to test the whole circuit, programming and development of the application (again) soon, even if it is with an Android and a Windows borrowed πŸ˜‚πŸ˜‚.

Individual Assignment

PROCESSING

Processing is a programming language, before to Arduino, that is used for teaching and producing multimedia and interactive digital designs. It uses a Java-based programming language and uses variables and libraries, as in Arduino.

(29/01/2020 11:28 a.m.) At the beginning of last year, before the global confinement, in the Master of Fashion Futures that I did in the fab lab IED Madrid, I had the opportunity to meet David Cuartielles (co-founder of Arduino) who gave us a beginning masterclass on Processing and Arduino. 😍😍

With David, I started doing my first steps in programming, but when the pandemic hit, our classes were reduced and I did not use processing again. 😞

(06/06/2021 10:47 a.m.) However, for this week, AdriÑn + Nuria + Pablo have advised us to follow a tutorial-masterclass by their fab academy alumni, Marta Verde, where she explains very well the basic concepts to use processing and do incredible things! 😍😍.

One of the things I liked the most about Marta's tutorial was a phrase that she said: Everything can be, random!! So following that concept, I started doing some quick tests following this advice. Here you can see a video about the first tests I did with Processing:

  • PROCESSING + ACCELEROMETER SENSOR (ARCELI ADXL345 MODULE)

(07/05/2021 09:34 a.m.) For this assignment, we have to create an application connected to an input or output that we have developed during the previous weeks. In my case, I have chosen to use the ADXL 345 accelerometer with my Hello PiΓ±a! 3.0.

(07/05/2021 09:48 a.m.) Since I am confinament, I do not have access to Alberto's computer to program πŸ˜‚. But I have discovered that my FTDI is recognized by Arduino on the iMac computer.

So, the first thing I do is check if I can program an acclerometer sensortest code with the Hello PiΓ±a! 3.0. I think everything works, so, once programmed, I connect the accelerometer, open the monitor, and... data runs! The FTDI works perfectly on this computer! πŸŽ‰πŸŽ‰

Now in Processing, I have to know some basics concepts. First, to receive data from the accelerometer and receive it in processing, we have to add the variable processing.serial.*; , which can be found in File > Examples > Libraries > Serial > SimpleRead, or directly in Scketch > Import Library ... > Serial.

Second, we have to define two important parts of the code:

  • void setup (){}. It is where the fixed elements are defined and we can later modify and make them do certain actions.
  • void draw () {}. It is the actions that are repeated in loop on the fixed elements defined.

I define the precode that will collect the data obtained by the accelerometer for the application does things

In void setup () {}, it is very important to define what is the name of the Serial.Port. In my case, I go to Arduino, and type the name that appears as my FTDI port name. And then, other important things is that the baud rate , both in the Arduino and Processing code, match then and are 115200.

I want to check my accelerometer works correctly when I connect it to Processing. So, I'm going to use a Processing code from AdriΓ‘n + Marta Verde did together for Adrianino + ADXL 345 accelerometer 😍😍. When I changed the Port name and pushed "Run" it worked!πŸŽ‰πŸŽ‰ Here you can watch a short video showing how the accelerometer works with Processing:

After this, for the following applications that I am going to do, I am going to define two different codes for data reading: one that reads the data variations in XYZ axes and the other that reads the global data variations of the sensor.

  • PROCESSING + ACCELEROMETER SENSOR: XYZ VALUES VARIATION

Using AdriΓ‘n + Marta VerdeΒ΄s code, developed for Adrianino + the ADXL 345 accelerometer, I am going to make a series of modifications to make a data reading in XYZ with circles increase or decrease depending on the movement of the accelerometer.

To do this, I am going to describe some basic commands to explain what I am doing:

  • float ;. It is a variable declaration of something.
  • color []; It is a color variable.
  • myPort = new Serial(this, "/dev/cu.usbserial-AB0L7ISD", 115200);. Definition of FTDI port.
  • size();. It is the size of the window in pixels.
  • background(); It is the color in RGB, of the window background.
  • fill();. It is the fill color of something.
  • textSize();. It is the size of the text font.
  • noStroke();. Removes the outlines of objects.
  • map();. It is the variables mapping and assigns a number from one range to another.
  • ellipse();. Definition of an ellipse or circle.

Below this is the final code:


//Fab Academy 2021
//Lorena Delgado
//ADXL 345 3 axis Accelerometer Module
//Hello Pinya 3.0 (ATtiny1614)

import processing.serial.*;

Serial myPort;  // The serial port
int cycles = 0;

float in_x;
float in_y; 
float in_z;
float x, y, z;
color[] col = new color[4];

void setup() {
  size(1000, 450); //size of the window
  background(0);   // set initial background black
  myPort = new Serial(this, "/dev/cu.usbserial-AB0L7ISD", 115200); //serial port
  myPort.bufferUntil('\n');   // don't generate a serialEvent() unless you get a newline character:
  initGraph();  //we start variable for graphs
}

void draw() {
  background(0);   // set initial background black
  
   //X 
 
    //text
      fill(188,188,188);
      textSize(15);
      text("X", 200, 320);
        fill(255);
        textSize(20);
        text(in_x, 160, 350);

    //Xellipse
      noStroke();
      fill(col[1]);
      x = map(in_x, 0, 200, 0, 600);
      ellipse(200, 200, 150-x, 150-x);
      
   //Y 

      //text
        fill(188,188,188);
        textSize(15);
        text("Y", 500, 320);
         fill(255);
         textSize(20);
         text(in_y, 460, 350);

      //Yellipse 
        noStroke();
        fill(col[2]);
        y = map(in_y, 0, 200, 0, 600);
        ellipse(500, 200, 150-y, 150-y);
      
    //Z 
    
      //text
        fill(188,188,188);
        textSize(15);
        text("Z", 800, 320);
         fill(255);
         textSize(20);
         text(in_z, 760, 350);

      //Zellipse 
        noStroke();
        fill(col[3]);
        z = map(in_z, 0, 200, 0, 600);
        ellipse(800, 200, 150-z, 150-z);

}

void initGraph() { //the colors of the graph
  col[0] = color(255, 127, 31);
  col[1] = color(230,93,39);
  col[2] = color(137, 188, 197);
  col[3] = color(49, 135, 156);
}

void serialEvent(Serial myPort) {
  String inString = myPort.readStringUntil('\n');
  if (inString != null) {
    inString = trim(inString);
    float[] values = float(split(inString, ","));
    if (values.length >=3) {
      in_x = values[0]; 
      in_y = values[1];
      in_z = values[2];
    }
  }
}
                        

Here you can watch a video, where depending on the accelerometer movement, the circles size change depending on the XYZ variables:

Now, with this code I am going to make a variation so that depending on the accelerometer values, the spheres change their size and color randomly. To do it, I use these commands:

  • pushMatrix(); and popMatrix();. They are "keys" where a series of commands are applied ans only affect the object or group of objects that are between each other, without affecting the rest of the code.
  • random();. Generates random numbers. Each time the random() function is called, it returns an unexpected value within the specified range.

When I apply these new functions, the circles start to change in size and color interchangeably.

Here you can watch a video where moving the accelerometer very random things happen. It looks like a disco πŸ’ƒπŸ€£πŸ€£.

  • PROCESSING + ACCELEROMETER SENSOR: RANDOM SHAPES VARIATION
    _ DRAWING RANDOM RECTANGLES SIZES

For this new application, I want to see how I can drawing the monitor space with randomly square geometric figures, from the global accelerometer data.

To do this, I define the global data variables of the accelerometer (int_acc), the size of the squares (tamanho) and the rotation variable (rotate). For this occasion, I am going to use three new commands:

  • rotate();. Rotate objects in radians.
  • smooth();. Draws all geometry with smooth (anti-aliased) edges.
  • rect();. Definition of a rectangle or square.

This is the final code:


//Fab Academy 2021
//Lorena Delgado
//ADXL 345 3 axis Accelerometer Module
//Hello Pinya 3.0 (ATtiny1614)

import processing.serial.*;

Serial myPort;  // The serial port

float int_acc;     
float tamanho;
float rotate1;

void setup() {
  size(1000, 1000); //size of the window
  background(0); // set initial background black
  smooth(2); //anti-aliased edges
  myPort = new Serial(this,"/dev/cu.usbserial-AB0L7ISD", 115200); //serial port
  myPort.bufferUntil('\n'); // don't generate a serialEvent() unless you get a newline character:

}

void draw() { 
  rotate(rotate1);
  tamanho = map(int_acc, 200, 1000, 200, 1000); //map size
  noStroke();
  fill(random(255), 0, random(255), 150); //color fill of random rectangles
  rect(random(width), random(height), tamanho, tamanho); //place and sizes of the rectangles
  
  rotate1 = rotate1 + PI/4;
  
}

void serialEvent(Serial myPort) {
  String inString = myPort.readStringUntil('\n');
  if (inString != null) {
    inString = trim(inString);
    float[] values = float(split(inString, ","));
    if (values.length >=1) {
      int_acc = values[0];
    }
  }
}
                                

Here you can see a video where depending on the movements of the accelerometer, squares or rhombuses are drawn randomly:

    _ CIRCLE VARIATION

In this new exercise, on a fixed basic shape, I am going to vary the size and colors of the object depending on the global values of the accelerometer in motion.

This code is a remix between several previously commands used, except for a new command:

  • frameRate();. Specifies the number of frames to be displayed every second.

This is the final code:


//Fab Academy 2021
//Lorena Delgado
//ADXL 345 3 axis Accelerometer Module
//Hello Pinya 3.0 (ATtiny1614)

import processing.serial.*;

Serial myPort;  // The serial port

float int_acc;     
float tamanho;


void setup() {
  size(1000, 1000); //size of the window
  background(0); // set initial background black
 frameRate(5);
  myPort = new Serial(this,"/dev/cu.usbserial-AB0L7ISD", 115200); //serial port
  myPort.bufferUntil('\n'); // don't generate a serialEvent() unless you get a newline character:

}

void draw() { 
  tamanho = map(int_acc, 100, 500, 100, 500); //map size
  noStroke();
  fill(random(255), 0, random(255), 150); //color fill of random circles
  ellipse(width/2, height/2, 35*(tamanho), 35*(tamanho)); //place and sizes of the circle
  
   
}

void serialEvent(Serial myPort) {
  String inString = myPort.readStringUntil('\n');
  if (inString != null) {
    inString = trim(inString);
    float[] values = float(split(inString, ","));
    if (values.length >=1) {
      int_acc = values[0];
    }
  }
}
                                

Here you can watch a short video where, depending on the movement, the frames of the circles grow or decrease in different colors:

  • PROCESSING + ACCELEROMETER SENSOR: IMAGE ROTATION

Finally, I want to do one last test to rotate images based on the global movement of the accelerometer. So, it occurred to me to make an animation with a pinwheel.

From an image that I downloaded from google and modified in Illustrator, I export it as a .png, to maintain the transparency of the image.

In addition, I am going to print the global values of the accelerometer at the animation, to make a simulation of what the acceleration of the pinwheel is when we move the sensor.

For this new code I use these new commands:

  • photo = loadImage("thephoto.png");. Upload an image.
  • imageMode();. Image location in drawing space

 //Fab Academy 2021
//Lorena Delgado
//ADXL 345 3 axis Accelerometer Module
//Hello Pinya 3.0 (ATtiny1614)

import processing.serial.*;

Serial myPort;  // The serial port
PImage photo;

 float rotate1;
 float int_acc;
 float speed;

 void setup(){
   size(1000, 1000);
   photo = loadImage("molinillo.png");
   background(0);
   
    myPort = new Serial(this,"/dev/cu.usbserial-AB0L7ISD", 115200); //serial port
    myPort.bufferUntil('\n'); // don't generate a serialEvent() unless you get a newline character:
   
   
   
}  
 void draw(){
   
 background(0);
 
       //text
        fill(188,188,188);
        textSize(15);
        text("Speed", 480, 100);
         fill(255);
         textSize(20);
         text(int_acc, 465, 135);
    
  pushMatrix();
    noStroke();
    fill(189,147,107); //color fill of random rectangles
    rect(500, 500, 20, 500); //place and sizes of the rectangles
  popMatrix();
   
  imageMode(CENTER);
  
  speed = map(int_acc, 0, width, 0, 1);
  println(int_acc);
  
  pushMatrix();
    translate(width/2, height/2);
    rotate(rotate1);
    image(photo, 0, 0, photo.width/1.5, photo.height/1.5);
  popMatrix();

  rotate1 = rotate1 + int_acc; 
  


}

void serialEvent(Serial myPort) {
  String inString = myPort.readStringUntil('\n');
  if (inString != null) {
    inString = trim(inString);
    float[] values = float(split(inString, ","));
    if (values.length >=1) {
      int_acc = values[0];
    }
  }
}            
                         

This is a short video where depending on the movement to one side or the other of the accelerometer, the pinwheel rotates clockwise or vice versa:

FILES

MIT APP INVENTOR

  • App "Neil Says".APK
  • TRY TO CONNECT ESP8266 MODULE + RELAY WITH MIT APP INVENTOR

  • ESP8266 + Relay + App "Neil Says".ino
  • PROCESSING + ACCELEROMETER SENSOR: XYZ VALUES VARIATION

  • Circle XYZ Variation Values.pde
  • Circle XYZ Multicolor Variation Values.pde
  • PROCESSING + ACCELEROMETER SENSOR: RANDOM SHAPES VARIATION

  • Drawing Rectangles Sizes.pde
  • Circle Variation Size.pde
  • PROCESSING + ACCELEROMETER SENSOR: IMAGE ROTATION

  • Pinwheel Rotation.pde
  • REFERENCES

  • Using ESP8266 and MIT App Inventor to control a Relay
  • Adrianino. Accelerometer 3 axis. Processing Code (AdriΓ‘n Torres + Marta Verde)
  • Marta Verde. Interface and Application Programming Assignment