interface and application programming

After a few slow weeks this is the week i'm picking up pace again. Our partnerlab from Sarbonne visited us in the Waag and together we watched the global lecture. The next day we all worked on our individual and group assignments. After the global lecture I was super confused since I heard so many familiar concepts and languages but I couldn't really bring them together.
Bringing the physical world and digital world together is the goal for this week. Since I consider myself a novice I followed the advice Henk (local instructor) gave to the group: use processing!

Input devices

Hardware
USB 2.0 hub
HelloHenk board (attiny44)
FabISP

Software
Arduino
Processing

Group assignment

For the group assignment we looked at different tools - Joey worked on a few different examples to showcase the different options and Henk showed us his application from last year.

The different options we looked at were:

Python
Node.js/Javascript
Processing

Everything we looked at has the same basic capabillities. The most noticable difference is the enviroment. Node.js/javascript work really well for building (local) websites. While Python and Processing can be used to make and design your own window. Processing was by far the easiest to quickly get results from. The language is similar to Arduino and it has a nice familiar interface. I'd love to learn more on how to get data from and to an Arduino using Javascript.

Individual assignment

I made a interface that works with my Hello Henk board that I made in week 7. The board runs on a ATTINY44 chip and has a button, a light sensor and LED. Since I already programmed it in week 9 using Arduino language I decided to stick with it the same language for this week. For the interfacing part I used Processing since it's fairly similar. For the interface my idea was to quickly make it work and than keep adding a bits and pieces to make it more complex.

There are a few basic concepts and steps that I learned this week. First one being - the microcontroller should output data, it now feels a bit silly to write this down. I never really thought about it or questioned it. Second step is that the microcontroller that outputs data can be read - not just from the Arduino IDE console but also from other frameworks like Processing. This works in both directions, you can also send/write data to your microcontroller. Last but not least - all these proccesses are already super familiar but hidden in plain sight. Think about your smart phone or smart sound system.

Exp 1: receiving data in Processing

I started by programming my ATTINY in a way that outputs the sensor data to the console. I recycled the code from week 9 and tweaked it specifically for this purpose.
The first bit starts with the setup for serial communication, constants and variables. In the setup the LED is switched on during the callibration period for the sensor. Once this is done the loop starts and the sensor value is being printed as a (remapped) value in the 0-255 range.

//Serial communication
#include <SoftwareSerial.h>         // import serial library 
#define rxPin 0                     // link  rx and tx pins to library
#define txPin 1
SoftwareSerial serial(rxPin, txPin);

const int sensorPin = A2;    // pin that the sensor is attached to
const int ledPin = A7;        // pin that the LED is attached to

// variables
int sensorValue = 0;         // the sensor value
int sensorMin = 1023;        // minimum sensor value
int sensorMax = 0;           // maximum sensor value


void setup() {
  // turn on LED to signal the start of the calibration period:
  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);

  // calibrate during the first five seconds
  while (millis() < 5000) {
    sensorValue = analogRead(sensorPin);

    // record the maximum sensor value
    if (sensorValue > sensorMax) {
      sensorMax = sensorValue;
    }

    // record the minimum sensor value
    if (sensorValue < sensorMin) {
      sensorMin = sensorValue;
    }
  }

  // signal the end of the calibration period
  digitalWrite(ledPin , LOW);

  // Serial communication
  pinMode(rxPin, INPUT);
  pinMode(txPin, OUTPUT); 
  serial.begin(9600);    
}

void loop() {
  // read the sensor:
  sensorValue = analogRead(sensorPin);

  // apply the calibration to the sensor reading
  sensorValue = map(sensorValue, sensorMin, sensorMax, 0, 255);

  // in case the sensor value is outside the range seen during calibration
  sensorValue = constrain(sensorValue, 0, 255);

  // fade the LED using the calibrated value:
  analogWrite(ledPin, sensorValue);

  // serial print

  serial.println(sensorValue);
  delay(50);
}

In Arduino IDE I checked if I was receiving the data that I was expecting in the Serial Plotter. Using a flashlight I can bring the value all the way up to 255 (dark) and down to 0 (light).

Data from sensor board output in serial plotter range 0 - 255

Now it's time to see if this data can be received by processing. In processing I started a new sketch to read the data from the serial port. In the setup there are only two lines, the first assigns the port to a variable. Since I only have 2 ports on my computer I just started from 0 to see if it worked by uploading the sketch. The second line gives it a name and speed. In the setup the value is printed in the Processing console.

import processing.serial.*; // import library

Serial myPort;  
String val; 

void setup()
{
  String portName = Serial.list()[1];  
  myPort = new Serial(this, portName, 9600);
}

void draw()
{
  if ( myPort.available() > 0) 
  { 
  val = myPort.readStringUntil('\n');         // read it and store it in val
  } 
println(val); //print it out on a new line in the Processing console.
}

Serial monitor in processing for first test.

Exp 2: receiving data on Arduino

In the second Exp I tried to switch a LED on and off from Processing. In Processing I created a 200x200 px canvas that can be clicked. If it's clicked '1' is written to the serial port. If it's not pressed '0' is send. When the sketch is running there is a pop up canvas that changes color when it's pressed.

 import processing.serial.*;

 Serial myPort;  // Create object from Serial class
 String val;     // Data received from the serial port


 void setup()
 {
   //size(500,500); //make our canvas 500 x 500 pixels big
   fullScreen(); // make canvas full screen 
   String portName = Serial.list()[1]; 
   myPort = new Serial(this, portName, 9600);
 }

 void draw() {
   if (mousePressed == true) 
   {                           //if we clicked in the window
    myPort.write('1');         //send a 1   
    background(255, 204, 0);
   } else 
   {                           //otherwise
   myPort.write('0');          //send a 0
   background(51);
   }

   println(mousePressed);
 }

Next it's time to make Arduino listen to what processing is sending via serial communication. The code consists of a few parts - importing the library and declaring the ports for serial communication, setting up LED and creating a variable for the 'received message'(val). When a 1 is received the LED is switched on, when anything but 1 is received it switches of again.

 //Serial communication

#include <SoftwareSerial.h>         // import serial library 
#define rxPin 0                     // link  rx and tx pins to library
#define txPin 1
SoftwareSerial serial(rxPin, txPin);


 char val; // Data received from the serial port
 int ledPin = A7; // Set the pin to digital I/O 13

void setup() {
  // put your setup code here, to run once:
   pinMode(ledPin, OUTPUT); // Set pin as OUTPUT
  //      digitalWrite(ledPin, LOW); // turn the LED on
serial.begin(9600);
}

 void loop() {
   if (serial.available()) 
   { // If data is available to read,
     val = serial.read(); // read it and store it in val
   }
   if (val == '1') 
   { // If 1 was received
     digitalWrite(ledPin, HIGH); // turn the LED on
   } else {
     digitalWrite(ledPin, LOW); // otherwise turn it off
   }

   delay(10); // Wait 10 milliseconds for next reading
}

Exp 3: sending and receiving

For the last experiment I combined the previous learnings - I made a line that moves depending on the value we get from the light sensor and if you click the Processing canvas the LED on the board switches on. This is what it looks like. The GUI is generated from Processing and is a 250px x 250px window that is fully clickable. It can also be changed into a fullscreen interface by commenting and uncommenting these lines. The clicking is detected by the mousePressed() function - this checks for clicks within the window.

size (250,250);
// fullScreen();

The code looks a bit more difficult but it combines the previous two experiements. The line is created and changes position depending on the light sensor value (0-255). To make it move a bit more fluid Joey helped me with some SmoothingMagic. Y is used as the value to change the position. Y is calculated with value and a float named 'smoothing' - so it changes relative steps instead of absolute steps.

  y = int (y * smoothing + val * ( 1-smoothing));

Arduino code:

//Serial communication
#include <SoftwareSerial.h>         // import serial library 
#define rxPin 0                     // link  rx and tx pins to library
#define txPin 1
SoftwareSerial serial(rxPin, txPin);


char val; 
int ledPin = A7; 
int sensorPin = A2;

// variables:
int sensorValue = 0;         // the sensor value
int sensorMin = 1023;        // minimum sensor value
int sensorMax = 0;           // maximum sensor value
byte sendValue;

void setup() 
{


  serial.begin(9600);

  pinMode(ledPin, OUTPUT); 

   //sensor callibration 
    while (millis() < 5000) {
    sensorValue = analogRead(sensorPin);
    digitalWrite(ledPin, HIGH);

    // record the maximum sensor value
    if (sensorValue > sensorMax) {
      sensorMax = sensorValue;
    }

    // record the minimum sensor value
    if (sensorValue < sensorMin) {
      sensorMin = sensorValue;
    }
  }
   digitalWrite(ledPin, LOW);
}
void loop()
{

  if (serial.available() > 0) { // If data is available to read,
    val = serial.read(); // read it and store it in val

  if (val == '1') 
   { // If 1 was received
     digitalWrite(ledPin, HIGH); // turn the LED on
   } else {
     digitalWrite(ledPin, LOW); // otherwise turn it off
   }

  } 

     // read the sensor:
  sensorValue = analogRead(sensorPin);

  // apply the calibration to the sensor reading
  sensorValue = map(sensorValue, sensorMin, sensorMax, 0, 255);

  // in case the sensor value is outside the range seen during calibration
//  sensorValue = constrain(sensorValue, 0, 255);
sendValue = constrain(sensorValue, 0, 255);

  // serial print

// serial.print(sensorValue);
serial.write(sendValue);
  delay(50);
    }

And processing code:

  import processing.serial.*; //import the Serial library
 Serial myPort;  //the Serial port object
// since we're doing serial handshaking, 
// we need to check if we've heard from the microcontroller
boolean firstContact = false;

int val;      // Data received from the serial port
int y = 250;
float smoothing = 0.92; //smooth transistion line

void setup() 
{

  size (250,250);
  // fullScreen();
  String portName = Serial.list()[1];
  myPort = new Serial(this, portName, 9600);
  stroke(255); 
  frameRate(30);
}

void draw()
{
  if ( myPort.available() > 0) {  // If data is available,
    val = myPort.read();         // read it and store it in val
    println(y);
  }
  background(0);             // Set background to white

  y = int (y * smoothing + val * ( 1-smoothing)); 
  if (y < 0) { 
    y = height; 
  } 
  line(0, y, width, y);  



   if (mousePressed == true) 
   {                           //if we clicked in the window
    myPort.write('1');         //send a 1   
    background(255, 204, 0);
   } else 
   {                           //otherwise
   myPort.write('0');          //send a 0
   background(51);
   }

}

Isssues

The first issue I had was not knowing what port to use. I solved this quick and dirty by trial and error. The better solution would have been to list all ports using the following code by Tom Igoe.

// Example by Tom Igoe import processing.serial.*; // The serial port Serial myPort; // List all the available serial ports printArray(Serial.list());

LED failing - a few times the LED wasn't behaving in the way I expected. I found a few different causes for this. The most common was that the microcontroller got disconnected while the Processing sketch was running. Solution was to close and reopen the window. If this doesn't work it's most likely an error in the code. Go back to the previous code where it was still working by commenting all new lines. See if it works and uncomment line by line to see where the issue is.
Going full screen. The line get's a value between 0 and 255 from the microcontroller and this is directly connected to the Processing sketch. As long as the height of the window is also 255px this works really well. For fullscreen the height of the window changes to a different height depending on the screen of the computer. Without changing anything else the line will stay in this range and not use the whole screen. To fix this fix the mapping of the arduino to the height of the screen. A even better solution is to use percentages of the height of the screen so it works on any screen. Unfortunatly I didn't have time to get this working properly.

Files

All original code in documentation above.