16. Interface and application programming

This week has been pretty hectic so far. With school finals coming up as well as all of our group work for the machine design and automation weeks right before and after this week, it’s been quite crammed. I apologize in advance for any notable errors in my work.

This week, my individual assignment was to create an interface application that can communicate directly from my computer to a microcontroller input or output board of my creation. Our group assignment was to compare as many tool options for application programming as possible.

Group Assignment

For this week’s group project, we decided to divide up the different tools between the four of us. Once we all individually researched our tools, we shared our research and took notes.

MIT App Inventor (Kai)

Mit App Inventor

Chili Peppr (Kai)

Chili Peppr is a “Javascript software workspaces that talk to your hardware”. We plan to use it in conjunction with the GRBL workspace for our gCode.

Processing (Kai)

Processing is a visual, user friendly interface tool. Arduino IDE is built on Processing. It works with Java or Python; there are additional adaptations of the architecture linked below.

Different Adaptations of the Program: - Processing Javascript - Processing Python - Processing Android - Processing Pi

Processing Python (Maxine)

Processing Python is an interaction of Processing.

Firefly (Maxine)

Simulink, built around simulation and model based design. It is an evolution and add on of Matlab

Wiring (Will)

More about Wiring here

-open source programming framework for microcontrollers -used to create all kinds of creative coding, interactive objects, spaces or physical experiences -includes support for many different hardware architectures -used for learning, prototyping, and finished professional work production

The site is well documented with many helpful resources that can be found here

-Wiring builds off of processing which Kai documented above.

Arduino Firmata (Katie)

Firmata is an Arduino library that utilizes the Firmata Protocol to software communication.

*These photos were found from this source

Here is some example code listed on the Firmata Arduino page.

This code starts the Firmata library.

begin(); //start the library
begin(long); //start the library and override the default baud rate
begin(Stream &s); // start the library using a [Stream](http://www.arduino.cc/en/Reference/Stream) other than Serial (eg Serial1 or EthernetClient)
printVersion(); //send the protocol version to the host computer
blinkVersion(): //blink the protocol version on the build in LED (typically pin 13)
printFirmwareVersion(); //send the firmware name and version to the host computer
setFirmwareVersion(byte major, byte minor); //set the firmware name and version, using the sketch's filename, minus the '.ino'
setFirmwareNameAndVersion(const char *name, byte major, byte minor); //set both the name and version of the firmware

This code sends messages.

sendAnalog(byte pin, int value); //send an analog message
sendDigitalPort(byte portNumber, int portData); //send an 8-bit port in a single digital message
sendString(const char* string); //send a string to the host computer
sendString(byte command, byte bytec, byte *bytev); //send a string to the host computer using a custom command type
sendSysex(byte command, byte bytec, byte* bytev); //send a command with an arbitrary array of bytes
write(byte c); //write a byte to the Stream

This code receives the messages.

available(); //check to see if there are any incoming messages in the buffer
processInput(); //process incoming messages from the buffer, sending the data to any registered callback functions
attach(byte command, callbackFunction myFunction); //attach a function to an incoming message type
detach(byte command); //detach a function from an incoming message type

This examples is for sending and receiving messages.

#include <Firmata.h>

byte analogPin;

void analogWriteCallback(byte pin, int value)
{
  pinMode(pin, OUTPUT);
  analogWrite(pin, value);
}

void setup()
{
  Firmata.setFirmwareVersion(FIRMATA_MAJOR_VERSION, FIRMATA_MINOR_VERSION);
  Firmata.attach(ANALOG_MESSAGE, analogWriteCallback);
  Firmata.begin();
}

void loop()
{
  while (Firmata.available()) {
    Firmata.processInput();
  }
  for (analogPin = 0; analogPin < TOTAL_ANALOG_PINS; analogPin++) {
    Firmata.sendAnalog(analogPin, analogRead(analogPin));
  }
}

Firmata Protocol

The Firmata Protocol is a protocol that aids in software communication with microcontrollers to computers. The Firmata Protocol can be used on any microcontroller and any computer!

Firmata Protocol is “based on the midi message format in that commands bytes are 8 bits and data bytes are 7 bits” (soundanalogous). Ex. “MIDI Channel Pressure (Command: 0xD0) message is 2 bytes long, in Firmata the Command 0xD0 is used to enable reporting for a digital port (collection of 8 pins)” (soundanalogous). “In Firmata, the number of bytes in a message must conform with the corresponding midi message” (soundanalogous).

MIDI Messages Format

This link provides a table for MIDI Messages. It is in binary order!

Python / mbed SDK / RPC (Katie)

To understand interfacing with Python, I looked at this [tutorial] (https://os.mbed.com/cookbook/Interfacing-with-Python).

The tutorial referenced this developmental kit. It is called the mbed SDK. It is a C/C++ platform, but I read the tutorial for interfacing with Python.

Here are the different platforms that can be used to interface with Python.

Windows:

Mac:

Linux: - Python is probably already installed

RPC

mbed libraries (this is the company that the tutorial came from) support RPC so you can interface using that. I referred to this tutorialto learn about RPC interfacing.

HTTP

This is an HTTP example of Hello World code which references the Python RPC library.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
python
>>> from mbedrpc import *
>>> mbed = HTTPRPC("192.168.0.4")
>>> x = DigitalOut(mbed,"LED1")#These objects should already exist on mbed
>>> z = DigitalOut(mbed,"LED2") 
>>> ain = AnalogIn(mbed, "LED3")
>>> x.write(1)
>>> z.write(0.5)
>>> ain.read()
0.786757474
>>>

I decided to focus on RPC with serial.

Serial

Here is how to connect to the COM port.

1
2
3
4
5
6
python
>>> import serial
>>> serdev = 15
>>> s = serial.Serial(serdev)
>>> s.write("hello")
>>> s.close(

This is a Serial example of RPC that receives the commands from the serial port and then pass them.

#include "mbed.h"
#include "mbed_rpc.h"

/**
 *  This example program has been updated to use the RPC implementation in the new mbed libraries.
 *  This example demonstrates using RPC over serial
 */

//Use the RPC enabled wrapped class  - see RpcClasses.h for more info
RpcDigitalOut myled(LED4,"myled");

Serial pc(USBTX, USBRX);
int main() {
    //The mbed RPC classes are now wrapped to create an RPC enabled version - see RpcClasses.h so don't add to base class

    // receive commands, and send back the responses
    char buf[256], outbuf[256];
    while(1) {
        pc.gets(buf, 256);
        //Call the static call method on the RPC class
        RPC::call(buf, outbuf); 
        pc.printf("%s\n", outbuf);
    }
}

Individual Assignment

For my individual assignment this week, I wanted to create an interface for my computer that would let me specifically turn on Leds in a charlieplexed array. In the past, most of my original charlieplexing designs have been their own individual breakout boards that operated from a commercial arduino board. This week, I wanted to make my own board that would act similar to an arduino, but that also had a charlieplexed array directly on it and ready to go.

Board Creation

Before I could jump into the interface programming, I had to make the board that I would be communicate with. I had seen many of my classmates using the Satshakit design for their own projects, so I chose to try that for myself this week.

I started by trying to understand what a Satchakit really was in the first place. From what I’ve heard around the lab, it seems to be an easily fabbable microcontroller board that works and acts just like an Arduino Uno. Since I used an arduino to control my charlieplexed array before, I figured this would be the best step for me to try next.

After some searching around I eventually found this Github page that has a full set of instructions for making your own satshakit.

At first, I tried just using this image as a reference to create my own schematic and board layout files with.

It was all going pretty well for a while, here’s the schematic I made for the board before adding in the array.

I got about this far into my board design before I saw an issue.

For some reason, my design rules were showing that the pads for my Atmega328p chip were all to close together for the milling machine to cut it out properly.

The red dashed areas in between all of the pads are showing that those areas are too small for the milling machine to be able to cut out. I was confused by this because I thought I was using the same chip file that everyone else had used, and all of their boards had worked without issue. This wasn’t good because it meant that I wouldn’t be able to mill out this board on our milling machines, even with the tiny 1/64th inch bit.

I went back onto the github page and looked around some more until I eventually just found the raw code that makes up the schematic and board files for the satshakit. I put these into eagle and pulled up the correct files, meaning that now most of my work had already been done for me.

So, I added this into the schematic for my charlieplexed array.

I also condensed the wiring for the satshakit while fitting my aray onto the board nicely. I made the board so that the charlieplexing leads would be directly connected to their specific pins on the Atmega chip.

Board production

I cut out the board on our milling machine again, and since nothing bad happened during the cut, I don’t have a ton to document for this section. (I guess no news is good news in this case).

Soldering

Soldering this board was a bit more of a challenge than my past ones have been, especially because of how tiny all of the pins on the Atmega chip are.

While I was soldering, I noticed one spot on my board that I must have missed during the design process, because there was a trace that wasn’t able to be fully cut out.

I was able to use an exacto knife and cut the trace by hand, though, and that ended up working just fine.

I got the rest of the components soldered on, and the board came out looking great!

I washed it off with acetone to clean it up a bit.

Then my board was finally done.

Programing

To program the board, I followed this tutorial. It actually didn’t end up being nearly as difficult as I expected.

One thing to note before I start talking about how I programmed the board is that I had forgotten my FTDI header at the lab when i brought all of this home with me. Lucky for me, though, I did have an arduino uno with me. I ended up using the arduino in place of the FTDI for power and ground, and it all worked out in the end.

So, I used this diagram to see how to wire up my board the first time.

In actuality, though, the wiring was a bit more confusing than that. I needed to provide power and ground to both the satshakit and the usbtinyisp programmer at the same time, because the satshakit didnt have an isp header that would be able to give both power and ground. To do this, I used an actual arduino uno 3 that had all of the necessary connection pins.

On the usb tiny pins, I had power and ground coming in from the arduino uno, and I had MOSI, MISO, SCK, and RST going out to my satshakit.

Then, lastly, I had power and ground coming into the satshakit from the arduino and I had MOSI, MISO, SCK, and RST coming from the usb tiny.

With this triangle of wires and boards all set up, I plugged in the usb tiny and burned on the arduino bootloader using the arduino IDE.

Then I tried uploading an old charlieplexing sketch that I used during my outputs week. To my great excitement, it worked perfectly on the first try.

Everything went perfectly. I was really happy to see that my board was working exactly as intended, so I tried another one of my favorite charlieplexing patterns next.

Making the Interface

I’m expecting things to get harder here, especially since I’ve never programed an interface before in my life. All I really know is that I want to use processing to do this because it had a lot of recommendation from people around the lab.

I started with this video here. It is the first in a series that will hopefully help me understand how all of this works.

Here’s some of the things I took away from the first video.

-Computer displays are based off of a coordinate system, with the origin being the top left of the screen.

-Need to start seeing things on the visual system by their coordinate position, not the physical distance. For example, rather than saying that a box is two inches from the left of the screen, you might say that it at coordinate (120, 240).

-to draw a line across the screen, you might say to make a line from (20,40) to (250,500).

Then, I found this tutorial that helped me get my first thing made in processing. It’s pretty simple compared to what I want to do in the end, but for now, I’ll take whatever I can get.

I followed their instructions and basically just copied in this code that lets me draw stuff out on the screen. This code creates a window in which I can move the mouse around. Wherever I move the mouse to, it creates a circle, and when I click down on my mouse, it makes the circle black, but when I don’t click the mouse, the circles are white.

I thought the window was a little small, though, so I made my first edits in processing by increasing the size of the frame from 480 by 120, to 2000 by 1000.

Now, that’s better. After that, I started messing around with the framework of the code above and just tried to see what all I could do with it. Here’s a few of the fun things I found.

Then I started messing around with some of the other example codes. For example, I did one that changes the camera view based off of where the mouse is on the screen by using the mouse position as the y input value for the camera coordinates.

After some messing around I understood that fairly well, but I still had no idea where to start for creating an interface that communicates with one of my boards. Even though I was effectively doing stuff in processing now, the gap between what I was doing and what I wanted to do seemed incredibly immense.

I jumped over to Mary Fabian’s page for some guidance on where to go next, and after reading through it for a bit I’m realizing that my next step now should be to download something called Firmata. In all honesty, I don’t really know what firmata is or even what it does, but it looks promising for what I’m trying to do.

So we’re getting somewhere now. I downloaded firmata here Then I added it into my libraries folder in the processing files, and was able to open up the example called “Arduino_Output”. I figured this would be a good place to start because I was using my satshakit board as an output device in this case.

The example had some text written beforehand that explained how to start using the example effectively. Following the instructions at the top, I opened up the arduino software, pulled up the example code in arduino called “StandardFirmata”, and uploaded that to my “Arduino” (really it’s just my satshakit, but the computer can’t tell a difference).

Then, I followed the rest of the instructions in the code and changed one value to match the serial port that my board is plugged into.

From here, I can use the interface to set the general IO pins on the board to either “HIGH” or “LOW”. The video below shows me using this to turn on the pin that controls my onboard LED, as well as the pins that control the charlieplexed array. Without the complex tri-state logic being used, the charlieplexed LEDs all turn on at random when I turn on some of the inputs and outputs, but I’ll be able to fix this down the road.

Now, I think I’m going to try to modify the example code to work for my board, I’ll add some more after giving it a shot.

I’ve been staring at the same example code for a while now trying to work with it and understand what it all really does. Some of the things I’ve gathered so far are detailed below.

This is a for loop. In the code, it is taking all of the GIO pins on the chip and initializing them to be an output. 

  for (int i = 0; i <= 13; i++)
    arduino.pinMode(i, Arduino.OUTPUT);

I’m pretty sure that this section here is just used for the interface side of things, not for the actual pinmodes of the arduino pins. I think that it is what the code uses to know when to change the buttons to on or off, (filled or no infill), in the GUI.

void draw() {
  background(off);
  stroke(on);

  for (int i = 0; i <= 13; i++) {
    if (values[i] == Arduino.HIGH)
      fill(on);
    else
      fill(off);

    rect(420 - i * 30, 30, 20, 20);
  }
}

After coming back to this the next day, I think I’m going to try to make just one of the charlieplexed LEDs turn on individually. This will require me to set all four of the charlieplexed pins to either output or input, and it will require me to initialize the two output pins as either HIGH or LOW.

I’m starting to think that I might have been a little ambitious at first with all of my charlieplexing plans. It’s hard enough as is to get just a single LED going, much less all twelve in specific orders. Rather, I think I’ll for something a little more simple to try.

After seeing what people have been doing in the past, I wanted to try changing some sort of object in the user interface by adjusting a potentiometer connected to my board. To do this, I used the framework of the inputs example from firmata and adjusted it to specifically work for what I’m doing.

Here’s what my processing code looked like

import processing.serial.*;

import cc.arduino.*;

Arduino arduino;

color off = color(150, 0, 20);
color on = color(20, 140, 220);

void setup() {
  size(970, 400);

  println(Arduino.list());

  arduino = new Arduino(this, "COM13", 57600);
}

void draw() {
  background(off);
  stroke(on);

  fill(10, 70, 110);
  for (int i = 0; i <= 5; i++) {
    ellipse(300 + i * 90, 145, arduino.analogRead(i) / 8, arduino.analogRead(i) / 8);
  }
}

The video below shows me working with my GUI. It uses the analogue input to set the size of the circles so that the circles all change size as you adjust the potentiometer.

Next, I wanted to at least get this to look a little better before I moved back to struggling with the charlieplexing. To do this, I centered one single circle and then added others inside of it to make all of the circles concentric.

Here’s the code that I used to do this.

import processing.serial.*;

import cc.arduino.*;

Arduino arduino;

color off = color(200, 120, 255);
color on = color(20, 140, 220);

void setup() {
  size(2052, 1368);

  println(Arduino.list());

  arduino = new Arduino(this, "COM13", 57600);
}

void draw() {
  background(off);
  stroke(on);

  for (int i = 0; i <= 0; i++) {
    ellipse(1026, 684, arduino.analogRead(i) / 1, arduino.analogRead(i) / 1);
    fill(10, 50, 200);

     ellipse(1026, 684, arduino.analogRead(i) / 1.25, arduino.analogRead(i) / 1.25);
    fill(8, 40, 175);

     ellipse(1026, 684, arduino.analogRead(i) / 1.5, arduino.analogRead(i) / 1.5);
    fill(6, 30, 150);

     ellipse(1026, 684, arduino.analogRead(i) / 1.75, arduino.analogRead(i) / 1.75);
    fill(4, 20, 125);

     ellipse(1026, 684, arduino.analogRead(i) / 2, arduino.analogRead(i) / 2);
    fill(2, 10, 100);

         ellipse(1026, 684, arduino.analogRead(i) / 2.25, arduino.analogRead(i) / 2.25);
    fill(0, 0, 75);

         ellipse(1026, 684, arduino.analogRead(i) / 2.5, arduino.analogRead(i) / 2.5);
    fill(0, 0, 50);

            ellipse(1026, 684, arduino.analogRead(i) / 2.75, arduino.analogRead(i) / 2.75);
    fill(0, 0, 25);
  }
}

And here is my GUI, all nice and cleaned up.

Because of the backup of other work that I have, I’m going to end this week here for now. If I get time later on during the machine week I will come back and keep trying to make my charlieplexed interface work but for now, this will have to do.

Here are all of my files from this week.

Click Me