Interface + application programming

This week's goal:

"interface a user to a device that you've made"

How to do that:

  • pick a language
  • interface to a device
  • visualize the results for the user

Group assignment

Visualization with Python

I followed this tutorial (with the help of Bas) to translate the readings of Henkduino into a graph.

  1. Copy over this code from the tutorial (python-test.py).
  2. Install pyserial and matplotlib using pip3.
  3. Open Arduino IDE and open the Henkduino code (henkduino.ino)
  4. The Python file doesn't know which COM port to read - find that in Arduino: /dev/cu.usbmodem1101.
  5. Close Arduino.
  6. Open the terminal in VSC and run $ python3 ./python-test.py.

This is the output:

PT reading

Digital files

P5.js sketch

I found an example on the p5.js site that allowed users to paint away one photo to reveal another photo underneath.

I made a sketch where the user can paint color into Patrik's life, but I am unable to get it to show on my documentation (issues are detailed below).

painting

patty 2

patty

Painting patrik code

// Define the global variables: bottomImg and topImg.
let bottomImg, topImg;

function preload() {
  // Preload the images from the canvas's assets directory.
  // The bottomImg is the photograph with color,
  // and the topImg is the black-and-white photograph.
  bottomImg = loadImage('/images/patrik.jpeg');
  topImg = loadImage('/images/patrik.png');
}
function setup() {
  describe(
    'Black-and-white photograph of a parrot. The cursor, when dragged across the canvas, adds color to the photograph.'
  );

  createCanvas(720, 400);

  // Hide the cursor and replace it with a picture of
  // a paintbrush.
  // noCursor();
  // cursor('/images/patrik.jpeg', 20, -10);

  // Load the top image (the black-and-white image).
  image(topImg, 0, 0);
}
function mouseDragged() {
  // Using the copy() function, copy the bottom image
  // on top of the top image when you drag your cursor
  // across the canvas.
  copy(bottomImg, mouseX, mouseY, 20, 20, mouseX, mouseY, 20, 20);
}

Showing sketches in my documentation

I tried to follow Bas' example where he embeds the p5js sketch on his site, but I wasn't able to get that workgin. (Eventually I decided to use an iframe).

I started by adding this to my mkdocs.yml file so that this would be in the <head> of my pages:

extra_javascript:
  - https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js
  - https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/addons/p5.sound.min.js

Below is an screenshot where I'm inspecting my documentation page from the browser to make sure what's in the <head>.

bas help

I added the .js file to my repository and updated the function setup() to what Bas had in his example.

bas help

Then I added the HTML from his example to this week's documentation, but nothing worked, sadly.

bas help

I reached out to him on Mattermost and he responded with this:

bas

I attempted to make the fixes, but was unable to get them working and had to focus on my final project.

IFrame

In the end, a simple iframe worked.

NB. The iframe seems to not work with code that involves photos.

Both my final and individual project both use photos of Patrik...

After further testing, I was able to get very simple sketches showing with a bit of HTML -- but it didn't work for more complicated ones.

html

This is the grey background with a black line that I was able to draw with HTML.

Coding side quest

pymdownx.highlight

To get my code blocks looking spiffy, I went back into my code and set it up based on this guide.

NB. Pymdown Highlight documentation.

Here's an example:

$a = array("foo" => 0, "bar" => 1);

It may not look like much, but it felt really nice being able to figure out how to make it work -- and knowing why I was placing certain lines of code in different places.

pymdownx.blocks.details

I also learned how to use the Details Pymdown extension:

Some summary

$a = array("foo" => 0, "bar" => 1);

Here's the documentation around this extension.

Individual project

This week was mostly focused on working on my final project. I tried to be time conscious. If I had the whole week, I would hopefully have figured out a better way to show how my p5.js sketch runs. As you may have read above, that wasn't easy.

Below is a gif of me running the sketch in the p5.js editor, connecting my phototransistor via serial, and having my sensors data translated into changes of the size of the image of Patrik.

sensor-patrik

The code below opens a prompt in your Chromium-based browser (I use Brave browser) that allows you to connect a Serial port.

let port;
const BAUDRATE = 9600;

function setup() {
  createCanvas(400, 400);
  port = createSerial();

  // fetch the list of serial ports that were previously approved
  let usedPorts = usedSerialPorts();

  if (usedPorts.length > 0) {
    port.open(usedPorts[0], BAUDRATE);
  }

  else { 
    // if no ports have been approved yet
    // add a button to open the serial port

    let button = createButton('open serial port');

    button.mousePressed(() => {
      // open the port
      port.open(BAUDRATE); 
      // this will show a popup window asking the user to select a serial port
    });
  }
}

I wasn't quite sure what these functions were doing in Bas' code, I had to look them up.

!== (Strict inequality)

true if the operands are of different types OR have different values

false if the operands are of the same type AND have the same value

!= (Non-strict Inequality)

!== (strict inequality): Does NOT perform type conversion before comparison

!= (non-strict inequality): Performs type conversion before comparison

5 !== "5"    // Returns true (different types)
5 != "5"     // Returns false (converts types first)

My plan was to take the readings from my phototransistor and use them to change a shape in my browser. I coded a circle and had it's size change by what my sensorValue was reading.

function draw() {
  background(220);
  if (port.available() > 0) {
    let data = port.readUntil('\n');
    if (data !== null) {
      let sensorValue = Number(data);
      circle(200, 200, sensorValue*10); // circle size = sensorValue (made 10x bigger)
    }
  }
}

sensor read

Because I combined a few examples to get my code working, I asked AI to help me better understand what was what. It all made sense.

Code breakdown (AI) 
if (port.available() > 0) {
    let data = port.readUntil('\n');
    if (data !== null) {
      let sensorValue = Number(data);

if (port.available() > 0) {

  • port is a serial port object, typically created using the p5.js serial library
  • port.available() checks how many bytes of data are available to read from the serial port

  • 0 means "if there is any data available at all"

  • This condition ensures we only try to read data when there's actually something to read

let data = port.readUntil('\n');

  • port.readUntil('\n') reads data from the serial port until it encounters a newline character (\n)
  • This is a common pattern when reading line-by-line data from serial devices
  • The newline character typically marks the end of a complete message
  • The read data is stored in the variable data

if (data !== null) { let sensorValue = Number(data);

  • if (data !== null) checks if we actually received valid data
  • Number(data) converts the received string data to a numeric value
  • This is necessary because serial communication transmits text, but we often want to work with actual numbers
  • The converted numeric value is stored in the variable sensorValue
  • This pattern is commonly used when working with hardware sensors that send readings over serial communication. The code ensures data is available, reads a complete line ending with a newline character, and then converts the text data to a usable numeric value.

I found this "simple" p5js tutorial quite useful.

Then, in some example code I found, I replaced the circle with a photo of Patrik.

let patrik;

function setup() {
  patrik = loadImage('/images/patrik.png')

patrik(200, 200, sensorValue*10);

The code uses a map() function, so here are notes on better understanding that.

map() function (AI) 
scaleFactor = map(mouseX, 0, width, 0.5, 3);

The map() function takes five parameters:

  • Value to be mapped (mouseX): This is the current X-coordinate of the mouse pointer on the canvas.
  • Input minimum (0): The lowest possible value for mouseX, which is 0 (the left edge of the canvas).
  • Input maximum (width): The highest possible value for mouseX, which is the width of the canvas (the right edge).
  • Output minimum (0.5): The lowest value in the target range - in this case, a scaling factor of 0.5 (half size).
  • Output maximum (3): The highest value in the target range - in this case, a scaling factor of 3 (triple size).

This was my final function draw() code:

function draw() {
  background(220);

  let scaleFactor = 1;

  if (port.available() > 0) {
    let data = port.readUntil('\n');

    if (data !== null) {
      let sensorValue = Number(data);

      // Scale based on mouse position or other input
      scaleFactor = map(sensorValue, 0, 500, 0.5, 10);

      push();
      scale(scaleFactor);
      image(patrik, 0, 0);
      pop();
    }
  }
}