Week 14: Interface and Application Programming

Tasks for This Week

Group assignment

• Compare as many tool options as possible.
• Document your work on the group work page and reflect on your individual page what you learned.

Individual assignment

• Write an application that interfaces a user with input and/or output device(s) on a board that you made.

Group Assignment

Link to Group Assignment

We both had an interest in generative art this week. I think it is a great way to display data in a different way and it is even better when you can interact with it.

When I was in Japan I went to the teamLAB Planets exhibition in Tokyo. One part involved generative art that adapted to people in the space using cameras. This made people a part of the artwork and made it come alive. This demonstrated the power of interaction and creative visualisation coming together to give a memorable experience. Perhaps we can use this to make boring data, more interesting?

Plan for this Week

We have some upcoming workshops at the Lake Mac Libraries Fab Lab on data literacy and interpretation. One of the areas we are looking at is visualising weather data. I have decided this week to therefore connect a temperature sensor to the Xiao RP2040 and create generative art based on the temperature of the sensor, which a user can interact with by taking their finger on/off the sensor.

I have had a little experience with javascript in the past so decided to use that as the generator language.

Temperature Sensor

I decided to use the PiicoDev Precision Temperature Sensor

I connected the temperature sensor to the Xiao RP2040 using a PiicoDev connection cable.

Code

# PiicoDev TMP117 minimal example code
# This program reads the temperature from the PiicoDev TMP117 precision temperature sensor
# and displays the result in Degrees Celsius, Farenheit and Kelvin

from PiicoDev_TMP117 import PiicoDev_TMP117
from PiicoDev_Unified import sleep_ms # cross-platform compatible sleep function

tempSensor = PiicoDev_TMP117() # initialise the sensor

while True:
    # Read and print the temperature in various units
    tempC = tempSensor.readTempC() # Celsius
    tempF = tempSensor.readTempF() # Farenheit
    tempK = tempSensor.readTempK() # Kelvin

    # Convert temperature into a string and print the data
    print(str(tempC) + " °C")

    sleep_ms(1000) # delay 1 second

As the Xiao is not standard for the pin reference in PiicoDev Unify, I needed to modify this code to work:

# PiicoDev TMP117 minimal example code
# This program reads the temperature from the PiicoDev TMP117 precision temperature sensor
# and displays the result in Degrees Celsius, Farenheit and Kelvin
from machine import Pin, I2C
from PiicoDev_TMP117 import PiicoDev_TMP117
from PiicoDev_Unified import sleep_ms # cross-platform compatible sleep function

tempSensor = PiicoDev_TMP117(1, scl=Pin(7), sda=Pin(6), freq=200000) # initialise the sensor

while True:
    # Read and print the temperature in various units
    tempC = tempSensor.readTempC() # Celsius
    # tempF = tempSensor.readTempF() # Farenheit
    # tempK = tempSensor.readTempK() # Kelvin

    # Convert temperature into a string and print the data
    print((tempC))

    sleep_ms(1000) # delay 1 second

Temperature sensor is all ready to send data over USB Serial.

Application/Interface Code

I asked Chat GPT 3.5 (May 24 Version) to:
write a guide on creating abstract 3D generative art using javascript on a Windows PC that changes based on live data coming from a temperature sensor that is sending its data over serial USB via a Seeed Xiao RP2040

Certainly! Here's a guide to creating abstract 3D generative art using JavaScript on a Windows PC, with live data from a temperature sensor connected to a Seeed Xiao RP2040:

Materials needed: - Windows PC - Code editor (e.g. Visual Studio Code) - Seeed Xiao RP2040 - Temperature sensor (e.g. LM35) - USB cable - Breadboard and jumper wires - Basic knowledge of JavaScript

Steps:

1. Connect the temperature sensor to the Seeed Xiao RP2040 using the breadboard and jumper wires. The LM35 sensor has three pins: Vcc, GND, and Vout. Connect Vcc to 3.3V on the Xiao RP2040, GND to GND, and Vout to an analog input pin (e.g. A0) on the Xiao RP2040.
2. Write a script in JavaScript to read the temperature data from the serial port of the Xiao RP2040. You can use the serialport library to communicate with the Xiao RP2040. Here's an example script:

const SerialPort = require('serialport');
const Readline = SerialPort.parsers.Readline;

const port = new SerialPort('COM3', { baudRate: 9600 });
const parser = port.pipe(new Readline({ delimiter: '\r\n' }));

parser.on('data', (data) => {
  const temperature = parseFloat(data);
  console.log(`Temperature: ${temperature}°C`);
});

This script opens a serial connection to the Xiao RP2040 on COM3 at a baud rate of 9600. It then sets up a parser to read data from the serial port and parse it as a string with a newline delimiter. When data is received, the script converts it to a floating-point number and logs it to the console.

1. Install the three library for 3D graphics rendering in JavaScript. You can install it using npm by running npm install three.
2. Write a script to generate the abstract 3D generative art based on the temperature data. You can use the three library to create a 3D scene, add shapes and textures, and animate them. Here's an example script that creates a spinning cube with a texture that changes based on the temperature:

import * as THREE from 'three';

const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);

const renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);

const textureLoader = new THREE.TextureLoader();
let texture = textureLoader.load('texture1.png');

const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshBasicMaterial({ map: texture });
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);

camera.position.z = 5;

function animate() {
  requestAnimationFrame(animate);

  // Read temperature data from serial port
  parser.on('data', (data) => {
    const temperature = parseFloat(data);

    // Update texture based on temperature
    if (temperature < 20) {
      texture = textureLoader.load('texture1.png');
    } else if (temperature >= 20 && temperature < 30) {
      texture = textureLoader.load('texture2.png');
    } else {
      texture = textureLoader.load('texture3.png');
    }

    material.map = texture;
    material.needsUpdate = true;
  });

  cube.rotation.x += 0.01;
  cube.rotation.y += 0.01;

  renderer.render(scene, camera);
}

I knew that the above was a start of a direction to follow.

I had heard about using three.js to create javascript visualisations before, so I googled for examples and found one I liked by Shuya Tamaru called Popping Square:
Article can be found at https://styublog.com/popping-sphere-animation/
Demo can be found at https://codepen.io/shuya-tamaru/pen/wvmJjbX
Tamaru provides a good explanation on how her code works in the article.

I took the original javascript code from Tamaru

window.addEventListener("load", init);

function init() {
  let width = window.innerWidth;
  let height = window.innerHeight;

  const scene = new THREE.Scene();
  scene.background = new THREE.Color(0x050505);

  const camera = new THREE.PerspectiveCamera(45, width / height, 0.1, 1000);
  cameraSetBackDist = 7;
  camera.position.z = cameraSetBackDist;

  const light = new THREE.PointLight(0xffffff, 2);
  light.position.set(10, 10, 10);
  scene.add(light);

  const renderer = new THREE.WebGLRenderer();
  renderer.setSize(width, height);
  document.body.appendChild(renderer.domElement);

  const geometry = new THREE.IcosahedronGeometry(1, 10);
  const geometryPos = geometry.getAttribute("position").array;
  const mesh = [];
  const normalDirection = [];

  for (let i = 0; i < geometryPos.length; i += 9) {
    const geometry2 = new THREE.BufferGeometry();

    const vertices = new Float32Array([
      geometryPos[i],
      geometryPos[i + 1],
      geometryPos[i + 2],
      geometryPos[i + 3],
      geometryPos[i + 4],
      geometryPos[i + 5],
      geometryPos[i + 6],
      geometryPos[i + 7],
      geometryPos[i + 8]
    ]);

    geometry2.setAttribute("position", new THREE.BufferAttribute(vertices, 3));
    geometry2.setAttribute("normal", new THREE.BufferAttribute(vertices, 3));

    const normal = new THREE.Vector3(
      (geometryPos[i] + geometryPos[i + 3] + geometryPos[i + 6]) / 3,
      (geometryPos[i + 1] + geometryPos[i + 4] + geometryPos[i + 7]) / 3,
      (geometryPos[i + 2] + geometryPos[i + 5] + geometryPos[i + 8]) / 3
    );

    normal.normalize();
    const icoSphereGeometry = new THREE.IcosahedronGeometry(0.1, 1);
    const material = new THREE.MeshBasicMaterial({
      wireframe: false,
      color: 0xc100eb
    });

    const sphere = new THREE.Mesh(icoSphereGeometry, material);
    mesh.push(sphere);

    normalDirection.push(normal);

  }

  let loopSpeed = 0;
  let rot = 0;
  const clock = new THREE.Clock();

  const tick = () => {
    rot += 0.3;
    const cameraAngle = (rot * Math.PI) / 180;
    let z = cameraSetBackDist * Math.cos(cameraAngle);
    let x = cameraSetBackDist * Math.sin(cameraAngle);
    camera.position.set(x, 0, z);
    camera.lookAt(0, 0, 0);

    const elapsedTime = clock.getElapsedTime();

    mesh.map((spheremesh, index) => {
      const coordinateAverageValue =
        (normalDirection[index].x +
         normalDirection[index].y +
         normalDirection[index].z) / 3;
      const addAngle = coordinateAverageValue * elapsedTime * 1;
      const distance = 1;
      loopSpeed += 0.002;
      const radians = (loopSpeed * Math.PI) / 180;
      const angle = radians + addAngle;
      const loop = (Math.sin(angle) + 1) * distance;
      const scale = (Math.sin(angle) + 1.1) * 0.3;

      spheremesh.position.set(
        normalDirection[index].x * loop,
        normalDirection[index].y * loop,
        normalDirection[index].z * loop
      );
      spheremesh.scale.set(scale, scale, scale);

      const h = Math.abs(Math.sin(angle)) * 360;

      const s = 100;
      const l = 70;
      const color = new THREE.Color(`hsl(${h},${s}%,${l}%)`);
      spheremesh.material.color = color;

      scene.add(spheremesh);
    });
    requestAnimationFrame(tick);
  };
  tick();

  window.addEventListener("resize", () => {
    width = window.innerWidth;
    height = window.innerHeight;
    camera.aspect = width / height;
    camera.updateProjectionMatrix();
    renderer.setSize(width, height);
});


  function animate() {
    requestAnimationFrame(animate);
    renderer.render(scene, camera);
  }

  animate();
}

I then played around with adding the reference to temperature as a variable. But could not get the code to work. I gave it to Chat GPT to see if it could fix it.

To Chat GPT 3.5 May 24 Version I asked:
check the below code for errors

window.addEventListener("load", init);

function init() {

    const serialOptions = {
      baudRate: 9600,
    };

    let material;
    let texture;

    async function connectSerial() {
      try {
        const port = await navigator.serial.requestPort();
        await port.open(serialOptions);
        const reader = port.readable.getReader();

        while (true) {
          const { value, done } = await reader.read();
          if (done) break;
          const textDecoder = new TextDecoder();
          const data = textDecoder.decode(value);
          const temperature = parseFloat(data);
          console.log(`Temperature: ${temperature}°C`);



  let width = window.innerWidth;
  let height = window.innerHeight;

  const scene = new THREE.Scene();
  scene.background = new THREE.Color(0x050505);

  const camera = new THREE.PerspectiveCamera(45, width / height, 0.1, 1000);
  cameraSetBackDist = 7;
  camera.position.z = cameraSetBackDist;

  const light = new THREE.PointLight(0xffffff, 2);
  light.position.set(10, 10, 10);
  scene.add(light);

  const renderer = new THREE.WebGLRenderer();
  renderer.setSize(width, height);
  document.body.appendChild(renderer.domElement);

  const geometry = new THREE.IcosahedronGeometry(1, 10);
  const geometryPos = geometry.getAttribute("position").array;
  const mesh = [];
  const normalDirection = [];

  for (let i = 0; i < geometryPos.length; i += 9) {
    const geometry2 = new THREE.BufferGeometry();

    const vertices = new Float32Array([
      geometryPos[i],
      geometryPos[i + 1],
      geometryPos[i + 2],
      geometryPos[i + 3],
      geometryPos[i + 4],
      geometryPos[i + 5],
      geometryPos[i + 6],
      geometryPos[i + 7],
      geometryPos[i + 8]
    ]);

    geometry2.setAttribute("position", new THREE.BufferAttribute(vertices, 3));
    geometry2.setAttribute("normal", new THREE.BufferAttribute(vertices, 3));

    const normal = new THREE.Vector3(
      (geometryPos[i] + geometryPos[i + 3] + geometryPos[i + 6]) / 3,
      (geometryPos[i + 1] + geometryPos[i + 4] + geometryPos[i + 7]) / 3,
      (geometryPos[i + 2] + geometryPos[i + 5] + geometryPos[i + 8]) / 3
    );

    normal.normalize();
    const icoSphereGeometry = new THREE.IcosahedronGeometry(0.1, 1);
    const material = new THREE.MeshBasicMaterial({
      wireframe: false,
      color: 0xc100eb
    });

    const sphere = new THREE.Mesh(icoSphereGeometry, material);
    mesh.push(sphere);

    normalDirection.push(normal);

  }

  let loopSpeed = 0;
  let rot = 0;
  const clock = new THREE.Clock();

  const tick = () => {
    rot += 0.3;
    const cameraAngle = (rot * Math.PI) / 180;
    let z = cameraSetBackDist * Math.cos(cameraAngle);
    let x = cameraSetBackDist * Math.sin(cameraAngle);
    camera.position.set(x, 0, z);
    camera.lookAt(0, 0, 0);

    const elapsedTime = clock.getElapsedTime();

    mesh.map((spheremesh, index) => {
      const coordinateAverageValue =
        (normalDirection[index].x +
         normalDirection[index].y +
         normalDirection[index].z) / (temperature/2);
      const addAngle = coordinateAverageValue * elapsedTime * temperature;
      const distance = 1;
      loopSpeed += 0.002;
      const radians = (loopSpeed * Math.PI) / 180;
      const angle = radians + addAngle;
      const loop = (Math.sin(angle) + 1) * distance;
      const scale = (Math.sin(angle) + 1.1) * 0.3;

      spheremesh.position.set(
        normalDirection[index].x * loop,
        normalDirection[index].y * loop,
        normalDirection[index].z * loop
      );
      spheremesh.scale.set(scale, scale, scale);

      const h = Math.abs(Math.sin(angle)) * 360;

      const s = 100;
      const l = 70;
      const color = new THREE.Color(`hsl(${h},${s}%,${l}%)`);
      spheremesh.material.color = color;

      scene.add(spheremesh);
    });
    requestAnimationFrame(tick);
  };
  tick();

  window.addEventListener("resize", () => {
    width = window.innerWidth;
    height = window.innerHeight;
    camera.aspect = width / height;
    camera.updateProjectionMatrix();
    renderer.setSize(width, height);
});


  function animate() {
    requestAnimationFrame(animate);
    renderer.render(scene, camera);
  }

  animate();
}
                }
      }
    }

The code provided seems to have some inconsistencies. Here's the modified code with the corrections:

<!DOCTYPE html>
<html>
<head>
  <title>Morphing Sphere</title>
  <style>
    body { margin: 0; overflow: hidden; }
    canvas { display: block; }
  </style>
</head>
<body>
  <button onclick="connectSerial()">Connect Serial</button>
  <script src="https://cdn.jsdelivr.net/npm/three@0.134.0/build/three.min.js"></script>
  <script>
    const serialOptions = {
      baudRate: 9600,
    };

    let material;
    let texture;

    async function connectSerial() {
      try {
        const port = await navigator.serial.requestPort();
        await port.open(serialOptions);
        const reader = port.readable.getReader();

        while (true) {
          const { value, done } = await reader.read();
          if (done) break;
          const textDecoder = new TextDecoder();
          const data = textDecoder.decode(value);
          const temperature = parseFloat(data);
          console.log(`Temperature: ${temperature}°C`);

          // Update the morphing sphere based on temperature
          mesh.forEach((sphere, index) => {
            const coordinateAverageValue =
              (normalDirection[index].x +
              normalDirection[index].y +
              normalDirection[index].z) / (temperature / 2);
            const addAngle = coordinateAverageValue * elapsedTime * temperature;
            const distance = 1;
            loopSpeed += 0.002;
            const radians = (loopSpeed * Math.PI) / 180;
            const angle = radians + addAngle;
            const loop = (Math.sin(angle) + 1) * distance;
            const scale = (Math.sin(angle) + 1.1) * 0.3;

            sphere.position.set(
              normalDirection[index].x * loop,
              normalDirection[index].y * loop,
              normalDirection[index].z * loop
            );
            sphere.scale.set(scale, scale, scale);

            const h = Math.abs(Math.sin(angle)) * 360;

            const s = 100;
            const l = 70;
            const color = new THREE.Color(`hsl(${h},${s}%,${l}%)`);
            sphere.material.color = color;
          });

          // Render the scene
          renderer.render(scene, camera);
        }
      } catch (error) {
        console.error('Error:', error);
      }
    }

    let width = window.innerWidth;
    let height = window.innerHeight;

    const scene = new THREE.Scene();
    scene.background = new THREE.Color(0x050505);

    const camera = new THREE.PerspectiveCamera(45, width / height, 0.1, 1000);
    const cameraSetBackDist = 7;
    camera.position.z = cameraSetBackDist;

    const light = new THREE.PointLight(0xffffff, 2);
    light.position.set(10, 10, 10);
    scene.add(light);

    const renderer = new THREE.WebGLRenderer();
    renderer.setSize(width, height);
    document.body.appendChild(renderer.domElement);

    const geometry = new THREE.IcosahedronGeometry(1, 10);
    const geometryPos = geometry.getAttribute('position').array;
    const mesh = [];
    const normalDirection = [];

    for (let i = 0; i < geometryPos.length; i += 9) {
      const geometry2 = new THREE.BufferGeometry();

      const vertices = new Float32Array([
        geometryPos[i],
        geometryPos[i + 1],
        geometryPos[i + 2],
        geometryPos[i + 3],
        geometryPos[i + 3],
        geometryPos[i + 4],
        geometryPos[i + 5],
        geometryPos[i + 6],
        geometryPos[i + 7],
        geometryPos[i + 8]
      ]);

      geometry2.setAttribute('position', new THREE.BufferAttribute(vertices, 3));
      geometry2.setAttribute('normal', new THREE.BufferAttribute(vertices, 3));

      const normal = new THREE.Vector3(
        (geometryPos[i] + geometryPos[i + 3] + geometryPos[i + 6]) / 3,
        (geometryPos[i + 1] + geometryPos[i + 4] + geometryPos[i + 7]) / 3,
        (geometryPos[i + 2] + geometryPos[i + 5] + geometryPos[i + 8]) / 3
      );

      normal.normalize();
      const icoSphereGeometry = new THREE.IcosahedronGeometry(0.1, 1);
      const material = new THREE.MeshBasicMaterial({
        wireframe: false,
        color: 0xc100eb
      });

      const sphere = new THREE.Mesh(icoSphereGeometry, material);
      mesh.push(sphere);

      normalDirection.push(normal);
    }

    let loopSpeed = 0;
    let rot = 0;
    const clock = new THREE.Clock();

    function tick() {
      rot += 0.3;
      const cameraAngle = (rot * Math.PI) / 180;
      let z = cameraSetBackDist * Math.cos(cameraAngle);
      let x = cameraSetBackDist * Math.sin(cameraAngle);
      camera.position.set(x, 0, z);
      camera.lookAt(0, 0, 0);

      const elapsedTime = clock.getElapsedTime();

      mesh.forEach((spheremesh, index) => {
        const coordinateAverageValue =
          (normalDirection[index].x +
          normalDirection[index].y +
          normalDirection[index].z) / (temperature / 2);
        const addAngle = coordinateAverageValue * elapsedTime * temperature;
        const distance = 1;
        loopSpeed += 0.002;
        const radians = (loopSpeed * Math.PI) / 180;
        const angle = radians + addAngle;
        const loop = (Math.sin(angle) + 1) * distance;
        const scale = (Math.sin(angle) + 1.1) * 0.3;

        spheremesh.position.set(
          normalDirection[index].x * loop,
          normalDirection[index].y * loop,
          normalDirection[index].z * loop
        );
        spheremesh.scale.set(scale, scale, scale);

        const h = Math.abs(Math.sin(angle)) * 360;

        const s = 100;
        const l = 70;
        const color = new THREE.Color(`hsl(${h},${s}%,${l}%)`);
        spheremesh.material.color = color;

        scene.add(spheremesh);
      });

      renderer.render(scene, camera);

      requestAnimationFrame(tick);
    }
    tick();

    window.addEventListener('resize', () => {
      width = window.innerWidth;
      height = window.innerHeight;
      camera.aspect = width / height;
      camera.updateProjectionMatrix();
      renderer.setSize(width, height);
    });
  </script>
</body>
</html>

In this modified code, the morphing sphere animation is updated based on the temperature value received from the serial port. The connectSerial function is responsible for reading the temperature value, and the tick function is responsible for updating the position, scale, and color of each sphere based on

Chat GPT and I went back a forth on some errors. We got there, but there wasn't a real noticeable change when the temperature was changing. I decided to ask Chat GPT to:
Modify the above so that color changes with temperature, make this color change a gradient from blue for the coldest, less than 15 degrees, to red for the hottest, more than 30 degrees

Certainly! Here's the modified code that incorporates color changes with temperature:

<!DOCTYPE html>
<html>
<head>
  <title>Morphing Sphere</title>
  <style>
    body { margin: 0; overflow: hidden; }
    canvas { display: block; }
  </style>
</head>
<body>
  <button onclick="connectSerial()">Connect Serial</button>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
  <script>
    const serialOptions = {
      baudRate: 9600,
    };

    let material;
    let texture;

    async function connectSerial() {
      try {
        const port = await navigator.serial.requestPort();
        await port.open(serialOptions);
        const reader = port.readable.getReader();

        while (true) {
          const { value, done } = await reader.read();
          if (done) break;
          const textDecoder = new TextDecoder();
          const data = textDecoder.decode(value);
          const temperature = parseFloat(data);
          console.log(`Temperature: ${temperature}°C`);

          // Update the morphing sphere based on temperature
          mesh.forEach((sphere, index) => {
            const coordinateAverageValue =
              (normalDirection[index].x +
              normalDirection[index].y +
              normalDirection[index].z) / (temperature / 2);
            const addAngle = coordinateAverageValue * temperature;
            const distance = 1;
            loopSpeed += 0.002;
            const radians = (loopSpeed * Math.PI) / 180;
            const angle = radians + addAngle;
            const loop = (Math.sin(angle) + 1) * distance;
            const scale = (Math.sin(angle) + 1.1) * 0.3;

            sphere.position.set(
              normalDirection[index].x * loop,
              normalDirection[index].y * loop,
              normalDirection[index].z * loop
            );
            sphere.scale.set(scale, scale, scale);

            // Calculate color based on temperature gradient
            const minTemperature = 15;
            const maxTemperature = 30;
            const t = (temperature - minTemperature) / (maxTemperature - minTemperature);
            const color = new THREE.Color().setHSL(0.67 - t * 0.67, 1, 0.5);
            sphere.material.color = color;
          });

          // Render the scene
          renderer.render(scene, camera);
        }
      } catch (error) {
        console.error('Error:', error);
      }
    }

    let width = window.innerWidth;
    let height = window.innerHeight;

    const scene = new THREE.Scene();
    scene.background = new THREE.Color(0x050505);

    const camera = new THREE.PerspectiveCamera(45, width / height, 0.1, 1000);
    const cameraSetBackDist = 7;
    camera.position.z = cameraSetBackDist;

    const light = new THREE.PointLight(0xffffff, 2);
    light.position.set(10, 10, 10);
    scene.add(light);

    const renderer = new THREE.WebGLRenderer();
    renderer.setSize(width, height);
    document.body.appendChild(renderer.domElement);

    const geometry = new THREE.IcosahedronGeometry(1, 10);
    const geometryPos = geometry.getAttribute('position').array;
    const mesh = [];
    const normalDirection = [];

    for (let i = 0; i < geometryPos.length; i += 9) {
      const geometry2 = new THREE.BufferGeometry();

      const vertices = new Float32Array([
        geometryPos[i],
        geometryPos[i + 1],
        geometryPos[i + 2],
        geometryPos[i + 3],
        geometryPos[i + 4],
        geometryPos[i + 5],
        geometryPos[i + 6],
        geometryPos[i + 7],
        geometryPos[i + 8]
      ]);

      geometry2.setAttribute('position', new THREE.BufferAttribute(vertices, 3));
      geometry2.setAttribute('normal', new THREE.BufferAttribute(vertices, 3));

      const normal = new THREE.Vector3(
        (geometryPos[i] + geometryPos[i + 3] + geometryPos[i + 6]) / 3,
        (geometryPos[i + 1] + geometryPos[i + 4] + geometryPos[i + 7]) / 3,
        (geometryPos[i + 2] + geometryPos[i + 5] + geometryPos[i + 8]) / 3
      );

      normal.normalize();
      const icoSphereGeometry = new THREE.IcosahedronGeometry(0.1, 1);
      const material = new THREE.MeshBasicMaterial({
        wireframe: false,
      });

      const sphere = new THREE.Mesh(icoSphereGeometry, material);
      mesh.push(sphere);

      normalDirection.push(normal);
    }

    let loopSpeed = 0;
    let rot = 0;
    const clock = new THREE.Clock();

    function tick() {
      rot += 0.3;
      const cameraAngle = (rot * Math.PI) / 180;
      let z = cameraSetBackDist * Math.cos(cameraAngle);
      let x = cameraSetBackDist * Math.sin(cameraAngle);
      camera.position.set(x, 0, z);
      camera.lookAt(0, 0, 0);

      const elapsedTime = clock.getElapsedTime();

      mesh.forEach((spheremesh, index) => {
        const coordinateAverageValue =
          (normalDirection[index].x +
          normalDirection[index].y +
          normalDirection[index].z) / (temperature / 2);
        const addAngle = coordinateAverageValue * elapsedTime * temperature;
        const distance = 1;
        loopSpeed += 0.002;
        const radians = (loopSpeed * Math.PI) / 180;
        const angle = radians + addAngle;
        const loop = (Math.sin(angle) + 1) * distance;
        const scale = (Math.sin(angle) + 1.1) * 0.3;

        spheremesh.position.set(
          normalDirection[index].x * loop,
          normalDirection[index].y * loop,
          normalDirection[index].z * loop
        );
        spheremesh.scale.set(scale, scale, scale);

        // Calculate color based on temperature gradient
        const minTemperature = 15;
        const maxTemperature = 30;
        const t = (temperature - minTemperature) / (maxTemperature - minTemperature);
        const color = new THREE.Color().setHSL(0.67 - t * 0.67, 1, 0.5);
        spheremesh.material.color = color;

        scene.add(spheremesh);
      });

      renderer.render(scene, camera);

      requestAnimationFrame(tick);
    }
    tick();

    window.addEventListener('resize', () => {
      width = window.innerWidth;
      height = window.innerHeight;
      camera.aspect = width / height;
      camera.updateProjectionMatrix();
      renderer.setSize(width, height);
    });
  </script>
</body>
</html>

Again we went back and forth with an error, then we got the final working code:

<!DOCTYPE html>
<html>
<head>
    <!--This Code was adapted from Tamaru 2022 https://styublog.com/popping-sphere-animation/ by Chat GPT and Claire Chaikin-Bryan in May 2023-->
  <title>Morphing Sphere</title>
  <style>
    body { margin: 0; overflow: hidden; }
    canvas { display: block; }
  </style>
</head>
<body>
  <button onclick="connectSerial()">Connect Serial</button>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
  <script>
    const serialOptions = {
      baudRate: 9600,
    };

    let material;
    let texture;

    let temperature; // Declare the temperature variable

    async function connectSerial() {
      try {
        const port = await navigator.serial.requestPort();
        await port.open(serialOptions);
        const reader = port.readable.getReader();

        while (true) {
          const { value, done } = await reader.read();
          if (done) break;
          const textDecoder = new TextDecoder();
          const data = textDecoder.decode(value);
          temperature = parseFloat(data);
          console.log(`Temperature: ${temperature}°C`);

          // Update the morphing sphere based on temperature
          mesh.forEach((sphere, index) => {
            const coordinateAverageValue =
              (normalDirection[index].x +
              normalDirection[index].y +
              normalDirection[index].z) / (temperature / 2);
            const addAngle = coordinateAverageValue * temperature;
            const distance = 1;
            loopSpeed += 0.002;
            const radians = (loopSpeed * Math.PI) / 180;
            const angle = radians + addAngle;
            const loop = (Math.sin(angle) + 1) * distance;
            const scale = (Math.sin(angle) + 1.1) * 0.3;

            sphere.position.set(
              normalDirection[index].x * loop,
              normalDirection[index].y * loop,
              normalDirection[index].z * loop
            );
            sphere.scale.set(scale, scale, scale);

            // Calculate color based on temperature gradient
            const minTemperature = 15;
            const maxTemperature = 30;
            const t = (temperature - minTemperature) / (maxTemperature - minTemperature);
            const color = new THREE.Color().setHSL(0.67 - t * 0.67, 1, 0.5);
            sphere.material.color = color;
          });

          // Render the scene
          renderer.render(scene, camera);
        }
      } catch (error) {
        console.error('Error:', error);
      }
    }

    let width = window.innerWidth;
    let height = window.innerHeight;

    const scene = new THREE.Scene();
    scene.background = new THREE.Color(0x050505);

    const camera = new THREE.PerspectiveCamera(45, width / height, 0.1, 1000);
    const cameraSetBackDist = 7;
    camera.position.z = cameraSetBackDist;

    const light = new THREE.PointLight(0xffffff, 2);
    light.position.set(10, 10, 10);
    scene.add(light);

    const renderer = new THREE.WebGLRenderer();
    renderer.setSize(width, height);
    document.body.appendChild(renderer.domElement);

    const geometry = new THREE.IcosahedronGeometry(1, 10);
    const geometryPos = geometry.getAttribute('position').array;
    const mesh = [];
    const normalDirection = [];

    for (let i = 0; i < geometryPos.length; i += 9) {
      const geometry2 = new THREE.BufferGeometry();

const vertices = new Float32Array([
  geometryPos[i],
  geometryPos[i + 1],
  geometryPos[i + 2],
  geometryPos[i + 3],
  geometryPos[i + 4],
  geometryPos[i + 5],
  geometryPos[i + 6],
  geometryPos[i + 7],
  geometryPos[i + 8]
]);

geometry2.setAttribute('position', new THREE.BufferAttribute(vertices, 3));
geometry2.setAttribute('normal', new THREE.BufferAttribute(vertices, 3));

const normal = new THREE.Vector3(
  (geometryPos[i] + geometryPos[i + 3] + geometryPos[i + 6]) / 3,
  (geometryPos[i + 1] + geometryPos[i + 4] + geometryPos[i + 7]) / 3,
  (geometryPos[i + 2] + geometryPos[i + 5] + geometryPos[i + 8]) / 3
);

normal.normalize();
const icoSphereGeometry = new THREE.IcosahedronGeometry(0.1, 1);
const material = new THREE.MeshBasicMaterial({
  wireframe: false,
});

const sphere = new THREE.Mesh(icoSphereGeometry, material);
mesh.push(sphere);

normalDirection.push(normal);
}

let loopSpeed = 0;
let rot = 0;
const clock = new THREE.Clock();

function tick() {
rot += 0.3;
const cameraAngle = (rot * Math.PI) / 180;
let z = cameraSetBackDist * Math.cos(cameraAngle);
let x = cameraSetBackDist * Math.sin(cameraAngle);
camera.position.set(x, 0, z);
camera.lookAt(0, 0, 0);

const elapsedTime = clock.getElapsedTime();

mesh.forEach((spheremesh, index) => {
  const coordinateAverageValue =
    (normalDirection[index].x +
    normalDirection[index].y +
    normalDirection[index].z) / 3;
  const addAngle = coordinateAverageValue * elapsedTime;
  const distance = 1;
  loopSpeed += 0.002;
  const radians = (loopSpeed * Math.PI) / 180;
  const angle = radians + addAngle;
  const loop = (Math.sin(angle) + 1) * distance;
  const scale = (Math.sin(angle) + 1.1) * 0.3;

  spheremesh.position.set(
    normalDirection[index].x * loop,
    normalDirection[index].y * loop,
    normalDirection[index].z * loop
  );
  spheremesh.scale.set(scale, scale, scale);

  // Calculate color based on temperature gradient
  const minTemperature = 15;
  const maxTemperature = 30;
  const t = (temperature - minTemperature) / (maxTemperature - minTemperature);
  const color = new THREE.Color().setHSL(0.67 - t * 0.67, 1, 0.5);
  spheremesh.material.color = color;

  scene.add(spheremesh);
});

renderer.render(scene, camera);

requestAnimationFrame(tick);
}
tick();

window.addEventListener('resize', () => {
width = window.innerWidth;
height = window.innerHeight;
camera.aspect = width / height;
camera.updateProjectionMatrix();
renderer.setSize(width, height);
});
</script>
</body>
</html>

Final Result

I was able to get the javascript application on my computer to change based on the temperature sensor connected to the XIao RP2040, which was connected via USB to the computer.

When the application starts you need to select the COM port that the Xiao is connected to. This needs to be done to comply with security around access of the browser to USB Serial and visa versa.

I would have liked more time to explore more colourful and other pattern options, but ran out of time. I do however have a base to work from in the future.

Files

app.html

Code for temperature sensor on Xiao
PiicoDev_TMP117.py
PiicoDev_Unified.py
temp.py