Introduction

Connecting your embedded devices to the internet can have some real benefits. One thing that always remains a nuisance is to find your device on the world wide web of connected computers by determining it’s IP address. To obtain the IP address, a user must often resort to having a display or opening up another communication channel to your device, typically a serial connection.

I have this problem quite often when students have to connect a large number of devices simultaneously in a classroom setting. This made me start looking for an answer to the following question:

How can I communicate a device’s IP address or other unique identifier to an end-user?

The solution should be:

• cheap
• easy to use
• without access to the connected router’s DHCP table
• speed isn’t an issue for now.

Ways to do this would be:

• send the address to a known external server
  • which would have security and reliability issues
• print the identifier
  • When it’s static this would work. For large amounts of devices this would mean a lot of work.
• a display
  • Which would defeat the purpose of having a small IoT device
• bluetooth
  • which is a bit of a hassle, with pairing and everything. Also, not all wifi-capable devices also have bluetooth.
• another radio protocol
  • Which would need specialized hardware on receiving side
• a special LED Matrix “QR-code”
  • Would be nice, quite cheap but make your devices rather bulky

Eventually I chose to explore whether it would be possible to encode data in the flashing pattern of an RGB LED

I was happy enough with the result I decided to present it at the Open Review.

The final result

The microcontroller flashes an RGB LED in a specific pattern. A JavaScript application is written to analyze the pattern, interpret the bits and bytes and convert it back into readable data.

Besides sending text, the microcontroller can be programmed to send out it’s IP address as separate bytes.

The circuit

For the code examples to run out of the box, it’s necessary to:

• have an Arduino compatible device
• have the AdaFruit NeoPixel library (or equivalent) installed
• have a WS2812b LED with its data-in pin connected to whatever the LED_PIN points to, in my case ESP8266 pin D3.

I chose to connect a WS2812b RGB LED to an ESP8266 microcontroller. I already had a PCB lying around which I designed to drive little classroom robots.

The schematic for the “Little-Endian” classroom robot	The PCB for the “Little-Endian” classroom robot. The PCB was designed and manufactured before FabAcademy.

A lot of components on this PCB are not significant for this demonstration but this is what I used for testing.

Firmware

For the RGB-LED to reliably send data a synchronous communication protocol was chosen. An RGB LED is basically three LEDS in one so each color channel is able to play a different role. A little serial communication protocol was made to turn the red, green and blue channels of the LED on in different combinations and the point it towards my webcam and see what color it returned.

if (Serial.available())
  {
    val = Serial.read();
    switch (val)
    {
      case 'r': r = BRIGHTNESS_R; break;
      case 'e': r = 0; break;
      case 'g': g = BRIGHTNESS_G; break;
      case 'f': g = 0; break;
      case 'b': b = BRIGHTNESS_B; break;
      case 'v': b = 0; break;
      case 's':
        sending = true;
        bitIndex = 0; clockBit = 0; break;
      case 'S': sending = false; break;
    }
  }

When the character 's' is sent, the ws2812 starts flashing in a certain pattern:

When flashing the bytes, the FLASH_INTERVAL will determine how fast the LED will flash it’s bits. The sampling rate of the camera that picks up the signal is assumed to be between 24 and 30fps. Flashing bits at a rate of over half this frequency will result in aliasing. Keeping the FLASH_INTERVAL at 75ms will keep it slightly above the Nyquist frequency of 15Hz.

Determining whether to flash the bit channel is done by shifting the byte to send:

// Creates a unit32_t that represents whether the bit at <bitIndex> is high or low
uint32_t createDataColor(uint8_t dataByte, uint8_t bitIndex)
{
  return BRIGHTNESS_R << 16 | // Set the red (clock) channel to HIGH
         //determine wether the bit at <bitIndex> in <dataByte> is high
         BRIGHTNESS_B * ((dataByte & (1 << (7 - bitIndex))) > 0); 
}

Timing is done using millis() and determining the state (flashing a bit, clock or byte) is done using an ugly pile of nested if-statements.

Javascript and Webcam

The Webcam code should run in a browser, as this is an application to turn flashes of light into an IP address. Using P5.js makes it easy to get to activate a webcam and to draw shapes on a canvas. I am using the Facetime camera of my MacBook Pro 13" 2019.

The end-user should point a camera to the RGB-LED. I don’t have to scan the entire image, so I defined a specific region of the screen in which I scan for color changes. The average color in the region was determined to see whether separate flashes could be picked up with the webcam.

The first iteration looked like this. The average red, green and blue components were drawn below the web-cam image as separate circles

At a later stage some code was introduced to handle rudimentary signal processing and a better way to plot timeseries.

As it turns out, activating separate colors of the LED will not purely activate the corresponding channels of the webcam. This is probably due to the webcam being smart and dynamically trying to correct colors and dynamic white balancing. There is a certain amount of cross-talk between the color channels. Especially the blue and green channels affect each other. Red is quite unaffected by both. Luckily there is enough information to differentiate the signals.

The conversion from raw signals to bits and bytes is done by using some simple rules and statistics.

A bit is detected by determining the state of the bit signal at two samples before a rising edge of the clock signal. A byte is detected at the rising edge of the byte signal. When a byte event is detected, all eight previously received bits are converted to a byte. One thing to look out for is the unavailability of unsigned data types in JavaScript. When interpreting the received byte as a number, 128 should be added to the byte in order to convert it to an unsigned byte. When converting the byte to an ASCII character, the byte should be left alone.

Hello World Demo

Conclusion

It is possible to send data to a device with a webcam using a cheap RGB LED. By sending an IP address from the microcontroller to a device with a web browser it is shown that sending data this way can be useful. Other useful applications might be:

• Sending simple device ID information, such as the last few bytes of the MAC address, which helps you identifying your device on any website that supports it.
• Sending simple data over a large distance. Setting up an RGBeacon on large festivals and scanning them from afar.
• Indoor positioning. Analyzing a video stream filming multiple RGBeacons can help localizing objects, for instance in a warehouse. Use TCP/IP to tell all your RGBeacons to blink their ID will tell your webcam where to find your devices spatially.

At this point sending data using and RGB LED and a webcam is quite slow and error prone but there is a lot of room for refinements and improvements:

On the hardware side:

• The environmental light conditions determine the way the camera perceives light levels. The light level of the LED could dynamically react on the amount of environment light. Either using a sensor or maybe assume a certain light level at a certain time of day.
• The data could be encoded in another fashion. Doing it asynchronously one might triple the speed because each color component could represent one bit of data.
• Some error correction could be built in. Maybe sacrificing one entire channel for parity bits, making a tradeoff between speed and accuracy.
• Standardize protocols for integers, ip addresses, text, geo coordinates, etc.

On the webcam side:

• Cross-talk between channels could be minimized by correcting them using linear algebra
• Parameters that should be explored to make the application more stable:
  • Threshold levels
  • Timing
  • Window size for SD and average
• Error checking on the webcam side. Using checksums

Eventually this should be turned into an Arduino library and a nicely encapsulated JavaScript library for use in actual applications. This is purely a proof of concept.

Group Assignment

For the group project I prepared a tutorial for my fellow students to get experienced in Processing and connect Processing applications to your microcontroller using serial communication. The tutorial can be found here.

The result was a particle system that emitted particles based on a serial input value.

Furthermore, Bente showed us how to use WebSockets to communicate with a web-page. Using an ESP8266 and a websocket library we were able to send data to a website when a button was pressed.

Files

- Particle system analog input

• RGBeacon Hello World - RGBeacon ip address link - RGBeacon Webserver Firmware - RGBeacon Hello World firmware