Summary
This week I finally had all the knowledge to piece all the parts together to get my 3 Axis accelerometer to work. With some nicely formatted data being sent to the serial port in my computer I was able to bridge the gap between that, a local web server and show it in a web page.
What I thought I knew before
A little bit of a few different relevant computer languages through my studies and work on web design. A bit of Java, PHP, Javascript should give me a little bit of head start. The biggest challenge will how to get the data from the terminal into something where i can manipulate it.
Learning Outcomes
Theory and use of things i learnt from this assignment:
Interpret and implement design and programming protocols to create a Graphic User Interface (GUI).
Group Assignment
Apache server working!!
Compare as many tool options as possible.
What we did
As a group we were a little overwhelmed by the amount of languages and ‘tools’ there were that were capable of connecting with the serial port of the computer. Nadieh visualises her data professionally with D3. She offered to give a little impromptu presentation of this framework and how it connects to data sources and can be used as a application interface.
D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG, and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.
As part of her talk she suggested that another javascript framework was simpler to learn called P5. P5.js is a JavaScript library for creative coding, with a focus on making coding accessible and inclusive for artists, designers, educators, beginners, and anyone else!
Using the metaphor of a sketch, p5.js has a full set of drawing functionality. Not limited to your drawing canvas (an area on your web page). The whole browser page is your sketch, including HTML5 objects for text, input, video, webcam, and sound.
So as part of a group task, Nadieh walked us through how to create our own web server on our computers that would mean we could run our own websites from our hard drives.
Guide:
A web server is included in every Mac OS system. Its just a case of ‘waking it up’ to start using it. It can also need re-configuring after each major (or possible minor) OS upgrade which will follow the same basic procedure.
- Apache Server
MAc OS comes with the Apache HTTP Server. It provides a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards. Launched in 1995 and it has been the most popular web server on the Internet since April 1996.
Not only does it contain a HTTP web server, it also includes a MySQL server (for databases) and PHP to talk between them.
In the Terminal type:
sudo apachectl start
Open a browser and type:
‘localhost’ or ‘127.0.0.1’ in the address bar.
If it works you should see a ‘It works’ message.
- Make ‘Sites’ folder
You need to create a ‘Sites’ folder under your [username/ login name] folder. This will be your document root for any web-related files.
If you need to check what your username is, type whoami in the Terminal window.
Go to Mac HDD > Users > [your account folder]. Create a folder with the name ‘Sites’. When the folder is created, it will generate a folder with a ‘compass’ image in the centre.
- Modify configuration files
To be able to recognize the files put into the Sites folder several configuration files were created. To configure the server’s settings to do what you want it to do, you will need modify some of the code. This is nothing more difficult than commenting out sometimes lines of code and adding usernames where necessary.
You need to make a config file called ‘username.conf’ where you add your username if that doesn’t exist already. Going through the Terminal type: cd /etc/apache2/users
Or in the Finder window menu [Go] > [Go to Folder], type in cd /etc/apache2/users.
Make a backup if there is already a file with your username. Otherwise make a new one and include the following:
<Directory "/Users/developer/Sites/">
AllowOverride All
Options Indexes MultiViews FollowSymLinks
Require all granted
</Directory>
To configure the httpd.conf file you need to be in the ‘/etc/apache2’ folder to find the original. Modifying the file will include uncommenting and commenting out a number of modules.
Detailed instructions can be found here: Tech-cookbook.com.
To configure the httpd-userdir.conf file you need to be in the ‘/etc/apache2/extra’ folder to find the original. Modifying the file will include uncommenting a module.
Detailed instructions can be found here: Tech-cookbook.com.
- Restart and begin
In Terminal type: sudo apachectl restart. (this step will restart the Apache server to take effect of the changes made in the config file).
Visiting the ‘localhost’ address in your browser should now show a list of the folders / web files in your ‘Sites’ folder if you have any there.
Individual Assignment
Write an application that interfaces a user with an input &/or output device that you made.
What I did
Choice of ‘Tool’
A long time ago, in a galaxy far away… well last century at least, i use MAX in my first undergraduate degree (Sonic Art). I remember enjoying this using this visual programming language to create interactive sound and multimedia applications. I remember that MIDI used to control virtual instruments and music run on values from 0 to 255, numbers i’m very familiar with now with embedded programming. One student I remember, triggered music samples when a dancer moved over different pressure pads. So it will be interesting to see how this programming language has developed with technology in the last 20+ years…. (it hurts to say that).
How best to integrate this into my final project? Well, at it’s simplest version (the proof of concept i’m trying to make) there doesn’t need an application or user interface. In future spiral developments, an interface for a platform would be needed to collect and manipulate data collected from the sensors.
However, taking that old example above and one of the example user case studies for my final project (the dancer), I could leverage the 3 axis accelerometer to control sound synthesis or something for this week.
Another option would be to use another of the user cases, as a therapeutic exercise would be to be able to control and see feedback from movements from the sensor. This could be used in rehabilitation situations.
(Unfortunately, with the extra problems i came across this week (see below) there wasn’t anytime to look in to MAX.)
But this all relies on me finally getting a 3 axis accelerometer working connected to a board and talking in I2C. See Week 10 to see the working sensor.
The sensor now sends some serial data to the computer. The next step it to make a link to another coding language environment where I can visualise this data e.g. a web browser.
Installing
To try an get Neil’s python example to work I needed to make sure I had it installed in some version.
-
Install python
Python is a general-purpose coding language—which means that, unlike HTML, CSS, and JavaScript, it can be used for other types of programming and software development besides web development. That includes back end development, software development, data science and writing system scripts among other things. Skillcrush
-
Install pyserial
This module encapsulates the access for the serial port. It provides backends for Python running on Windows, OSX, Linux, BSD (possibly any POSIX compliant system) and IronPython. The module named “serial” automatically selects the appropriate backend.
I had problems with this (see below) so I changed my focus on another option. Javascript libraries to also connect to the serial port and can also be used in web pages.
-
Install p5
A p5.js library that enables communication between your p5 sketch and Arduino (or another serial enabled device). p5.serialport more or less clones the Processing Serial Library API. As JavaScript in a browser can not interact directly with a serial port, this library solves this. p5.serialport comes in two flavors; one is a simple app, this is good for all skill levels and is the easiest to use; second is Node.js based WebSocket server, this is for more skilled advanced users or someone who needs heavy customization.
-
p5.serial App
This application incorporates p5.serialserver in a GUI application for MacOS and Windows.
-
-
Install Node.js
To install Node.js i needed to first install Node Version Manager - a POSIX-compliant bash script to manage multiple active node.js versions.
- install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
- install latest version of Node.js
nvm install node
Check that they’re installed:
# check for node
node -v
# check for nvm
nvm -v
# check for npm
npm -v
npm is the Node package manager, which is installed automatically with Node.js.
Connecting serial to web browser
- Using p5
Open up the application. Pick the port that matches your serial port (it helpfully lists all your available ports). and press ‘Open’. This opens the connection between that port and the p5 server. It will also handily give you some example code for you to place into a javascript file. Containing things like functions calls with your serial port already in place.
After running the p5.serialcontrol application, you need to include the p5 client side library in your web page file. Using a combination of Javascript and HTML code (and files), you can program an interface for the serial data.
- Write a javascript file
This is where you will setup the link between the serial communication and the p5.serialcontrol app, making the data available to your web page. Javascript will provide any interaction, animation or anything outside the possibilities of plain HTML for your webpage.
The most important functions being:
serial = new p5.SerialPort();
Creating a new instance of the p5.SerialPort object - creating the possibility of communication with the serial port.
serial.open('/dev/tty.usbserial-YOURSERIALPORT');
Opening that connection with your serial port;
serial.on('data', gotData);
What to do when you receive data.
- Write a html file
In the ‘’ section import the p5 javascript libraries your script depends on. If you have the files locally:
<!-- JavaScript files -->
<script src="lib/p5.min.js"></script>
<script src="lib/p5.serialserver.js"></script>
Or direct from the web:
<script language="javascript" type="text/javascript" src="https://cdn.jsdelivr.net/npm/p5.serialserver@0.0.28/lib/p5.serialport.js"></script>
In the ‘’ section make a call to your javascript file.
<!-- Custom JavaScript -->
<script src="myTestScript.js"></script>
The interface
To keep it as simple as possible, and use the 3 points of data that the sensor send out, I decided to map the output to an RGB value of the background of a webpage. As the user would interact with the sensor it would change the colour of the page background.
To get to data points of between 0 and 255 from the sensor meant i would have to do a little bit of manipulation to the data first before it can be sent to the web browser. This was easiest to do in the IC, and send correct data to the p5 server where there wouldn’t need any other manipulation in javascript.
- Mapping values
The sensor has a particular range of values it can send based on the resolution of forces that you want to measure, from +/- 2G to +/- 16G (you’d be dead accelerating at those forces, but its nice to measure anyhow!). Checking the values that came out of the sensor when i tested it, I worked out what were the likely minimum and maximum values i would be expecting and then mapping those to 8 bit (0-255).
- Reversing negative values
One problem with the values were that some were in the negative (e.g. -30). This wouldn’t work very well with the mapping and the range of colours i would get in the end. So any negative values that come out of the sensor I multiplied by -1 to make the values positive.
With some useable values coming from the sensor, i had do something with them in the browser (javascript). The IC was sending the values as a long string. Which wasn’t the best idea when the the RGB code was looking for integers. In the Arduino sketch I added a ‘,’ between the data values so i could see where the data was separated in Javascript.
- Split serial output
So I split the string from the serial port in Javascript using the String.split() function, adding the ‘,’ comma as the delimiting sign between data values. This function would split the string into pieces and add them to an array and turn them into integers. Very useful!!
- Use the separate data values
Then it was just a simple task of filling in RGB function with the contents of the array.
background(myDataArray[0], myDataArray[1], myDataArray[2]);
Et Voila! a changing background on my web page.
What I should’ve done
I don’t think i could’ve done anything different to make it easier to get serial output from the port into a web page. It could’ve been more of a struggle I’m sure!
Mistakes & Issues
Python: import Serial error
When trying to run Neil’s python script for the accelerometer, i kept getting a “line 17, import Serial" error. After checking that i had the right version of Python (3) (pre-installed on Mac OSX) and Pyserial installed, it still wouldn’t work. Re-installing and updating these didn’t help. No results of significance showed relating to this when I searched the internet. ONly problems related to the ESP chip. Which I don’t think I could take any helpful information from.
I was thinking that maybe this is another “your computer processor and OSX is too new” problem again. So I tried on my older computer instead.
Dangerous files
Following the instructions to download and install p5, Chrome thought the files were ‘dangerous’ and wouldn’t complete the download. A look at the ‘download’ section of Chrome gave to option to finish the download and disregard the warning.
Serial from the sensor stops working, then UPDI and FTDI problems, and then everything!!!
I was at the point of getting some serial response from the board and sensor, which i was trying to refine. Then suddenly, after updating the board with Arduino over the UPDI, i failed to get any serial response when i changed to the FTDI. :(. The only physical thing I had done was change the connection. The adjustment to the embedded programming consisted of commenting out extra ‘serial.print’ statements.
So began a few hours of trying to find the source of the error. Because, when i tried to upload any sketches i got a ‘failed to initialise UPDI’ error. So what was wrong? the UPDI, the FTDI, the cables, the programming…
I had just made a new UPDI a couple of days ago so i doubted it was that (i didn’t have a spare), and the FTDI had never gone wrong in the past (but i did have a spare). I swapped everything i could, cables, other boards, tried to upload the program to boards with the same chip…..
After many hours of testing and racking my brain, i localised the problem to the sensor board. Every time it was attached to the main board it wouldn’t allow uploading via a UPDI, or it would stop any serial output from an FTDI. A continuity test of the cables between the boards were good. So I tested the few tracks and components of the sensor board. All was fine apart from there was a beep when I placed the probes either side of the capacitor (which doesn’t usually happen I think?). Capacitors on other boards made no sound in tests, but this maybe due to it being such a small value capacitor this time?
I swapped capacitors on the board, and still the problem persisted. I removed it completely and there was still a connection between VCC and GND. There wasn’t many tracks to test, but a continuity test showed that the only possible problem component was the sensor itself. How this can spontaneously stop working I don’t know? But reflowing the soldered underneath it fix the problem!
It was pushing out expected serial values from which i could work with :).
CONCLUSION
Making an interface / application depends on firstly getting a some useful data from your circuit board into your computer. A bridge between that and something humans can interact is needed, with it depending on which particular ‘tool’ or language you intend to visualise it. For which there are 600+.
FILES
| FILE | DESCRIPTION |
|---|---|
| Javascript+HTML | The Javascript + HTML files. |
| Arduino sketch | The Arduino sketch. |