<h1>Networking and Communications</h1> <h4>Wednesday 25 April, 2018</h4> ## Preamble It's crunchy time now, so I don't really have a lot of preamble this week ("thank goodness" I hear you say!) but I did want to share a quick thought I had while doing my laps of this tiny island (and climbing both volcanoes a couple times each) #### The idea of Open Source A lot of Aussies I consult with tend to be afraid of exposing their "million dollar idea". It's a bit of a weird frame of mind to be in. Naturally I'm a bit adverse to sharing designs *so* openly but I wouldn't consider myself to be exactly in the same boat as the generalisation I just made. My perspective on it is to not be so protective but to be competitive and outperform any copycats. But I get that not everyone has the energy for that. We are a laid back country after all. But when it comes to Open Source specifically there's a terrible reputation for some open source fields where fundamental designs are just ill-advised. I think that eventuates from not having a vetting process for that sort of stuff - anyone can just post their files up and things that might work but for all the wrong reasons can get undeserving traction. A lot of "professionals" want to stay away from Open Source tech (though conversely they don't realise how much of their world is seated on top of OS technologies)... However, there is some fruit to the sharing principle (Fab Academy and a plenty of other good examples. I had a thought for the way scientific research is shared, and the way we build upon the work of others. As Neil says, "we want to be standing on the shoulders of giants, not on their toes". In a similar vain, I thought to myself that in nature, divergence and evolution is nature's learning algorithm. We start to use it now with new-tech concepts like machine learning, where we just try a bunch of stuff and see what works. More educated guesses get you closer to the ideal solution, but the real power is that simply *more guesses* gets you even closer! Just exploring the weird and wonderful, or sometimes simply stupid, gets you away from local maxima and only then can you really **SOLVE** a problem. It's a radical thought, but I've been known as a heretic anyway so I'm just putting it there. A few select, really great OS works eventually get refined by the community into something actually beautiful. These are few and far between however. I wonder how we can make that happen better, like maybe making better guesses from the start or opening the source earlier to get more (and more qualified) eyeballs on it before people start trying to use it. Anyway that's not really going anywhere it was just a weird thought I had. Have a fun week! *** Oh, and by the way, here's my favourite quote for the week: > It's like dating or religion, everybody has different tastes for what they like. - Neil 'the wise' Gershenfeld <div style="width:40%; float:right"> <video width="100%" autoplay loop controls muted><source src="https://i.imgur.com/wL4OrFZ.mp4" type="video/mp4"/>Your browser does not support the video tag.</video> <legend>"The Raspberry-Mega - a fan and GPIO shield I put together.<br>It features two power sources so you can transition between a battery and other power source easily, two capacitors for handling external power demand at 3.3V and 5V, and a blower fan mainly because it fits but also it could help with overclocking."</legend> </div> ##This week's assignment: <legend>"<a href="media/week14/joker-sending-a-message-meme.png">Design and build a wired and/or wireless network connecting at least two processors.</a>"</legend> I was pretty keen about all this network stuff - it's always been a phobia of mine but a very powerful topic to understand, even a little bit. My mentor <a href="http://fabacademy.org/archives/2015/as/students/harmsworth.daniel/index.html">Daniel Harmsworth</a> from the 2015 class (also incidentally the person who insisted I take Fab Academy, travel to Shenzhen, meet Wendy in Dubai and then meet Frosti in Vestmannaeyjar) is a Network Engineer and since working with him I've had a profound respect for network engineering. Among many, many other things, Daniel taught me (or namely, inspired me to Google) how to use <a href="http://osxdaily.com/2014/03/27/send-data-across-network-netcat/">Netcat</a> and the power of a simple ethernet cable. As usual I'd had a lot of tangential experience with networking but not necessarily in the embedded systems scene. This week I decided to cement my understanding by attempting a couple of things, and I'll go through them in order. ###Webserver hosting with an ESP32 ####How to underutilise (almost) any embedded controller A re-hash of the ESP32 project I mentioned in <a href="assignment13.html">Week 13</a>, which I actually built for the <a href="assignment06.html">Hackathon Week</a>. I used this <a href="http://www.instructables.com/id/ESP32-BLE-Android-App-Arduino-IDE-AWESOME/ ">Instructable</a> as the starting point, which basically connects to a hard-coded WiFi network on boot and then serves HTML via API depending on what has been coded. You can find the code in my <a href="design-files.html">Design Files</a>, which if I remember correctly currently has two API. One for reading a dummy sensor and one for turning on an LED. The main issues with this were that if the IP address of the device changed for any reason I would have to either use a WiFi network port scanner app such as <a href="https://www.fing.io/">Fing.io</a>, or plug the ESP32 in and read the IP address announcement from Serial Monitor. Not a deal breaker, but still annoying enough to stress me out before the live demo during our hackathon pitch! ###Tx/Rx or RS-232 Transistor-Transistor-Logic (TTL) ####A modified Gersh-xample Firstly I tried Neil's tx/rx examples, and changed the code to include the Carl-signature triple flash when that chip's address is called out. The code for this can be found in <a href="design-files.html">Design Files</a>, under the hello-45 directory. I had different scripts for the bridges and nodes, and you just change the address at the top of each script before uploading. The code also returns the value of the node ID - which if you're looking at it seems pretty mundane in that it just returns the number you typed, but the board itself is what actually responds so to me that's pretty cool. After fixing Neil's makefile (again) and arranging the C programs for bridge and node into separate folders I did the following: # Do the thing make -f hello.bus.45.make avrdude -p t45 -c usbtiny -P usb -U flash:w:hello.bus.45.c.hex:i I had planned to eventually re-implement this code where if I call out a node it returns the reading of a sensor that's attached to it, but I was put off by the "straight-C" implementation. Eventually (and at a later date) I managed to carve through the bit-bashing and properly understand Neil's code. Just for those doing FabAcademy currently, I found this writeup by Arduino's CosineKitty about bit-bashing. ***I wish I had found this earlier***, perhaps Neil will include it in his future lecture notes. http://playground.arduino.cc/Code/BitMath <div style="width:40%; float:right"> <img src="media/week14/network-boards-ttl.jpg" style="width:90%; float:right; padding:5px 5px 5px 5px;"/> <legend>"Boards for the TTL bridge set up. They're pretty much identical except that one of the boards has FTDI exposed."</legend> <legend>"The designs for the Tiny45's using a scratchbuilt implementation of UART and reading the on-board temperature sensor attached to ADC4 on the chip."</legend> <img src="media/week14/schematic-temp-adc4-tiny45.png" style="width:90%; float:right; padding:5px 5px 5px 5px;"/> <img src="media/week14/board-temp-adc4-tiny45.png" style="width:90%; float:right; padding:5px 5px 5px 5px;"/> </div> I ended up reverse engineering Neil's UART code written in bitmath, and to understand the fundamentals I'll explain the code snippets below. I used the following because I had a persistent clock problem (characters were getting interpreted at half the speed and I wasn't even sure what the actual TX baud rate of the board was running at). /* Serial-t45 (UART and Temp sensor only) Onboard temperature with UART(TTL) monitoring at 9600 Baud // I included below because I had a problem with the clock and had to reset fuses avrdude -p t45 -c usbtiny -P usb -U flash:w:hello.bus.45.c.hex:i avrdude -p t45 -c usbtiny -U lfuse:w:0x62:m avrdude -p t45 -c usbtiny -U hfuse:w:0xDF:m avrdude -p t45 -c usbtiny -U efuse:w:0xFF:m */ I then used a mini DSO to get my pulses near 104us. These were the numbers that came out in the end, but only after correctively setting the clock prescaler. #define bit_delay() _delay_us(70) // 9600 Baud - my clock is wrong. Neil's delay was 102 #define half_bit_delay() _delay_us(35) // Sounds obvious but make sure to update this too #define unit_id 0 void init_prescaler() { // Set prescaler division CLKPR = (1 << CLKPCE); CLKPR = (0 << CLKPS3) | (0 << CLKPS2) | (1 << CLKPS1) | (1 << CLKPS0); } I decided to play a little with the direct port manipulation, this code sets up the output pins and then shows a simplified digitalWrite(). Later on I did a bunch of stress tests and found that this can be between 20x and 50x faster than using digitalWrite() from the Arduino IDE. void init_outputs() { // Set pin to high and then output PORTB |= (1 << PB4) | (1 << PB0); //PB4 is UART out, PB0 is LED DDRB |= (1 << PB4) | (1 << PB0); } void triple_flash() { int i; for (i = 0; i < 3; i++) { PORTB &= ~(1 << PB0); _delay_ms(100); PORTB |= (1 << PB0); _delay_ms(100); } } Next I used Neil's RX and TX logic. This took me a while to translate, but basically you go through each bit and store/send it. There would be a bit more complex logic required for reading data of arbitrary size, so I left that job for another time. void serial_rx(volatile unsigned char *pins, unsigned char pin, char *rxbyte) { // Read character into rxbyte on PINB // Inverts bits (Assumes line driver) *rxbyte = 0; // Wait for start bit while (*pins & pin) {}; half_bit_delay(); // Delay half a bit int i; for (i = 0; i < 8; i++) { if (*pins & pin) { *rxbyte |= (1 << i); } else { *rxbyte |= (0 << i); } half_bit_delay(); // Not sure why, but after checking with a scope the full bit delay is too long } // Wait for stop bit bit_delay(); half_bit_delay(); } void serial_tx_string(volatile unsigned char *port, unsigned char pin, PGM_P str) { // Prepare a string of characters to send through UART TX static char chr; int i = 0; do { chr = pgm_read_byte(&(str[i])); serial_tx(&PORTB, (1 << PB4), chr); ++i; } while (chr != 0); } void serial_tx(volatile unsigned char *port, unsigned char pin, char txchar) { // Send a character through UART TX // Inverts bits (Assuming a line driver) // Start bit *port &= ~pin; bit_delay(); // Write data bits int i; for (i = 0; i < 8; i++) { if (txchar & (1 << i)) { *port |= pin; } else { *port &= ~pin; } bit_delay(); } // Stop bit *port |= pin; bit_delay(); // Char delay bit_delay(); } This next bit was just me having some fun; I wanted to see if there was any performance increase to manually reading the ADC as opposed to using analogRead() . Turns out there is minimal difference for analogRead(), however if you were looking at digitalRead() you can expect to see a 20x to 30x increase in read time from a digital pin. You can also read many pins on a single port at the same time which is suuuuper handy! This code sets the ADC to ADC4 on the ATTiny45, which is actually an inbuilt temperature sensor. The sensor needs a 1.1V reference so I've set that here, and also I'm expecting to take a 10-bit reading. I did a check to see if you get a speed advantage measuring 8- or 10-bits and as far as I could tell it had no effect. I'd presume that this is because the ADC loads all 10 bits when you do a conversion whether you read the low bit (ADCL) or not. uint16_t adc_lo, adc_hi; void init_adc() { // Initialise ADC //ADMUX = (0 << REFS2) | (0 << REFS1) | (0 << REFS0) // Set reference voltage as Vcc ADMUX = (0 << REFS2) | (1 << REFS1) | (0 << REFS0) // Set ref. voltage as 1.1V for using ADC4 temp. sensor onboard | (0 << ADLAR) // No left shift for 10-bit | (1 << MUX3) | (1 << MUX2) | (1 << MUX1) | (1 << MUX0); // ADC4 (Onboard temperature sensor) //Note: 20x gain diff. input and external ref. voltage is available ADCSRA = (1 << ADEN) // ADC Enable | (1 << ADPS2) | (1 << ADPS1) | (1 << ADPS0); // Prescaler to 128 } void read_adc() { ADCSRA |= (1 << ADSC); // Set ADSC bit to 1 to start conversion while (ADCSRA & (1 << ADSC)) {}; // Wait until the conversion is complete adc_lo = ADCL; // Read ADCL first adc_hi = ADCH; // Read ADCH } The main for this script basically does what Neil's did, except returns the value of the ADC when this unit's ID is called via the FTDI interface. For that I just used Arduino IDE's serial monitor because it's convenient. It doesn't *actually* give you a human readable value for the ADC yet (I ran outta time and didn't work that one out), but the framework is there. int main(void) { // Set up a char for RX pin static char chr; // Init all the things init_prescaler(); init_outputs(); init_adc(); triple_flash(); // Send data frame serial_tx(&PORTB, (1 << PB4), B01010101); // Should come up as U while(1) { // Read the contents of RX serial_rx(&PINB, (1 << PB3), &chr); triple_flash(); if (chr == unit_id) { triple_flash(); // Do this again so you can see if interpretation is good read_adc(); // Send a string DDRB |= (1 << PB4); // Prepare for output static const char message[] PROGMEM = "ADC4 reads:\t"; serial_tx_string(&PORTB, (1 << PB4), (PGM_P) message); serial_tx(&PORTB, (1 << PB4), &adc_lo); serial_tx(&PORTB, (1 << PB4), &adc_hi); DDRB &= ~(1 << PB4); } _delay_ms(10); } return 1; } Lastly, I found this awesome way to do what's called "debug scaffolding". Basically you can use it super fast to chase down errors and understand the execution flow of your program. If you set DEBUG to NDEBUG or DEBUG0 it allows the code to compile completely without your debug lines, meaning the code doesn't suffer for additional debug lines. // //float tlc; //#define DEBUG // Comment this out to disable debug scaffold //#ifdef DEBUG //// Debug for checking code execution, time since last call in float milliseconds // #define db(say) Serial.print(micros()/1000.000-tlc);Serial.print("\t");Serial.println(say);tlc=micros()/1000.000; // #define dbi(insert) insert;db("code insertion") //// Direct insert something // #define dbo(instead) instead;db("code override")while(0) //// Skip a block of code //#else // #define db(say) // Compiles to blank line // #define dbi(insert) // #define dbo(instead) //#endif // db("main()") // To use debug scaffold e.g.: <div style="width:60%; float:right"> <img src="media/week14/serial-explanations.jpg" style="width:90%; float:right; padding:5px 5px 5px 5px;"/> </div> ###Long distance SPI ####A very regular tale of romance between two chips #####But over an unreasonable distance So next I wanted to advance on one of my sub-projects - the Raspbuino. To do this I was hoping I could attach my CopyCat board from <a href="assignment11">Input Week</a> to a Raspberry Pi and display the sensor readings at high speed. Furthermore, I had intended to be able to do this with multiple sensor boards, where I call out the address of one and it yells back the response indiscriminately over the bus. <a href="https://i.imgur.com/lxuhZ34.gifv">Trying to get my head around SPI</a> took a very long time **The following are notes from Neil, after having <a href="https://www.facebook.com/madouc/videos/10156134787461166/ ">not one</a>, <a href="https://www.facebook.com/madouc/videos/10156134784646166/ ">but two</a> in depth chats with Bas and some reading I did around the interwebs.** The links may not work because they're attached to Bas' facebook account but I will work to have them hosted publicly if possible. ####Networking and Communications notes: Serial bus - hard coded addresses Different types: + RS-232, 422, 485 etc is "TTL, Transitor-transistor logic" + I2C - Inter-Integrated Circuit, also known as Two-Wire Interface (TWI) + Synchronous Serial + Bit-banging - pushing the pins up and down + Data lines and clock toggling + Tx/rx/clk SPI - Serial Peripheral Interface + Synchronous serial + MISO/MOSI/Chip Select USB + USB (ATMEGA 'u' series supports this) + V-USB OSI Layers - Open Systems Interconnection Model + 7: application (HTTP) + 6: presentation (SSL) + 5: session (RPC) + 4: transport (TCP, UDP) + 3: network (IP) + 2: data link (MAC) + 1: physical (PHY) + Joke - https://www.ietf.org/rfc/rfc2324.txt + Sockets - streams of data packets ####\#Interrupt What I discovered was that SPI was not a great option for this, but I wanted to learn about it and I'd already worked with TWI/I2C previously. I jumped in on QA with Neil on the weekly <a href="http://archive.fabacademy.org/archives/2018/lectures/">mid-Wednesday show</a> and asked him about how people would use SPI on an embedded controller as a slave instead of master. All the implementations I could find were related to master-only implementation, and though Neil said it was quite easy and he would be able to go through with me how write a script at the end of the class, he didn't. He did however provide me this little gem of advice, which consistently with all the advice I'd been getting previously from him and also Bas, was basically to stop doing what I was trying to do and do something else. "Life is short, my advice is to ditch SPI." - N. Gershenfeld (02 May, 2018) In all fairness, I have a tendency to pick the hardest ways to solve problems - owing to the fact that my pedigree is in generalist engineering with a major in <a href="https://en.wikipedia.org/wiki/Mechatronics">nothing particularly specific</a>. So it's understandable that none of the stuff I try is the best or nicest method. The trick with being a generalist is to find the way that's easiest, and most of the time that means giving up efficiencies every-which-way. You can see that with the advent of Arduino and despite the bloat associated with it, people outside the discipline are able to implement fundamental MCU functions. This accessibility is something that's really powerful, even though to the hardcore embedded programmers like Neil and Bas it may appear inefficient and limited. <div style="width:40%; float:left"> <img src="media/week14/output-inversion.jpg" style="width:90%; float:left; padding:5px 5px 5px 5px;"/> <img src="media/week14/export-quick-holes.jpg" style="width:90%; float:left; padding:5px 5px 5px 5px;"/> <img src="media/week14/post-processing-circuit.jpg" style="width:90%; float:left; padding:5px 5px 5px 5px;"/> <img src="media/week14/better-circuit.jpg" style="width:90%; float:left; padding:5px 5px 5px 5px;"/> </div> ####Setting up the Raspberry Pi without a display Of course, I had to start small. Since I didn't pack a monitor in my suitcase I decided to set up my Raspberry Pi in "headless" configuration, where I log in and interact with it entirely through Secure SHell (SSH). To do this we have to: + Download and unpack Raspbian + Enable SSH for the OS + Connect the device to a known WiFi network automagically + Log into the device + Change some settings + Install some utilities that help with SSH + Overclock for no reason # Installing Raspbian Lite from disk image diskutil list # Check your disk number diskutil unmountDisk /dev/disk3 # Your disk here. Don't get this one wrong cd ~/Desktop/Raspbuino # The path to your *.img sudo dd if=2018-04-18-raspbian-stretch-lite.img | pv -s 2G -ptebar | sudo dd bs=4m of=/dev/disk3 # Setting up automagic wifi connect from boot media ls /Volumes/ touch /Volumes/boot/ssh # Enable SSH for the OS sudo nano /Volumes/boot/wpa_supplicant.conf # Then chuck the following into the *.conf country=GB ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev update_config=1 network={ ssid="YOUR WIFI IF OPEN" proto=RSN key_mgmt=NONE } network={ ssid="YOUR SECURE WIFI" psk="THE PASS KEY" } <div class="row"></div> <div style="width:40%; float:right"> <img src="media/week14/raspberry-pi-fan-shield-bare.jpg" style="width:90%; float:left; padding:5px 5px 5px 5px;"/> <img src="media/week14/raspberry-pi-fan-shield.jpg" style="width:45%; float:left; padding:5px 5px 5px 5px;"/> <img src="media/week14/raspberry-pi-shield-fit-test.jpg" style="width:45%; float:left; padding:5px 5px 5px 5px;"/> <img src="media/week14/raspberry-pi-fan-shield-v2.jpg" style="width:90%; float:left; padding:5px 5px 5px 5px;"/> </div> --- # Eject and boot RPi diskutil eject /dev/disk3 # Connecting via ssh to RPi ssh-keygen -R raspberrypi.local ssh pi@raspberrypi.local # SSH via new login and enable relevant services sudo nano /boot/config.txt arm_freq=1300 over_voltage=5 gpu_freq=500 sdram_freq=500 sdram_schmoo=0x02000020 over_voltage_sdram_p=6 over_voltage_sdram_i=4 over_voltage_sdram_c=4 sudo raspi-config # Change the hostname and password. Do it. # From your SSH terminal you can now use the hostname and password you set ssh pi@raspi1.local sudo apt-get install htop # Htop lets you see the status of your remote machine, I usually have it running on another SSH just so I can see if the machine has crashed, running slow for some reason or network drop-out sudo apt-get update -y && sudo apt-get upgrade -y && sudo apt-get install git -y sudo rpi-update # Check installed apps sort by size (Good for removing bloat apps) dpkg-query -W --showformat='${Installed-Size;10}\t${Package}\n' | sort -k1,1n # Cloning disk image sudo dd if=/dev/disk3 of=./clone-rpi.dmg <br> <br>