lecture notes

global review previous week

Neil helped setting up a multidisciplinary global think-tank to encourage the acceleration of Interspecies Communication called interspecies.io
Through this, he recently visited fluentPet, a company that makes buttons that allow dogs to speak.
Press coverage on this:
https://www.theverge.com/21557375/bunny-the-dog-talks-researchers-animal-cognition-language-tiktok
https://www.nytimes.com/2021/05/27/style/bunny-the-dog-animal-communication.html

Great app that creates graphics for LCDs (works on arduino!) is lopaka.app. Animator.wokwi.com can do something similar.

this weeks lecture

This week is about programming applications, not on embedded systems, but on e.g. a computer (desktop application) or smartphone (app) that can communicate to your embedded system. This could e.g. be to visualize data or other types of applications.
Because C let’s you write programs with memory leaks and buffer overflows, a whole plethora of other languages appeared: Go, Rust, Flutter, Dart, Kotlin, Swift. All focussing mainly on safety.
Perl and Ruby are spiritually in the same group as python.
Python was developed to be a language that implements a lot of good stuff from other languages. Python has packages. Neil uses Conda as package management system.
Arduino emerged as embedded version of Processing (which is used for desktop applications). P5.js is the javascript version of this.
JavaScript is for applications within a browser. Node is JavaScript applications outside of the browser. NPM is a package manager for javascript.
WebAssembly lets you compile other languages into things you can run in a browser.
Visual applications using data flows (dataflow applications) are e.g. labview, Max, Mods, Blockly (by Google) and node-RED. Dataflow programming is all about blocks that are connected through lines.
Node-RED is a framework to build a dataflow interface. It uses flows with modules. The modules can create events.

Interfacing with the embedded processor can be done in multiple ways, e.g. serial, I2C, USB, network.
MQTT: typical for IoT systems. You have a cloud of devices (sensors, actuators) that should talk to each other. MQTT is simple and lightweight. Mosquitto is a broker for MQTT. The broker hosts the communication. You can send messages into and out of the broker. It is at the heart of a MQTT system. There is a MQTT broker specifically for fabacademy on mosquitto.org. See issue #18.
The idea of MQTT is that there are channels to which you can subscribe and to which you can publish to.

TOML is a descendant of JSON with a clean syntax.

Tk is a widget library and Tkinter is the binder for Python. With this you create a window with a drawing canvas and some graphical element that you can update.
wxWidgets is a family of widgets designed to look like the local OS.
Mods uses HTMLForms. Nicer looking widget library for HTML is for example jQuery, Bootstrap, Flat UI, etc.

With more complex applications you have multiple files, multiple packages, etc. Frameworks to handle dependencies between these are e.g. Backbone, Require, Angular, Ember, etc.

Canvas is bitmaps using javascript. Originates from Apple. You’re basically pushing pixels to the screen. You can make animations using this. Good if you just want to push pixels to the screen.
SVG is 2D graphics on the web. It is object based (can be done using javascript). It is vectors so resolution independent.

WebGL is the standard for 3D graphics (low level programming!).

With three.js you can push pixels, use SVGs (2D) and you can also do 3D objects where you add a camera, place it and point it. It interfaces very efficiently to the graphics processor of your device. Three.js also has build-in support for VR goggles and AR on phones.

With HTML5 you can use the Audio context to create sound. The sound can be created using javascript. See documentation and Neils example.

Analyse data is done using Math and machine learning. Math in Python is done using Numpy. With Matplotlib you can show it in a graph.
Anaconda lets you make interactive notebooks. It uses Python to code and you can add your own notes and comments to it.
In javascript, plotting graphs can be done using plotly (works in multiple languages btw).
D3JS is a javascript library to visualize data in interesting, creative ways. See the work of Nadieh Bremer.
Machine learning in Python can be done in PyTorch, heavily used in research. Tensorflow is a bit older and heavily used in production. Neil teaches a class on machine learning. Here are his lecture notes.

Python itself is slow because it is interpreted. Numba compiles Python so it can run faster.
Javascript is faster. Micropython itself knows how to do this, called the Native code emitter. See the micropython docs. The trade-off for the improved performance (roughly twices as fast as bytecode) is an increase in compiled code size.
You can get Python to run on the GPU instead of on the CPU or get it to run on multiple CPUs using Numba, Cuda and and Taichi.