Embedded Networking and Communications
Group assignment
- Send a message between two projects
Individual assignment
- Design, build, and connect wired or wireless node(s) with network or bus addresses
Learning outcomes:
- Demonstrate workflows used in network design
- Implement and interpret networking protocols and/or communication protocols
Have you:
- Linked to the group assignment page
- Documented your project
- Documented what you have learned from implementing networking and/or communication protocols
- Explained the programming process/es you used.
- Outlined problems and how you fixed them
- Included design files (or linked to where they are located if you are using a board you have designed and fabricated earlier) and original code.
Software Used
- VS Code
- PlatformIO
- Arduino IDE
- ArduinoJson Website
- Autodesk Eagle
- PCB-Gcode ULP for Autodesk Eagle
- Auto Leveller
Files
- GitLab Fabcloud project page containing the PCB, enclosure and firmware.
- Firmware
- Board Files
- Enclosure
Group Assignment
Go to the group assignment page.
Index
- Software Used
- Files
- Group Assignment
- Network Diagram
- PCB
- Designing my Irrigation System Board
- Milling the PCB
- Soldering and testing/Debugging
- Burning the firmware
- Debugging Again
- Network
- Network Coding
- Connecting C++ and the HTML with ArduinoJson
- Hero Shot
Network Diagram
Designing my Irrigation System Board
After tinkering with an ESP32 Development kit for a while, I had an idea of what I needed for my PCB. So let's see the list of capabilities I want for it:
- Control:
- 3 valve relays;
- 1 pump relay;
- Sense:
- Temperature and humidity sensor;
- Ammeter expansion header (for pump);
- Battery charging/sensing;
- RTC clock expansion header or solder pad;
With the requirements in hand, I went looking for an Eagle library for the ESP-WROOM-32 and found several:
After reviewing all of them, I decided to use the one from MacroYau/MacroYau.
Because I want maximum compatibility with the ESP32 dev kits I was used to, I went looking for, and found, an ESP32-WROOM pinout that I used to name the pins.
The next step was to go and attach net wires to each pin. I wish I had done this in the order I am presenting, as it would have saved me redoing many wires I connected wrongly in the first place. The lesson is that logical pin numbers and physical pin numbers are not always the same, so with that in mind, I connected them correctly this time.
That's the end result:
The relay I found at snapeda is not the same model as the one I have, but the footprint is.
I found the relay circuit online here at web.archive; schematics below.
With the schematics in hand, I drew the following in Eagle:
For the battery circuit, I followed a circuit on the web page Random Nerd Tutorials.
Since I don't have the TP4056 module for now, my circuit will contain only the sensing part.
The voltage divider resistor was calculated using the online calculator here.
For the future, I found the library for the TP4056 here thanks to a tip from Luis Carvão, my remote instructor.
The humidity sensor library can be found here. With it in hand, I simply had to net some wires to ESP32 pin 32.
As for the current sensor, I chose to use the ACS712. I found the library on this GitHub. The finished schematics are below:
I got the real-time clock library at SnapEDA as well, but for now, I will not use it, so the full schematics I drew are below:
After some time staring at the schematics, I realized a few errors. I forgot to connect the relays' LED resistor to VCC, and I also forgot to include a reset and a program button, necessary to put the ESP in programming mode, as well as headers for the serial mode and for expandability. Finally, I decided to include a crystal for better timekeeping.
I then drew the Reset and Program buttons:
Here I corrected the relays by adding the VCC, and I also added an optocoupler between the signal and the relays to prevent any feedback from the relays.
Here I added a TX RX header, a crystal for better internal RTC timekeeping, and ESP32DEVKIT compatibility headers.
So, hopefully, finally my final schematics...
Now I will start the routing work on the board. I have to say I am a bit anxious as I have never made such a complex PCB!
How encouraging is what I see when I move to the board mode in Eagle!!!
The first thing I do is move the devkit footprint to the center, along with the ESP32.
After trying for a while to connect by hand, I give the autorouter a try. I had to stop after a few minutes, and the result is laughable. I will try again moving the ESP32 up, and if that doesn't work, I will not attempt ESP32devkit compatibility.
So I gave up on the DevKit compatibility and decided to move slowly this time. I created a backup of my work that I opened in another Eagle window, and I deleted all the components I was not working on. The schematics and board looked like this. My idea is to copy the components back from the backup as I finish each step. So below is how step one looks.
On the board, you can see #1 the relay, #2 the current sensor circuit and #3 the onboard LED.
So now I copied the first valve relay, and when I went to the board, Eagle presented me with the pasted components. I believe this way is much easier.
So far, the board looks like this. I haven't routed any power wires, including GND, 3.3V and 5V. I will try the autorouter now to get an idea, and maybe route manually. The entire routing process was best described in the Input Devices week.
After finishing the layout of the board, I decided to add the logo. Click on File, Import, and Bitmap (currently, Eagle only accepts .bmp).
Choose the color to be imported.
Then select the unit, mm in my case, and the size, and click OK.
Because I wanted to mill the logo as opposed to stamp it, I had to move it to the top layer.
The finalized board layout with holes and milling contours, as described in the Output Devices week.
Milling
Here I show the visualization of the G-code routes, with red as the trace closest to the copper to be left.
As usual, and again best described in the Input Devices week, here follows a glimpse of the autolevel software.
Soldering and testing/Debugging
Inspecting the PCB under the light.
In order to verify if the tracks are all good, I place the light under the PCB and review everything looking for shorts or missing tracks.
The underside being populated.
I place the big components last so as not to make soldering the small SMD components hard.
Reading from Serial Monitor/Burning Firmware Schematics
After soldering everything, I connect the board to my serial adapter. Below are the schematics.
I pasted the first output from the ESP32 with factory firmware. It boots!
--- Available filters and text transformations: colorize, debug, default, direct, esp32_exception_decoder, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at http://bit.ly/pio-monitor-filters
--- Miniterm on COM5 115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun 8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0x00
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0008,len:8
load:0x3fff0010,len:3480
load:0x40078000,len:7804
ho 0 tail 12 room 4
load:0x40080000,len:252
entry 0x40080034
␛[0;32mI (45) boot: ESP-IDF v2.0-3-gbef9896 2nd stage bootloader␛[0m
␛[0;32mI (45) boot: compile time 05:59:45␛[0m
␛[0;32mI (45) boot: Enabling RNG early entropy source...␛[0m
␛[0;32mI (64) boot: SPI Speed : 40MHz␛[0m
␛[0;32mI (77) boot: SPI Mode : DIO␛[0m
␛[0;32mI (89) boot: SPI Flash Size : 4MB␛[0m
␛[0;32mI (101) boot: Partition Table:␛[0m
␛[0;32mI (113) boot: ## Label Usage Type ST Offset Length␛[0m
␛[0;32mI (136) boot: 0 phy_init RF data 01 01 0000f000 00001000␛[0m
␛[0;32mI (159) boot: 1 otadata OTA data 01 00 00010000 00002000␛[0m
␛[0;32mI (182) boot: 2 nvs WiFi data 01 02 00012000 0000e000␛[0m
␛[0;32mI (205) boot: 3 at_customize unknown 40 00 00020000 000e0000␛[0m
␛[0;32mI (228) boot: 4 ota_0 OTA app 00 10 00100000 00180000␛[0m
␛[0;32mI (252) boot: 5 ota_1 OTA app 00 11 00280000 00180000␛[0m
␛[0;32mI (275) boot: End of partition table␛[0m
␛[0;32mI (288) boot: Disabling RNG early entropy source...␛[0m
␛[0;32mI (305) boot: Loading app partition at offset 00100000␛[0m
␛[0;32mI (1481) boot: segment 0: paddr=0x00100018 vaddr=0x00000000 size=0x0ffe8 ( 65512) ␛[0m
␛[0;32mI (1481) boot: segment 1: paddr=0x00110008 vaddr=0x3f400010 size=0x1c5f0 (116208) map␛[0m
␛[0;32mI (1498) boot: segment 2: paddr=0x0012c600 vaddr=0x3ffb0000 size=0x0215c ( 8540) load␛[0m
␛[0;32mI (1528) boot: segment 3: paddr=0x0012e764 vaddr=0x40080000 size=0x00400 ( 1024) load␛[0m
␛[0;32mI (1551) boot: segment 4: paddr=0x0012eb6c vaddr=0x40080400 size=0x1b028 (110632) load␛[0m
␛[0;32mI (1630) boot: segment 5: paddr=0x00149b9c vaddr=0x400c0000 size=0x00034 ( 52) load␛[0m
␛[0;32mI (1631) boot: segment 6: paddr=0x00149bd8 vaddr=0x00000000 size=0x06430 ( 25648) ␛[0m
␛[0;32mI (1648) boot: segment 7: paddr=0x00150010 vaddr=0x400d0018 size=0x7a56c (501100) map␛[0m
␛[0;32mI (1675) heap_alloc_caps: Initializing. RAM available for dynamic allocation:␛[0m
␛[0;32mI (1697) heap_alloc_caps: At 3FFBA6B8 len 00025948 (150 KiB): DRAM␛[0m
␛[0;32mI (1719) heap_alloc_caps: At 3FFE8000 len 00018000 (96 KiB): D/IRAM␛[0m
␛[0;32mI (1740) heap_alloc_caps: At 4009B428 len 00004BD8 (18 KiB): IRAM␛[0m
␛[0;32mI (1761) cpu_start: Pro cpu up.␛[0m
␛[0;32mI (1773) cpu_start: Single core mode␛[0m
␛[0;32mI (1786) cpu_start: Pro cpu start user code␛[0m
␛[0;32mI (1846) cpu_start: Starting scheduler on PRO CPU.␛[0m
␛[0;32mI (2085) uart: queue free spaces: 10␛[0m
Bin version:0.10.0
I (2087) wifi: wifi firmware version: c604573
I (2087) wifi: config NVS flash: enabled
I (2088) wifi: config nano formating: disabled
I (2096) wifi: Init dynamic tx buffer num: 32
I (2097) wifi: wifi driver task: 3ffc4eac, prio:23, stack:3584
I (2102) wifi: Init static rx buffer num: 10
I (2106) wifi: Init dynamic rx buffer num: 0
I (2110) wifi: Init rx ampdu len mblock:7
I (2114) wifi: Init lldesc rx ampdu entry mblock:4
I (2118) wifi: wifi power manager task: 0x3ffca254 prio: 21 stack: 2560
I (2125) wifi: wifi timer task: 3ffcb2d4, prio:22, stack:3584
␛[0;31mE (2130) phy_init: PHY data partition validated␛[0m
␛[0;32mI (2152) phy: phy_version: 329, Feb 22 2017, 15:58:07, 0, 0␛[0m
I (2152) wifi: mode : softAP (3c:71:bf:10:66:e1)
I (2155) wifi: mode : sta (3c:71:bf:10:66:e0) + softAP (3c:71:bf:10:66:e1)
I (2159) wifi: mode : softAP (3c:71:bf:10:66:e1)
Burning the firmware:
In order to burn my firmware, I use Microsoft VS Code with PlatformIO.
Before starting the burning process, I first had to configure the PlatformIO environment. I describe this here in the Project Development week.
Next, I hold down the GPIO0 switch and press the reset button to enter the firmware burning mode. After that, I press on the icon to burn the firmware, and it burns. Success!
Debugging the board.
I ended up removing all the components in order to review the signals directly produced by the microcontroller.
The result of the debugging was that I found out there was nothing wrong with the code, the microcontroller soldering or the immediate tracks.
What I found out was that I needed to add 220 ohm resistors between the signal and the anode, pin number 1, of each optocoupler. It turns out the optocoupler has an internal LED, and the current used to activate it needs to be limited.
Then R=(3.3V-1.2V)/20mA=105ohms, or
I then cut the tracks and soldered the 220 ohm resistors, because that was what I had on hand, and it worked, but the right size is 120 ohms, as the online resistor calculator shows.
And this is what the schematics for the relay look like:
Network Code in the firmware
There are two types of communication happening on my board that I can describe.
- The communication between the phone and the ESP32 (TCP/IP protocol)
- The communication between the C++ and the HTML (ArduinoJson)
Phone and the ESP32
The network between the phone and the ESP32 is established thanks to a few libraries working together:
- WiFi.h
It is described in its documentation:
"Enables network connection (local and Internet) using the ESP32 built-in WiFi. With this library, you can initiate Servers, Clients, and send/receive UDP packets through WiFi. The shield can connect either to open or encrypted networks (WEP, WPA). The IP address can be assigned statically or through a DHCP. The library can also manage DNS."
- AsyncTCP.h
It is described in its documentation:
"This is a fully asynchronous TCP library, aimed at enabling a trouble-free, multi-connection network environment for Espressif's ESP32 MCUs. This library is the base for ESPAsyncWebServer"
- ESPAsyncWebServer.h
It is described in its documentation:
"Async HTTP and WebSocket Server for ESP8266 Arduino"
So basically, in my project, these libraries are responsible for establishing a network connection to a router, as well as creating an access point.
They also create a web server as well as a DHCP server, and with that, it issues an IP address in case one connects to it via the access point created with this code:
//Soft Wifi Access point setup WiFi.softAP("softap", "password_here"); IPAddress IP = WiFi.softAPIP();The connection to the router is established with this code:
//start wifi sessions as a client. //Wifi client setup const char* ssid = doc["data"]["ssid"]; const char* password = doc["data"]["pass"]; ssid = "fabfarm"; password = "password_here"; WiFi.begin(ssid, password);Warning: the
ssidandpasswordread from the JSON are overwritten right away by the fixed values on the next two lines, so the credentials in the JSON file are never used. Remove the two fixed assignments if you want the JSON values to count.The web server is initiated with this code:
AsyncWebServer server(81);
Connecting C++ and the HTML with ArduinoJson
JSON is the format I chose for the communication between the web interface and the microcontroller.
ArduinoJson.h is described in its documentation as:
ArduinoJson is a C++ JSON library for Arduino and IoT (Internet Of Things)
A great source of documentation, and a great assistant to create the parsing deserializeJson() program and the serializing serializeJson(doc, Serial) program, is the ArduinoJson website.
In order to use the library, I first had to create a json file. This file will be used by the assistant in order to generate code for the C++.
Having created the JSON file by hand, it's time to create the programs in C++. The assistant will then create C++ code to serialize as well as parse data to and from the JSON file that is inside the file system.
In the dialog box on the left, I pasted the json file contents.
The website generates the code automatically.
Below I show the code generated in the assistant.
Parsing program (Deserialize/decode)
const size_t capacity = JSON_ARRAY_SIZE(1) + JSON_ARRAY_SIZE(2) + 2*JSON_ARRAY_SIZE(3) + JSON_OBJECT_SIZE(2) + 6*JSON_OBJECT_SIZE(4) + JSON_OBJECT_SIZE(5) + 3*JSON_OBJECT_SIZE(6) + 540;
DynamicJsonDocument doc(capacity);
const char* json = "{\"data\":{\"currentTime\":\"Wednesday, June 24 2020 23:49:23\",\"temperature\":\"22.30\",\"humidity\":\"74.00\",\"override\":0,\"ssid\":\"rato\",\"pass\":\"imakestuff\"},\"relays\":[{\"name\":\"Fruit Tree\",\"pin\":25,\"isRunning\":1,\"isEnabled\":0,\"times\":[{\"startTime\":\"01:01\",\"duration\":0,\"hour\":1,\"min\":1}]},{\"name\":\"Vegie Garden\",\"pin\":26,\"isRunning\":1,\"isEnabled\":0,\"status\":1,\"times\":[{\"startTime\":\"02:02\",\"duration\":32,\"hour\":2,\"min\":2},{\"startTime\":\"02:02\",\"duration\":32,\"hour\":2,\"min\":2}]},{\"name\":\"Cypress Hill\",\"pin\":33,\"isRunning\":1,\"isEnabled\":0,\"status\":1,\"times\":[{\"startTime\":\"03:03\",\"duration\":60,\"hour\":3,\"min\":3},{\"startTime\":\"03:03\",\"duration\":60,\"hour\":3,\"min\":3},{\"startTime\":\"03:03\",\"duration\":60,\"hour\":3,\"min\":3}]}]}";
deserializeJson(doc, json);
JsonObject data = doc["data"];
const char* data_currentTime = data["currentTime"]; // "Wednesday, June 24 2020 23:49:23"
const char* data_temperature = data["temperature"]; // "22.30"
const char* data_humidity = data["humidity"]; // "74.00"
int data_override = data["override"]; // 0
const char* data_ssid = data["ssid"]; // "rato"
const char* data_pass = data["pass"]; // "imakestuff"
JsonArray relays = doc["relays"];
JsonObject relays_0 = relays[0];
const char* relays_0_name = relays_0["name"]; // "Fruit Tree"
int relays_0_pin = relays_0["pin"]; // 25
int relays_0_isRunning = relays_0["isRunning"]; // 1
int relays_0_isEnabled = relays_0["isEnabled"]; // 0
JsonObject relays_0_times_0 = relays_0["times"][0];
const char* relays_0_times_0_startTime = relays_0_times_0["startTime"]; // "01:01"
int relays_0_times_0_duration = relays_0_times_0["duration"]; // 0
int relays_0_times_0_hour = relays_0_times_0["hour"]; // 1
int relays_0_times_0_min = relays_0_times_0["min"]; // 1
JsonObject relays_1 = relays[1];
const char* relays_1_name = relays_1["name"]; // "Vegie Garden"
int relays_1_pin = relays_1["pin"]; // 26
int relays_1_isRunning = relays_1["isRunning"]; // 1
int relays_1_isEnabled = relays_1["isEnabled"]; // 0
int relays_1_status = relays_1["status"]; // 1
JsonObject relays_1_times_0 = relays_1["times"][0];
const char* relays_1_times_0_startTime = relays_1_times_0["startTime"]; // "02:02"
int relays_1_times_0_duration = relays_1_times_0["duration"]; // 32
int relays_1_times_0_hour = relays_1_times_0["hour"]; // 2
int relays_1_times_0_min = relays_1_times_0["min"]; // 2
JsonObject relays_1_times_1 = relays_1["times"][1];
const char* relays_1_times_1_startTime = relays_1_times_1["startTime"]; // "02:02"
int relays_1_times_1_duration = relays_1_times_1["duration"]; // 32
int relays_1_times_1_hour = relays_1_times_1["hour"]; // 2
int relays_1_times_1_min = relays_1_times_1["min"]; // 2
JsonObject relays_2 = relays[2];
const char* relays_2_name = relays_2["name"]; // "Cypress Hill"
int relays_2_pin = relays_2["pin"]; // 33
int relays_2_isRunning = relays_2["isRunning"]; // 1
int relays_2_isEnabled = relays_2["isEnabled"]; // 0
int relays_2_status = relays_2["status"]; // 1
JsonArray relays_2_times = relays_2["times"];
JsonObject relays_2_times_0 = relays_2_times[0];
const char* relays_2_times_0_startTime = relays_2_times_0["startTime"]; // "03:03"
int relays_2_times_0_duration = relays_2_times_0["duration"]; // 60
int relays_2_times_0_hour = relays_2_times_0["hour"]; // 3
int relays_2_times_0_min = relays_2_times_0["min"]; // 3
JsonObject relays_2_times_1 = relays_2_times[1];
const char* relays_2_times_1_startTime = relays_2_times_1["startTime"]; // "03:03"
int relays_2_times_1_duration = relays_2_times_1["duration"]; // 60
int relays_2_times_1_hour = relays_2_times_1["hour"]; // 3
int relays_2_times_1_min = relays_2_times_1["min"]; // 3
JsonObject relays_2_times_2 = relays_2_times[2];
const char* relays_2_times_2_startTime = relays_2_times_2["startTime"]; // "03:03"
int relays_2_times_2_duration = relays_2_times_2["duration"]; // 60
int relays_2_times_2_hour = relays_2_times_2["hour"]; // 3
int relays_2_times_2_min = relays_2_times_2["min"]; // 3Serializing Program (encode)
const size_t capacity = JSON_ARRAY_SIZE(1) + JSON_ARRAY_SIZE(2) + 2*JSON_ARRAY_SIZE(3) + JSON_OBJECT_SIZE(2) + 6*JSON_OBJECT_SIZE(4) + JSON_OBJECT_SIZE(5) + 3*JSON_OBJECT_SIZE(6);
DynamicJsonDocument doc(capacity);
JsonObject data = doc.createNestedObject("data");
data["currentTime"] = "Wednesday, June 24 2020 23:49:23";
data["temperature"] = "22.30";
data["humidity"] = "74.00";
data["override"] = 0;
data["ssid"] = "rato";
data["pass"] = "imakestuff";
JsonArray relays = doc.createNestedArray("relays");
JsonObject relays_0 = relays.createNestedObject();
relays_0["name"] = "Fruit Tree";
relays_0["pin"] = 25;
relays_0["isRunning"] = 1;
relays_0["isEnabled"] = 0;
JsonArray relays_0_times = relays_0.createNestedArray("times");
JsonObject relays_0_times_0 = relays_0_times.createNestedObject();
relays_0_times_0["startTime"] = "01:01";
relays_0_times_0["duration"] = 0;
relays_0_times_0["hour"] = 1;
relays_0_times_0["min"] = 1;
JsonObject relays_1 = relays.createNestedObject();
relays_1["name"] = "Vegie Garden";
relays_1["pin"] = 26;
relays_1["isRunning"] = 1;
relays_1["isEnabled"] = 0;
relays_1["status"] = 1;
JsonArray relays_1_times = relays_1.createNestedArray("times");
JsonObject relays_1_times_0 = relays_1_times.createNestedObject();
relays_1_times_0["startTime"] = "02:02";
relays_1_times_0["duration"] = 32;
relays_1_times_0["hour"] = 2;
relays_1_times_0["min"] = 2;
JsonObject relays_1_times_1 = relays_1_times.createNestedObject();
relays_1_times_1["startTime"] = "02:02";
relays_1_times_1["duration"] = 32;
relays_1_times_1["hour"] = 2;
relays_1_times_1["min"] = 2;
JsonObject relays_2 = relays.createNestedObject();
relays_2["name"] = "Cypress Hill";
relays_2["pin"] = 33;
relays_2["isRunning"] = 1;
relays_2["isEnabled"] = 0;
relays_2["status"] = 1;
JsonArray relays_2_times = relays_2.createNestedArray("times");
JsonObject relays_2_times_0 = relays_2_times.createNestedObject();
relays_2_times_0["startTime"] = "03:03";
relays_2_times_0["duration"] = 60;
relays_2_times_0["hour"] = 3;
relays_2_times_0["min"] = 3;
JsonObject relays_2_times_1 = relays_2_times.createNestedObject();
relays_2_times_1["startTime"] = "03:03";
relays_2_times_1["duration"] = 60;
relays_2_times_1["hour"] = 3;
relays_2_times_1["min"] = 3;
JsonObject relays_2_times_2 = relays_2_times.createNestedObject();
relays_2_times_2["startTime"] = "03:03";
relays_2_times_2["duration"] = 60;
relays_2_times_2["hour"] = 3;
relays_2_times_2["min"] = 3;
serializeJson(doc, Serial);With the help of my friend Jeff Knight, without whom I would not have been able to do it alone, the generated code was simplified for my program. I then created a few functions, and the following picture shows the serial monitor printing the result of a deserialization of the JSON file that was read.
I made this diagram so I don't forget how it works:
Hero shot
In the end, it works like this:
← (Apr 29) Molding and Casting · all weeks · Interface and Application Programming (May 13) →