15. Interface and application programming¶

Assignments¶

Neil’s Assignment¶

individual assignment: write an application that interfaces a user with an
input &/or output device that you made
group assignment:
compare as many tool options as possible. I used DroidScript Alan explored Tkinter, mods, and Processing.

Nueval Assignment¶

(What Local and Global Evaluators grade as pass-fail)

UNIT DESCRIPTION
Task: Interface and Application Programming

Group assignment:¶

Compare as many interface and application tool options as possible.

Individual assignment:¶

Write an application that interfaces a user with an input and/or output device that you made

LEARNING OUTCOMES
Interpret and implement design and programming protocols to create a Graphic User Interface (GUI).

Implement and interpret networking protocols and/or communication protocols
100% Complete

My Assignment Plan¶

I will create an android based gui to turn an led on and off on my LEDuo board from week 07 Electronics Design via WiFi or Bluetooth.

Research¶

I found a great tutorial that I could modify code for to turn LEDs on and off on my ESP32 based LEDou board via and android based interface (Graphical User Interface or GUI)! Remotely Controlling ESP32 Using an Android App by blhurren

I tried the new Arduino IDE 2.0.0-rc9.1 for the activity below. I’m not sure how that will go, but let’s see!

Building a GUI¶

New Arduino IDE 2.0 completed tasks¶

The following tasks I had already completed in previous Assignments 1. Install Arduino IDE 2. Install the ESP32 Boards following Installing ESP32 Board in Arduino IDE 2.0 3. 4. In the Arduino IDE add ESP32 boards to: Tools, Board Manager 5. Arduino IDE Select: Tools, Board, ESP32 Arduino, ESP32 Dev Module 6. Add dependency to run ESPAsyncWebServer: AsyncTCP(click Code, then Download ZIP ) 7. In the Arduino IDE navigate to: Sketch, Include library, Add.ZIP Library and navigate to where you put the file above. 6. Add the ESPAsyncWebServer library per link above under Research (click Code, then Download ZIP ) 7. In the Arduino IDE navigate to: Sketch, Include library, Add.ZIP Library and navigate to where you put the file above.

Arduino code¶

Code #1¶

Next I needed to send a hello world test message via WiFi and my ESP32 based LEDou board! Note I have not replaced the required SSID and password in the code below with my setting to protect my network. You will need to change those lines to work on your network.

#include "WiFi.h"
#include "ESPAsyncWebServer.h"

const char* ssid = "myRouter";  //replace
const char* password =  "myPassword"; //replace

AsyncWebServer server(80);

int relayPin = 23;

void setup(){

  pinMode(relayPin, OUTPUT);
  digitalWrite(relayPin, HIGH);

  Serial.begin(115200);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println(WiFi.localIP());

  server.on("/hello", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", "Hello World");
  });

  server.on("/relay/off", HTTP_GET   , [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", "ok");
    digitalWrite(relayPin, HIGH);
  });
   server.on("/relay/on", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain","ok");
    digitalWrite(relayPin, LOW);
  });

  server.on("/relay/toggle", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain","ok");
    digitalWrite(relayPin, !digitalRead(relayPin));
  });

  server.on("/relay", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", String(digitalRead(relayPin)));
  });

  server.begin();
}

void loop(){}

/*********
  Rui Santos
  Complete project details at https://RandomNerdTutorials.com/vs-code-platformio-ide-esp32-esp8266-arduino/
*********/

#include <Arduino.h>

#define LED 2

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  pinMode(LED, OUTPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  digitalWrite(LED, HIGH);
  Serial.println("LED is on");
  delay(1000);
  digitalWrite(LED, LOW);
  Serial.println("LED is off");
  delay(1000);
}

I first verified the code above to make sure all libraries needed were installed and working. The code worked. If your does not don’t worry, look up the error messages and search online on how to fix the error.

Next uploaded the code to my LEDuo board. Remember you may need to press your reset and programming buttons to put your ESP Board into programming mode.

I opened the serial monitor to see if the code above was working. I then pressed the reset button to knock the board out of programming mode and into run mode. I got these weird characters ��*�'��'��. Oops the baud rate was on 9600, I need to bump it up to? 19200, nope. 115200, yes! I got this:

Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..

Hmm not really what I wanted. Time to reset again. Then I realized my network SSID was wrong and was set for my 5G wifi, the ESP32 only uses 2G WiFi, I added a _2G (This is how I differentiate my 2G SSID) to the end of my WiFi SSID… put my board into program mode again, and re-uploaded the code. I noted that with serial monitor running that the ESP32 actually let us know rst:0x1 (POWERON_RESET),boot:0x3 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_REO_V2)) waiting for download` when the reset button is held and then the program button is held and released. I’m not sure why I never noticed this before now.

And the following appeared:

Sketch uses 739990 bytes (56%) of program storage space. Maximum is 1310720 bytes.
Global variables use 38576 bytes (11%) of dynamic memory, leaving 289104 bytes for local variables. Maximum is 327680 bytes.
esptool.py v3.0-dev
Serial port /dev/ttyUSB0
Failed uploading: uploading error: exit status 1Traceback (most recent call last):
  File "/home/meistertek/.local/lib/python3.8/site-packages/serial/serialposix.py", line 322, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
OSError: [Errno 16] Device or resource busy: '/dev/ttyUSB0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/meistertek/.arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool.py", line 3682, in <module>
    _main()
  File "/home/meistertek/.arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool.py", line 3675, in _main
    main()
  File "/home/meistertek/.arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool.py", line 3329, in main
    esp = chip_class(each_port, initial_baud, args.trace)
  File "/home/meistertek/.arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool.py", line 263, in __init__
    self._port = serial.serial_for_url(port)
  File "/home/meistertek/.local/lib/python3.8/site-packages/serial/__init__.py", line 90, in serial_for_url
    instance.open()
  File "/home/meistertek/.local/lib/python3.8/site-packages/serial/serialposix.py", line 325, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 16] could not open port /dev/ttyUSB0: [Errno 16] Device or resource busy: '/dev/ttyUSB0'

I Remember reading somewhere that you must make sure that the processor settings must be correct for memory, speed etc. So I opened up my old version 1.8 of Arduino and took a look at the settings that worked. Hmmm all of them were the same, what was wrong?

I decided to look carefully at the last error message:

raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 16] could not open port /dev/ttyUSB0: [Errno 16] Device or resource busy: '/dev/ttyUSB0'
Failed uploading: uploading error: exit status 1

I searched the error online and didn’t even have open the links that came up. You cannot have serial monitor open while trying to program a board, there is only one serial port and you have to either monitor it or close it to program. LOL :) This explains why I never saw the status via serial monitor rst:0x1 (POWERON_RESET),boot:0x3 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_REO_V2)) waiting for download before. Next time I will Remember to close serial monitor once I see that message again :)

Well what do you know, with serial monitor closed, the errors were gone! Here is the Output status:

Writing at 0x00074000... (100 %)
Wrote 740112 bytes (419958 compressed) at 0x00010000 in 7.6 seconds (effective 774.6 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 128...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.0 seconds (effective 1526.8 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

ok reset button pressed and serial monitor back open again! and , sh*t!

Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..

What am I doing wrong? Back to look carefully at the code. hmmm I couldn’t find anything, I tried my guest network next. I wondered is the _ underscore in my SSID was whacking stuff out?

Nope! same result?

Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..

Let’s try some simpler tests, this time I turned off security on my guest network. That did not work either. Jeesh! Next I upped the security to WPA2/WPA3. Well what do you know that worked. Does this code use WPA3? Wow that’s pretty good! Note: I found out later that if I simply reset to board a few times it will eventually connect to WiFi properly. So it has nothing to do with WPA2 or WPA3 settings.

Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
192.168.0.205

I tried pinging the board via a terminal and got, host unreachable. ??? oh wait my computer is on my secure network, that’s good that I can’t see my board on the guest network. Time to try my secure network…and…

Connecting to WiFi..
Connecting to WiFi..```
192.168.0.205

Reset my board and via terminal ping got this:

meistertek@meistertek-ThinkPad-X1-Yoga-2nd:~$ ping 192.168.0.205
PING 192.168.0.205 (192.168.0.205) 56(84) bytes of data.
64 bytes from 192.168.0.205: icmp_seq=1 ttl=255 time=77.9 ms
64 bytes from 192.168.0.205: icmp_seq=2 ttl=255 time=97.6 ms
64 bytes from 192.168.0.205: icmp_seq=3 ttl=255 time=19.0 ms
64 bytes from 192.168.0.205: icmp_seq=4 ttl=255 time=42.3 ms
64 bytes from 192.168.0.205: icmp_seq=5 ttl=255 time=66.8 ms

Awesome! Hmm is my password obtainable from and ESP32, how is it stored. I think someone would have to sue Bluetooth or serial to get my code of the ESP32, which would require physical access. I plan to learn more about ESP32 security.

Jeesh, this was a lot more trouble to get this far then I first thought.

Back to the instructions at Remotely Controlling ESP32 Using an Android App

Code #2¶

At this point I wanted to combine some of the code I used in week 7 Electronics Design with the code above. Below you can see the WiFi code followed by code that will blink the two LEDs on the LEDuo board continuously after connecting to WiFi I think? I also added the Electronics Design code under the void setup() area and the int a = 0; code just above that to declare integer a variable and to set it to zero.

#include "WiFi.h"
#include "ESPAsyncWebServer.h"

const char* ssid = "242_2G";  //replace
const char* password =  "fredshalamarpudding139506QRXK"; //replace

AsyncWebServer server(80);

int relayPin = 23;
int a = 0;

void setup(){

  pinMode(relayPin, OUTPUT);
  digitalWrite(relayPin, HIGH);
  pinMode(16,OUTPUT);
  pinMode(32,OUTPUT);

  Serial.begin(115200);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println(WiFi.localIP());

  server.on("/hello", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", "Hello World");
  });

  server.on("/relay/off", HTTP_GET   , [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", "ok");
    digitalWrite(relayPin, HIGH);
  });
   server.on("/relay/on", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain","ok");
    digitalWrite(relayPin, LOW);
  });

  server.on("/relay/toggle", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain","ok");
    digitalWrite(relayPin, !digitalRead(relayPin));
  });

  server.on("/relay", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", String(digitalRead(relayPin)));
  });

  server.begin();
}

void loop() {
  // put your main code here, to run repeatedly:
  // flash LEDou LEDs in a loop after code above runs, indicating WiFi connection is established.
  digitalWrite(16,1);
  delay(50);
  digitalWrite(16,0);
  delay(50);
  a++;
  if(a>5){
    digitalWrite(32,1);
  }
  if(a>10){
    a=0;
    digitalWrite(32,0);
  }
}

Frack it worked! What’s interesting is that being a bit of a distance away from my WiFi router and with the small WiFi antenna I noticed it could take a bit of time to connect to WiFi. Sometimes I had to reset the board a few times before it would connect.

I also found the website Processing, Reference quite useful for better understanding the various bits of code. For example I wanted to know about Int so the area Processing, Reference, Int was where I could find more info.

Code #3¶

The next block of code would allow me to connect with the Android GUI on my phone to the ESP32 chips IP address and control the LEDs with screen buttons!

#include "WiFi.h"
#include "ESPAsyncWebServer.h"

const char* ssid = "myRouter";  //replace with your router SSID
const char* password =  "myPassword"; //replace your router password

AsyncWebServer server(80);

int relayPin = 32; //LEDuo left LED is D1 on GPIO32, right LED is D2 on GPIO16

void setup(){

  pinMode(relayPin, OUTPUT);
  digitalWrite(relayPin, HIGH);

  Serial.begin(115200);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println(WiFi.localIP());

  server.on("/hello", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", "Hello World");
  });

  server.on("/relay/off", HTTP_GET   , [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", "ok");
    digitalWrite(relayPin, HIGH);
  });
   server.on("/relay/on", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain","ok");
    digitalWrite(relayPin, LOW);
  });

  server.on("/relay/toggle", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain","ok");
    digitalWrite(relayPin, !digitalRead(relayPin));
  });

  server.on("/relay", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", String(digitalRead(relayPin)));
  });

  server.begin();
}

void loop(){}

I copied and pasted the code above into the Arduino IDE and changed the SSID and Router password as my comments above indicate in the code. I then uploaded the code to my LEDou board after pressing the buttons to put into program mode and made sure serial monitor was closed.

Once the code above was done uploading, I opened serial monitor again and pressed the reset button on the LEDuo board to run the code.

The serial monitor displayed:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:10944
load:0x40080400,len:6388
entry 0x400806b4
Connecting to WiFi..
Connecting to WiFi..
Connecting to WiFi..
192.168.0.205

ok time to program the GUI!

DroidScript¶

DroidScript is a free JavaScript IDE for UI creation on android devices. I will use it to create an app that runs on my adroid phone that has buttons to turn on and off an LED on my LEDou board I fabricated in week 7 Electronics Design.

I installed DroidScript - JavaScript IDE on my Android phone using Play Store. Opening the DroidScript App I then said yes to access permissions. I touched the bottom of the screen to close the twitter links about DroidScript apps that popped up. This brought me this screen:

Next I touched the wifi symbol in the DroidScript on my phone to program from my computer. I typed the ip address and password that showed on my cell phone into my chrome browser on my computer. This makes programming a DroidScript much easier. Here is what that looks like:

Then I selected the Grid icon in the DroidScript window that appeared:

Then I selected New JavaScript App and named it LEDuoGUI left the drop down selection on Simple and clicked Ok. This loaded a code window where I cut and pasted the following java JavaScript code:

var url = "http://192.168.0.205"; //change to your ESP32s IP address

//Called when application is started.
function OnStart()
{
  //Create a layout with objects vertically centered.
  lay = app.CreateLayout( "linear", "VCenter,FillXY" );   
  app.AddLayout( lay );


      //Create a button to send request.
  btn = app.CreateButton( "State", 0.3, 0.1 ); 
  btn.SetMargins( 0, 0.05, 0, 0 ); 
      btn.SetOnTouch( btn_OnTouch ); 
  lay.AddChild( btn ); 

  //Create a button to send request.
  btnON = app.CreateButton( "LED ON", 0.3, 0.1 ); 
  btnON.SetMargins( 0, 0.05, 0, 0 ); 
      btnON.SetOnTouch( btnON_OnTouch ); 
  lay.AddChild( btnON ); 

  //Create a button to send request.
  btnOFF = app.CreateButton( "LED OFF", 0.3, 0.1 ); 
  btnOFF.SetMargins( 0, 0.05, 0, 0 ); 
      btnOFF.SetOnTouch( btnOFF_OnTouch ); 
  lay.AddChild( btnOFF ); 
}

function btn_OnTouch()
{
        //Send request to remote server.
        var path = "/relay";

        //var params = "data=" + data.replace("\r","");
        app.HttpRequest( "get", url, path, "", HandleReply );
}

function btnON_OnTouch()
{
        //Send request to remote server.
        var path = "/relay/on";

        //var params = "data=" + data.replace("\r","");
        app.HttpRequest( "get", url, path, "", HandleReply );
}

function btnOFF_OnTouch()
{
        //Send request to remote server.
        var path = "/relay/off";

        //var params = "data=" + data.replace("\r","");
        app.HttpRequest( "get", url, path, "", HandleReply );
}

//Handle the servers reply.
function HandleReply( error, response )
{
      console.log(error);
      app.ShowPopup(response);
}

I then clicked the Save button (you can also press keys ctrl+s) I then clicked the Play icon (you can also use keys Alt+r) and this GUI appeared on my android phone. Wow that was way easier than I thought!

Hero Shots¶

Here is a video showing the control of D1 LED on the LEDou board with the Android GUI I programmed.

Here is code on my phone in the DroidScript app

References¶

Back to Home Page