Assignment

individual assignment
design, build, and connect wired or wireless node(s) with network or bus addresses

group assignment
send a message between two projects

Table of content

My Work

Group Assignment

send a message between two projects(link )

MQTT (ESP32 as a pubsub client - AWS IoT as a MQTT broker)

Researching for distributed machining idea (that allows participants who are in separated location to operate the separated machine for each local site of participant work altogether), I investigated how we can implement interfaces between remote location.

There are some ways to do dispatching message to remote location like WebSocket, MQTT, IFTTT.

Overview of MQTT

MQTT broker configuration

Select MQTT broker

Firstly, I tried to setup Heroku as it looks to be able to install in simple GUI. However, at the screen of Heroku's add-on, I got error of "the add-on is out of stock".

So, I moved to AWS IoT. It looks that Fee of AWS IoT Core is free in "free charge volume".

Names to be shared in client and MQTT broker

Same names need to be specified in client application and AWS IoT management console

Element name in client
(PubSubClient)		 Name configured both in client and MQTT broker Setting place in AWS IoT management console
device name(deviceId) FA2020JpDevice1 Manage > Things
arn (*endpoint) a2toz7cb5zl4er-ats.iot.ap-northeast-1.amazonaws.com Manage > Things > "FA2020JpDevice1" > Interact > HTTPS
publish topic name (*pubTopic) fa2020jp/topic*
(ex. fa2020jp/topic1, fa2020jp/topic2)		 Manage > Things > "FA2020JpDevice1" > Security > (attached certificate) > Policies > policy name("FA2020JpPolicy2")
{
  "Effect": "Allow",
  "Action": "iot:Publish",
  "Resource": "arn:aws:iot:ap-northeast-1:062759670692:topic/fa2020jp/topic*"
}
subscribe topic name(*subTopic) fa2020jp/topic*
(ex. fa2020jp/topic1, fa2020jp/topic2)		 Manage > Things > "FA2020JpDevice1" > Security > (attached certificate) > Policies > policy name("FA2020JpPolicy2")

Note that "subscribe" in policy needs to adopted to "subscribefilter" and "receive" in policy needs to adopted to "subscripted" topic name

{
  "Effect": "Allow",
  "Action": [
    "iot:Subscribe"
  ],
  "Resource": [
    "arn:aws:iot:ap-northeast-1:062759670692:topicfilter/fa2020jp/topic*"
  ]
},
{
  "Effect": "Allow",
  "Action": [
    "iot:Receive"
  ],
  "Resource": [
    "arn:aws:iot:ap-northeast-1:062759670692:topic/fa2020jp/topic*"
  ]
}

SSL certification for MQTT

Before embedding SSL certification files and key file, it's possible to check by openssl command

$ openssl s_client -connect a2toz7cb5zl4er-ats.iot.ap-northeast-1.amazonaws.com:8443 -CAfile AmazonRootCA1.pem -cert a35d03aab7-certificate.pem.crt -key a35d03aab7-private.pem.key

If above command works without error, the other setting like attachment of policy or naming (endpoint, device name or topic name might be wrong). In my case, failed by following reason.

check point for connecting to MQTT
  • Check you use the right certification that is attached
  • Check every content in certification or key file is embedded in program (you need to include new line code (\n) both for content and begin/end string.
  • policy did not allow "subscribefilter" or "subscribe" > change rule name
  • Check if "activate" the policy in AWS management console.
  • If the client application cannot subscribe the topic, PubSubClient disconnects the MQ connection. So, if the MQ connect is repeated, it's possible to be failed by incorrect subscription definition in policy.

Policy

For debugging, I attached a policy allowing everything as follows. If this works, something is wrong in policy setting. 

  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": "iot:*",
        "Resource": "*"
      }
    ]
  }

MQTT Pubsub client

Component and connection

Source code: mqtt_aws_v1.ino


Check Network connection

For connecting ESP32 to internet over Wifi, you need to do some steps (Tips of Wifi and SSL configuration).

Check port

MQTT default port is 8883. Instead, it looks to be able to assign 443 (it seems to require additional configuration as my application is failed just by setting 443 port)

Changed max packet size configuration in PubSubClient.h

At PubSubClient.h in library, Change #define MQTT_MAX_PACKET_SIZE 128 to #define MQTT_MAX_PACKET_SIZE 512

Experiment for publish to topic1 and topic2, then subscribe only from topic1

Overview of MQTT

Above experiment was done by tithering Wifi network of my android phone.

After above experiment, I changed my WIfi router at home and that supports 2.4Ghz Wifi network. In that environment, MQTT over AWS IoT performance is quite quick. Even with writing serial monitor both in publisher and subscriber, send publish - receive message costs under 40 msec

17:57:59.422 -> Publishing message to topic1 fa2020jp/topic1
17:57:59.422 -> {“channel”: {“1":{“tone”:“28"}}}
17:57:59.422 -> Publishing message to topic2 fa2020jp/topic2
17:57:59.422 -> {“channel”: {“2":{“tone”:“29"}}}
17:57:59.422 -> Published.
17:57:59.460 -> Received. topic=fa2020jp/topic1
17:57:59.460 -> {“channel”: {“1":{“tone”:“28"}}}

MQTT in work



Make program as a client template (with adding JSON function)

For easier implementation as a group project that can connect devices over MQTT in remote location, I made this like a template.

As a coding for other members, I specified some point to update in comment like:

  1. Set your ssid and password
  2. Configure your topic (or for your use to publish/receive message)
  3. Specify your topic name to subscribe in connectAWSIoT()
  4. Write your logic on receiving message in mqttCallback()
  5. Write your logic for publishing message in mqttLoop()

Also, I added JSON function using ArduinoJson library. It would be useful to retrieve necessary value from byte message in JSON format.

mqtt_aws_v2.ino(Gitlab, clean up security information)

For detail documentation of environmental setup and updating point in source in template, please see Group assignment page of networking and communications


MQTT (Node-RED - AWS IoT - ESP32)

Connect to Node for aws-iot

I installed and setup Node-RED on RaspberryPi, then connected it to MQTT Broker.

This image is the connection between dESP32 and subscriber over MQTT.

Flow in Node-RED

For accessing to MQTT Broker on AWS IoT, I made a simple flow that publishes and subscribes to a topic.


Add node for accessing to Node-RED

If you do not have node to access to AWS IOT, use node-red-contrib-aws-iot. You can see simple procedure to install it in the page of Node-RED Tips: Add Node for "AWS IoT"

In function node, "set message" from injected timestamp as a json format

In Json node, convert javascript object to json text

"payload" is pre-determined element to handle message content between nodes in Node-RED.

Security configuration for accessing to MQTT broker on AWS IoT

aws-iot-out" node is for setting to publish message to MQTT.

At property screen, set:

  • "Server" (same as "Endpoint" of MQTT Broker oo AWS IoT)
  • "Topic" (topic name should follow policy set configured on AWS IoT)
  • "QoS" (0 or 1. In my environment, both worked. If MQTT connection is bad, it's considerable to change this to 0 to 1)

"connection" tab in detail property of server, set:

  • "Port" as 8833 (default)
  • "Client ID" should be okay with blank (this depends on your policy in MQTT broker. In my case, if the device name(client id here) is duplicated with other devices, MQTT is repeated to be disconnected from MQTT broker after 3 seconds MQTT connect. Then I changed "statement" block in MQTT broker's policy to allow any client device like: 
    • "Statement": [
{
  "Effect": "Allow",
  "Action": [
  "iot:Connect"
],
  "Resource": [
  "arn:aws:iot:ap-northeast-1:062759670692:client/*"
]
},
  • "Keep Alive" as 60 seconds (default)

"connection" tab in detail property of server, set file path of:

  • private key
  • certification
  • CA root certification

Experiment: ESP32 and Node-RED over MQTT AWS IoT broker

ESP32 and Node-RED over MQTT AWS IoT broker

Experiment: multiple ESP32 boards(with the other students in remote place) and Node-RED over MQTT

Then I did experiment with the other students (from Fablab Kamakura) as group assignment (send a message between two project). For detail procedure, please see group assignment page link )

For experiment to publish message from Node-RED to ESP32 over MQTT, changed injected string charactors on Node-RED.

mqtt_aws_flows_fa2020jp_experiment.json

At first, we had error to connect to MQTT (connect and disconnect is repeated). The client ID of MQTT (device ID) cannot be the same when connecting to MQTT at the same time.

After we corrected device ID for each local ESP32 (ex. ID10: Oguri-san, ID11: Kimura-san, ID1: Me), it worked and we could publish/subscribe message from ESP32, RaspPi to multiple ESP32 at remote location.

mqtt_aws_v3.ino(Gitlab, clean up security information)

Generate device ID from Mac Address

For avoiding duplicated device ID of MQTT client, I changed the source code and use mac address of the device as follows. 

WiFi.macAddress(mac_addr);
sprintf(deviceId, "%02X:%02X:%02X:%02X:%02X:%02X", mac_addr[0], mac_addr[1], mac_addr[2], mac_addr[3], mac_addr[4], mac_addr[5]);)

I2C (ATtiny3216 input board - OLED display)

For my final project, I want to display the status that is input by tactile switches because it will change the mode of sound. Showing the values in display would also help debugging. I used 0.96 inches 128x64 dots OLED display

I designed and fabricated components for this

Input board for switches(ver.1.1)

Input board(ver.1.0)

Connection

Then I connected the boards by cables and programmed to dispaly the switch status programmed in input board.

Showing input status of siwthes

Test code for this is final_Input_brd_v0.30_i2c.ino

I also documented simple code example or added the link to video for experiment to show switches status changes in final project page.


References

Lessons Learned

  • MQTT is quite powerful infrastructure for networking multiple sensor device in terms of easy implementation and performance.
  • Node-RED provides fine user interface that allows to combine with MQTT and operate remote devices.

Files


to Top