Week 13: Networking and Communications
April 28, 2024
Networking and Communications
This week, I focused on setting up and testing the network system that will be used in my ToGo project. The primary goal was to establish a robust communication network between multiple ESP32-C3 microcontrollers and a Raspberry Pi 4, which acts as the central controller. This setup is crucial for ensuring responsiveness and reliability in signal handling as the ESP32 devices receive and execute commands based on numbers detected from Aruco codes through a connected camera.
Group Assigment Page
group assignment networking link 🚀
Project Objectives and Goals
- Establish Reliable Communication: Test the ability of the ESP32 devices to reliably receive and execute commands sent from the Raspberry Pi over HTTP and UDP protocols within a single network.
- System Scalability: Evaluate the network’s capability to manage multiple devices simultaneously, ensuring no significant delays or data loss.
- Integration with External Sensors: Assess the integration of real-time data processing from a camera detecting Aruco codes.
- User Interaction and Control: Develop and test a Flask web application that allows dynamic interaction with the network, enabling controls for device status checks, camera settings adjustments, and system resets.
Network Setup and Experiments
I experimented with an architecture consisting of 4 ESP32-C3 devices and 1 Raspberry Pi, all communicating within the same network using Flask to facilitate connections. This setup allowed for the following functionalities:
- Monitoring Device Connection: Check which ESP devices are connected and view the status of the Raspberry Pi and camera connections.
- Operational Control: Manage simulations by connecting from a designated diagram and controlling the system through a specified web address.
Project Architecture Overview
This section provides an overview of the network architecture and communication flow within the project:
Communication Flow
[Raspberry Pi] –HTTP/UDP–> [ESP32 Devices]
Master Controller: Raspberry Pi
- Function: Acts as the central control unit for managing ESP32 devices and processing image data from the connected camera.
- Responsibilities:
- Detecting Aruco codes using the connected camera.
- Sending commands to ESP32 devices based on detected Aruco codes.
- Serving a Flask application to monitor and control the network of devices.
Camera Conncetion
Camera Setting
Post - Send Packet
Aruco Detect
Code Section
Flask(app.py)
from flask import Flask, render_template, jsonify
import threading
import requests
import time
import cv2
from cv2 import aruco
import numpy as np
#import ledstrip
from picamera2 import Picamera2
app = Flask(__name__)
# Global variabels
picam2 = None
def setup_camera():
global picam2
picam2 = Picamera2()
camera_config = picam2.create_still_configuration(main={"size": (1920, 1080)}, lores={"size": (640, 480)}, display="lores")
picam2.configure(camera_config)
print("[INFO] Camera is configured.")
def take_photo_and_detect_aruco():
global picam2
print("[INFO] Taking picture...")
picam2.start()
frame = picam2.capture_array()
picam2.stop()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
aruco_dict = aruco.getPredefinedDictionary(aruco.DICT_4X4_50)
parameters = aruco.DetectorParameters_create()
corners, ids, rejectedImgPoints = aruco.detectMarkers(gray, aruco_dict, parameters=parameters)
if ids is not None:
# ArUco algılandığında LED'leri yeşil yap
#ledstrip.flash_green(10)
print(f"Detected ArUco IDs: {ids.flatten()}")
control_led_by_id(ids.flatten().tolist())
send_aruco_ids_to_esp(ids.flatten().tolist())
return ids.flatten().tolist()
else:
print("No ArUco markers detected")
return []
def send_aruco_ids_to_esp(ids):
ids_str = ','.join(map(str, ids))
try:
response = requests.get(f"http://192.168.68.120/show_aruco?ids={ids_str}")
print("ArUco IDs sent to ESP")
except Exception as e:
print(f"Error sending ArUco IDs to ESP: {e}")
def control_led_by_id(ids):
esp_control = {
"192.168.68.120": {"range": range(1, 7), "on": "http://192.168.68.120/D10/on", "off": "http://192.168.68.120/D10/off"},
"192.168.68.121": {"range": range(7, 13), "on": "http://192.168.68.121/D10/on", "off": "http://192.168.68.121/D10/off"},
}
for esp_ip, info in esp_control.items():
if any(id_ in info["range"] for id_ in ids):
try:
# LED'i aç
requests.get(info["on"], timeout=2)
print(f"LED turned on at ESP {esp_ip} for IDs {ids}")
# 10 saniye sonra LED'i kapat
threading.Timer(10, lambda esp_ip=esp_ip: requests.get(esp_control[esp_ip]["off"], timeout=2)).start()
except Exception as e:
print(f"Error controlling LED at ESP {esp_ip}: {e}")
@app.route('/')
def index():
return render_template('index.html')
@app.route('/connect_camera', methods=['GET'])
def connect_camera():
threading.Thread(target=setup_camera).start()
return jsonify({'message': 'Camera connection is initiated.'})
#threading.Thread(target=lambda: (setup_camera(), ledstrip.flash_yellow(5))).start()
#return jsonify({'message': 'Camera connection is initiated and LEDs flashed yellow for 5 seconds.'})
@app.route('/take_photo', methods=['GET'])
def photo_route():
ids = take_photo_and_detect_aruco()
if ids:
return jsonify({'detected_ids': ids})
else:
return jsonify({'message': 'No ArUco markers detected'})
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
Cube Code
#include <WiFi.h>
#include <U8g2lib.h>
#include <Adafruit_NeoPixel.h>
#ifdef U8X8_HAVE_HW_SPI
#include <SPI.h>
#endif
#ifdef U8X8_HAVE_HW_I2C
#include <Wire.h>
#endif
#define PIN D9 // Pin connected to the RGB LED
#define NUMPIXELS 1 // Number of LEDs
#define NORMAL_LED D10 // Pin connected to the normal LED
bool ledState = false; // LED initially off
Adafruit_NeoPixel pixels(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);
U8G2_SSD1306_128X64_NONAME_F_SW_I2C u8g2(U8G2_R0, SCL, SDA, U8X8_PIN_NONE);
const char* ssid = "*********";
const char* password = "**********";
WiFiServer server(80);
IPAddress local_IP(192, 168, 68, 122);
IPAddress gateway(192, 168, 68, 1);
IPAddress subnet(255, 255, 255, 0);
String header;
String arucoIDs = "";
void setup() {
Serial.begin(115200);
pixels.begin();
pixels.show(); // Turn off all LEDs initially
u8g2.begin();
u8g2.enableUTF8Print();
pinMode(NORMAL_LED, OUTPUT); // Set the normal LED pin as output
digitalWrite(NORMAL_LED, LOW); // LED initially off
if (!WiFi.config(local_IP, gateway, subnet)) {
Serial.println("STA Failed to configure");
}
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected.");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
server.begin();
}
void loop() {
WiFiClient client = server.available();
if (client) {
Serial.println("New Client.");
String currentLine = "";
while (client.connected()) {
if (client.available()) {
char c = client.read();
header += c;
if (c == '\n') {
if (currentLine.length() == 0) {
// Receive ArUco IDs and control LEDs
if (header.indexOf("GET /show_aruco?ids=") >= 0) {
int startPos = header.indexOf('=') + 1;
int endPos = header.indexOf(' ', startPos);
arucoIDs = header.substring(startPos, endPos);
Serial.println("Received ArUco IDs: " + arucoIDs);
digitalWrite(NORMAL_LED, HIGH); // Turn on the normal LED
// You can set the RGB LED to a specific color here
flashDifferentColors(); // Flash in different colors
pixels.clear(); // Turn off the LED
pixels.show();
}
if (header.indexOf("GET /D10/on") >= 0) {
Serial.println("Normal LED on");
digitalWrite(NORMAL_LED, HIGH); // Turn on the normal LED
ledState = true; // Update LED state
} else if (header.indexOf("GET /D10/off") >= 0) {
Serial.println("Normal LED off");
digitalWrite(NORMAL_LED, LOW); // Turn off the normal LED
ledState = false; // Update LED state
}
// Web server response
client.println("HTTP/1.1 200 OK");
client.println("Content-type:text/html");
client.println("Connection: close");
client.println();
client.println("<!DOCTYPE html><html>");
client.println("<head><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">");
client.println("</head><body>");
client.println("<h1>ESP32 Web Server</h1>");
client.println("</body></html>");
client.stop();
} else {
currentLine = "";
}
} else if (c != '\r') {
currentLine += c;
}
}
}
header = "";
Serial.println("Client Disconnected.");
}
// OLED screen updates and other processes...
// Print ArUco IDs and other information on the OLED
updateOledDisplay();
}
void updateOledDisplay() {
u8g2.clearBuffer(); // Clear the screen content
u8g2.setFont(u8g2_font_6x10_tf); // Set font size
// Print the WiFi MAC address
u8g2.setCursor(0, 10); // Position the cursor
u8g2.print("MAC: ");
u8g2.print(WiFi.macAddress());
// Print the WiFi IP address
u8g2.setCursor(0, 20); // Position the cursor
u8g2.print("IP: ");
u8g2.print(WiFi.localIP());
// Print ArUco IDs
u8g2.setCursor(0, 30); // Position the cursor
u8g2.print("ArUco IDs: ");
u8g2.print(arucoIDs);
// Print LED status
u8g2.setCursor(0, 40); // Position the cursor
u8g2.print("LED: ");
u8g2.print(ledState ? "On" : "Off"); // Print "On" or "Off" based on LED status
u8g2.sendBuffer(); // Display updates on the screen
}
void flashDifferentColors() {
// Flash in different colors at specified intervals
for(int i = 0; i < 3; i++) { // 3 cycles for each color
pixels.setPixelColor(0, pixels.Color(255, 0, 0)); // Red
pixels.show();
delay(1000); // Wait 1 second
pixels.setPixelColor(0, pixels.Color(0, 255, 0)); // Green
pixels.show();
delay(1000); // Wait 1 second
pixels.setPixelColor(0, pixels.Color(0, 0, 255)); // Blue
pixels.show();
delay(1000); // Wait 1 second
}
}
How It Works
Flask Application (app.py)
The Flask application operates on a Raspberry Pi and performs several key functions:
-
Camera Setup and Aruco Detection: The application configures and starts a camera using
Picamera2. It captures a photo to detect ArUco markers around. The detected ArUco IDs are returned as a list and sent to the ESP module. -
LED Control: Based on the detected ArUco IDs, HTTP requests are made to control LEDs on a specific ESP32 device. This function sends commands over the network to ESP32 to toggle LEDs on and off.
-
Flask Server Routes:
/: Returns the main page./connect_camera: Initiates the camera setup and informs the user that camera connection is started./take_photo: Captures a photo using the camera, detects ArUco markers, and returns the detected IDs or a message if no markers are detected.
@app.route('/')
def index():
return render_template('index.html')
@app.route('/connect_camera', methods=['GET'])
def connect_camera():
threading.Thread(target=setup_camera).start()
return jsonify({'message': 'Camera connection is initiated.'})
@app.route('/take_photo', methods=['GET'])
def photo_route():
ids = take_photo_and_detect_aruco()
if ids:
return jsonify({'detected_ids': ids})
else:
return jsonify({'message': 'No ArUco markers detected'})
ESP32 Code
The ESP32 code serves as a web server and is responsible for the following tasks:
-
Network Settings and Web Server Initialization: The ESP32 connects to the specified WiFi network and initializes a web server. The network settings and IP configuration are predefined in the code.
-
Receiving ArUco IDs and Controlling LEDs: The web server handles HTTP requests from the Flask application that include ArUco IDs. Based on these IDs, it controls the LEDs which flash in different colors.
-
OLED Display Updates: The OLED screen displays the detected ArUco IDs along with other relevant status information, enhancing user interaction by providing real-time feedback.
void loop() {
WiFiClient client = server.available(); // Check if a client has connected
if (client) { // If a client is connected, read the HTTP request
String currentLine = ""; // make a String to hold incoming data from the client
while (client.connected()) { // loop while the client's connected
if (client.available()) { // if there's bytes to read from the client,
char c = client.read(); // read a byte, then
header += c;
if (c == '\n') { // if the byte is a newline character
// if the current line is blank, you got two newline characters in a row.
// that's the end of the client HTTP request, so send a response:
if (currentLine.length() == 0) {
// HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
// and a content-type so the client knows what's coming, then a blank line:
client.println("HTTP/1.1 200 OK");
client.println("Content-type:text/html");
client.println("Connection: close");
client.println();
// turns the GPIOs on and off
if (header.indexOf("GET /D10/on") >= 0) {
digitalWrite(D10, HIGH); // Turn the LED on
} else if (header.indexOf("GET /D10/off") >= 0) {
digitalWrite(D10, LOW); // Turn the LED off
}
// Break out of the while loop
break;
} else { // if you got a newline, then clear currentLine
currentLine = "";
}
} else if (c != '\r') { // if you got anything else but a carriage return character,
currentLine += c; // add it to the end of the currentLine
}
}
}
// Clear the header variable
header = "";
// Close the connection
client.stop();
Serial.println("Client disconnected.");
}
}