Edge Processing
On-device sound classification and localisation aid using Edge Impulse (TensorFlow Lite Micro), reducing data transmission needs and enabling real-time analysis.
Hu-: Project Concept
A distributed intelligent sensing, classification, communication and learning platform
Project Vision
Hu- is an open-source, distributed intelligent sensing, classification, communication and learning platform designed to sense and communicate information within, between, and across FabLabs. Starting with a combination of acoustic sensor and TFT touchscreen display, the system will learn about the sounds in the Fablab and share this with users in and across the FabLab network. It is hoped that over time, and to meet the needs of various labs, the system will be expanded to include a range of senses and outputs; integrated with LLMs; provide acess to datasets; and increase the complexity of its network.
Quick Overview
- Project Type: Environmental Monitoring / Edge AI & IoT
- Key Technologies: XIAO ESP32-S3, Edge Impulse (TensorFlow Lite Micro), Meshtastic (LoRa), Wi-Fi, TFT Touchscreen
- Learning Objectives:
- Machine Learning: Training and deploying models for sound classification
- Edge Computing: Real-time processing on resource-constrained devices
- Distributed Sensing: Networked sensors for environmental monitoring
- Status: Concept Development
Concepts Explored
- Edge AI and Machine Learning
- Distributed Sensing Networks
- Low-Power IoT
- Environmental Data Collection
- Learning and Self-efficacy
- AI/Human Interaction
- Digital Conscious Awareness & its'qualia'
Project Concept Sketch
The conceptual sketch illustrates a single Hu- node
Prototype: System Architecture
The initial prototype focuses on a self-contained intelligent node capable of local processing and multiple communication pathways. The architecture can be broken down as follows:
Hu- Node Components
- Microcontroller: XIAO ESP32-S3 Sense (with onboard Wi-Fi capability and integrated digital microphone)
- Acoustic Sensing: Utilising the integrated MSM261D3526H1CPM digital microphone on the XIAO ESP32-S3 Sense. An artificial pinna may be used as part of the enclosure for acoustic directionality or protection, but not for complex sound localisation.
- LoRa Communication: WIO-SX1262 LoRa Module (primarily for the Meshtastic network).
- Display & Primary Input: Adafruit 2.4" Colour TFT Touchscreen.
- Audio Output: TPA2016 Amplifier + Speaker (for local audio feedback/alerts).
- Auxiliary Input: Single Tactile Switch.
- Power: LiPo Battery with USB charging and power management.
- Enclosure: Custom-designed 3D printed enclosure.
Communication Pathways
- Meshtastic Network (LoRa): Utilising the WIO-SX1262 module, nodes will form a decentralised mesh network for robust, long-range, low-power communication of alerts, sensor data summaries, and basic messages between nodes. This is ideal for local, off-grid communication within a FabLab or a regional cluster of nodes.
- Wi-Fi to Web Dashboard: Leveraging the ESP32-S3's onboard Wi-Fi, each node can connect to a standard Wi-Fi network to communicate with a central web-hosted dashboard (e.g., using Microdot on the ESP32-S3 for a lightweight web server/client). This pathway allows for richer data visualisation, remote monitoring of node status, historical data logging, and potentially more advanced configuration.
External Systems & Backend Infrastructure
- Meshtastic Network Peers: Other Hu- nodes or compatible Meshtastic devices forming the mesh.
- Web-Hosted Dashboard: A central server application providing a user interface for data exploration, device management, and alert review. The backend for this is detailed further in the Software Architecture page (typically involving a web framework like FastAPI/Flask, a database like PostgreSQL, and a frontend framework like React).
System Diagram
Overall system architecture of a Hu- node and its connections.
Key Features
Energy Efficient
Designed for low-power operation with LiPo battery, with optional solar charging for long-term deployment.
Dual Communication
LoRa (via Meshtastic) for long-range, low-power mesh networking and Wi-Fi for higher-bandwidth dashboard communication.
Data Visualization
Interactive web platform for exploring and analyzing collected sound data.
Potential Secondary Applications
Environmental Monitoring
- Wildlife activity tracking
- Biodiversity assessment
- Habitat health monitoring
- Climate change impact studies
Urban Applications
- Noise pollution monitoring
- Urban wildlife tracking
- Traffic pattern analysis
- Public safety monitoring