16. Interface and Application Programming - Week 14¶
16.1. Assignments¶
Interface and Application Programming
individual assignment
write an application that interfaces a user with an
input &/or output device that you made group assignment
compare as many tool options as possible
16.2. Group Assignment¶
I tried out the tooling with Rust and Slint as a UI-Framework. For the communication with my dev-board I used the TinyUSB library from the esp-idf and set up my board as a CDC-ACM Serial device.
And for debugging I used the UNIX commands cat and echo and for convenience ‘minicom’ to easily read, write and reconnect to the USB Serial device.
16.2.1. Understanding the USB Protocol¶
Note
Generated with deepseek-r1:14b (self-hosted) with following prompt:
Can you explain the Protocol behind USB?
The Universal Serial Bus (USB) protocol is a fundamental technology that enables communication between devices and computers. Here’s a detailed breakdown of its key components and functions:
History and Purpose:
Developed in the mid-1990s to replace older interfaces like parallel and serial ports.
Designed to be fast, versatile, and user-friendly, supporting a wide range of devices.
Key Components:
Host: Typically the computer or laptop that acts as the central controller.
Device: The peripheral connected to the host, such as a printer or keyboard.
Bus: The physical medium (cable) through which data is transmitted between devices.
Data Transfer Mechanisms:
Packets: Data is sent in packets, each containing information about its origin and destination.
Addresses: Assigned by the host to identify devices on the bus, facilitating communication.
Transfer Types:
Control Transfers: Used for device configuration and management.
Bulk Transfers: For large data transfers, like files or images.
Interrupt Transfers: Real-time communication, essential for devices like keyboards and mice.
Isochronous Transfers: Guaranteed bandwidth for time-sensitive data, such as audio or video.
Descriptors:
Endpoints: Channels for communication between the host and device, each with specific roles (e.g., input/output).
Configurations: Define how a device is set up and operates.
Power Management:
Devices can draw power from the host, with mechanisms in place to prevent overloading.
Power requirements are negotiated during connection setup.
Error Handling:
Protocols exist for detecting and recovering from data loss or corruption, including retransmission mechanisms.
Physical Aspects:
Cables and connectors adhere to specific standards (e.g., USB-A, USB-C) ensuring compatibility and reliable communication.
USB Versions:
USB 1.0/2.0/3.0: Offer different data transfer rates (e.g., USB 3.0 supports SuperSpeed).
Devices negotiate the highest supported speed during connection.
Hubs and Root Hubs:
Hub: Allows multiple devices to connect to a single USB port.
Root Hub: Located on the host, manages connections to other hubs and devices.
16.2.2. Setup¶
16.2.2.1. Rust + Slint¶
For Rust I used my already used rust-setup (I basically downloaded rustup with my package-manager and the normal rustc compiler and rest of the default toolchain from rustup ).
To get started I used this template, which has the needed structure for slint and a simple counter to learn the concepts.
16.2.2.2. TinyUSB + ESP-IDF¶
16.3. Individual Assignment¶
I wanted to control the LEDs via USB and also display the touch value, when I press the touchpads on my device.
16.3.1. USB Device Setup¶
First I setup an USB Serial device with this example.
Here the TinyUSB Code:
/*
* SPDX-FileCopyrightText: 2022-2024 Espressif Systems (Shanghai) CO LTD
*
* SPDX-License-Identifier: Unlicense OR CC0-1.0
*/
#include "driver/gpio.h"
#include "driver/touch_sensor.h"
#include "driver/touch_sensor_common.h"
#include "esp_log.h"
#include "freertos/FreeRTOS.h"
#include "freertos/queue.h"
#include "freertos/task.h"
#include "led_strip.h"
#include "portmacro.h"
#include "sdkconfig.h"
#include "tinyusb.h"
#include "tusb_cdc_acm.h"
#include "tusb_console.h"
#include <stdint.h>
#include <stdio.h>
static const char *TAG = "example";
static uint8_t rx_buf[CONFIG_TINYUSB_CDC_RX_BUFSIZE + 1];
#define TOUCH_DEFAULT_LEVEL 50000
#define TOUCH_MAX_LEVEL 800000
#define NUM_OF_TOUCHPADS 2
#define NUM_OF_LEDS 6
static const touch_pad_t button[NUM_OF_TOUCHPADS] = {
TOUCH_PAD_NUM1,
TOUCH_PAD_NUM2,
};
#define LED_V0 1
#define LED_V1 2
#define LED_V2 3
#define LED_G0 6
#define LED_G1 5
#define LED_G2 4
static const char *TOUCH_TAG = "CoolKeyboard";
static led_strip_handle_t led_strip;
union Leds {
struct {
uint8_t id;
uint8_t r;
uint8_t g;
uint8_t b;
} value;
uint32_t raw;
};
/**
* @brief Application Queue
*/
static QueueHandle_t app_queue;
typedef struct {
uint8_t buf[CONFIG_TINYUSB_CDC_RX_BUFSIZE + 1]; // Data buffer
size_t buf_len; // Number of bytes received
uint8_t itf; // Index of CDC device interface
} app_message_t;
int init_device() {
// TOUCH PAD INIT
touch_pad_init();
for (int i = 0; i < NUM_OF_TOUCHPADS; i++) {
touch_pad_config(button[i]);
}
touch_pad_set_measurement_interval(TOUCH_PAD_SLEEP_CYCLE_DEFAULT);
touch_pad_set_charge_discharge_times(TOUCH_PAD_MEASURE_CYCLE_DEFAULT);
touch_pad_set_voltage(TOUCH_PAD_HIGH_VOLTAGE_THRESHOLD,
TOUCH_PAD_LOW_VOLTAGE_THRESHOLD,
TOUCH_PAD_ATTEN_VOLTAGE_THRESHOLD);
touch_pad_set_idle_channel_connect(TOUCH_PAD_IDLE_CH_CONNECT_DEFAULT);
for (int i = 0; i < NUM_OF_TOUCHPADS; i++) {
touch_pad_set_cnt_mode(button[i], TOUCH_PAD_SLOPE_DEFAULT,
TOUCH_PAD_TIE_OPT_DEFAULT);
}
/* Denoise setting at TouchSensor 0. */
touch_pad_denoise_t denoise = {
/* The bits to be cancelled are determined according to the noise level.
* */
.grade = TOUCH_PAD_DENOISE_BIT4,
.cap_level = TOUCH_PAD_DENOISE_CAP_L4,
};
touch_pad_denoise_set_config(&denoise);
touch_pad_denoise_enable();
ESP_LOGI(TOUCH_TAG, "Denoise function init");
/* Enable touch sensor clock. Work mode is "timer trigger". */
touch_pad_set_fsm_mode(TOUCH_FSM_MODE_TIMER);
touch_pad_fsm_start();
// END TOUCHPAD
// LED STRIP INIT
led_strip_config_t strip_config = {
.strip_gpio_num = 3,
.max_leds = NUM_OF_LEDS, // at least one LED on board
};
led_strip_spi_config_t spi_config = {
.spi_bus = SPI2_HOST,
.flags.with_dma = true,
};
ESP_ERROR_CHECK(
led_strip_new_spi_device(&strip_config, &spi_config, &led_strip));
led_strip_clear(led_strip);
// END LED STRIP
return 1;
}
/**
* @brief CDC device RX callback
*
* CDC device signals, that new data were received
*
* @param[in] itf CDC device index
* @param[in] event CDC event type
*/
void tinyusb_cdc_rx_callback(int itf, cdcacm_event_t *event) {
/* initialization */
size_t rx_size = 0;
/* read */
esp_err_t ret =
tinyusb_cdcacm_read(itf, rx_buf, CONFIG_TINYUSB_CDC_RX_BUFSIZE, &rx_size);
if (ret == ESP_OK) {
app_message_t tx_msg = {
.buf_len = rx_size,
.itf = itf,
};
memcpy(tx_msg.buf, rx_buf, rx_size);
xQueueSend(app_queue, &tx_msg, 0);
} else {
ESP_LOGE(TAG, "Read Error");
}
}
/**
* @brief CDC device line change callback
*
* CDC device signals, that the DTR, RTS states changed
*
* @param[in] itf CDC device index
* @param[in] event CDC event type
*/
void tinyusb_cdc_line_state_changed_callback(int itf, cdcacm_event_t *event) {
int dtr = event->line_state_changed_data.dtr;
int rts = event->line_state_changed_data.rts;
ESP_LOGI(TAG, "Line state changed on channel %d: DTR:%d, RTS:%d", itf, dtr,
rts);
}
void app_main(void) {
// Create FreeRTOS primitives
app_queue = xQueueCreate(5, sizeof(app_message_t));
assert(app_queue);
app_message_t msg;
ESP_LOGI(TAG, "USB initialization");
const tinyusb_config_t tusb_cfg = {
.device_descriptor = NULL,
.string_descriptor = NULL,
.external_phy = false,
#if (TUD_OPT_HIGH_SPEED)
.fs_configuration_descriptor = NULL,
.hs_configuration_descriptor = NULL,
.qualifier_descriptor = NULL,
#else
.configuration_descriptor = NULL,
#endif // TUD_OPT_HIGH_SPEED
};
ESP_ERROR_CHECK(tinyusb_driver_install(&tusb_cfg));
tinyusb_config_cdcacm_t acm_cfg = {
.usb_dev = TINYUSB_USBDEV_0,
.cdc_port = TINYUSB_CDC_ACM_0,
.rx_unread_buf_sz = 64,
.callback_rx = NULL,
.callback_rx =
&tinyusb_cdc_rx_callback, // the first way to register a callback
.callback_rx_wanted_char = NULL,
.callback_line_state_changed = NULL,
.callback_line_coding_changed = NULL};
ESP_ERROR_CHECK(tusb_cdc_acm_init(&acm_cfg));
/* the second way to register a callback */
ESP_ERROR_CHECK(tinyusb_cdcacm_register_callback(
TINYUSB_CDC_ACM_0, CDC_EVENT_LINE_STATE_CHANGED,
&tinyusb_cdc_line_state_changed_callback));
#if (CONFIG_TINYUSB_CDC_COUNT > 1)
acm_cfg.cdc_port = TINYUSB_CDC_ACM_1;
ESP_ERROR_CHECK(tusb_cdc_acm_init(&acm_cfg));
// ESP_ERROR_CHECK(tinyusb_cdcacm_register_callback(
// TINYUSB_CDC_ACM_1, CDC_EVENT_LINE_STATE_CHANGED,
// &tinyusb_cdc_line_state_changed_callback));
#endif
// esp_tusb_init_console(TINYUSB_CDC_ACM_0);
ESP_LOGI(TAG, "USB initialization DONE");
init_device();
union Touch {
uint32_t touch[NUM_OF_TOUCHPADS];
uint8_t raw[4 * NUM_OF_TOUCHPADS];
};
while (1) {
union Touch touch_data = {.touch = {0, 0}};
uint32_t sum = 0;
for (int i = 0; i < NUM_OF_TOUCHPADS; i++) {
ESP_ERROR_CHECK(touch_pad_read_raw_data(button[i], &touch_data.touch[i]));
sum += touch_data.touch[i];
// ESP_LOGI(TAG, "Touch Data: %d: %d", i, touch_data.touch[i]);
}
if (sum > NUM_OF_TOUCHPADS * TOUCH_DEFAULT_LEVEL) {
int bytes =
tinyusb_cdcacm_write_queue(0, touch_data.raw, sizeof(union Touch));
ESP_LOGI(TAG, "%d bytes queued!", bytes);
esp_err_t err = tinyusb_cdcacm_write_flush(0, 50);
if (err != ESP_OK) {
ESP_LOGE(TAG, "CDC ACM write flush error: %s", esp_err_to_name(err));
}
}
if (xQueueReceive(app_queue, &msg, 100 / portTICK_PERIOD_MS)) {
if (msg.buf_len) {
union Leds leds[NUM_OF_LEDS] = {};
/* Print received data*/
ESP_LOGI(TAG, "Data from channel %d:", msg.itf);
ESP_LOG_BUFFER_HEXDUMP(TAG, msg.buf, msg.buf_len, ESP_LOG_INFO);
led_strip_set_pixel(led_strip, msg.buf[0], msg.buf[1], msg.buf[2],
msg.buf[3]);
led_strip_refresh(led_strip);
}
}
}
}
Here I had some problems first when sending because it waited at the QUEUE and I needed to change to blocking time: 100 / portTICK_PERIOD_MS which reduced the blocking time so it is interactive.
16.3.2. USB Rust Setup¶
As a first Approach I tried the library nusb And got the code working to find my device, but then I couldn’t find out, why I can’t send data via serial…
println!("Gathering USB Devices...");
let mut devices = list_devices().expect("Error Receiving Devices!");
for device in devices {
// dbg!(device.product_string());
println!("ID: {}:{}", device.vendor_id(), device.product_id());
}
let esp_devices = devices.find(|device| device.vendor_id() == 12346);
let keyboard: nusb::DeviceInfo = match esp_devices {
Some(k) => {
println!(
"Found keyboard: \n product_string: {} \n ID: {}:{}",
k.product_string().expect("No Product String avaibible"),
k.vendor_id(),
k.product_id()
);
k
}
None => {
println!("No keyboard found...");
std::process::exit(-1);
}
};
let keyboard_dev = keyboard.open()?;
let kb_interface = keyboard_dev.detach_and_claim_interface(0)?;
loop {
let buf = RequestBuffer::new(512);
let received = block_on(kb_interface.bulk_in(0x80, buf)).into_result();
dbg!(received);
}
… but I noticed, that this library is not used for CDC-ACM devices but for all other USB devices (or maybe there was also an option to use it but I couldn’t find it ._.).
Then I tried an explicit Serial Library: serialport which worked flawlessly (after learning, how to use ;] ).
To find (currently only the first) USB Device, I used:
use serialport::{available_ports, SerialPort, SerialPortType};
fn main() {
let devices = available_ports().expect("Unable to gather Serial Ports");
let keyboard_dev = devices
.iter()
.find(|&device| match device.port_type.clone() {
SerialPortType::UsbPort(usb) => {
println!("Found USB serial device '{}'", device.port_name);
dbg!(usb);
true
}
_ => false,
}).expect("No USB Serial Device found!");
let mut keyboard = serialport::new(&keyboard_dev.port_name, 115200).open()?;
}
Note
To find out the Baudrate (here 115200) I looked in the sdkconfig- file for ‘BAUD’ and found some parameter like: “CONFIG_CONSOLE_UART_BAUDRATE=115200”.
Then when its open, I could read the values and also write some back:
let id: i8: 0;
let id_prev: i8: 0;
loop {
let mut buf: [u8; 4 * NUM_OF_KEYS] = [0, 0, 0, 0, 0, 0, 0, 0];
match keyboard.read(&mut buf) {
Ok(l) => {
println!("Read {l} bytes:");
dbg!(buf);
let touch_keys: [u32; NUM_OF_KEYS] = [
u32::from_ne_bytes(buf[..4].try_into().expect("Could not convert to u32")),
u32::from_ne_bytes(buf[4..8].try_into().expect("Could not convert to u32")),
];
println!("Actual touch values: {}; {}", touch_keys[0], touch_keys[1]);
}
Err(_) => {}
};
let led_on: [u8; 4] = [id, 0, 255, 255];
let led_off: [u8; 4] = [id_prev, 0, 0, 0];
match keyboard.write(&led_on) {
Ok(l) => println!("Wrote {l} bytes to turn ON {id}!"),
Err(_) => {}
};
keyboard.flush();
match keyboard.write(&led_off) {
Ok(l) => println!("Wrote {l} bytes to turn OFF {id_prev}!"),
Err(_) => {}
};
keyboard.flush();
id_prev = id;
id = (id + 1) % 6;
sleep(time::Duration::from_millis(100));
}
This loop continuously reads the key presses and writes new values for the LEDs to switch around.
16.3.3. Rust Slint UI¶
A simple ui:
And ths DSL - Slint code
import { Button, VerticalBox, HorizontalBox, ComboBox, Slider } from "std-widgets.slint";
component LedsGrid inherits GridLayout {
property <int> led_count: 6;
property <int> board_count: 2;
min-height: 100px;
callback toggle_led(id: int, state: bool);
Button {
text: self.state ? "LED 0 ON" : "LED 0 OFF";
row: 0;
col: 0;
property <int> id: 0;
property <bool> state: false;
clicked => {
self.state = !self.state;
root.toggle_led(self.id, self.state);
}
}
Button {
text: self.state ? "LED 1 ON" : "LED 1 OFF";
row: 0;
col: 1;
property <int> id: 1;
property <bool> state: false;
clicked => {
self.state = !self.state;
root.toggle_led(self.id, self.state);
}
}
Button {
text: self.state ? "LED 2 ON" : "LED 2 OFF";
row: 1;
col: 0;
property <int> id: 2;
property <bool> state: false;
clicked => {
self.state = !self.state;
root.toggle_led(self.id, self.state);
}
}
Button {
text: self.state ? "LED 3 ON" : "LED 3 OFF";
row: 1;
col: 1;
property <int> id: 3;
property <bool> state: false;
clicked => {
self.state = !self.state;
root.toggle_led(self.id, self.state);
}
}
Button {
text: self.state ? "LED 4 ON" : "LED 4 OFF";
row: 2;
col: 0;
property <int> id: 4;
property <bool> state: false;
clicked => {
self.state = !self.state;
root.toggle_led(self.id, self.state);
}
}
Button {
text: self.state ? "LED 5 ON" : "LED 5 OFF";
row: 2;
col: 1;
property <int> id: 5;
property <bool> state: false;
clicked => {
self.state = !self.state;
root.toggle_led(self.id, self.state);
}
}
}
component Piano inherits HorizontalBox {
min-height: 50px;
in property <bool> key0: false;
in property <bool> key1: false;
Rectangle {
background: key0 ? green : black;
border-color: white;
border-width: 1px;
}
Rectangle {
background: key1 ? green : black;
border-color: white;
border-width: 1px;
}
}
export component AppWindow inherits Window {
in-out property <int> counter: 42;
in property <bool> key0: false;
in property <bool> key1: false;
min-height: 500px;
callback request-increase-value();
callback toggle_led <=> led_grid.toggle_led;
VerticalBox {
HorizontalBox {
Text {
text: "USB Device:";
}
ComboBox {
model: ["/dev/ttyACM0"];
width: 50%;
}
}
Text {
text: "LEDs:";
}
led_grid := LedsGrid { }
Text {
text: "Piano Keys";
}
piano := Piano {
key0 <=> key0;
key1 <=> key1;
}
}
}
And here the integration with rust:
fn start_ui(mut keyboard: Box<dyn SerialPort>) {
let ui = AppWindow::new().expect("Couldn't create a window!");
ui.on_request_increase_value({
let ui_handle = ui.as_weak();
move || {
let ui = ui_handle.unwrap();
ui.set_counter(ui.get_counter() + 1);
}
});
ui.on_toggle_led(move |id, state| {
println!("Toggle LED!");
println!("ID: {id}");
let buf = if state {
[
id.try_into().expect("Could not convert i32 to u8 for id!"),
255,
255,
255,
]
} else {
[
id.try_into().expect("Could not convert i32 to u8 for id!"),
0,
0,
0,
]
};
match keyboard.write(&buf) {
Ok(l) => println!("Wrote {l} bytes to turn ON {id}!"),
Err(_) => println!("Error while writing to keyboard!"),
};
keyboard.flush();
});
ui.run().expect("Error running UI!");
}
Unfortunately I didn’t manged to light up the Keys, only output on command line. I had some rust-language-problems X)