Interface & Applications

Group Assignment

Individual Assignments

WebSerial

WebSerial is a browser-based API that allows direct communication between web applications and serial devices, such as development boards. By leveraging WebSerial, developers can send and receive data over a serial connection without requiring additional software installations. This is particularly useful for debugging, configuring, or interacting with microcontrollers like the SAMD21 or ESP32. To use WebSerial, ensure your browser supports the API, and establish a connection by selecting the appropriate serial port. Once connected, you can exchange data in real-time, enabling seamless integration between hardware and web-based interfaces.

Open a serial port

The Web Serial API is asynchronous by design. This prevents the website UI from blocking when awaiting input, which is important because serial data can be received at any time, requiring a way to listen to it.

To open a serial port, you can use the following code:

async function connecttoSerial(){  //connecttoSerial is a use defined async function that opens a serial port.
    // Prompt user to select any serial port.
    const port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600}); //open the port with a baud rate of 9600.
}

Getting writer and reader

writer = port.writable.getWriter();  //allows you to send data to the serial device
reader = port.readable.getReader(); //allows you to read incoming data

Read incmoming data

//create a new instance of the TextDecoder class to decode incoming data.
const decoder = new TextDecoder(); 

//textDecoder is a built-in JavaScript class that decodes text from a stream of bytes(Uint8Array).

while(true) {
    //read and store incoming data(bytes) from the serial port into value
    const { value, done } = await reader.read(); 
    if (done) break;

    //decoding stored data(bytes) into a string.
    const string = decoder.decode(value);

    //append the decoded string to the outputMsg textarea.
    document.getElementById('outputMsg').textContent += string; 
}

Sending Serial Data

async function sendMessage() {
    //get the value of the input field with id 'input'.
    //store it in a variable called input.
    
    const input = document.getElementById('input').value;
    if (!writer) return;

    //create a new instance of the TextEncoder class to encode the input string into bytes.
    const encoder = new TextEncoder();

    //write the encoded input string to the serial port using the writer object.
    await writer.write(encoder.encode(input + '\n'));

    //clear the input field after sending the message.
    document.getElementById('input').value = ""
}

WebSockets

WebSockets is a communication protocol that enables full-duplex communication channels over a single TCP connection. It is designed to work over the web and allows for real-time data exchange between a client (usually a web browser) and a server. Unlike traditional HTTP requests, which are request-response based, WebSockets maintain an open connection, allowing both the client and server to send messages independently. This makes WebSockets ideal for applications that require low latency and real-time updates, such as chat applications, online gaming, and live data feeds.

Spotify API - ESP32

The ESP32 is a powerful microcontroller with built-in Wi-Fi and Bluetooth capabilities, making it ideal for IoT applications. By integrating the Spotify API with the ESP32, you can create a project that fetches and displays real-time song data, such as the currently playing track, artist, and album art. This can be achieved by using the Spotify Web API to authenticate and retrieve data, and then displaying the information on an OLED screen or sending it to a connected device. The project involves setting up OAuth2 for Spotify API access, using the ESP32 to make HTTP requests, and parsing the JSON responses to extract the desired information. This integration showcases the potential of combining hardware and cloud-based services to create interactive and dynamic applications.

Spotify Web API

The Spotify Web API is a powerful tool that allows developers to access and interact with Spotify's vast music catalog and user data. It provides endpoints for retrieving information about tracks, albums, artists, playlists, and user profiles. The API also supports features like searching for music, managing playlists, and controlling playback on Spotify clients. To use the Spotify Web API, developers need to register their application on the Spotify Developer Dashboard, obtain an access token through OAuth2 authentication, and make HTTP requests to the API endpoints. The API returns data in JSON format, which can be easily parsed and utilized in various applications, such as music discovery apps, personalized playlists, and more.

To use the Spotify API, you need to create an application on the Spotify Developer Dashboard. This will provide you with the necessary credentials (Client ID and Client Secret) to authenticate your requests. Once you have your credentials, you can use OAuth2 to obtain an access token, which is required for making API calls. The access token is typically valid for a limited time, so you'll need to refresh it periodically.

I am using a refresh token which allows the program to retrieve a new access token whenever the token expires.

You need to note your API KEY and SECRET for futher development

Connecting ESP32 with Spotiy API

To connect the ESP32 with the Spotify API, you can use the HTTPClient library to make GET and POST requests. The library allows you to send requests to the Spotify API endpoints and receive responses in JSON format. You can then parse the JSON data to extract relevant information, such as track names, artist names, and album art URLs. This data can be displayed on an OLED screen or sent to a connected device for further processing.

For this week I have used a library called Spotify_Esp32

To use the library, you need to include it in your Arduino sketch and initialize it with your Spotify credentials. The library provides functions to authenticate with the Spotify API, fetch track information, and control playback on connected devices.

Here is a simple example of how to use the library to fetch the currently playing track:

/*
    An example of how to authenticate with Spotify without using a refresh token and print Artist and Track via Serial

In this example your current track will be printed to the serial and as soon as you listen to a new track that tracks information will be printed.

    15.03.2024
    Created by: Finian Landes

    16.03.2024
    edited by: Sascha Seidel
        * added getting artist and trackname to print it Serial

    Documentation: https://github.com/FinianLandes/Spotify_Esp32
*/
// Include the required libraries
#include < Arduino.h> 
#include < WiFi.h> 
#include < SpotifyEsp32.h> 

char* SSID = "YOUR WIFI SSID";
const char* PASSWORD = "YOUR WIFI PASSWORD";
const char* CLIENT_ID = "YOUR CLIENT ID FROM THE SPOTIFY DASHBOARD";
const char* CLIENT_SECRET = "YOUR CLIENT SECRET FROM THE SPOTIFY DASHBOARD";
const char* REFRESH_TOKEN = "YOUR REFRESH TOKEN";

Spotify sp(CLIENT_ID, CLIENT_SECRET);

void setup() {
    Serial.begin(115200);
    connect_to_wifi();
    
    sp.begin();
    while(!sp.is_auth()){
        sp.handle_client();
    }
    Serial.println("Authenticated");
}

void loop() {
    static String lastArtist;
    static String lastTrackname;
    
    String currentArtist = sp.current_artist_names();
    String currentTrackname = sp.current_track_name();
    
    if (lastArtist != currentArtist && currentArtist != "Something went wrong" && !currentArtist.isEmpty()) {
        lastArtist = currentArtist;
        Serial.println("Artist: " + lastArtist);
    }
    
    if (lastTrackname != currentTrackname && currentTrackname != "Something went wrong" && currentTrackname != "null") {
        lastTrackname = currentTrackname;
        Serial.println("Track: " + lastTrackname);
    }
}

void connect_to_wifi(){
    WiFi.begin(SSID, PASSWORD);
    Serial.print("Connecting to WiFi...");
    while (WiFi.status() != WL_CONNECTED) {
        delay(1000);
        Serial.print(".");
    }
    Serial.printf("\nConnected to WiFi\n");
}

In this example, the ESP32 connects to Wi-Fi, authenticates with the Spotify API using the provided credentials, and continuously checks for the currently playing track. When a new track is detected, it prints the artist and track name to the serial monitor.

For my use case I wanted to use basically one function of the Web API - Start/Resume Playback

In this project, Flutter, the ESP32 microcontroller, and the Spotify Web API communicate using HTTP, a widely used protocol for data exchange over networks.

On the Flutter app side, HTTP POST requests are sent using the Dart http package. When the user provides a Spotify playlist or track URI, the app constructs an HTTP POST request targeting the ESP32s local IP address (e.g., http://192.168.xx.xx/uri). This request contains headers like Content-Type: application/x-www-form-urlencoded and a body with the required data (such as the playlist URI).

The ESP32, running a local web server, listens for these incoming requests on predefined routes (like /uri, /play, /pause). When a request arrives, it parses the data, acts on it, and sends back an HTTP response — typically including a status code (200 OK) and an optional message, confirming that the request was successfully received or processed.

Separately, the Spotify Web API uses HTTP over the internet (specifically HTTPS) to handle actions like playing, pausing, or fetching playlists. These requests must include an OAuth 2.0 Bearer token for authentication and are typically sent with JSON-formatted payloads. For example, sending a POST request to https://api.spotify.com/v1/me/player/play requires both proper headers and a correctly formatted body.

In summary, this project bridges local network communication (Flutter ↔ ESP32) with external cloud API services (Spotify Web API), both using standardized HTTP methods, but under very different security, authentication, and data format requirements.

Flutter and Android Studio

Flutter is an open-source UI software development toolkit created by Google. It is used to develop applications for Android, iOS, Linux, macOS, Windows, and the web from a single codebase. Flutter allows developers to create visually appealing and high-performance applications using a reactive programming model.

Android Studio is the official integrated development environment (IDE) for Google's Android operating system. It is based on JetBrains' IntelliJ IDEA software and is designed specifically for Android development. Android Studio provides a comprehensive set of tools for building, testing, and debugging Android applications, including a code editor, layout editor, and emulator.

In this week I have used Flutter and Android Studio to create a simple app that allows the user to send track, album or playlist links to the ESP32. The app uses the WebSerial API to establish a connection with the ESP32 and send data over the serial port.

Postman to ESP32

I had to test the http request communication to ESP32, so I used postman to communicate a POST request

Postman is a popular API development and testing tool that allows developers to send requests to APIs and view responses. It provides a user-friendly interface for creating and managing API requests, making it easier to test and debug APIs during development.

#include 
#include <WebServer.h>

const char* ssid = "SSID";
const char* password = "PASSWORD";

WebServer server(80);

void handleUriPost() {
	if (server.hasArg("uri")) {
	String uri = server.arg("uri");
	Serial.println("Received URI: " + uri);

	server.send(200, "text/plain", "URI received: " + uri);
	} else {
	server.send(400, "text/plain", "Missing 'uri' parameter");
	}
}

void setup() {
	Serial.begin(115200);
	WiFi.begin(ssid, password);
	Serial.print("Connecting to WiFi...");

	while (WiFi.status() != WL_CONNECTED) {
	delay(500);
	Serial.print(".");
	}

	Serial.println("\nConnected! IP address: ");
	Serial.println(WiFi.localIP());

	server.on("/uri", HTTP_POST, handleUriPost);
	server.begin();
	Serial.println("HTTP server started");
}

void loop() {
	server.handleClient();
}
	

Flutter App to ESP32

To create a Flutter app that communicates with the ESP32, you need to set up a Flutter project in Android Studio. Once the project is created, you can use the http package to access the WebSerial API and establish a connection with the ESP32. The app should have a simple user interface that allows users to input track, album, or playlist links and send them to the ESP32.

In the app, you can use a TextField widget to allow users to enter the URI and a button to send the URI to the ESP32. The app should also handle the response from the ESP32 and display it to the user.

The Serial Monitor when I send the Spotify link to the ESP32 from the App

Writing the URI to the NFC Card

Detecting the URI and sending it to Spotify Web API

MVP App

This app allows to send track, album or playlist links to the ESP32 and the esp32 sends a put request to Spotify Web API

In this app, I have used a simple text field to enter the URI and a button to send the URI to the ESP32. The app uses the http package to send a POST request to the ESP32 with the URI as a parameter.

Receiving Shared Text

The app has the ability to receive shared urls from Spotify

For receiving url from other apps I used this package

To use the package, you need to add it to your pubspec.yaml file and configure your AndroidManifest.xml file. The package provides a simple API to receive shared text and files from other apps.

Things you need to add in pubspec.yaml for making http request and to receive shared content

dependencies:
	flutter:
	  sdk: flutter
	http: ^0.13.6  #for http communication
	receive_sharing_intent_plus: ^1.0.1 #for reveiving shared content

Things you need to add in AndroidManifest.xml for making http request and to receive shared content

	<manifest xmlns:android="http://schemas.android.com/apk/res/android">
	
	<uses-permission android:name="android.permission.INTERNET"/> 
	<application....>
	.....
	</application>

In Android, apps appear in the share sheet if they declare an intent-filter that listens for shared content. Add this in the activity section of AndroidManifest.xml. There might be an existing intent-filter for launching the app ,it tells I'm a normal launchable app to android

<activity android:name=".MainActivity">
		<intent-filter>	
			<action android:name="android.intent.action.SEND"/>
			<category android:name="android.intent.category.DEFAULT"/>
			<data android:mimeType="text/plain">
			</intent-filter>

In the above code, we are telling android that this app can handle text/plain data type. You can also add other data types like image/*, audio/* etc.

You have to add this in the main.dart file

class _SpotifyInputScreenState extends State&SpotifyInputScreen> {
	String _sharedLink = '';
	bool _userEdited = false; // Flag to check if user has manually typed
	final TextEditingController _controller = TextEditingController();
	String _parsedUri = '';
	String _statusMessage = '';
  
	final esp32Ip = '192.168.33.175'; // Replace with your ESP32's IP address
  
	StreamSubscription<String>? _intentDataStreamSubscription;
  
	@override
	void initState() {
	  super.initState();
  
	  // Listen for incoming shared text while app is in memory
	  _intentDataStreamSubscription =
		  ReceiveSharingIntentPlus.getTextStream().listen((String value) {
			if (value.isNotEmpty && !_userEdited) {
			  setState(() {
				_sharedLink = value.trim();
				_controller.text = _sharedLink;
			  });
			}
		  }, onError: (err) {
			print("Error receiving shared text: $err");
		  });
  
	  // Get shared text if app was launched via sharing
	  ReceiveSharingIntentPlus.getInitialText().then((String? value) {
		if (value != null && value.isNotEmpty && !_userEdited) {
		  setState(() {
			_sharedLink = value.trim();
			_controller.text = _sharedLink;
		  });
		}
	  });
	}
  
	@override
	void dispose() {
	  _intentDataStreamSubscription?.cancel();
	  super.dispose();
	}
	......Rest of the code.......
	

Now, when you share a link from Spotify, the app will receive the link and you can use it to send the link to the ESP32.

Playback Controls

I wanted to control the playback using the App but I couldn't communicate the App directly with Spotify, so I communicated with the ESP32 to send Web API

This is code for the UI part of it, when pressed it sends a command to the ESP32 using the sendCommand function

	Row(
              mainAxisAlignment: MainAxisAlignment.spaceEvenly,
              children: [
                ElevatedButton(
                  onPressed: () => _sendCommand('prev'),
                  child: Icon(Icons.skip_previous),
                ),
                ElevatedButton(
                  onPressed: () => _sendCommand('pause'),
                  child: Icon(Icons.pause),
                ),
                ElevatedButton(
                  onPressed: () => _sendCommand('play'),
                  child: Icon(Icons.play_arrow),
                ),

                ElevatedButton(
                  onPressed: () => _sendCommand('next'),
                  child: Icon(Icons.skip_next),
                ),
              ],
            ),

This is _sendCommand function

void _sendCommand(String command) async {
    setState(() {
      _statusMessage = 'Sending $command...';
    });

    try {
      final response = await http.post(
        Uri.parse('http://$esp32Ip/$command'),
        headers: {'Content-Type': 'application/x-www-form-urlencoded'},
      );

      if (response.statusCode == 200) {
        setState(() {
          _statusMessage = 'ESP32 says: ${response.body}';
        });
      } else {
        setState(() {
          _statusMessage = 'HTTP Error ${response.statusCode}: ${response.body}';
        });
      }
    } catch (e) {
      setState(() {
        _statusMessage = 'Failed to send $command: $e';
      });
    }
  }

App User Interaction

App Final Video

In this video, I have shown the app in action. The app allows the user to send track, album or playlist links to the ESP32. The ESP32 then sends a PUT request to the Spotify Web API to start playback.