16. Interface and application programming

Ok, so in this week it’s about interfacing with the board from a computer or a phone. As other weeks I’m going to start very basic and from there I’ll be work more and more.

In these weeks I had problem with my cable and my boards so the comunication in serial with them has been very tricky (to upload the programs I need 15 minutes of trial and error and that’s very annoying) and I couldn’t establish a good serial connection to debug so, since this is a serial connection week, I have to do something with it!

Make the trex run with my button

The first thing I want to try (since they talk a lot of it) it’s to make a simple game.

In this case I’m going to use an existing game (the trex runner of chrome) and control it by my nodeMCU with a button using a serial communication.

In this case the game is already set in python using game.py so let’s try to use it.

The game is uploaded HERE and you can download the package and play with the .exe if you’re using windows.

screenshot of the project, I use windows 10 so it looks different

Make the sktech for the nodeMCU

While I reuploaded the sketch I found an error in my platformio because the serial monitor stops my laptop (even if it’s not the fastest laptop it shouldn’t get stuck by a serial monitor) so I tried the serial monitor or arduino IDE and that works well.

The sketch is the following. It’s the button example but I added that if you press the led changes situation.

#include <Arduino.h>

// the setup function runs once when you press reset or power the board
void setup() {
  // initialize digital pin LED_BUILTIN as an output.
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(10, INPUT);
  Serial.begin(9600);
}

// the loop function runs over and over again forever
void loop() {

  // read the input pin:
int buttonState = digitalRead(D8);
// print out the state of the button:

Serial.println(buttonState);
delay(1);        // delay in between reads for stability
//pin low or in depending on the button state.
if(buttonState==0)
{
  digitalWrite(LED_BUILTIN, LOW);
}
else
{
  digitalWrite(LED_BUILTIN, HIGH);
}
}

This produces an 0 if it’s not pressed and a 1 if it’s pressed.

And it works. Not a very big deal of a program, but with the barduino problems I have this is compelling.

The circuit follows this schema:

It’s still easier and faster to draw by hand and take a photo than try o make a digital drawing

Thankfully, there is an internal pull-up resistor (as the datasheet says) you can reduce the circuit to this. And since I have very limited number of cables and jumpers this is very convenient!

Yeah, I made this drawing first and then the other one. I called optimization of resources

But I still have struggles to pick the name of the pin. It’s something like the curse of the usb that you can’t never fit it properly.

something like this but guessing if the pin is 10 or 8 or A3

But finally this is what I get. Some 0 and 1 in the serial monitor

Making the game work from the source

I guess that this is obvious for people that are used to coding as to breathe but for me it’s not so simple so what I’m trying to do it’s to make the game run by the very python control so I need to create an enviroment where I can call game.py

The libraries that the author is using are several and I’m using conda so I can create a new enviroment

At the first of the python file I can see the import section.

import os
import sys
import pygame
import random
from pygame import *

When I run it it goes up to pygame that it’s not recognized. So I guess that os, sys are default packages in my Python 3.x installation

So what I’m going to do it’s to create a separate folder for this project so I can manipulate. I will keep the asset folder because I know I’ll need the sounds and the sprites.

Then I’m going to use cobra en that folder with this command

conda create -n trex pygame

It gives me an error so I’ll try to solve it

PackagesNotFoundError: The following packages are not available from current channels:

  - pygame

Basically it doesn’t find that package.

So it seems that Anaconda doesn’t install it for you, so I’ll install it via pip (that’s a extension manager for Python) using this command

pip install pygame

After that it doesn’t make the enviroment either, with the same error = (

But if I try to play the python it works. So at least I have the program running.

Understanding pygame of this game.

Well, when I tried this it gives me a lot of warning because there are some stuff deprecated. Specially some if that use “is” looking for a number and some methods that they ask for a integer and they recieve a float. So I’ll cast some of those errors to keep the warnings away even if the code is less clean.

Then the structure is, the following:

def main():
    isGameQuit = introscreen()
    if not isGameQuit:
        gameplay()

main()

So first works with the introscreen and then the gameplay. I look into the code and I see what it’s happening when you press space (The primary control that I want to substitute with the button of my board).

if event.type == pygame.KEYDOWN:
    if event.key == pygame.K_SPACE:
        if playerDino.rect.bottom == int(0.98*height):
            playerDino.isJumping = True
            if pygame.mixer.get_init() != None:
                jump_sound.play()
            playerDino.movement[1] = -1*playerDino.jumpSpeed

In this part I realize that this game works with events. And what I want to substitute it’s the event of the type KEYDOWN (that it’s a constant) and the key pygame.K_SPACE (that I found that it’s another constant)

Now to work on the serial connection I’d need two functions. One to open the communication (run once) and the other one to read the serial (0 for unpressed, 1 for pressed) and call an event if the button is pressed (the line in serial is 1).

I think it should be a delay of what’s happening but first let’s try if we can handle it and then, debuggit.

I found a “clock.tick(FPS)” line that maybe I can handle it

Ok, so I figure that I can open the port and then use read line? I try to follow the serial.py documentation but I’m not understanding it very well. So let’s figure out any examples I can find.

Ok, let’s try to use one of the simple python programs of Neil for input devices.

Here in the pyroelectric example we can find an example and the video.

There I can see these snippets. Here for starting the communication.

port = "COM7"
baudrate = 4800
#
# open serial port
#
ser = serial.Serial(port,baudrate)
ser.setDTR()
ser.flush()

I also realize that I want this to run before arriving to main so the object Serial is available. I’m not sure if I could have encapsulated it (I come from Java) but I thought that I’d need the Serial object to be accesible.

Ok, I can see things but they are not the same I recieve the on the arduino IDE serial monitor. So time to dig up.

Let’s see the difference between using Serial.println, Serial.write and serial.println

Serial.prntln doesn’t seem to work, serial.print seems that there is something weird. And also it depends on the casting of the casting.

No, I still can’t see anything different from 0 on python. Uhm. Maybe it’s a python problem?

Let’s try again.

Ok, so there seems that serial.py when you use serial.read() it gives you a byte, not a string. So I have to decode it as it says in stack overflow here.

Ok, now I have a String with this line.

readingFromThePort = ser.read(3).decode(encoding).strip()

encoding was UTF-8 defined as a global variable

So I take the server, I read the 3 last bytes. I decode them to convert them using UTF-8 and now I have a string. Then I strip it so I left the /n and the /r that I may have.

Then I compare to 1 so I know if I have to jump. And it works. With lots of delay.

By now the only thing it happens is that everytime that I check serial, I print what I recieve and print “jump”.

It does that but now I have to handle when does it happen. Let me try a different baud rate because maybe the fps it’s too fast.

I increased the baud rate from 9800 to 115200.

But the queue everytime I check get’s increased by 45-49 bytes. I know it by using the propierty in_waiting that it’s somehting that I can attach to my object Serial (ser) to get how much do they have to read yet.

When I reduce the baudrate to 4800 I get less.

But finally I realize that there is a handy solution that it’s, since the port it’s open always I can delete the buffer so I can delete the old values and have fresh ones. reset_input_buffer()

So I finally come up with this.

def checkSerial():
    #first read the port. 3 bytes of fun because there is that /n before and after.
    readingFromThePort = ser.read(3)
    #I have to make a try except because sometimes I can't decode properly the bytes. I also strip it so I only get the 0 or the 1
    try:
        strfromtheport = readingFromThePort.decode(encoding).strip()
    except UnicodeDecodeError:
        print("checkSerial: error decoding")
        return
    print("this is the string from the port")
    print(strfromtheport)
    print("this is the waiting buffer")
    print(ser.in_waiting)
    #here I check if I have to reset the input because there is too much inputs for this program.
    if (ser.in_waiting > 15):
        ser.reset_input_buffer()
    #finally if everything goes well, print "jump"
    if (strfromtheport == "1"):
        print("jump")
        #this is the jump

And that works. Well, that works on the console. Now I have to post an event of jumping. For that the first solution will be to post a Keypress event or create a custom event.

I’ll try the first because the program reads already keypress spacebar events.

Maybe later I’ll have to check the list if there is another event of pressing the spacebar so I don’t duplicate it.

–

Ok I finally made it!

The thing is, you have to create an event and post it on the queue of the event line and… don’t substract it. Because I was using “get” to debug, I was getting out of the list of pending events!

So finally the added things in the 500 lines of code are the following.

At the first part of the code I added this to set and open the port:

port = "COM7"
baudrate = 4800
#set encoding for the byte transmision
encoding = 'utf-8'
print("Using port" + port)
#
# open serial port
#
ser = serial.Serial(port,baudrate)
ser.setDTR()
ser.flush()

This is mostly copied from Neil files and I added the encoding variable to have it when I decode the bytes I read.

And then I make a checkSerial function like this.

def checkSerial():
    readingFromThePort = ser.read(3)
    try:
        strfromtheport = readingFromThePort.decode(encoding).strip()
    except UnicodeDecodeError:
        print("checkSerial: error decoding")
        return
    if (ser.in_waiting > 15):
        ser.reset_input_buffer()
    if (strfromtheport == "1" and not pygame.event.peek(pygame.KEYDOWN, False)):
        pressSpaceBecauseIPressedTheButton = pygame.event.Event(pygame.KEYDOWN, {'unicode': ' ', 'key': 32, 'mod': 0, 'scancode': 57, 'window': None})
        pygame.event.post(pressSpaceBecauseIPressedTheButton)

When this is called, it reads the last 3 bytes, try to decode them (sometimes it can, sometimes it gives the error)

If there is too much bytes waiting to be read, they are reset.

Lastly if the string is a “1” and there is not any other event of KEYDOWN (pressing the space bar) it creates an event of the type KEYDOWN with the key 32.(pygame.K_SPACE). I obtained that info via catching a event object of that type and printing it, because if not I coudln’t guess the details of the dictionary it needs.

Lastly I put the calls of this checkserial in several parts, when it’s needed to check if you need to press the button.

First in introscreen() in while not gameStart. In def gameplay(): in a while while not gameOver. And lastly also in gameplay(); in a while gameOver

Hero shot

I got up to 306 hi score using only the botton

What I’ve learnt from this

• Pygame and pyserial has dificult to read documentation in comparasion with other documentations I’ve seen. I didn’t have any example of how to create an existing event.

• How bytes work and exception works in Python.

• How objects can be more difficult to creat than I first thought and be careful with event queues.

How could it be improved.

• There is a button to jump but there is also a button to duck. I could try to make something else to make that ducking.
• The encoding doesn’t work properly and I don’t know how to improve it.
• Ther could be a menu to set the COM and the baudrate before starting the game. By now it’s just hardcoded.
• I still have problems with the conda enviroments.
• I still have problems with the pin naming.

Source file

Remember to use download link as.. to download the files.

Trex-runner-serial.py

Group assigment

Now we have to compare with other students how was our interfacing. Let’s check on them, have a chat and learn from each other!

Useful links

• Source of trex
• How to make an exe file from a .py
• Pygame documentation
• Pyserial documentation
• Manage enviroments with Conda
• W3schools for reference in python
• Libraries for ArduinoIDE