17. Applications and Implications

Propose a final project masterpiece that integrates the range of units covered, answering some questions.

Questions:


  • What will it do?
    • The Mexican Sign Language (LSM) translation glove will recognize the hand shapes and movements corresponding to some letters of the alphabet or words in LSM. This data will be processed to interpret the letters and convert them into speech with the sound emitted through a speaker, facilitating communication between deaf individuals and hearing individuals who do not know LSM. The purpose of the project is to give a functional glove that can be trained in every way to recognize different gestures.

  • Who's done what beforehand?
    • BrightSign Glove: They have made a glove that can translate different sign languages into any spoken language and lets the user read or see the other side of the conversation. [1]

    Enable talk: Ukrainian developers, have designed a pair of gloves with 15 flex sensors, an accelerometer, a gyroscope, and a compass to track the movements of the hand, then the data collected is processed and translated into spoken language trough a mobile device. [2]

  • What will you design?
    • From my design it will be the glove with the 4 flex sensors with a voltage divider, in the index finger using a gyro and accelerometer sensor (BNO08x) with I2C communication, all this will be connected to the Xiao NRF52480 which it has the trained model to recognize the gesture. The xiao will send the data trough serial to a laptop, when the laptop receives a value it will send it via Bt to the speaker module; this will have the speaker, a xiao rp2040, H bridge for generating the sound, power bank for power and an SD for reading a voice or a file. (to be implemented).

  • What materials and components will be used and its costs?


    • Material Cost Link for Buying
    Xiao NRF52840 $442.00 Buy Here
    4 Flex Sensors $724.00 Buy Here
    BNO08X $649.00 Buy Here
    Copper Tape $680.00 Buy Here
    Transparency Film $295.00 Buy Here
    Glove $170.00 Buy Here
    10 Magnets (4 x 2mm) $20.00 Buy Here
    Xiao RP2040 $242.00 Buy Here
    Copper Plate $100.00 Buy Here
    Resistors (N/A) $5
    Bluetooth HC-05 $100.00 Buy Here
    Speaker $20.00 Buy Here
    H Bridge (TB67H451AFNG) $26.00 Buy Here
    Power Bank $289.00 Buy Here
    Pin Headers (N/A) $0.00
    Cable (N/A) $0.00
    3D Filament (case) $40.00
    Total $0.00


  • What parts and systems will be made?
    • First the PCB is going to be designed for both modules (glove/speaker). For the glove for a better and comfortable use it will be made from cupper taper and the speaker from a cupper plate. All the components will be solder in place and tested for its correct function. In the case of the speaker, it will be needed to design a 3D printed case for encapsulating all the components with a ring to be easy to carry around, this case must be easy to repair and access with outside screws. For the glove, it will be bought then it will sew the flex sensors in a way that the back is hold and the end is loosen a bit for the flex to slide, for the index finger on the back will have a magnet and a Velcro to slide on the front; finally the main PCB will be attached by magnets to the glove for easy connect.

  • What processes will be used?
    • PCB fabrication
    • 3D printing
    • Tiny ML
    • Programming
    • Sewing
    • Data acquisition
  • What questions need to be answered?
    • How do the sensors in the glove detect hand and finger movements in LSM?

    What algorithms and techniques are used to recognize and classify LSM gestures?

    How is voice synthesized from interpreted alphabet letters?

  • How will it be evaluated?
    • Accuracy test: Conducting multiple tests, each consisting of a set of sign language gestures corresponding to different letters of the alphabet, performed by different participants wearing the gloves. For each test, it should be compared the translated output with the intended meaning of gestures, the anterior, using the following equation:

    Accuracy=(Number of correct translations)/(Total number of translations) ∙ 100%

    User feedback: Gathering feedback from participants in tests about their experience using the gloves, including any challenges encountered or suggestions for improvement.

    Modular: A big change and differences between this glove and the other it the capability to detach the main module from the flex sensors and the BNO08x sensor in case it have to be repaired or washed.

  • References
    • [1] BrightSign Technology Limited, “BrightSign - Translate any sign into any language,” brightsignglove, 2022. Reference_1

    [2] P. Olson, “The Amazing Digital Gloves That Give Voice To The Voiceless,” Forbes, Jan. 15, 2024. Accessed: May 23, 2024. [Online]. Available: Reference_2