System Integration


This week we were tasked with an individual set of activities. For our individual activities this week we were tasked with designing and documenting the system integration for our Final Project.


Week 16

Check List

This week's project requirements:

    Individual Assignment:

    • Design and document the system integration for your final project
    • Make a plan for system integration for your final project
    • Documented your plan with CAD and/or sketches for system integration
    • Implemented methods of packaging
    • Designed final project to look like a finished product
    • Documented system integration of your final project
    • Linked to systems integration documentaiton in your final project page

    Images/Files:

  • Include your original design files
    • Properly compress or use a zip folder if needed
    • Include hero shots of your results

Learning Outcomes

  • Define and apply system integration to your final project

Introduction



My final project combines embedded electronics, interactive sensors, animated displays, laser cutting, pogo style connectors, 3D printing, and physical fabrication to turn a traditional artist style Ball Jointed Doll into an interactive desk companion. This week I focused on making a game plan for finishing my project and planning how all my things will come together to create a functional and emotive friend.

Main Systems in My Final Project



      Physical Systems

      • 3D printed body design
      • Fully strung and articulated doll parts with standing, sitting, and pose holding capabilities
      • Magnetic access panel in the head for easy access to electronics
      • Embedded magnets in hand and feet for standing assistance and swappable hand and feet attachments
      • Laser cut wooden chair with Pogo style connectors for charging and data transfer
      • Embedded magnets in bottom to ensure proper alignment and connection to the chair for charging and data transfer
      • Wiring channels and compartments embedded in the head of the doll


      Electronics Systems

      • ESP32-C6 "Brain" board to receive signals from imput sensors and control/manage doll interactivity
      • ESP32-S3 Emotion/Screen Controller board (Built into screen module) to display animated facial expressions and control the screen
      • Input sensors such as push button and touch sensor to trigger different interactions and emotional responses from the doll
      • Pogo power and data transfer system for easy charging and programming access


      Software Systems

      • Processing IDE emotion and face animation design and project simulation
      • Animation engine that controls different facial expressions and animations based on inputs from the brain board
      • Serial communication between the brain board and the screen controller board to send signals and data back and forth to control the animations and interactivity of the doll
      • Custom wakeword and voice recognition to trigger different interactions and emotional responses from the doll based on voice commands and sounds
      • UI logic to control the different modes and states of the doll based on inputs and interactions


      Communication Systems

      • Wired board to board communication between the brain board and the screen controller board
      • Serial communication between the brain board and a computer for programming and debugging
      • Serial communication between the brain board and a computer to listen for voice commands/wakeup word


      User Interaction Systems

      • Touch interaction using a touch sensor on the top of the doll head to simulate petting the doll and trigger a unique face animation and interaction
      • Custom wakeword to wake up the doll and trigger a unique face animation and interaction
      • Emotional feedback system to provide emotional responses and interactions based on the different inputs and interactions with the doll
      • Charging animation that triggers when the doll is connected to the chair to provide visual confirmaiton of successful connection and charging
      • Push button interaction on left and right side of head to trigger different face animations and interactions depdning on how and when the buttons are pressed


Physical Systems


3D Printed Ball Jointed Body Design

The body is designed based off of traditional ball jointed dolls. This design allows for a wide range of motion and poseability, as well as the ability to stand and sit unassisted.

Doll Pose


Their is an internal channel system designed into the body pieces to allow for elastic string to be threaded through the body. The elastic string holds the joints together, provides tension on the parts, and allows for pose holding. Their are also internal compartments designed into the head of the doll to hold the electronics and wiring in place.

3D view of my Stringing Channels


Internal Stringing Method for Tension and Posing



Ball-Jointed Doll Stringing Method



For Triss's articulation, I used the traditional ball-jointed doll (BJD) stringing method. This system uses two sets of internal 3mm elastic strings to hold the individual body parts together while maintaining tension.

Sketch of my internal Stringing and wiring system


Tensioning and Assembly



As each set of elastic is threaded through the doll, tension is adjusted to keep the joints securely connected while still allowing movement. Once the desired tension is achieved, each string is tied off inside the head.

Using two separate stringing paths allows the arms and legs to move independently while keeping the entire doll assembled. This traditional BJD construction method allows a ball jointed doll the flexibility to sit, stand, and pose unassisted.

Like the traditional method, Triss uses two separate elastic stringing paths. One for the legs and body, and one for the arms

This was an intentional design choice. Since the bottom can be replaced between the two different charging wire methods it was annoying to have to remove the entire arm assembly each time I needed to swap the bum out.

Leg and Body Stringing



The first set of elastic runs through the main body and both legs. Starting at the head, the string travels down through the neck, through the torso, and through the waist.

At the bum, there are two channel holes. One for the right leg, and one for the left leg. The string travels down through the hole for the right leg, down the right leg to the foot, loops through the loop on the right foot, then back up the right leg. It then feeds out the right channel hole in the bum and over to the left channel hole. The string repeats the process down the left leg. After returning back up through the left leg channel hole in the bum, the string continues up the body back through the neck to the head.

Once the string is back in the head, the left and right side of the string is tugged to add additional tension and tied off inside the head.

Arm Stringing



The second set of elastic connects both arms to the upper body. Starting at the neck, the string travels part way into the top of the torso. There is a small hidden channel in the top of the torso. The bottom half of the torso is an open cylinder design to allow viewing into the torso.

I had intentionally designed the torso this way so that in the future I would have the ability to store and have easy access to a battery I can tuck into the torso.

Once in the hidden channel in the top of the torso, the string is pulled out the right side and fed through the right arm to the hand. At the hand, the string feeds through the hand loop and returns back up the right arm. The string travels straight across the hidden torso channel and repeats the same stringing method for the left arm.

Finally, the string is pulled back up through the neck and tied off in the head.

Physical Systems Continued


Their are magnets embedded into the hands and feet of the doll to allow for swappable attachments. Their is a hinge and a magnet set into the front and back of the head of the doll that allows for easy access to the electronics, charging ports, and stringing system. Their are magnets set into the bottom of the doll to align the doll with the chair for charging and data transfer.

Magnetic Hand Features
Head Hinge


Magnetic Bottom underside
Magnetic Bottom top side


Finally, thier is a laser cut wooden chair that astecially hides the charging wires and charging system.

Simple Laser Cut chair
Detailed and wired Laser Cut chair


Electronics Systems


This is my KiCAD schematic for my "Brain" Board which controls the input and output sensors and tells the screenboard what emotions or animations to display.

Brain Board design in KiCAD
Fabricated Brain and Input Board


This is my KiCAD schematic for my Sensor Boards. Each sensor is built onto its own breakout board which can be plugged into the screw terminals on the Brain board. This system allows for future additions to the project or to easily test out, or change, to different sensors and inputs.

Sensor Board design in KiCAD


Electronics Packaging in the Doll Head



One of the biggest considerations when designing Triss was figuring out how to package all of the electronics inside the head while keeping the body articulated and allowing the charging connections to run through the body.

The head was designed with several custom-fitted slots and holders to keep the electronics and sensors securely in place. The buttons and sensors are positioned along the sides of the head, with dedicated holders to prevent them from shifting during use.

Close up of sensors seated in head
Picture of the button on the outside of the head


The brain board has a dedicated slot behind the neck that holds it upright. I intentionally positioned the ESP32's USB-C port at the bottom of the board, facing the neck. This creates a direct connection point for the charging cable as it travels up through the body and reaches the electronics inside the head.

Picture of brain board seated in head
Picture of brain board seated in head


The screen board has its own slot at the bottom of the head, allowing it to sit upright. Once the top-front piece of the head is installed, additional holders keep the screen from moving or wiggling out of position. I also designed screw holes into the head that can accommodate heat-set inserts if additional security is needed. I ultimately did not install the inserts because the existing holders provided enough support, but the option is there if I decide to make adjustments in the future.

To keep the internal wiring organized, the sensor wires are bundled inside black wire-wrap sleeves.

Each button sensor has three available output wires, although only two are needed for the connections. Having the additional wire allows the same sensor design to be positioned on either side of the head without needing to cross wires when connecting to the terminal boards.

Button sensor board


The unused wire is simply left disconnected and tucked into the wire-wrap sleeve.

The head itself is constructed from three main pieces: the main-front, the bottom-back, and thetop-front. These pieces are assembled using strong magnets, making the head easier to open for electronics access and maintenance while keeping the exterior clean. Finally, by keeping all of the electronics contained into the head, I am also able to use and enjoy the doll in a smaller compact version by removing the doll body.

Software Systems


Processing Simulation Example of the emotions and interactions I hope to implement into to my final project.



This is a test of my custom wakeword and voice recognition system that I plan to implement into my final project. The video shows the system recognizing the wakeword "Hi Triss." Normal static or noise sits around 0.6, but when the wakeword is spoken the score drops to between 0.2 and 0.3.



How the Systems Work Together


My final project requires several different systems to work together to create my interactive ball jointed doll. My plan for systems integration is to treat each part of my system individually as well as a whole. By treating parts and sections individually, I can test and troubleshoot the individual sections before combining everything. This will save me the potential headaches of trying to solve problems that arise from the interaction of multiple systems.

The flowchart below shows how the different systems will interact from charging, to the initial user input, to the screens output. This flowchart helps me plan how the physical components, electronics, and software will work together to create Triss's interactive features.

Flowchart of system interactions