Week 14 — Interface and Application Programming

PIKPIK CARE webpage structure

I organized the PIKPIK CARE interface as a separate mini web application inside my Fab Academy site. The webpage files are stored in the public/pikpik-care/ folder, while the character images are stored in public/images/pikpik-care/.

public/
├── assignments/
│   └── week14.html
├── pikpik-care/
│   ├── pikpik-care.html
│   ├── pikpik-care.css
│   ├── pikpik-care.js
│   └── sounds/
│       ├── feed_cookie_crunch.mp3
│       ├── love_heart_pop.mp3
│       ├── play_shake_pop.mp3
│       ├── sleep_soft_breath.mp3
│       └── unhappy_warning.mp3
└── images/
    └── pikpik-care/
        ├── info.png
        ├── love.png
        ├── play.png
        ├── sleep.png
        ├── feed.png
        ├── unhappy.png
        └── cookie.png

Instead of using a code screenshot for the file structure, I documented it directly as a text-based project tree. This makes the structure easier to read, copy, and update.

Web interface design idea

Since Loco Pik is an emotional companion robot, I did not want the interface to look like a normal engineering control panel. Instead, I wanted the webpage to feel soft, playful, and close to the character language of my final project.

The main design idea was to translate technical hardware commands into emotional user actions. Instead of showing buttons such as LED_RED, BLINK, or SET_COLOR, I used actions like Info, Love, Play, Sleep, and Feed. These actions are closer to the way people interact with a digital pet or companion robot.

The webpage is not only a visual demo. It works as a testing and interaction layer for the final project. Through this interface, I can check whether each user action creates the correct screen effect, sound effect, HTTP command, and WS2812B LED light response.

Embedded preview version

I also added an embedded preview version of the PIKPIK CARE interface directly inside this Week 14 documentation page. This allows the interface to be viewed and tested without opening a separate page.

The embedded window shows the same PIKPIK CARE interface that I used during local testing. The interface can be previewed after the page is uploaded to Fab Academy. However, the live hardware control only works when the browser and the ESP32-S3 are connected to the same local Wi-Fi network.

Week 14 documentation page
        ↓
Embedded PIKPIK CARE preview window
        ↓
User taps Info / Love / Play / Sleep / Feed
        ↓
pikpik-care.js plays screen effects and sound effects
        ↓
pikpik-care.js sends HTTP request to ESP32-S3
        ↓
ESP32-S3 controls WS2812B emotional light

System overview

The system has three main layers: the user interface, the network command layer, and the physical LED output layer. The user interacts with the PIKPIK CARE webpage. The webpage sends HTTP requests. The ESP32-S3 receives and parses the requests. Finally, the WS2812B LED strip shows the emotional light effect.

Text-based system map

Phone / Laptop
        ↓
PIKPIK CARE Webpage
HTML + CSS + JavaScript interface
        ↓
HTTP requests over local Wi-Fi
        ↓
ESP32-S3
HTTP server receives emotional commands
        ↓
WS2812B-compatible LED Strip
Physical emotional light output

In this system, the phone or laptop does not control the LED strip directly. The user opens the PIKPIK CARE webpage in a browser. The webpage sends HTTP requests through the local Wi-Fi network. The ESP32-S3 receives these requests and translates them into temporary WS2812B LED light effects.

Control chain logic

The complete control chain starts from a user action in the PIKPIK CARE webpage and ends with the physical LED output. After a temporary light effect finishes, the LED strip automatically returns to the AI conversation emotion mode.

User taps an active item in PIKPIK CARE
        ↓
pikpik-care.js triggers screen animation and sound
        ↓
Browser sends HTTP request
        ↓
ESP32-S3 receives /love, /play, /feed, /sleep, or /unhappy
        ↓
LocoPicLightServer parses the command
        ↓
Ws2812bMoodLight plays the corresponding LED effect
        ↓
After 2 or 4 seconds
        ↓
The light automatically returns to the AI conversation emotion mode

Connecting PIKPIK CARE actions to ESP32-S3

In the PIKPIK CARE webpage, each user action is not only a screen interaction. The JavaScript also sends a matching HTTP command to the ESP32-S3. For example, when the user taps Love, the webpage plays the love animation and sound effect, and then sends a request to http://192.168.1.75/love.

This means the same action controls both the digital interface and the physical LED output. The webpage becomes a bridge between user intention and hardware behavior.

PIKPIK CARE action
        ↓
Screen animation and sound effect
        ↓
sendEsp32Command("love")
        ↓
GET http://192.168.1.75/love
        ↓
ESP32-S3 receives /love
        ↓
WS2812B LED strip shows love light effect

The relationship between one PIKPIK CARE action and one ESP32-S3 command was implemented in JavaScript. One user action can trigger screen feedback, sound feedback, and physical LED feedback.

const ESP32_BASE_URL = "http://192.168.1.75";

const actionCommandMap = {
  info: "info",
  love: "love",
  play: "play",
  sleep: "sleep",
  feed: "feed",
  unhappy: "unhappy"
};

function triggerPikpikAction(actionName) {
  const command = actionCommandMap[actionName];
  if (!command) return;

  playScreenEffect(actionName);
  playSoundEffect(actionName);
  sendEsp32Command(command);
}

function sendEsp32Command(command) {
  fetch(`${ESP32_BASE_URL}/${command}`, {
    method: "GET",
    mode: "no-cors"
  });
}

JavaScript HTTP command bridge

In pikpik-care.js, I added an HTTP command bridge. This function converts emotional action names into URL paths and sends GET requests to the ESP32-S3 local IP address.

const ESP32_BASE_URL = "http://192.168.1.75";

function sendEsp32Command(eventName) {
  if (!eventName) return;

  const command = eventName.toLowerCase();
  const url = `${ESP32_BASE_URL}/${command}`;

  fetch(url, {
    method: "GET",
    mode: "no-cors"
  })
    .then(() => {
      console.log(`ESP32 command sent: /${command}`);
    })
    .catch((error) => {
      console.error(`ESP32 command failed: /${command}`, error);
    });
}

I used mode: "no-cors" so that the browser can still send the command to the ESP32-S3 during local testing, even if the ESP32-S3 does not return CORS headers. In this mode, the webpage cannot read the response body, but the command can still reach the board.

The code above is the actual HTTP command bridge used in the webpage. I documented it as editable code instead of using a screenshot, so the logic can be read directly in the browser.

Interface command table

Each PIKPIK CARE action has a screen effect, sound effect, HTTP command, and physical LED output. This table helped me connect the user interface, emotional meaning, and hardware behavior together.

Local network setup

To test the interface on my phone, the laptop, phone, and ESP32-S3 needed to be connected to the same Wi-Fi network. I used VS Code Live Server to host the webpage on my Mac.

On the phone, I opened the webpage using the Mac's local network IP:

http://192.168.1.225:5501/pikpik-care/pikpik-care.html

However, the webpage actions send requests to the ESP32-S3:

http://192.168.1.75/love
http://192.168.1.75/play
http://192.168.1.75/feed
http://192.168.1.75/sleep
http://192.168.1.75/unhappy

Starting the HTTP server after Wi-Fi is ready

One important detail was that the ESP32-S3 HTTP server should not start too early. At first, I considered starting the light server directly inside the board constructor, but this could make the device unstable because Wi-Fi and TCP/IP were not fully initialized yet.

To solve this problem, I added a network-ready flow. The board waits until Wi-Fi is connected, then calls OnNetworkReady, and only then starts the Loco Pik light server.

ESP32-S3 boot
        ↓
Connect to Wi-Fi
        ↓
Network becomes ready
        ↓
OnNetworkReady()
        ↓
Start Loco Pik light server
        ↓
Receive HTTP commands from PIKPIK CARE

I changed the firmware structure so the light server starts only after Wi-Fi is ready. This avoids starting the HTTP service before the TCP/IP stack is initialized.

// board.h
class Board {
public:
  virtual void StartNetwork() = 0;
  virtual void OnNetworkReady() {}
};

// application.cc
void Application::Start() {
  auto& board = Board::GetInstance();

  board.StartNetwork();

  // After Wi-Fi and TCP/IP are ready, start board-specific services.
  board.OnNetworkReady();
}

// seeed_xiao_esp32s3_sense.cc
void OnNetworkReady() override {
  if (mood_light_ != nullptr) {
    loco_pic_light_server_.Start(mood_light_);
  }
}

ESP32-S3 HTTP server

The ESP32-S3 runs a small HTTP server. In loco_pic_light_server.cc, I registered several short command paths. All of these paths are handled by the same function, which parses the request and sends the event name to the LED class.

/info
/love
/play
/feed
/sleep
/unhappy

For example, when the browser visits http://192.168.1.75/love, the HTTP server receives the URI /love. The code removes the first slash and converts it into the event name love. Then it calls the LED control function.

Browser request
        ↓
GET /love
        ↓
LocoPicLightServer receives request
        ↓
Parse "/love" into event = "love"
        ↓
Call Ws2812bMoodLight
        ↓
Play Love LED effect

The HTTP server registers short URL paths as emotional commands. These paths are used by the webpage when the user taps an action.

bool LocoPicLightServer::Start(Ws2812bMoodLight* mood_light) {
  if (server_ != nullptr) {
    return true;
  }

  mood_light_ = mood_light;

  httpd_config_t config = HTTPD_DEFAULT_CONFIG();
  config.server_port = 80;
  config.max_uri_handlers = 16;
  config.lru_purge_enable = true;

  esp_err_t ret = httpd_start(&server_, &config);
  if (ret != ESP_OK) {
    ESP_LOGE(TAG, "Failed to start HTTP server: %s", esp_err_to_name(ret));
    server_ = nullptr;
    return false;
  }

  httpd_uri_t love = {
    .uri = "/love",
    .method = HTTP_GET,
    .handler = HandleAction,
    .user_ctx = this
  };

  httpd_uri_t play = love;
  play.uri = "/play";

  httpd_uri_t feed = love;
  feed.uri = "/feed";

  httpd_uri_t sleep = love;
  sleep.uri = "/sleep";

  httpd_uri_t unhappy = love;
  unhappy.uri = "/unhappy";

  httpd_register_uri_handler(server_, &love);
  httpd_register_uri_handler(server_, &play);
  httpd_register_uri_handler(server_, &feed);
  httpd_register_uri_handler(server_, &sleep);
  httpd_register_uri_handler(server_, &unhappy);

  return true;
}

WS2812B mood light logic

The LED output is handled by the Ws2812bMoodLight class. This class manages both the AI emotion light mode and the temporary webpage-triggered light effects.

Even when a webpage-triggered light effect is playing, the latest AI emotion is still recorded. When the temporary effect finishes, the LED strip can return to the most recent AI emotion instead of an old state.

The important members of the Ws2812bMoodLight class manage AI emotion mode, temporary webpage effects, and automatic recovery.

class Ws2812bMoodLight {
public:
  Ws2812bMoodLight(gpio_num_t gpio, int led_count);
  ~Ws2812bMoodLight();

  bool IsReady() const { return led_strip_ != nullptr; }

  void SetEmotion(const std::string& emotion);
  bool SetColorByName(const std::string& color);
  void SetRgb(uint8_t r, uint8_t g, uint8_t b);
  bool SetLocoPicAction(const std::string& action);

  void SetEmotionMode(bool enabled);
  bool IsEmotionMode();

private:
  led_strip_handle_t led_strip_ = nullptr;
  int led_count_ = 0;

  bool emotion_mode_ = true;
  uint32_t effect_generation_ = 0;
  std::string last_emotion_;

  std::mutex mutex_;

  void SetAll(uint8_t r, uint8_t g, uint8_t b);
  void ApplyEmotionLocked(const std::string& emotion);
  void RestoreEmotionLocked();
  void StartTemporaryEffect(const std::string& action, int duration_ms);

  static void TemporaryEffectTask(void* arg);
};

Temporary light effect and AI emotion recovery

The LED class manages two different kinds of light behavior. One is the AI conversation emotion light, and the other is the temporary light effect triggered by the PIKPIK CARE webpage.

When a webpage action is received, the system temporarily pauses the AI emotion light mode and plays the selected LED effect. After the effect duration ends, the LED strip automatically returns to the latest AI conversation emotion.

The effect_generation_ variable is important when the user taps different actions quickly. For example, if the user taps Love and then taps Sleep one second later, the old Love task will not restore the light after it finishes, because the current effect generation has already changed. This prevents Love from interrupting the Sleep effect.

User taps Love
        ↓
Love effect starts
        ↓
User taps Sleep before Love finishes
        ↓
Sleep effect becomes the latest generation
        ↓
Old Love task finishes
        ↓
Old Love task does not restore the light
        ↓
Sleep continues correctly

The effect_generation_ check prevents older temporary effects from restoring the LED state after a newer effect has already started.

void Ws2812bMoodLight::StartTemporaryEffect(
    const std::string& action,
    int duration_ms) {
  std::lock_guard<std::mutex> lock(mutex_);

  emotion_mode_ = false;
  effect_generation_++;

  auto* context = new TemporaryEffectContext{
      this,
      action,
      effect_generation_,
      duration_ms
  };

  xTaskCreate(
      TemporaryEffectTask,
      "loco_light_effect",
      4096,
      context,
      4,
      nullptr
  );
}

void Ws2812bMoodLight::TemporaryEffectTask(void* arg) {
  auto* context = static_cast<TemporaryEffectContext*>(arg);
  auto* light = context->light;
  const std::string action = context->action;
  const uint32_t generation = context->generation;
  const int duration_ms = context->duration_ms;
  delete context;

  auto still_current = [light, generation]() {
    std::lock_guard<std::mutex> lock(light->mutex_);
    return light->effect_generation_ == generation;
  };

  if (action == "play") {
    const int step_ms = 160;
    for (int elapsed = 0;
         elapsed < duration_ms && still_current();
         elapsed += step_ms) {
      std::lock_guard<std::mutex> lock(light->mutex_);

      if (light->effect_generation_ != generation) {
        break;
      }

      if ((elapsed / step_ms) % 2 == 0) {
        light->SetAll(48, 36, 0);
      } else {
        light->SetAll(4, 3, 0);
      }

      vTaskDelay(pdMS_TO_TICKS(step_ms));
    }
  }

  {
    std::lock_guard<std::mutex> lock(light->mutex_);

    if (light->effect_generation_ == generation) {
      light->emotion_mode_ = true;
      light->RestoreEmotionLocked();
    }
  }

  vTaskDelete(nullptr);
}

Real hardware test with phone control

I recorded a real hardware test to show the complete interaction. In this test, I opened the PIKPIK CARE webpage on my phone and tapped different emotional actions. The webpage sent HTTP requests to the ESP32-S3, and the ESP32-S3 controlled the WS2812B LED strip to show the corresponding light effects.

This test confirmed that the interface was not only a screen prototype. The phone webpage, Wi-Fi connection, ESP32-S3 HTTP server, and WS2812B LED output were working together as one interaction system.

Phone opens PIKPIK CARE
        ↓
Tap Love / Play / Feed / Sleep
        ↓
HTTP request is sent to ESP32-S3
        ↓
ESP32-S3 receives the command
        ↓
WS2812B LED strip changes color or effect