Introduction

Recently I created a tracking device for the dog using a Xaio ESP32S3, GPS receiver and a Li-Po battery - I was curious how fast he runs...

To be able to see the data I wanted to be able to connect to it by Wi-Fi and use the browser to see the data and a map:

Rather than trying to create a full web application in C++ on the ESP32, I felt that serving up a Single Page Application (SPA) that can make some basic API calls to receive the data would be nicer (both development and UX wise) and potentially a bit easier.

My front end framework of preference is React.

This post explores building, embedding and serving a React App from the ESP32 using the PlatformIO tooling.

The Web Application

The React App is fairly standard - there's no special preparation required for the ESP32 (but size is a consideration, as covered further on).

The source is in a folder called web alongside src (note that web has a src folder too):

In this case I've created a Typescript app (using the create-react-app template) but Javascript will work equally well.

  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build && mv build/static/* build/ && rmdir build/static && rm build/**/*.LICENSE.txt",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },

This modified build script above, when run with npm run build, will build the app, and move the static folders (js and css) to be relative to the index page in the build folder to keep the file paths as short as possible as the SPIFFS file system has path length limits.

In my project I found that license files also needed to be removed to minimise size and extra unnecessary files:

Building React for the ESP32

PlatformIO can be instructed to rebuild the filesystem image using the build in actions/commands, however this will just create the image from the ./data folder.

Naturally it would be nice and convenient if the web app was automatically re-built when the Filesystem Image is built.

This can be done using the extra_scripts directive in PlatformIO (docs).

In platformio.ini the extra_scripts line is added, pointing to a python files as per the example below:

[env:seeed_xiao_esp32s3]
platform = espressif32@6.2.0
board = seeed_xiao_esp32s3
framework = arduino
monitor_speed = 115200
lib_deps = 
	mikalhart/TinyGPSPlus@^1.0.3
	bblanchon/ArduinoJson@^6.21.2
    https://github.com/me-no-dev/ESPAsyncWebServer.git
extra_scripts = extra_script.py

The extra_script.py is as follows:

Import("env")
import os

def before_build_spiffs(source, target, env):
    print("Building React App...")
    env.Execute("cd web && npm run build")
    print("React App built!")

    print("Removing old SPIFFS image...")
    env.Execute("rm -rf data")

    print("Copying React App to SPIFFS...")
    env.Execute("cp -r web/build data")    

env.AddPreAction("$BUILD_DIR/spiffs.bin", before_build_spiffs)

The python file can have various other actions, along with this, for example loading .env files.

The script above builds the web app from the web directory, then removes the data folder (which is where the SPIFFS Filesystem Image is built from) and finally copies the web/build folder to data which essentially resets the folder with the new contents.

This then is invoked when the Build Filesystem Image Task, and other Tasks that depend on it (for example Upload Filesystem Image) is run from PlatformIO.

The File System Image is built and uploaded independently of the code, but that only needs doing when the Web App code changes.

Serving the App from the ESP32

In this section I'm skipping any networking setup - it presumes the ESP can be accessed by it's IP Address from a Web Browser.

Using the ESPAsyncWebServer library makes it easy to run a Web Server and serve up the static files.

The following includes are required:

#include <ESPAsyncWebServer.h>
#include "SPIFFS.h"

An AsyncWebServer instance needs creating:

AsyncWebServer server(80);

In the setup() function there are a few things to configure.

First set up SPIFFS:

if (!SPIFFS.begin(true))
  {
    Serial.println("An Error has occurred while mounting SPIFFS");
    return;
  }

Then, given the nature of this hobby project, we can work around CORS restrictions restrictions so that the the Web App can talk to the ESP32 during local development.

These allow any URL, rather than the same URL (which is the default), to make API calls to the API endpoints:

 DefaultHeaders::Instance().addHeader("Access-Control-Allow-Origin", "*");
  DefaultHeaders::Instance().addHeader("Access-Control-Allow-Methods", "GET, POST, PUT");
  DefaultHeaders::Instance().addHeader("Access-Control-Allow-Headers", "Content-Type");

Then serve the Filesystem:

  server.serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");
  server.serveStatic("/static/", SPIFFS, "/");

The Filesystem is served on both / (root) and /static since index.html will be looking in /static for the js and css folders - as with moving the files around in the prior steps, this helps keep the path name short without re-writing the actual paths compiled into the web app.

The index.html file is also specified for the default page.

We can serve up data using any endpoint, in the example below a GET request to /gps will send back a JSON document (of GPS data). The doc is populated elsewhere on a regular basis.

  server.on("/gps", HTTP_GET, [](AsyncWebServerRequest *request)
            {
        String out;
        serializeJson(doc, out);
        request->send(200, "text/json", out); });

A custom onNotFound handler needs to be added to allow the HTTP pre-flight checks to pass:

server.onNotFound(notFound);

Where notFound is implemented as follows:

void notFound(AsyncWebServerRequest *request)
{
  if (request->method() == HTTP_OPTIONS)
  {
    request->send(200);
  }
  else
  {
    request->send(404, "application/json", "{\"message\":\"Not found\"}");
  }
}

Finally, the server can be started:

server.begin();

Memory and File System Space

Depending on the ESP32 model the total memory will vary. The Xaio ESP32-S3 that I was using has 4Mb of flash.

Once I'd added a few dependencies to the React App (Google Maps mostly) I started hitting memory limits.

One option is to slim down the dependencies, which is potentially challenging - another is to change the partition layout.

There is a detailed post on how to do that here: https://blog.espressif.com/how-to-use-custom-partition-tables-on-esp32-69c0f3fa89c8.

The default layout is as follows:

This means that the SPIFFS partition is 0x160000 bytes long or 1,441,792 bytes - about 1.375 Mb. Each app slot is 1.25 Mb each - there are two - this table defines the the layout for 4 Mb of Flash.

Ideally I'd create a custom partition layout to meet my needs, but as the post lined above explains, that involves creating the CSV and then converting it to binary.

A number of default options are defined and documented here: https://github.com/espressif/arduino-esp32/tree/master/tools/partitions.

The no_ota.csv table is as follows:

This gives us a single 2 Mb app partition and a 1.875 Mb SPIFFS partition against a 4 Mb total table size (note the coredumps are at the same offset 64k shy of 4 Mb with a 64k size).

To use the no_ota layout we can specify it in the platformio.ini file:

board_build.partitions = no_ota.csv

This provides a bit more space compared to the default. An 8 Mb ESP32 with a custom table would be even more flexible.

Note: The default 8 Mb table appears to have a pair of 3.2 Mb app partitions and a 1.5 Mb SPIFFS partition.

Conclusion

These snippets combined with a functioning code-base that sets up the Wi-Fi either as an Access Point or a Client, allows for an SPA web application to be served from an ESP32, and this app in turn can make API calls back to the ESP to send and receive data.

The CORS tweaks also make development easier, allowing the web app to talk to the ESP directly when making and testing changes.

The SPA provides a convenient and powerful way to build a web front end for the project, avoiding the need to serve HTML strings, or JavaScript directly from the C++ code.

Space is at a premium so care must be taken when selecting 3rd party libraries and dependencies to include but trade-offs can be made with the storage configuration to allow more space for the File System data.