Remotely updating the firmware of embedded devices is a delicate operation prone to both operational and security risks. In this post, we present a safe and easy mechanism to seamlessly update the Python firmware of Zerynth powered devices connected to Amazon Web Services (AWS) IoT endpoints.
Considerations about FOTA updates
First of all, some considerations on the implications of supporting firmware over the air (FOTA) updates. In order to be able to change its own firmware, a device:
- Needs a partitioned internal flash with at least two separate slots to store both the current running firmware and the new one
- Must reserve a zone of flash or other non-volatile memory to store information about which firmware has to be run after device reset, together with any additional information on its properties
- Must withstand connectivity issues in retrieving and storing the new firmware
- Should employ an encrypted channel to retrieve the firmware
- Must be able to correctly communicate with the cloud endpoint in every phase of the FOTA flow
In the case at hand, thanks to the integration of Zerynth and AWS IoT, point 1 and 2 are implemented under the hood by the FOTA enabled Zerynth Virtual Machine. The other points can be addressed by using the appropriate AWS modules provided by Zerynth.
FOTA updates for AWS: overview and workflow
The standard AWS mechanism to let devices perform tasks is represented by IoT Jobs. A job can be viewed as a set of information (called the job document) that is sent to a device with instruction on the task to perform. The device can, in turn, change the status of the job to reflect the various phases of task execution (see point 5). The nice thing about IoT Jobs is that the document can contain links to files in an S3 bucket that are “pre-signed”. This means that only the device receiving the job document can correctly retrieve the new firmware through a secure HTTPS connection.
The FOTA flow, therefore, requires the device to:
- Retrieve the list of jobs
- Check if there is a FOTA job in progress
- If the current firmware is matching the one described in the job, the job is marked as successful
- Otherwise is marked as failed
- Check if there is a new FOTA job not yet started
- Retrieve the job document
- Mark the job as in progress
- Download the new firmware
- Check that the CRC of the downloaded firmware matches the one in the job document
- Reset the device
- Publish data to the MQTT and periodically check for new jobs (back to 1)
Getting started with FOTA updates for Amazon Web Services
Let’s follow the entire FOTA flow through a working example.
1. Download Zerynth Studio
First of all, you need to download and install Zerynth Studio, our free and cross-platform IDE for programming microcontrollers in Python or C/Python language.
2. Connect an ESP32 board
Then connect one of the ESP32 based devices supported by Zerynth to your PC. Here is the list of supported boards (search for “ESP32”). For example, you can use an ESP32 DevKitC. You can find here more info about this board.
3. Virtualize the ESP32 board
Now you have to register and virtualize your ESP32 board. In this case, you have to create and use a FOTA enabled Zerynth Virtual Machine.
Take note of the VM uid printed in the console during the VM creation phase. You’ll need it later.
4. Clone the example “FOTA AWS”
Zerynth Studio includes a huge list of useful examples that you can “clone” (aka copy) with just a few clicks.
In this case, you have to clone the example FOTA AWS. You can find it searching for “FOTA AWS” in the “Examples Browser” panel.
Let’s take a look at the main.py file:
5. Configuration for AWS connection
The Zerynth AWS IoT Library can be used to ease the connection to the AWS IoT platform.
It allows to make your device act as an AWS IoT Thing which can be registered through AWS tools or directly from the Zerynth Toolchain. Note that the Zerynth Toolchain assumes AWS command line tool to be available and configured to handle AWS resources.
Once cloned the example, the proper configuration of the device (Thing in the AWS jargon) must be provided as a Thing certificate and a Thing private key and a Thing name (thing_name). Such credentials partially resolve point 4 by forcing the device to exchange data through a secure TLS connection.
The main steps are:
- Copy/paste the “custom endpoint” from AWS console to the “thing.conf.json” of your project
- Create a new policy. More info about AWS IoT policies here. Following you can see the policy we’ve used for this example.
- Bind the AWS IoT Thing to the Zerynth project through the Zerynth Toolchain. More info here.
See the following video to better understand the workflow. You can stop watching at min 4:35.
6. Uplink the Python script to your ESP32 board
Now you have to uplink the code to your device remembering to set the wifi network name and password at line 36. Then open the serial monitor and check that it connects to the AWS MQTT broker.
7. Change the firmware and prepare a new Bytecode
Now change the version (from “0” to “1”) at line 30.
Then issue the following command from a shell:
This will compile the project to the file bytecode-1.vbo. Such file contains the result of the compilation and it is almost ready to be sent to the device. Before doing that, another command must be given:
This command will read the bytecode, modify it in order to be run by the VM on slot 1 and output the FOTA ready firmware. Note that here is the point of the workflow when you need to insert the vm uid, annotated during VM creation phase.
8. Start a FOTA AWS Job for your ESP32
The following last command will start a FOTA Job for the running device (job handling messages will be printed in the console).
The command assumes some configuration of AWS IoT has been made, so before running it ensure that:
- The AWS IoT endpoint has an IAM role “s3-read-role” (the name is not important, name it as you prefer) that allows access for Things to an S3 bucket.
- The S3 bucket named “fota-aws-bucket” exists (again, name it as pleased). More info about how to create S3 bucket here.
If such conditions are verified, the above command will create a JSON file with enough info for the Zerynth AWS IoT FOTA Library to initiate and finish the update, upload both the JSON and the firmware to the s3 bucket, and create a new IoT job immediately sent to the device.
As a result, at a certain point you will see printed “Hello, I am firmware version: 1” in the serial monitor. Enjoy your brand new firmware over-the-air uploaded to your ESP32!
If a new firmware update is needed, just repeat the command “ztc link” with a different slot (0 or 1). You can also play with FOTA by sending incorrect updates and check that the devices readily ignore them marking the jobs as failures.