Industrial Linux

Now it's real(ly) time!

OTA Requirements

An update is to be carried out either if an error or an attack threat has been identified in the host operating system, if a new version is to be installed, or if the applications that are active on the device require updates. Whether an update needs to be performed, the end device can learn by request to the update server, by a message from the server (for example, in the case of a critical update), by user action or by some other action.

A proper update system is able to provide this functionality and thus help to save a lot of money.

Flexibility

An update system should be able to exchange any component of an embedded system. This includes the boot loader, kernel, rootfs, binary files for flash components, software for additional CPUs, both in heterogeneous systems and in a cascading structure (gateway), and the application(s) themselves. Also should it take only those components from an update container that are needed for its own hardware. The system should be adaptable (with minimal adjustments) to different CPU architectures, storage media, storage layouts (UBIFS, eMMC, HDDs, ...) and so on.

Whether the application is part of the OS image or an autonomous partition and thus updateable independently, should be the user's decision and not determined by the update system. We have provided a separate partition for that purpose in our standard layout (see Figure 1).

It should also be able to use a wide variety of media for data transfer - be it a USB stick, an SD card or the internet in the form of a cable, WiFi or LTE / 5G. And it should handle the case where the embedded device is not constantly accessible and hence there might be identical devices with different software versions.

An update has to work without operator intervention, even in the case of an incomplete or faulty transfer of an update file (container) or insufficient memory. If an update cannot be started, a roll-back to a prior, functional image must be carried out automatically.

To meet all these requirements, an update must be atomic, which means do it all or nothing.

Carrying out an update may not affect the system, e.g. stop an application or delay its execution. Two approaches have emerged to achieve this: A/B update (symmetrical update) or Android update, also called asymmetrical update, with a "normal" and an update instance of the system.

Update Approaches

A/B update - here the update is written to the inactive partition and then the boot loader is changed to point to this partition. This newly written partition is started during the next boot process - so it is obvious that this approach requires a reboot for the update to take effect. A potential disadvantage is the large amount of memory required, since two complete images must be available at all times. The major advantage is that an executable image is always available.

Symmetric Update
Fig. 3: Default memory layout at symmetric update
Asymmetric Update
Fig. 4: Standard layout asymmetric Update
Asymmetrical update with one productive and one small update partition - here the memory requirement is less, but the downtime is significantly longer, as the productive system is inactive during the actual update process.

DELTA UPDATE

Regardless of whether symmetrical or asymmetrical, it should be possible to limit the amount of data to be transmitted. Especially if the connection is to take place via bandwidth-limited channels. Delta updates (differential) limit the size of the data to be transmitted. And also the storage space on the device required for the update.

Delta Update
Delta Update
Delta Update Principle
Delta Update Principle

Optimized bandwith 

Deduplication reduces the amount of data even further. And the packaging into equally sized chunks optimises the utilisation of the networks of the CDNs (content delivery providers).

Result: Minimum data transfer, minimum time requirement, minimum costs.

Roll-Back

In both scenarios, the success of the update is checked by a watchdog. If the system does not start after an update, it is switched to the latest executable version after an adjustable number of attempts during the A/B update. In the case of an asymmetrical update, the update system can be restarted, if necessary with a corresponding message to the user.

On-device check

Before an update file is completely loaded, it should be possible to check whether it matches the hardware and whether it contains a new, updated version of the software. The complete download of an update container only takes place after a positive check. This saves unnecessary data transfer and eliminates the danger of downgrading by transmitting an outdated update.

For authentication, the first two components of the update file are loaded and evaluated. There you will find information about the supported hardware and the software version(s) of the content. Furthermore, there is a signed file with the hashes of all update images that the update container carries along - see also Figure 4.

Pre- and Postprocessing

Before or after the actual update, it should be possible to perform certain user-definable operations. For example, in the case of an A/B update, the boot loader must receive information after the actual update as to which partition is to be booted next.

Feedback

After an update, a response should be possible so that the update server can verify whether the update was successfully installed and then synchronise the information about the device. It should also be possible to query the current status of the device and also other information, if needed.

security

Update Container

To increase security against manipulation, it is recommended that the update container is signed and that the content can as well be encrypted. The signature is the guarantee that the content has not been manipulated on its way to the device (man-in-the-middle attacks). At the same time, it serves as authentication, i.e. proof that the update originates from an authorised body. Our CMS signing supports more than one root-of-trust (RoT) for verification. This allows configurations with more than one update signing instance, e.g. the OEM supplier can deliver an update and also another supplier who delivered an application. Both updates are recognised as validly signed and applied.

Encryption prevents reading and increases the level of security, as only the person in possession of the correct key can make use of the content. For performance reasons, a symmetrical method should be chosen for encryption.

Such an approach provides data security even if the update is to be installed locally via a USB stick, for example.

Encrypting only the transport route using TLS is not sufficient. In this case, an update could be read in plain text after delivery.
If available, HSM or TPM modules or suitable structures of the SoCs such as Trusted Zone (including the use of Trusted Execution Environment) should be applied for the secure storage of certificates or keys. Using pkcs#11 simplifies the handling from within the update programme.

Signature Flow

Gui

Should the device receiving an update have a graphical user interface (GUI), it would be beneficial if the update procedure could use this GUI for, for example, selecting the update, obtaining permission to perform the update, displaying the process and checking the success.

The diagram illustrates the process of an update on the device.

Embedded Update Process

Boot up time

An update system should not affect the boot up time of the device.

Secure / Measured Boot

The update system ought to be functionally applicable even in these cases.

Multi-CPU Update and Gateway Functionality

Most current devices consist of more than one multi- or single-core CPU. Modern SoCs, so-called heterogeneous multi-core SoCs, also have more than one CPU. The update system must be able to extract the images from the update file and write them to the correct memory locations.

A similar, but not identical case is a device consisting of several (different) components which should receive updates but do not have access to the server. In this case, the device with access to the update server is serving as a gateway. It then distributes the corresponding images to the subordinate units. However, this will only work if all units use the same update software.

Fleet Management

EAn update system should be easy to integrate with a management system for automatically rolling out updates. The flexibility of our solution allows the connection to different device management systems.

Tooling

It should be possible to create the update file independently of the build system. In a further step, it should be possible to sign and, if necessary, encrypt the update file.
The update software for the end device should be integrable in Debian as well as in Yocto.

Expandability

Future requirements for an update system should be easy to integrate. Complex security requirements, such as those fulfilled by Uptane, should be feasible.

resale

An update system should not impede the neutralisation of the unit or the resetting to the delivery state for the purpose of resale.