SPMC¶
This document describes the SPMC (S-EL1) implementation for OP-TEE. More information on the SPMC can be found in the FF-A specification can be found in the FF-A spec.
SPMC Responsibilities¶
The SPMC is a critical component in the FF-A flow. Some of its major responsibilities are:
- Initialisation and run-time management of the SPs:
The SPMC component is responsible for initialisation of the Secure Partitions (loading the image, setting up the stack, heap, …).
- Routing messages between endpoints:
The SPMC is responsible for passing FF-A messages from normal world to SPs and back. It also responsible for passing FF-A messages between SPs.
- Memory management:
The SPMC is responsible for the memory management of the SPs. Memory can be shared between SPs and between a SP to the normal world.
This document describes OP-TEE as a S-EL1 SPMC.
Secure Partitions¶
Secure Partitions (SPs) are the endpoints used in the FF-A protocol. When OP-TEE is used as a SPMC SPs run primarily inside S-EL0.
OP-TEE will use FF-A for it transport layer when the OP-TEE CFG_CORE_FFA=y
configuration flag is enabled.
The SPMC will expose the OP-TEE core, privileged mode, as an secure endpoint
itself. This is used to handle all GlobalPlaform programming mode operations.
All GlobalPlatform messages are encapsulated inside FF-A messages.
The OP-TEE endpoint will unpack the messages and afterwards handle them as
standard OP-TEE calls. This is needed as TF-A (S-EL3) does only allow
FF-A messages to be passed to the secure world when the SPMD is enabled.
SPs run from the initial boot of the system until power down and don’t have any built-in session management compared to GPD TEE TAs. The only means of communicating with the outside world is through messages defined in the FF-A specification. The context of a SP is saved between executions.
The Trusted Service repository includes the libsp libary which export all needed functions to build a S-EL0 SP. It also includes many examples of how to create and implement a SP.
SPMC Program Flow¶
SP images are either embedded into the OP-TEE image or loaded from the FIP by
BL2. This makes it possible to start SPs during boot, before the rich OS is
available in the normal world. ldelf
is used to load and initialise the SPs.
Starting SPs¶
SPs are loaded and started as the last step in OP-TEE’s initialisation process.
This is done by adding sp_init_all()
to the boot_final
initcall level.
sp_init_all()
:Initialise all SPs which have been added by the
SP_PATHS
compiler option and run themthread_ffa_msg_wait()
:All SPs are loaded and started. A
FFA_MSG_WAIT
message is sent to the Normal World.
Each SP is loaded into the system using ldelf
and started. This is based
around the same process as loading the early TAs.
All SPs are run after they are loaded and run until a FFA_MSG_WAIT
is sent
by the SP.
init_with_ldelf()
:Load the SP
sp_init_info()
:Initialise the
struct ffa_init_info
. Thestruct ffa_init_info
is passed to the SP during it first run.sp_init_set_registers()
:Initialise the registers of the SP
sp_msg_handler()
:Handle the SPs FF-A message
Once all SPs are loaded and started we return to the SPMD and the Normal World is booted.
SP message handling¶
The SPMC is split into 2 main message handlers:
thread_spmc_msg_recv()
thread_spmc.c:Used to handle message coming from the Normal World.
sp_msg_handler()
spmc_sp_handler.c:Used to handle message where the source or the destination is a SP.
When a FFA_MSG_SEND_DIRECT_REQ
message is received by the SPMC from the
Normal World, a new thread is started.
The FF-A message is passed to the thread and it will call the
sp_msg_handler()
function.
Whenever the SPMC (sp_msg_handler()
) receives a message not intended for
one of the SPs, it will exit the thread and return to the Normal World
passing the FF-A message.
Currently only a FFA_MSG_SEND_DIRECT_REQ
can be passed from the Normal
World to a SP.
Every message received by the SPMC from the Normal World is handled in the
thread_spmc_msg_recv()
function.
When entering a SP we need to be running in a OP-TEE thread. This is needed to
be able to push the TS session (We push the TS session to get access to the SP
memory).
Currently the only possibility to enter a SP from the Normal world is via a
FFA_MSG_SEND_DIRECT_REQ
. Whenever we receive a FFA_MSG_SEND_DIRECT_REQ
message which doesn’t have OP-TEE as the endpoint-id, we start a thread and
forward the FF-A message to the sp_msg_handler()
.
The sp_msg_handler()
is responsible for all messages coming or going
to/from a SP. It runs in a while loop and will handle every message until it
comes across a messages which is not intended for the secure world.
After a message is handled by the SPMC or when it needs to be forwarded to a SP,
sp_enter()
is called.
sp_enter()
will copy the FF-A arguments and resume the SP.
When the SPMC needs to have access to the SPs memory, it will call
ts_push_current_session()
to gain access and ts_pop_current_session()
to release the access.
Running and exiting SPs¶
The SPMC resumes/starts the SP by calling the sp_enter()
. This will set up
the SP context and jump into S-EL0.
Whenever the SP performs a system call it will end up in sp_handle_svc()
.
sp_handle_svc()
stores the current context of the SP and makes sure that we
don’t return to S-EL0 but instead returns to S-EL1 back to sp_enter()
.
sp_enter()
will pass the FF-A registers (x0-x7) to
spmc_sp_msg_handler()
. This will process the FF-A message.
RxTx buffer managment¶
RxTx buffers are used by the SPMC to exchange information between an endpoint
and the SPMC. The rxtx_buf struct is used by the SPMC for abstracting buffer
management.
Every SP has a struct rxtx_buf
wich will be passed to every function that
needs access to the rxtx buffer.
A separate struct rxtx_buf
is defined for the Normal World, which gives
access to the Normal World buffers.
Configuration¶
SPMC config options¶
To configure OP-TEE as a S-EL1 SPMC with Secure Partition support, the following flags should be set for optee_os:
CFG_CORE_SEL1_SPMC=y
CFG_SECURE_PARTITION=y
CFG_DT=y
CFG_MAP_EXT_DT_SECURE=y
Furthermore TF-A should be configured as the SPMD, expecting a S-EL1 SPMC:
SPD=spmd
SPMD_SPM_AT_SEL2=0
ARM_SPMC_MANIFEST_DTS=<path to SPMC manifest dts>
SP loading mechanism¶
OP-TEE SPMC supports two methods for finding and loading the SP executable images. Currently only ELF executables are supported. In the build repo the loading method can be selected with the SP_PACKAGING_METHOD option.
Embedded SP¶
In this case the early TA mechanism of optee_os is reused: the SP ELF files are
embedded into the main OP-TEE binary. Each ELF should start with a specific
section (.sp_head) containing a struct which describes the SP (UUID, stack size,
etc.). The images can be added to optee_os using the SP_PATHS
config option,
the build repo will set this up automatically when
SP_PACKAGING_METHOD=embedded
is selected. The images passed in SP_PATHS
are processed by ts_bin_to_c.py
in optee_os and linked into the main binary.
At runtime the for_each_secure_partition()
macro can iterate through these
images, so a particular SP can be found by UUID and then loaded by ldelf.
The SP manifest file [1] used by the SPMC to setup SPs is also handled by
ts_bin_to_c.py
, it will be concatenated to the end of the SP ELF.
FIP SP¶
In this case the SP ELF files and the corresponding SP manifest DTs are
encapsulated into SP packages and packed into the FIP. The goal of providing
this alternative flow is to make updating SPs easier (independent of the main
OP-TEE binary) and to get aligned with Hafnium (S-EL2 SPMC). For more
information about the FIP, please refer to the TF-A documentation [2]. The SP
packaging process and the package format is provided by TF-A, detailed
description is available at [3]. In the build repo this method can be
selected by SP_PACKAGING_METHOD=fip
, it covers all the necessary setup
automatically. In case of using another buildsystem, the following steps should
be implemented:
TF-A config
SP_LAYOUT_FILE
: provide a JSON file which describes the SPs (path to SP executable and corresponding DT, example [4]). The TF-A buildsystem will create the SP packages (using sptool) based on this and pack them into the FIP.TF-A config
ARM_BL2_SP_LIST_DTS
: provide a DT snippet which describes the SPs’ UUIDs and load addresses (example: [5]). This will be injected into the SP list inTB_FW_CONFIG
DT of TF-A, and BL2 will load the SP packages based on this. Note that BL2 doesn’t automatically load all images from the FIP: it’s necessary to explicitly define them inTB_FW_CONFIG
(using this injected snippet or manually editing the DT).TF-A config
ARM_SPMC_MANIFEST_DTS
: provide the SPMC manifest (example: [6]). This DT is passed to the SPMC as a boot argument (in the TF-A naming convention this is theTOS_FW_CONFIG
). It should contain the list of SP packages and their load addresses in thecompatible = "arm,sp_pkg"
node.
At boot optee_os will parse the SP package load addresses from the SPMC manifest
and find the SP packages already loaded by BL2. Iterating through the SP
packages, based on the SP package header in each package it will map the SP
executable image and the corresponding manifest DT and collect these to the
fip_sp_list
list. Later when initialising the SPs, the for_each_fip_sp
macro is used to iterate this list and load the executables using ldelf, just
like for the embedded SP case.
[1] https://trustedfirmware-a.readthedocs.io/en/v2.6/components/ffa-manifest-binding.html
[5] https://github.com/OP-TEE/build/blob/master/fvp/bl2_sp_list.dtsi
[6] https://github.com/OP-TEE/build/blob/master/fvp/spmc_manifest.dts