2405: Add Rust Build Support [Rebase & FF]

[makubacki]

Preface

A series of commits to add Rust build support to edk2 build tools.

Closes #864

• Impacts functionality?
• Impacts security?
• Breaking change?
• Includes tests?
• Includes documentation?

---

PATCH 1: Add Rust config files

Two environment configuration files are added to be used in the edk2
repository for Rust development but to also serve as a configuration
reference for downstream repositories for the settings being used in
edk2.

A goal in these changes is support a standalone Rust build experience
using the command line where that build support can be leveraged
within the edk2 build system. A component for achieving this is the
"cargo-make task runner" that allows the "cargo make" command to be
used to directly build and test individual Rust packages with simple
commands. It also allows the details to be captured in a single
makefile.

These files have no impact on users not building Rust code.

• Makefile.toml - Defines the tasks and environment settings used
  to build and test code.

For more information:
https://github.com/sagiegurari/cargo-make?tab=readme-ov-file#usage

• rustfmt.toml - Defines Rust formatting options used.
  Automatically read by the cargo fmt command in the workspace.

For more information:
https://github.com/rust-lang/rustfmt

• rust-toolchain.toml - Defines the exact Rust toolchain supported
  in this repository. This ensures developers build with a toolchain
  that is used by all other developers and CI.

For more information:
https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file

---

PATCH 2: BaseTools: Add Rust build support

Updates BaseTools to be able to build Rust modules using the normal
edk build process.

Based on changes from edkii-rust.
https://github.com/tianocore/edk2-staging/tree/edkii-rust

---

Architecture and Toolchains Supported

• IA32, X64, and AARCH64 targets have been tested and supported.
  • Other targets can be further enabled in the future as necessary.
• GCC (Linux) and Visual Studio (Windows) builds are tested and
  supported.

---

Rust Build Combinations Supported

The simplest case is a new user approaching an EDK II package that
supports Rust modules. In that case, just build the package like
"normal" (e.g., stuart build) and the "normal" build output
(i.e., EFI drivers and firmware volumes) will be produced. Rust
code will automatically be built in the build process.

The supported combinations of Rust code are shown below:

1. C source + Rust source mixed in INF (Library or Module)

   • Rust source code is supported by build rule – Rust-To-Lib-File
     (.rs => .lib)

   • Limitation: Rust code cannot have external dependencies.

2. Pure Rust Module only.

   • A Cargo.toml file is added to the INF file [Sources]
     section.

   • Rust Module build is supported by build rule –
     Toml-File.RUST_MODULE (Toml => .efi)

   • Limitation: Runtime might be a problem, such as virtual address
     translation for Rust code internal global variables.

3. Pure Rust Module + Pure Rust Library with Cargo Dependency.

   • Cargo dependency means the Rust lib dependency declared in
     Cargo.toml.

4. Pure Rust Module + C Library with EDK II Dependency.

   • Rust Module build is supported by build rule – Toml-File
     (Toml => .lib)

   • The EDK II dependency means the EDK II lib dependency declared
     in INF.

     • If a Rust module is built with C, the cargo must use
       staticlib. Otherwise, rlib is used.

     • A simple example that specifies staticlib in the package
       Cargo.toml file using crate-type = ["staticlib"] in the
       [lib] section is shown below.

       [lib]
       crate-type = ["staticlib"]

5. C Module + Pure Rust Library with EDK II Dependency.

   • Rust library build is supported by build rule – Toml-File.
     (Toml => .lib)

6. Pure Rust Module + Pure Rust Library with EDK II Dependency.

   • Same as (4) + (5).

---

Command-Line Rust Build and Testing Commands

Note: "Pacakge" in this section refers to Rust packages not EDK II
packages. See the following explanation of packages and crates:

https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html

Individual Rust packages can be built and tested direcly on the
command-line. This provides a much faster development loop to build
and test the Rust code than performing a full EDK II build.

To build an individual Rust package:

cargo make build

The following command line options are available:

1. -p PROFILE [development|release]. DEFAULT =
   development (debug)
2. -e ARCH=[IA32|X64|AARCH64|LOCAL]. DEFAULT = X64
3. -e TARGET_TRIPLE=[triple].

• ARCH=LOCAL is used to build any locally executable tools
  associated with a Rust library package (e.g., a dual-purpose
  executable and library).

• TARGET_TRIPLE=<triple> is used to cross-compile locally
  executable tools associated with a Rust library package.

• Note: A Rust package must be specified.

• The output location is:

  • target/[x86_64-unknown-uefi|i686-unknown-uefi]/[debug|release]/
module_name.[efi|rlib]

To test an individual Rust package (run Rust unit tests):

cargo make test <Optional: Package Name>

To get Rust unit test coverage for an individual package:

cargo make coverage <Optional: Package Name>

• Note: If a package is not specified, all packages will be tested.
  Multiple packages can be provided, comma separated.
• Use cargo make coverage to generate coverage results on top of
  testing.

---

Generally Getting Started with Rust Development

It is recommended to:

1. Update the repo to ensure it includes the changes needed for Rust
   build support. Delete the /Conf directory to pick up the latest
   build rule changes.

2. Download and install Rust and cargo from
   https://www.rust-lang.org/learn/get-started

3. Verify cargo is working:

   >cargo --version

4. Install the desired Rust tool chain. The version is found in the
   rust-toolchain.toml file at the root of the edk2 repo.

   • Example: 1.74.0 x86_64 toolchain

   • Windows:

     >rustup toolchain install \
1.74.0-x86_64-pc-windows-msvc

     >rustup component add rust-src \
--toolchain 1.74.0-x86_64-pc-windows-msvc

   • Linux:

     >rustup toolchain install \
1.74.0-x86_64-unknown-linux-gnu

     >rustup component add rust-src --toolchain \
1.74.0-x86_64-unknown-linux-gnu

5. Install cargo make

   >cargo install --force cargo-make

6. Install cargo tarpaulin

   >cargo install --force cargo-tarpaulin

At this point, the essential Rust applications are installed.

---

PATCH 3 .pytool/Plugin: Add Rust Host Unit Test CI plugin

Adds a CI plugin, RustHostUnitTestPlugin, with the scope rust-ci
that runs all Rust unit tests, ensuring they pass. If they pass, code
coverage is calculated, which must meet the requirements specified in
a package's ci.yaml file (default is 75% code coverage). The plugin
will generate a coverage.xml file in the Build directory.

The 75% default code coverage bar is used to help ensure that new
Rust code has at least 3/4 of the code covered by tests.

Note: Add the rust-ci scope to your settings python file for
stuart_ci_build to activate the plugin. Add toml to your
pip-requirements.txt file to satisfy the plugin's dependency on the
toml PIP module.

Other notes:

• The plugin will fail when a Rust test fails to compile or cargo
  tarpaulin (used to generate Rust code coverage) fails to run.
• The plugin logs the reason for failing as a warning in addition to
  logging the output to the XML file.
• The plugin will filter results to only include results from Rust
  crates inside the EDK II package being tested.
• Regardless of the scope set in the build, if the plugin detects
  that no Rust crates are present in an EDK II package, code coverage
  will not be generated.
• The plugin will turn off code coverage if it detects it is running
  on a Windows Arm host machine since tarpaulin does not support that
  type of device at this time.

---

PATCH 4: BaseTools/Plugin: Add Rust Environment Check build plugin

Background

Firmware developer's machines are often not setup for Rust. As more
firmware developers begin working with Rust for the first time, this
plugin is used to provide early and direct feedback about the
developer's environment so it can successfully build Rust code using
the tools commonly used in the firmware build process.

The plugin is run when the rust-ci scope is specified which is used
by build scripts to opt into Rust plugin support.

The entire plugin takes ~1.4 sec to run on average so build time is
not meaningfully impacted.

---

Configuration

A feature that the plugin provides is the ability to set certain
tool exclusions. Using this, a build wrapper sets an environment
variable (chosen as the input method for its simplicity) to exclude a
list of tools.

For example, a GitHub workflow that only runs CodeQL for C code,
might need cargo installed for Rust compilation as part of the
package build, but might not need cargo tarpaulin for code coverage.

The GitHub workflow can set the environment variable on the build
step to opt out of verifying those tools.

This exclusion option is not intended to be used often as most local
developers and build environments are expected to have the base set
of tools needed to build and test the code when the plugins scope is
present.

It is very simple to set the environment variable in a CI environment
like an Azure Pipeline step as shown below for a step that builds
from information in a matrix with CodeQL enabled.

    - name: CI Build
      env:
        RUST_ENV_CHECK_TOOL_EXCLUSIONS: "cargo fmt, cargo tarpaulin"
        STUART_CODEQL_PATH: ${{ steps.cache_key_gen.outputs.codeql_cli_ext_dep_dir }}
      run: stuart_ci_build -c .pytool/CISettings.py -t DEBUG -p ${{ matrix.package }} -a ${{ matrix.archs }} TOOL_CHAIN_TAG=${{ matrix.tool_chain_tag }} --codeql

Another feature the plugin provides is the ability to enforce version
requirements for some Rust related tools needed for build. Repository
owners can optionally require specific versions of tools to be
installed. This is helpful because it pins the supported toolset at a
given point in the repos timeline. As an example, if a repository is
on Rust 1.73 (as specified in the rust-toolchain.toml file), "cargo
install" on the latest tools might not work as they might require
Rust 1.76. With this check, the user can simply be information to
install the tool version known to be compatible at that time in the
repo rather than receive ambiguous compilation failures when trying
to install the latest tools.

These tool version can be specified in the rust-toolchain.toml file
at the root of the repo. The following example updates the install
command suggestion output from the plugin to include the specific
versions given for cargo-make and cargo-tarpaulin.

[toolchain]
channel = "1.76.0"

[tool]
cargo-tarpaulin = "0.27.3"
cargo-make = "0.37.9"

How This Was Tested

Builds of INF files containing Rust Cargo.toml files. For example,
UefiHidDxeV2 is built with these changes.

See other usages in the Rust Build Combinations Supported section for
combinations supported.

All commands in the cargo make config file have been tested including
its usage with the rust-analyzer plugin in VS Code.

Integration Instructions

These changes should be backward compatible with existing build support
while additionally enabling Rust modules to be built. Follow the instructions
in each section for more information about the given plugin or build combination
supported. The release/202311 mu_plus branch has Rust build examples,
search for Cargo.toml in the repo to find the modules.