Overview
This project demonstrates how to boot to Linux on the Agilex SoC HPS using eMMC instead of the standard SD card.
The main changes vs. the standard SD card flow are:
- Hardware project is modified to target the eMMC daughtercard
- U-Boot device tree is patched to match the eMMC daughtercard
- Linux device tree is patched to match the eMMC daughtercard
- eMMC image is downloaded over TFTP and written by U-Boot which is ran by the debugger
Prerequisites
You will need the following items:
- Linux host PC (Ubuntu 22.04LTS was used for developing this project, but other versions may work too)
- Internet access (for downloading files attached to this page, and cloning git trees from github)
- TFTP server running on host computer (or other accessible computer on the local network)
- Quartus Prime Pro v24.1
- Agilex SoC Development Board production version with
- NAND/eMMC HPS Daughtercard
- SDM QSPI Bootcard with MT25QU02G flash device
Development Kit
See
https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/kit-agf-si.html for more details about the development kit.
NAND/eMMC Daughtercard
The NAND/eMMC HPS daughtercard that comes with the S10 SoC DevKit supports both NAND and eMMC.
To configure the card for eMMC operation, please set the jumpers as shown below:
| Jumper |
Setting |
|---|
| J2 |
unpopulated |
| J9 |
unpopulated |
| J10 |
populated 1-2 |
| J11 |
populated 2-3 |
For reference, the NAND/eMMC schematic is available at
https://www.intel.com/content/dam/altera-www/global/en_US/support/boards-kits/arria10/soc/hps_io48_nand_dc.pdf.
Deliverables
The following files are available as part of this example:
Building Instructions
Setting Up Environment
Create a top folder for this example, as the rest of the commands assume this location:
sudo rm -rf agilex-emmc-boot
mkdir agilex-emmc-boot
cd agilex-emmc-boot
export set TOP_FOLDER=`pwd`
Compilation of the software relies on downloading the build toolchain and adding it to the PATH variable:
cd $TOP_FOLDER
wget https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/\
gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz
tar xf gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz
rm -f gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz
export PATH=`pwd`/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/bin:$PATH
export ARCH=arm64
export CROSS_COMPILE=aarch64-none-linux-gnu-
Enable Quartus tools to be called from command line:
export QUARTUS_ROOTDIR=~/intelFPGA_pro/24.1/quartus/
export PATH=$QUARTUS_ROOTDIR/bin:$QUARTUS_ROOTDIR/linux64:$QUARTUS_ROOTDIR/../qsys/bin:$PATH
Building Hardware Design
The hardware design is downloaded from Github, then it's configured to enable the eMMC HPS card, then it's compiled:
cd $TOP_FOLDER
rm -rf ghrd-socfpga agilex_soc_devkit_ghrd
git clone -b QPDS24.1_REL_GSRD_PR https://github.com/altera-opensource/ghrd-socfpga
mv ghrd-socfpga/agilex_soc_devkit_ghrd .
rm -rf ghrd-socfpga
cd agilex_soc_devkit_ghrd
# select the emmc hps daughtercard
export DAUGHTER_CARD=devkit_dc_emmc
# use fpga configuration first instead of hps boot first
export BOOTS_FIRST=fpga
# disable sgmii and partial reconfiguration - to decrease build time
export HPS_ENABLE_SGMII=0
export ENABLE_PARTIAL_RECONFIGURATION=0
make scrub_clean_all
make generate_from_tcl
make all
cd ..
The following relevant files are created:
- $TOP_FOLDER/agilex_soc_devkit_ghrd/output_files/ghrd_agfb014r24b2e2v.sof - FPGA configuration file, without HPS FSBL
- $TOP_FOLDER/agilex_soc_devkit_ghrd/software/hps_debug/hps_debug.ihex - HPS Debug FSBL
- $TOP_FOLDER/agilex_soc_devkit_ghrd/output_files/ghrd_agfb014r24b2e2v_hps_debug.sof - FPGA configuration file, with HPS Debug FSBL
Building Build Arm Trusted Firmware
cd $TOP_FOLDER
rm -rf arm-trusted-firmware
git clone https://github.com/altera-opensource/arm-trusted-firmware
cd arm-trusted-firmware
# comment out next line to use the latest default branch
git checkout -b test -t origin/socfpga_v2.10.0
make -j 64 bl31 PLAT=agilex DEPRECATED=1
cd ..
Building U-Boot
cd $TOP_FOLDER
wget https://www.rocketboards.org/foswiki/pub/Projects/AgilexSoCWithEMMCStorageOnHPS/u-boot-socfpga-emmc.patch
rm -rf u-boot-socfpga
git clone https://github.com/altera-opensource/u-boot-socfpga
cd u-boot-socfpga
# comment out next line to use the latest default branch
git checkout -b test -t origin/socfpga_v2023.10
# change device tree to account for board differences
patch -p 1 < ../u-boot-socfpga-emmc.patch
# enable dwarf4 debug info, for compatibility with arm ds
sed -i 's/PLATFORM_CPPFLAGS += -D__ARM__/PLATFORM_CPPFLAGS += -D__ARM__ -gdwarf-4/g' arch/arm/config.mk
# use 'Image' for kernel image instead of 'kernel.itb'
sed -i 's/kernel\.itb/Image/g' include/configs/socfpga_soc64_common.h
# only boot from SD, do not try QSPI and NAND
sed -i 's/u-boot,spl-boot-order.*/u-boot\,spl-boot-order = \&mmc;/g' arch/arm/dts/socfpga_agilex_socdk-u-boot.dtsi
# disable NAND in the device tree
sed -i '/&nand {/!b;n;c\\tstatus = "disabled";' arch/arm/dts/socfpga_agilex_socdk-u-boot.dtsi
# remove the NAND configuration from device tree
sed -i '/images/,/binman/{/binman/!d}' arch/arm/dts/socfpga_agilex_socdk-u-boot.dtsi
# link to atf
ln -s $TOP_FOLDER/arm-trusted-firmware/build/agilex/release/bl31.bin .
# Create configuration custom file.
cat << EOF > config-fragment-agilex
# - Disable NAND/UBI related settings from defconfig.
CONFIG_NAND_BOOT=n
CONFIG_SPL_NAND_SUPPORT=n
CONFIG_CMD_NAND_TRIMFFS=n
CONFIG_CMD_NAND_LOCK_UNLOCK=n
CONFIG_NAND_DENALI_DT=n
CONFIG_SYS_NAND_U_BOOT_LOCATIONS=n
CONFIG_SPL_NAND_FRAMEWORK=n
CONFIG_CMD_NAND=n
CONFIG_MTD_RAW_NAND=n
CONFIG_CMD_UBI=n
CONFIG_CMD_UBIFS=n
CONFIG_MTD_UBI=n
CONFIG_ENV_IS_IN_UBI=n
CONFIG_UBI_SILENCE_MSG=n
CONFIG_UBIFS_SILENCE_MSG=n
# - Disable distroboot and use specific boot command.
CONFIG_DISTRO_DEFAULTS=n
CONFIG_HUSH_PARSER=y
CONFIG_SYS_PROMPT_HUSH_PS2="> "
CONFIG_USE_BOOTCOMMAND=y
CONFIG_BOOTCOMMAND="bridge enable;run mmcload;run linux_qspi_enable;run rsu_status;run mmcboot"
CONFIG_CMD_FAT=y
CONFIG_CMD_FS_GENERIC=y
CONFIG_DOS_PARTITION=y
CONFIG_SPL_DOS_PARTITION=y
CONFIG_CMD_PART=y
CONFIG_SPL_CRC32=y
CONFIG_LZO=y
CONFIG_CMD_DHCP=y
# Enable more QSPI flash manufacturers
CONFIG_SPI_FLASH_MACRONIX=y
CONFIG_SPI_FLASH_GIGADEVICE=y
CONFIG_SPI_FLASH_WINBOND=y
EOF
# build U-Boot
make clean && make mrproper
make socfpga_agilex_defconfig
# Use created custom configuration file to merge with the default configuration obtained in .config file.
./scripts/kconfig/merge_config.sh -O ./ ./.config ./config-fragment-agilex
make -j 64
cd ..
The following files will be created:
- $TOP_FOLDER/u-boot-socfpga/spl/u-boot-spl-dtb.hex - FSBL/SPL hex file
- $TOP_FOLDER/u-boot-socfpga/u-boot.itb - U-Boot and ATF combined image file
Building the QSPI Image
cd $TOP_FOLDER
quartus_pfg -c agilex_soc_devkit_ghrd/output_files/ghrd_agfb014r24b2e2v.sof \
flash_image.jic \
-o device=MT25QU128 \
-o flash_loader=AGFB014R24B2E2V \
-o hps_path=u-boot-socfpga/spl/u-boot-spl-dtb.hex \
-o mode=ASX4
The following file will be created:
- $TOP_FOLDER/flash_image.jic
Building Linux
The following instructions are used to build Linux:
cd $TOP_FOLDER
wget https://www.rocketboards.org/foswiki/pub/Projects/AgilexSoCWithEMMCStorageOnHPS/linux-socfpga-emmc.patch
rm -rf linux-socfpga
git clone https://github.com/altera-opensource/linux-socfpga
cd linux-socfpga
# comment out next line to use the latest default branch
git checkout -b test -t origin/socfpga-6.1.68-lts
patch -p 1 < ../linux-socfpga-emmc.patch
make clean && make mrproper
make defconfig
make -j 64 Image dtbs
cd ..
The following relevant files are created:
- $TOP_FOLDER/linux-socfpga/arch/arm64/boot/Image - Linux kernel image
- $TOP_FOLDER/linux-socfpga/arch/arm64/boot/dts/intel/socfpga_agilex_socdk.dtb - Linux Device Tree Blob
Building Rootfs
A root file system is required to boot Linux. There are a lot of ways to build a root file system, depending on your specific needs. This section shows how to build a small root file system using Yocto.
1. First, make sure you have Yocto system requirements met:
https://docs.yoctoproject.org/3.4.1/ref-manual/system-requirements.html#supported-linux-distributions.
The command to install the required packages on Ubuntu is:
sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath \
socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \
python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit \
mesa-common-dev zstd liblz4-tool
Note: You can also use a Docker container to build the Yocto recipes, refer to
https://rocketboards.org/foswiki/Documentation/DockerYoctoBuild for details. When using a Docker container, it does not matter what Linux distribution or packages you have installed on your host, as all dependencies are provided by the Docker container.
2. Run the following commands to build the root file system:
cd $TOP_FOLDER
rm -rf yocto && mkdir yocto && cd yocto
git clone -b nanbield http://git.yoctoproject.org/poky
git clone -b nanbield http://git.yoctoproject.org/meta-intel-fpga
git clone -b nanbield http://git.openembedded.org/meta-openembedded
source poky/oe-init-build-env ./build
echo 'MACHINE = "agilex7_dk_si_agf014ea"' >> conf/local.conf
echo 'BBLAYERS += " ${TOPDIR}/../meta-intel-fpga "' >> conf/bblayers.conf
echo 'BBLAYERS += " ${TOPDIR}/../meta-openembedded/meta-oe "' >> conf/bblayers.conf
echo 'IMAGE_FSTYPES = "tar.gz"' >> conf/local.conf
echo 'CORE_IMAGE_EXTRA_INSTALL += " fio"' >> conf/local.conf
bitbake core-image-minimal
After the build completes, which can take a few hours depending on your host system processing power and Internet connection speed, the following root file system archive will be created:
- $TOP_FOLDER/yocto/build/tmp/deploy/images/agilex7_dk_si_agf014ea/core-image-minimal-agilex7_dk_si_agf014ea.rootfs.tar.gz
Refer to the guides from Rocketboards Getting Started
https://rocketboards.org/foswiki/Main/GettingStarted page for details about building Linux.
Creating the eMMC Image
The eMMC image is created with the same procedure used for creating SD card images:
cd $TOP_FOLDER
rm -rf sd_card
mkdir sd_card && cd sd_card
wget https://releases.rocketboards.org/2021.04/gsrd/tools/make_sdimage_p3.py
# remove mkfs.fat parameter which has some issues on Ubuntu 22.04
sed -i 's/\"\-F 32\",//g' make_sdimage_p3.py
chmod +x make_sdimage_p3.py
mkdir fat && cd fat
cp $TOP_FOLDER/u-boot-socfpga/u-boot.itb .
cp $TOP_FOLDER/linux-socfpga/arch/arm64/boot/Image .
cp $TOP_FOLDER/linux-socfpga/arch/arm64/boot/dts/intel/socfpga_agilex_socdk.dtb .
cd ..
mkdir rootfs && cd rootfs
sudo tar xf $TOP_FOLDER/yocto/build/tmp/deploy/images/agilex7_dk_si_agf014ea/core-image-minimal-agilex7_dk_si_agf014ea.rootfs.tar.gz
sudo rm -rf lib/modules/*
cd ..
sudo python3 make_sdimage_p3.py -f \
-P rootfs/*,num=2,format=ext3,size=42M \
-P fat/*,num=1,format=fat32,size=42M \
-s 96M \
-n emmc-image-agilex.img
cd ..
The following image file will be created:
- $TOP_FOLDER/sd_card/emmc-image-agilex.img
Flashing Instructions
Writing the QSPI Image
1. Set the MSEL to JTAG & power cycle the board
2. Use Quartus Programmer to program the QSPI flash:
cd $TOP_FOLDER
quartus_pgm -c 1 -m jtag -o "pvi;flash_image.jic"
3. Set MSEL back to QSPI
Note: If there was no previous design running, or if the previous design has been running fine, the MSEL does not need to be set to JTAG. It can remain set to QSPI, and you can still program QSPI flash succesfully. In case there was a previous design running, run "jtagconfig" to determine if the Agilex device is first or second in the JTAG scan chain. If it is second, change the above command to:
cd $TOP_FOLDER
quartus_pgm -m jtag -o "pvi;./flash_image.jic@2"
Writing the eMMC Image
The flow for writing the eMMC image is:
- Configure the FPGA over JTAG with an image containing a dummy HPS SPL, which just stays in an infinite loop
- Load and run SPL and U-Boot to HPS over JTAG
- From U-Boot command prompt, transfer the eMMC image to HPS SDRAM over TFTP
- From U-Boot command prompt, write the eMMC image from SDRAM to eMMC
1. Run U-Boot from the debugger, as shown here:
https://rocketboards.org/foswiki/Documentation/BuildingBootloader#Agilex_45_Run_U_45Boot_from_Debugger
2. The above will cause the U-Boot to start running on HPS. Stop U-Boot counter by pressing any key on the serial console.
3. Copy the eMMC image to the TFTP folder on your host:
cd your_tftp_folder
cp $TOP_FOLDER/sd_card/emmc-image-agilex.img .
4. Determine the IP of your host computer running TFPT, by running 'ifconfig' command.
5. In U-Boot connect to the local network, and set the server ip parameter, where the tftp commands will connect to:
setenv autoload no
dhcp
setenv serverip your_host_ip_address
6. In U-Boot, download the eMMC image and write it to flash:
tftp ${loadaddr} emmc-image-agilex.img
setexpr blkcnt ${filesize} / 0x200
mmc write ${loadaddr} 0 ${blkcnt}
Booting Linux
1. Set the MSEL to JTAG
2. Power cycle the board.
3. Linux will boot up. Use 'root' as username, a password will not be required:
U-Boot SPL 2023.10-31805-g69d5338be0-dirty (Apr 05 2024 - 21:03:35 -0500)
Reset state: Cold
MPU 1200000 kHz
L4 Main 400000 kHz
L4 sys free 100000 kHz
L4 MP 200000 kHz
L4 SP 100000 kHz
SDMMC 50000 kHz
DDR: 8192 MiB
SDRAM-ECC: Initialized success with 1722 ms
QSPI: Reference clock at 400000 kHz
WDT: Started watchdog@ffd00200 with servicing every 1000ms (10s timeout)
Trying to boot from MMC1
## Checking hash(es) for config board-0 … OK
## Checking hash(es) for Image atf … crc32+ OK
## Checking hash(es) for Image uboot … crc32+ OK
## Checking hash(es) for Image fdt-0 … crc32+ OK
NOTICE: BL31: v2.10.0 (release):QPDS24.1_REL_GSRD_PR
NOTICE: BL31: Built : 21:03:06, Apr 5 2024
U-Boot 2023.10-31805-g69d5338be0-dirty (Apr 05 2024 - 21:03:35 -0500)socfpga_agilex
CPU: Intel FPGA SoCFPGA Platform (ARMv8 64bit Cortex-A53)
Model: SoCFPGA Agilex SoCDK
DRAM: 2 GiB (effective 8 GiB)
Core: 27 devices, 22 uclasses, devicetree: separate
WDT: Started watchdog@ffd00200 with servicing every 1000ms (10s timeout)
MMC: dwmmc0@ff808000: 0
Loading Environment from FAT... Unable to read "uboot.env" from mmc0:1...
In: serial0@ffc02000
Out: serial0@ffc02000
Err: serial0@ffc02000
Net:
Warning: ethernet@ff804000 (eth0) using random MAC address - a2:df:c5:c0:c1:26
eth0: ethernet@ff804000
Hit any key to stop autoboot: 5 4 3 2 1 0
38623744 bytes read in 1027 ms (35.9 MiB/s)
17897 bytes read in 3 ms (5.7 MiB/s)
SF: Detected mt25qu02g with page size 256 Bytes, erase size 64 KiB, total 256 MiB
Enabling QSPI at Linux DTB...
Working FDT set to 8000000
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
QSPI clock frequency updated
RSU: Firmware or flash content not supporting RSU
RSU: Firmware or flash content not supporting RSU
RSU: Firmware or flash content not supporting RSU
RSU: Firmware or flash content not supporting RSU
## Flattened Device Tree blob at 08000000
Booting using the fdt blob at 0x8000000
Working FDT set to 8000000
Loading Device Tree to 000000007eafd000, end 000000007eb04fff … OK
Working FDT set to 7eafd000
Starting kernel …
Deasserting all peripheral resets
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[ 0.000000] Linux version 6.1.68-248670-ge9428cbd9991-dirty (radu@radu2-linux-lab) (aarch64-none-linux-gnu-gcc (GNU Toolchain for the Arm Architecture 11.2-2022.02 (arm-11.14)) 11.2.1 20220111, GNU ld (GNU Toolchain for the Arm Architecture 11.2-2022.02 (arm-11.14)) 2.37.20220122) #1 SMP PREEMPT Fri Apr 5 21:10:03 CDT 2024
[ 0.000000] Machine model: SoCFPGA Agilex SoCDK
…
[ 4.467043] socfpga-dwmac ff804000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[ 4.475862] socfpga-dwmac ff804000.ethernet eth0: registered PTP clock
[ 4.482853] socfpga-dwmac ff804000.ethernet eth0: configuring for phy/rgmii link mode
udhcpc: started, v1.36.1
udhcpc: broadcasting discover
udhcpc: broadcasting discover
[ 8.580612] socfpga-dwmac ff804000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
udhcpc: broadcasting discover
udhcpc: no lease, forking to background
done.
Starting syslogd/klogd: done
Poky (Yocto Project Reference Distro) 4.3.4 agilex7_dk_si_agf014ea /dev/ttyS0
agilex7_dk_si_agf014ea login: root
[ 16.837602] socfpga-dwmac ff804000.ethernet eth0: Link is Down
root@agilex7_dk_si_agf014ea:~#
RocketBoards