Overview

Datamover example design sets to demonstrate design practices and software solutions to achieve high performance real time application with HPS ARM processor. It also serves as example design for users to evaluate their real time needs with the proposed software solutions.

Datamover project was carried out to establish the data communication between FPGA logic and HPS SDRAM. Packet processing time was captured using an externally developed “jitter monitor” module. The loop time is defined as total duration for a packet to enter SoC, being touched, and lastly transferred onto onchip memory in FPGA. Both DMA transfer durations are also monitored. Jitter of each measurement, loop time, ingress DMA time, and egress DMA time are also calculated.

The software section consists of three major parts, Linux with core affinity, VxWorks with core affinity and as well as baremetal solution utilizing hardware libs. All three software implementations yield different results and statistics that will help user to decide the best software solution to their real time needs.

Process Flow

1.)Nios II will initial mSGDMA0 to move data into HPS SDRAM through the F2H AXI bridge.

2.) Data passing from HPS SDRAM into FPGA Onchip Memory can be done via either of the following paths:

i.) Using mSGDMA 1 to move data from On-chip ram to FPGA Onchip Memory via F2H AXI Bridge,
ii.) Using mSGDMA 0 to move data from On-chip ram to FPGA Onchip Memory via F2H AXI Bridge

3.) Time tracking will be done at Nios II. The Nios II will track the time taken for each DMA transfer to HPS to returning of DMA data to FPGA. 3 time stamps will be used to track the complete loop of the data transfer.

1.) Total looptime (FPGA → HPS → FPGA)
2.) DMA transfer time (FPGA → HPS)
3.) DMA transfer time (HPS → FPGA)

4.) Nios II will start 2 performance counter associate with the DMA transfer (FPGA → HPS) and total looptime. Once the DMA completes the transfer, the HPS will stop the counter associate with DMA transfer (FPGA→HPS) and start the performance counter for DMA transfer (HPS→ FPGA) before initiating the DMA transfer (HPS → FPGA). Once the total loop is completed, Nios II will be responsible to stop the counter for DMA transfer (HPS → FPGA) and for total loop.

5.) In order to record the time durations for an interleaf data transfer, two sets of performance counters will be used to track the duration of two ongoing DMA transfer.

Prerequiites

Hardware requirement

1.) Cyclone V SoC Development Kit
2.) 4GB Micro SDHC flash card
3.) Host PC running Linux
4.) Quartus II Version 14.0

Deliverables

FPGA,Linux and Nios II Binaries

The reference design deliverables include binaries that can be used to run the reference design directly. The Linux binaries are delivers as an archive file accessible at <a href="https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz"

Folder	Folder	Folder	File	Description
cv_datamover_ed	bin	Linux	altera-datamover-image-socfpga_cyclone5.tar.gz	Zipped Root filesystem
^	^	^	sdimage.bin	SD Card Image
^	^	^	altera-datamover-image-socfpga_cyclone5.dtb	Device Tree Binary
^	^	^	Preloader-mkimage.bin	Preloader Image
^	^	^	vmlinux	Linux Kernel Executable
^	^	^	zImage	Linux Kernel Compressed Image
^	^	FPGA	Datamover_5csxfc6.sof	FPGA Configuration SOF
^	^	^	Datamover_5csxfc6.flash	FPGA Configuration Flash
^	^	Nios II	Datamover_demo.elf	Nios II Application Executable
^	^	^	Datamover_demo.flash	Nios II Application Flash Image

VxWorks and HWLibs Binaries

The VxWorks and hwlibs binaries are deliver as an archive files accessible at https://www.altera.com/support/support-resources/download/rtos_tools.html

The binaries archive contains the following items:

Folder	Folder	Folder	Item	Description
Altera_datamover_ed	bin	vxworks	vxw_datamover_sdmmc.img	VxWorks SD Card Image
			vxWorks	VxWorks Kernel

Hardware Design

The hardware design is delivered as an archive file accessible at https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz

Some of the relevant included files and folders are:

Folder	Fodler	Item	Description
cv_datamover_ed	hw	ip/	Folder containing IP files
		datamover_5csxfc6.qpf	Quartus Project File
		datamover_5csxfc6.qsf	Quartus Setting File
		datamover_5csxfc6.qsys	Qsys File
		soc_system_timing.sdc	Timing file
		soc_system_board_info.xml	SoC system board XML file
		hps_common_board_info.xml	HPS XML file
		datamover_top.v	Top Level Verilog File

Nios II Application Source

The Nios II application source is delivered as an archive file accessible at https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz

Folder	Folder	Folder	Item
altera-datamover-image-socfpga_cyclone5.tar.gz	Zipped Root filesystem		^	^
^	^	^	sdimage.bin	SD Card Image
^	^	^	altera-datamover-image-socfpga_cyclone5.dtb	Device Tree Binary
^	^	^	Preloader-mkimage.bin	Preloader Image
^	^	^	vmlinux	Linux Kernel Executable
^	^	^	zImage	Linux Kernel Compressed Image
^	^	FPGA	Datamover_5csxfc6.sof	FPGA Configuration SOF
^	^	^	Datamover_5csxfc6.flash	FPGA Configuration Flash
^	^	Nios II	Datamover_demo.elf	Nios II Application Executable
^	^	^	Datamover_demo.flash	Nios II Application Flash Image

VxWorks and HWlibs Binaries

The VxWorks and hwlibs binaries are deliver as an archive files accessible at https://www.altera.com/support/support-resources/download/rtos_tools.html The binaries archive contains the following items:

Folder	Folder	Folder	File	Description
Altera_datamover_ed	bin	vxworks	vxw_datamover_sdmmc.img	VxWorks SD Card Image
^	^	^	vxWorks	VxWorks Kernel

Hardware Design

The hardware design is delivered as an archive file accessible at https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz. Some of the relevant included files and folders are:

Folder	Folder	Item	Description
cv_datamover_ed	hw	ip/	Folder containing IP files
^	^	datamover_5csxfc6.qpf	Quartus Project File
^	^	datamover_5csxfc6.qsf	Quartus Setting File
^	^	datamover_5csxfc6.qsys	Qsys File
^	^	soc_system_timing.sdc	Timing file
^	^	soc_system_board_info.xml	SoC system board XML file
^	^	hps_common_board_info.xml	HPS XML file
^	^	datamover_top.v	Top Level Verilog File

Nios II Application Source

The Nios II application source is delivered as an archive file accessible at https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz

Folder	Folder	Folder	Item	Description
cv_datamover_ed	sw	niosii_hal	datamover_demo.c	Nios II Application Source

Linux patch

The Linux patch is delivered as an archive file accessible at https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz

Folder	Folder	Folder	Item	Description
cv_datamover_ed	sw	linux	altera_datamover_yocto.patch	datamover yocto patch

Linux Yocto Recipes

All the necessary files to build the Linux kernel, drivers, applications and root filesystem are delivered as a set of Yocto recipes accessible through the git trees at https://github.com/altera-opensource/.

Folder	Folder	Tag
Yocto Recipes	Poky-socfpga.git	ACDS14.0_REL_GSRD_PR

VxWorks

The VxWorks source is delivered as an archive file accessible at https://www.altera.com/support/support-resources/download/rtos_tools.html

Folder	Folder	Folder	Item	Description
Altera_datamover_ed	sw	vxworks	datamover_dkm.zip	Datamover DKM Project

HwLibs

The Hwlibs source is delivered as an archive file accessible at <u>https://www.altera.com/support/support-resources/download/rtos_tools.html

Folder	Folder	Folder	Item	Description
Altera_datamover_ed	sw	hwlibs	Altera-SoCFPGA-DataMover-CV-GNU.tar.gz	Datamover HwLibs Example Application

Getting Started

Obtain FPGA, Linux and NiosII Binaries

Download the example design archive from https://releases.rocketboards.org/2014.04/dwamp-rd/cv-dwamp-rd-bin.tar.gz and save it to user home folder.Unzip the example design archive:

$ cd ~
$ tar xzf cv_datamover_ed.tar.gz

The following files will be used in order to run the Reference Design:

Folder	Folder	Folder	File	Description
cv_datamover_ed	bin	Linux	sdimage.bin	SD Card Image
^	^	FPGA	Datamover_5csxfc6.flash	FPGA Configuration Flash
^	^	Nios	Datamover_5csxfc6.flash	NiosConfiguration Flash

Obtain VxWorks Binaries

Download the VxWorks binaries from https://www.altera.com/support/support-resources/download/rtos_tools.html, and save it to user home folder.

Unzip the example design archive:

$ cd ~
$ unzip Altera_datamover_ed.zip

The following files will be used in order to run the Reference Design:

Folder	Folder	Folder	File	Description
Altera_datamover_ed	bin	vxworks	vxw_datamover_sdmmc.img	Vxwork image

Board Setup

J5	open
J6	shorted
J7	shorted
J9	open
J13	shorted
J16	open
J26	right shorted
J27	right shorted
J28	right shorted
J29	right shorted
J30	left shorted
J31	open
Jumper	Setting

Switches:

SW1	All OFF
SW2	All OFF
SW3	1:ON 2:OFF 3:ON 4:ON 5:OFF 6:ON
SW4	1:OFF 2:OFF 3:ON 4:ON
Switch	Setting

Configuring Board to Use EPCQ

By default the board is configured to use the onboard FPGA configuration device as EPCS.

If not already done so, please follow the instructions posted at http://www.altera.com/support/kdb/solutions/rd11192013_118.html in order to configure the board to use the configuration device as EPCQ. Note that these steps are only needed to be done once. After that the board will be configured correctly, with the configuration stored in flash.

For a step-by-step tutorial on how to configure board to use EPCQ please click here.

Flashing Board

Write FPGA Flash Image by open the Nios II Command shell

$ cd ~/cv_datamover_ed
$ nios2-configure-sof bin/fpga/datamover_5csxfc6.sof –d 2
$ nios2-flash-programmer --base=0x10040000 --epcs bin/fpga/datamover_5csxfc6.flash 

Write Nios II Flash Image

$ nios2-flash-programmer --base=0x10040000 --epcs bin/nios/Datamover_5csxfc6.flash

After successful write the fpga and Nios flash image into EPCQ, the FPGA flash and Nios II flash will automatic boot from EPCQ after power-up the device.

$ nios-terminal

Hardware Component

The Datamover design system requires having the following components:
* FPGA * Nios II * Modular Scatter Gather Direct Memory Access (mSGDMA) * Performance Counter * Mailbox * Payload RAM * HPS (ARM Cortex-A9)

Components Nios II

A Nios II processor is placed in the FPGA as a traffic controller to perform the necessary data transfer between FPGA and HPS. The Nios II processor also serves as a time tracker to monitor the time taken to perform each data transfer.

Modular Scatter Gather Direct Memory Access (mSGDMA)
The mSGDMA is a soft logic to perform the necessary data transfer between FPGA and HPS via the FPGA to HPS (H2F) bridge. Two sets of mSGDMA are placed in the FPGA to perform the data from FPGA to HPS and HPS to FPGA.

Performance Counter
The performance counter soft logic in FPGA serves as a performance timer. It is accessible by Nios II to perform the performance monitoring of each data transfer. Each performance counter consist of one global timer and three performance timer to monitor the time taken for ingress, egress and total loop time of the data transfer

Mailbox
The mailboxes act as a command passing mechanism between FPGA and HPS. It also allows synchronization of the software between Nios II and HPS. Payload RAM
The payload RAM is an on chip ram in the FPGA as a temporary memory to store the data to be transferred between FPGA and HPS.

Software Flow

There are 3 different platforms employed for the same software flow which are Linux SMP, VxWorks SMP and Baremetal running on HPS.

Performance result between 3 platforms will be benchmarked against each other. The result will be shown in NiosII terminal.

Start Guide –Linux

1.) Please follow the board setup and flash board steps as stated above.
2.) Prepare SD Card Image by execute the following commands.

$ cd ~/cv_datamover_ed
$ sudo dd if=bin/linux/sdimage.bin of=/dev/sdx 

* *Please replace ‘dev/sdx’ with the name of the SD card device on your host computer. **Please ensure that the name of SD card device on your host computer is correct. You will not get the SD card image if you write into wrong drive

$ sudo sync

3.) Slot the SD card into the CV SoC dev kit and power-up the board.
4.) After booting process is finish, please login as *root at kernel terminal.
5.) Follow steps below to run datamover application.

$ modprobe datamover

6.) If you have already opened the Nios II terminal, you should observe the result as below.
Example output from Nios II terminal :


7.) Example output from UART terminal when datamover is executed in HPS:

8.) To continue test the program by press *CPU_RST button on the board to obtain stable and consistent result.

Start Guide –VxWorks

1.) Please follow the board setup and flash board steps as stated above.
2.) Prepare the VxWorks image by execute the following steps.
3.) Copy the SD card image from /bin/vxworks/vxw_datamover_sdmmc.img file into your local directory
4.) Use Win32DiskImager to write the SD image to the flash card * At the Image File, browse to the vxw_datamover_sdmmc.img directory * At the Device, browse the SD card driver. * Click Write to perform write the image to SD card.
5.) Slot in the SD card into the development kit after successful write.
6.) Power-up the board.
7.) You will need to perform a configuration for VxWorks environment for the 1st time. Please follow the steps below to modify the boot parameters.
8.) Boot the board and hit a key to stop the boot count-down process.
9.) Use the ‘c’ command to check and modify to each of the boot parameters.
10.) Enter the correct value and then hit <enter> to move to the next boot parameter.
11.) You cannot edit the existing values. You will need to enter the whole string.

[VxWorks Boot]: c

boot device: fs
processor number: 0
host name: host
file name: /sd0:1/vxWorks
inet on ethernet (e) : 192.168.192.204:ffffff00
inet on backplane (b):
host inet (h): 192.168.192.1
gateway inet (g):
user (u): target
ftp password (pw): vxTarget
flags (f): 0x0
target name (tn): alt_soc_gen5
startup script (s):
other (o): emac1

[VxWorks Boot]:

To delete the field use '.'
To go back to the previous parameter use '-'
To go to the next parameters use <enter>
To backspace use <ctrl>H

11.) After finish configuration, boot the VxWorks image by pressing the `<b>@</b>` key or power-up the development kit and wait for count-down.
12.) After VxWorks is booted, you should see the VxWorks splash screen as below.

13.) Follow steps below to perform run datamover design
Type “datamover_start”

14.) If you have already opened the Nios II terminal, you should observe the result as below.
Example output from Nios II terminal :


15.) Example output from UART terminal when the VxWorks is executed in HPS


16.) Continue to test the program by press CPU_RST button on the board to obtain stable and consistent result.
17.) You can also exercise processor core#1 by enter the following command:
Type “datamover_busy”
18.) By doing this, user can observe whether the performance of processor core#0 will be affected or not if processor core#1 is busy.

Steps to import VxWorks source and Image into Wind River Workbench

If you need to view and modify the VxWorks source code, then you can follow the steps below to import the archived files.

VxWorks Software Prerequisites

Wind River Workbench version : 3.3.4

Create VxWorks Source Build Project

This section shows how to create a source build project (VSB) to be able to incorporate the new BSP changes. 1.) Open WindRiver Workbench.
2.) Go to File → New → VxWorks Source Build (Kernel Library) Project
3.) Enter project name as VSB_SMP. Click Next.
4.) Base the project on board support package: alt_soc_gen5. Press Finish.
5.) Edit VSB Options
* Enable Symmetric Multiprocessing: SMP = y.
* Enable processsort specific optimizations: Symbol: ARMV7_CORE_CTX_A9_VSB = y.
6.) Save VSB Project: File → Save All.
7.) Build Project: Project → Build Project.

Create VxWorks Image Build Project

This section shows how to create ai image buid project (VIP) to be able to incorporate the new BSP changes. 1.) Open WindRiver Workbench.
2.) Go to File → New →VxWorks Image Project.
3.) Enter project name as VIP_SMP. Click Next.
4.) Base the project on a source build project and select VSB_SMP. Select the toolchain to be diab. Check the Enable WDB Target Agent to enable easy debugging. Uncheck Add support to project for the BSP validation test suite. Click Next.
5.) Click Next on the Options screen
6.) On the Configuration Profile window, select the Profile PROFILE_STANDALONE_DEVELOPMENT. Click Finish.
7.) Import the datamover source file into the VIP_SMP project.
8.) Go to File → Import →General →Archive File as shown in the diagram below


9.) Click “Next”.
10.) At the from archive file, browse to the datamover_dkm.zip directory.
11.) At the Into folder, browse to the VIP_SMP project.
12.) Click “Finish”.
13.) You should able to see the datamover_dkm folder has import into your project.

14.) Select the VIP_SMP project and build it by going to Project → Build All

Start Guide – HWLibs

1.) Please follow the board setup and flash board steps as stated above.
2.) Copy the *Altera-SoCFPGA-DataMover-CV-GNU.tar.gz into your local directory.
3.) Open ARM DS-5 Development Studio
4.) Import the HWlibs achieved file into ARM DS-5 Development Studio:
i.) File → Import
ii.) Select Existing Projects into Workspace and click Next.

5.) In the Import Projects, browse to the HWlibs achieved project directory.
6.) Select the archive file and browse to the *Altera-SoCFPGA-DataMover-CV-GNU.tar.gz directory
7.) click Finish.
8.) You should able to see the imported project in the Projects Explore.
9.) Go to Window → Open Perspective → Others
i.Select DS-5 Debug and click Ok.
10.) Right-click on the Altera-SoCFPGA-DataMover-CV-GNU project and select build project.
11.) After finish build project, right-click on the project and select Debug As → Debug Configurations
12.) Expand the DS-5 Debugger and click on the project name.
i.In the connection tab, select *Debug Cortex-A9_0
ii.Target Connection : USB-BlasterII
iii.In the Bare Metal Debug, ensure the connection is connecting correctly.
13.) Click on *Debug*
14.) After successful, you should able to see the design is connected to device.


15.) Click on the “Play” button and the HWLibs design will be executed.
16.) If you have already opened the Nios II terminal, you should observe the result as below.
Example output from Nios II terminal :


17.) Continue to test the program by press CPU_RST button on the board to obtain stable and consistent result.

Building the reference design

Software Development Flow Setting Up Yocto Environment

Setting Up Yocto Environment

Please follow this link to setup Yocto environment before starting this section
link : http://www.rocketboards.org/foswiki/Documentation/GSRD140YoctoUserManual**

Please follow this Documentation.GSRD140YoctoUserManual to setup Yocto environment before starting this section http://www.rocketboards.org/foswiki/Documentation/GSRD140YoctoUserManual

Apply Yocto Recipes and Patch for Datamover design example

Use below command to acquire poky source from Rocketboards Git:

$ git clone http://git.rocketboards.org/poky-socfpga.git
$ cd poky-socfpga
$ git checkout -b datamover-ed ACDS14.0_REL_GSRD_PR

Download the patch from release content section, apply the patch with below command:

$ git am cv_datamover_ed/sw/linux/altera_datamover_yocto.patch

Build U-Boot/Kernel/Rootfs:

$ source altera-init
$ bitbake virtual/bootloader
$ bitbake virtual/kernel
$ bitbake altera-datamover-image

Output files:

u-boot-socfpga_cyclone5.img	U-Boot image
zImage	Compressed kernel image
altera-datamover-image-socfpga_cyclone5.ext3	Root Filesystem as ext3 image
altera-datamover-image-socfpga_cyclone5.tar.gz	Root Filesystem in tar gzip archive format

Generate Device Tree Blob

Use below command to generate device tree blob

$ sopc2dts --input datamover_5csxfc6.sopcinfo --output <dtb.file.name>.dtb --type dtb --board soc_system_board_info.xml –board hps_common_board_info.xml --bridge_removal all –clocks

Output file: <dtb.file.name>.dtb

For more details about device tree generation, please refer to GSRD User Manual - Generating the Device Tree

Generate Preloader

For more details about preloader generation, please refer to GSRD - Generating and Compiling the Preloader.

Output file: preloader-mkpimage_cyclone5.bin

Build SD Card Image

Replace all required component into SD Card, or build and replace the whole image.

$ wget https://releases.rocketboards.org/2014.06/gsrd/tools/make_sdimage.py
$ mkdir rootfs
$ cd rootfs
$ sudo tar xzf ../altera-datamover-image-socfpga_cyclone5.tar.gz
$ cd ..
$ sudo ./make_sdimage.py -f -P preloader-mkimage.bin,u-boot-socfpga_cyclone5.img,num=3,format=raw,size=10M,type=A2 -P rootfs/*,num=2,format=ext3,size=1500M -P zImage ,socfpga.dtb,num=1,format=vfat,size=500M -s 2G -n sd_card_image_cyclone5.bin

For more details about SD card image generation, please refer to GSRD - Creating and Updating SD Card.


</noautolink>