Overview

The Datamover example design uses a Cyclone V SoC development kit to demonstrate data communication between the FPGA logic and SDRAM controlled through the Hard Processor System (HPS) portion of the device. The design uses a Nios II processor along with an mSGDMA in the FPGA fabric to move data through the F2H bridge. The Nios II is also used to track the time for each DMA transfer as well as the loop time, which is defined as the total duration for a packet to enter the SoC, be processed, and then transferred into on-chip memory in the FPGA fabric. The jitter of each measurement, ingress DMA time, and egress DMA time are also calculated.

The software for this design is discussed with respect to implementation using Linux with core affinity, VxWorks with core affinity, and a baremetal solution using hardware libs. All three software implementations yield different results and statistics that can assist a user in deciding the best software solution to meet the real-time requirements of a design.

Prerequisites

The following hardware and software components are required to implement this design:

Hardware

• Cyclone V SoC Development Kit
• 4GB Micro SDHC flash card

Software

• Quartus II Version 14.0
• Wind River Workbench version : 3.3.4
• ARM DS-5 Development Studio
• Win32DiskImager

Hardware Implementation

The following diagram shows the hardware implementation of the design in the FPGA and the required connections between the FPGA and the HPS.

The Datamover design system requires the following components:

• FPGA
  • Nios II
  • Two Modular Scatter Gather Direct Memory Access (mSGDMA)
  • Performance Counter
  • Two Mailboxes
  • Payload RAM
• HPS (ARM Cortex-A9)

Nios II
A Nios II processor is placed in the FPGA as a traffic controller to perform the necessary data transfer between FPGA and HPS. The Nios II processor also serves as a time tracker to monitor the time taken to perform each data transfer.

Modular Scatter Gather Direct Memory Access (mSGDMA)
The mSGDMA is a soft logic component that is used to perform the necessary data transfer between the FPGA and the HPS via the FPGA to HPS (H2F) bridge. Two mSGDMA components are placed in the FPGA to perform the data from the FPGA to the HPS and the HPS to the FPGA.

Performance Counter
The performance counter soft logic in the FPGA serves as a performance timer. It is configured by the Nios II to perform the performance monitoring of each data transfer. Each performance counter consists of one global timer and three performance timers to monitor the time taken for ingress, egress and total loop time of the data transfer.

Mailboxes
The mailboxes act as a command passing mechanism between the FPGA and the HPS and allow synchronization of the software between the Nios II and the HPS.

Payload RAM
The payload RAM is an on-chip RAM in the FPGA used as a temporary memory to store the data to be transferred between the FPGA and the HPS.

Software Flow

1. The Nios II processor configures mSGDMA0 to move data through the HPS SDRAM controller using the F2H AXI bridge.
2. Either mSGDMA0 or mSGDMA1 is configured to move data from the HPS on-chip RAM into the FPGA on-chip memory.
3. The Nios II processor is configured to track the time for each DMA transfer to the HPS and for returning the DMA data to the FPGA. Three time stamps are used to track the complete loop of the data transfer:
   • Total looptime (FPGA → HPS → FPGA)
   • DMA transfer time (FPGA → HPS)
   • DMA transfer time (HPS → FPGA)
4. Nios II starts the performance counters associated with the total looptime and the FPGA → HPS DMA transfer time. Once the DMA completes the transfer, the HPS stops the counter associated with the FPGA → HPS DMA transfer and starts the performance counter for the HPS → FPGA DMA transfer before initiating the DMA transfer. Once the total loop is completed, Nios II stops the counter for the HPS → FPGA DMA transfer (HPS → FPGA) and for the total loop. The figure below illustrates these times using the vertical gray bars.

The software flow is shown in the following figure:

There are 3 different platforms employed for the same software flow:

• Linux SMP
• VxWorks SMP
• Baremetal running on HPS.

The performance results of the three platforms are benchmarked against each other and are shown in the NiosII terminal.

Getting Started

Obtain FPGA, Linux and NiosII Binaries

Download cv_datamover_ed.tar.gz from https://releases.rocketboards.org/2014.09/cv-datamover-ed and save it to the user home folder.

Unzip the example design archive:

$ cd ~
$ tar xzf cv_datamover_ed.tar.gz

The following files are used to run the Reference Design:

Folder	Folder	Folder	File	Description
cv_datamover_ed	bin	Linux	sdimage.bin	SD Card Image
^	^	FPGA	Datamover_5csxfc6.flash	FPGA Configuration Flash
^	^	Nios	Datamover_5csxfc6.flash	Nios Configuration Flash

Obtain VxWorks Binaries

Download the VxWorks binaries from https://www.altera.com/support/support-resources/download/rtos_tools.html and save to the user home folder. Unzip the example design archive:

$ cd ~
$ unzip Altera_datamover_ed.zip

The following files are used to run the Reference Design:

Folder	Folder	Folder	File	Description
Altera_datamover_ed	bin	vxworks	vxw_datamover_sdmmc.img	Vxwork image

Board Setup

This section presents the board settings required for running the Datamover design.

Jumpers:

Jumper	Setting
J5	open
J6	shorted
J7	shorted
J9	open
J13	shorted
J16	open
J26	right shorted
J27	right shorted
J28	right shorted
J29	right shorted
J30	left shorted
J31	open

Switches:

Switch	Setting
SW1	All OFF
SW2	All OFF
SW3	1:ON 2:OFF 3:ON 4:ON 5:OFF 6:ON
SW4	1:OFF 2:OFF 3:ON 4:ON

Configuring Board to Use EPCQ

By default the board is configured to use the onboard FPGA configuration device as EPCS. In order to configure the board to use the configuration device as EPCQ, follow the instructions posted at http://www.altera.com/support/kdb/solutions/rd11192013_118.html. Note that these steps are only required to be performed one time. Once the board has been configured to use EPCQ, this configuration is stored in flash memory.

For a step-by-step tutorial on how to configure board to use EPCQ, refer to click here.

Programming the Board

Open the Nios II Command shell and write the FPGA Flash Image into EPCQ:

$ cd ~/cv_datamover_ed
$ nios2-configure-sof bin/fpga/datamover_5csxfc6.sof –d 2 
$ nios2-flash-programmer --base=0x10040000 --epcs bin/fpga/datamover_5csxfc6.flash 

Write the Nios II flash image into EPCQ:

$ nios2-flash-programmer --base=0x10040000 --epcs bin/nios/Datamover_5csxfc6.flash

After the FPGA and Nios flash image have been written successfully into EPCQ, the FPGA flash and Nios II flash automatically boot from EPCQ after the device is powered up:

$ nios-terminal

Getting Start Guide – Linux

After following the board set up and programming steps above, prepare the SD Card image by executing the following commands:

$ cd ~/cv_datamover_ed 
$ sudo dd if=bin/linux/sdimage.bin of=/dev/sdx   
* Please replace ‘dev/sdx’ with the name of the SD card device on your host computer.  
* Please ensure that the name of SD card device on your host computer is correct. You will not get the SD card image if you write into wrong drive

$ sudo sync

Insert the SD card into the CV SoC dev kit and power up the board. After the booting process is finished, login as root at the kernel terminal and follow the steps below to run the Datamover application:

$ modprobe datamover

The following is example output from Nios II terminal:

The following is example output from the UART terminal when the Datamover is executed in the HPS:

Continue testing the program by pressing the CPU_RST button on the board to obtain stable and consistent results.

Getting Start Guide –VxWorks

After following the board set up and programming steps above, copy the SD card image from /bin/vxworks/vxw_datamover_sdmmc.img file into your local directory.
Use Win32DiskImager to write the SD image to the flash card:

• For the Image File, browse to the vxw_datamover_sdmmc.img directory.
• For the Device, browse to the SD card driver.
• Click Write to write the image to the SD card.

Insert the SD card into the development kit after it has been successfully written. Power up the board.

Perform the configuration for the VxWorks environment by following the steps below to modify the boot parameters:

• Boot the board and hit a key to stop the boot count-down process.
• Use the "c" command to check and modify each of the boot parameters.
• Enter the correct value and then hit <enter> to move to the next boot parameter.
• Do not edit the existing values. You will need to enter the whole string.

[VxWorks Boot]: c

boot device: fs 
processor number: 0 
host name: host 
file name: /sd0:1/vxWorks 
inet on ethernet (e) : 192.168.192.204:ffffff00 
inet on backplane (b): 
host inet (h): 192.168.192.1 
gateway inet (g): 
user (u): target 
ftp password (pw): vxTarget 
flags (f): 0x0 
target name (tn): alt_soc_gen5 
startup script (s): 
other (o): emac1

[VxWorks Boot]:

To delete the field use '.' 
To go back to the previous parameter use '-' 
To go to the next parameters use <enter> 
To backspace use <ctrl>H

After finishing the configuration, boot the VxWorks image by pressing the "@" key or power up the development kit and wait for the count down. After VxWorks is booted, you should see the VxWorks splash screen as below:

Type “datamover_start” and you should see the screen below:

Example output from the Nios II terminal:

Example output from the UART terminal when VxWorks is executed in HPS

Continue to test the program by pressing the CPU_RST button on the board to obtain stable and consistent results.

Type “datamover_busy” to exercise processor core 1. In this way, you can observe whether the performance of processor core 0 is affected when processor core 1 is busy.

Getting Started Guide – HWLibs

After following the board set up and programming steps above, copy the Altera-SoCFPGA-DataMover-CV-GNU.tar.gz file into your local directory. Open ARM DS-5 Development Studio. Import the HWlibs archive file into ARM DS-5 Development Studio:

• File → Import
• Select Existing Projects into Workspace
• Click Next.

In the Import Projects dialog box, browse to the HWlibs archive project directory. Select the archive file and browse to the Altera-SoCFPGA-DataMover-CV-GNU.tar.gz directory. Click Finish. You should able to see the imported project in the Projects Explorer.

Go to Window → Open Perspective → Others

• Select DS-5 Debug and click Ok.

Right-click on the Altera-SoCFPGA-DataMover-CV-GNU project and select build project. After the project has finished building, right-click on the project and select Debug As → Debug Configurations. Expand the DS-5 Debugger and click on the project name.

• In the connection tab, select Debug Cortex-A9_0
• Target Connection: USB-BlasterII
• In the Bare Metal Debug, ensure the connection is connecting correctly.

Click on Debug, after entering debug mode, you should see the debugger is connected to the device.

Click on the “Play” button and the HWLibs design is executed.

Example output from Nios II terminal :

Continue to test the program by pressing the CPU_RST button on the board to obtain stable and consistent results.

Building the Linux example design

Software Development Flow Setting Up Yocto Environment

Follow the link to set up the Yocto environment before starting this section.

Apply Yocto Recipes and Patch

Use the commands below to acquire the poky source from Rocketboards Git:

$ git clone http://git.rocketboards.org/poky-socfpga.git

$ cd poky-socfpga $ git checkout -b datamover-ed ACDS14.0_REL_GSRD_PR

Download the patch from the release content section and apply the patch with this command:

$ git am cv_datamover_ed/sw/linux/altera_datamover_yocto.patch

Note:

Replace the file meta-alera/recipes-refdes/datamover-altera/datamover-altera_1.0.bb, replace "refdes-socfpga.git" with "linux-refdesigns.git"

Build U-Boot/Kernel/Rootfs:

$ source altera-init $ bitbake virtual/bootloader

$ bitbake virtual/kernel

$ bitbake altera-datamover-image

The output files are:

u-boot-socfpga_cyclone5.img	U-Boot image
zImage	Compressed kernel image
altera-datamover-image-socfpga_cyclone5.ext3	Root Filesystem as ext3 image
altera-datamover-image-socfpga_cyclone5.tar.gz	Root Filesystem in tar gzip archive format

Generate Device Tree Blob

Use the command below to generate the device tree blob:

$ sopc2dts --input datamover_5csxfc6.sopcinfo --output <dtb.file.name>.dtb --type dtb --board soc_system_board_info.xml –board hps_common_board_info.xml --bridge_removal all –clocks

Output file: <dtb.file.name>.dtb
For more details about device tree generation, refer to GSRD User Manual - Generating the Device Tree

Generate Preloader

For more details about preloader generation, refer to GSRD User Manual- Generating and Compiling the Preloader
Output file: preloader-mkpimage_cyclone5.bin.

Build SD Card Image

Replace all required components in the SD Card or build and replace the whole image:

$ wget https://releases.rocketboards.org/2014.06/gsrd/tools/make_sdimage.py 
$ mkdir rootfs $ cd rootfs $ sudo tar xzf ../altera-datamover-image-socfpga_cyclone5.tar.gz 
$ cd .. 
$ sudo ./make_sdimage.py -f -P preloader-mkimage.bin,u-boot-socfpga_cyclone5.img,num=3,format=raw,size=10M,type=A2 -P rootfs/*,num=2,format=ext3,size=1500M -P zImage ,
socfpga.dtb,num=1,format=vfat,size=500M -s 2G -n sd_card_image_cyclone5.bin

For more details about SD card image generation, refer to GSRD User Manual - Creating and Updating SD Card.

Steps to import VxWorks source and Image into Wind River Workbench

To view and modify the VxWorks source code, follow the steps below to import the archived files.

VxWorks Software Prerequisites

Wind River Workbench version : 3.3.4

Create the VxWorks Source Build Project

This section shows how to create a source build project (VSB) to incorporate the new BSP changes.

1. Open WindRiver Workbench.
2. Go to File → New → VxWorks Source Build (Kernel Library) ProjectEnter project name as VSB_SMP. Click Next.
3. Base the project on the board support package: alt_soc_gen5. Press Finish.
4. Edit VSB Options
5. Enable Symmetric Multiprocessing: SMP = y.
6. Enable processsort specific optimizations: ARMV7_CORE_CTX_A9_VSB = y.
7. Save VSB Project: File → Save All.
8. Build Project: Project → Build Project.

Create VxWorks Image Build Project

This section shows how to create an image buid project (VIP) to incorporate the new BSP changes.

1, Open WindRiver Workbench.
2. Go to File → New →VxWorks Image Project.
3. Enter project name as VIP_SMP. Click Next.
4. Select VSB_SMP tab. Select the toolchain to be diab. Check the Enable WDB Target Agent to enable easy debugging. Uncheck Add support to project for the BSP validation test suite. Click Next.
5. Click Next on the Options screen
6. On the Configuration Profile window, select the Profile PROFILE_STANDALONE_DEVELOPMENT. Click Finish.
7. Import the datamover source file into the VIP_SMP project.
8. Go to File → Import → General → Archive File as shown in the diagram below:

9. Click “Next.”
10. At From archive file, browse to the datamover_dkm.zip directory.
11. At Into folder, browse to the VIP_SMP project.
12. Click “Finish.”
13. You should able to see that the datamover_dkm folder has imported into your project.

14. Select the VIP_SMP project and build it by going to Project → Build All

Deliverables

FPGA, Linux, and Nios II Binaries

The reference design deliverables include binaries that can be used to run the reference design directly. The Linux binaries are delivered as an archive file accessible at https://releases.rocketboards.org/2014.09/cv-datamover-ed

Folder	Folder	Folder	File	Description
cv_datamover_ed	bin	Linux	altera-datamover-image-socfpga_cyclone5.tar.gz	Zipped Root filesystem
^	^	^	sdimage.bin	SD Card Image
^	^	^	altera-datamover-image-socfpga_cyclone5.dtb	Device Tree Binary
^	^	^	Preloader-mkimage.bin	Preloader Image
^	^	^	vmlinux	Linux Kernel Executable
^	^	^	zImage	Linux Kernel Compressed Image
^	^	FPGA	Datamover_5csxfc6.sof	FPGA Configuration SOF
^	^	^	Datamover_5csxfc6.flash	FPGA Configuration Flash
^	^	Nios II	Datamover_demo.elf	Nios II Application Executable
^	^	^	Datamover_demo.flash	Nios II Application Flash Image

VxWorks and HWLibs Binaries

The VxWorks and HWLibs binaries are delivered as an archive file accessible at https://www.altera.com/support/support-resources/download/rtos_tools.html

The binaries archive contains the following items:

Folder	Folder	Folder	Item	Description
Altera_datamover_ed	bin	vxworks	vxw_datamover_sdmmc.img	VxWorks SD Card Image
^	^	^	vxWorks	VxWorks Kernel

Hardware Design

The hardware design is delivered as an archive file accessible at https://releases.rocketboards.org/2014.09/cv-datamover-ed

Some of the relevant included files and folders are:

Folder	Folder	Item	Description
cv_datamover_ed	hw	ip/	Folder containing IP files
^	^	datamover_5csxfc6.qpf	Quartus Project File
^	^	datamover_5csxfc6.qsf	Quartus Setting File
^	^	datamover_5csxfc6.qsys	Qsys File
^	^	soc_system_timing.sdc	Timing file
^	^	soc_system_board_info.xml	SoC system board XML file
^	^	hps_common_board_info.xml	HPS XML file
^	^	datamover_top.v	Top Level Verilog File

Nios II Application Source

The Nios II application source is delivered as an archive file accessible at https://releases.rocketboards.org/2014.09/cv-datamover-ed

Folder	Folder	Folder	Item	Description
cv_datamover_ed	sw	niosii_hal	datamover_demo.c	Nios II Application Source

Linux Patch

The Linux patch is delivered as an archive file accessible at https://releases.rocketboards.org/2014.09/cv-datamover-ed

Folder	Folder	Folder	Item	Description
cv_datamover_ed	sw	linux	altera_datamover_yocto.patch	datamover yocto patch

Linux Yocto Recipes

All the necessary files to build the Linux kernel, drivers, applications, and root filesystem are delivered as a set of Yocto recipes accessible through the git trees at http://rocketboards.org/gitweb

Folder	Folder	Tag
Yocto Recipes	Poky-socfpga.git	ACDS14.0_REL_GSRD_PR

VxWorks

The VxWorks source is delivered as an archive file accessible at https://www.altera.com/support/support-resources/download/rtos_tools.html

Folder	Folder	Folder	Item	Description
Altera_datamover_ed	sw	vxworks	datamover_dkm.zip	Datamover DKM Project

HWLibs

The Hwlibs source is delivered as an archive file accessible at https://www.altera.com/support/support-resources/download/rtos_tools.html

Folder	Folder	Folder	Item	Description
Altera_datamover_ed	sw	hwlibs	Altera-SoCFPGA-DataMover-CV-GNU.tar.gz	Datamover HwLibs Example Application

Reference

1. GSRD User Manual