1. Overview

This pase shows how to implement Debian GNU/Linux based on Sodia GHRD for OpenVINO with Intel Neural Compute Stick 2.

2. Prerequisite

2.1 Equipment

• Sodia Board
• Intel® Neural Compute Stick 2 (NCS2)
• USB Hub (Recommend external power input for stability of NCS2)
• USB Camera
• SD Card (Over 16GB / Recommend Class 10)
• Monitor (1280x780@60p / DVI Input)
• Key Board & Mouse
• Ethernet (Can be accessed to Internet）

2.2 Development Environment

You must need to prepare Linux host machine to create Linux image for Cyclone V SoC with OpenVINO.

2.2.1 Windows 7/10 Host

• Quartus Prime v18.1 Standard Edition
• SoC FPGA Embedded Development Suite v18.1

2.2.2 Linux Host

• Quartus Prime v18.1 Standard Edition
• SoC FPGA Embedded Development Suite v18.1

3 Create Root File System for OpenVINO

This section shows how to create Debian Root File System for Cyclone V SoC.

3.1 Open terminal and create work directory

This page uses ~/vino as working directory.

$ mkdir ~/vino
$ cd vino

3.2 Download Root File System

Download Debian 9.0 (Stretch) Root File System from Linaro web site.

$ wget https://releases.linaro.org/debian/images/developer-armhf/latest/linaro-stretch-developer-20170706-43.tar.gz

3.3 Extract Root File System

Extract archive file using following command.

$ sudo tar zxf linaro-stretch-developer-20170706-43.tar.gz

* If you need to use proxy for internet access or something you need to customize. Please modify a file inside Root File System at this time.

4. Create Linux Kernel

4.1 Open a terminal and move to work directory

$ cd ~/vino

4.2 Download Build Tool Chain

Compiler for Debian 9.0 is GCC 6.3, use same version of GCC to build linux kernel.

$ wget https://releases.linaro.org/components/toolchain/binaries/6.3-2017.05/arm-linux-gnueabihf/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf.tar.xz

4.3 Extract Build Tool Chain

$ tar jxfv gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf.tar.xz

4.4 Clone Linux Kernel Repository

Using latest stable repository from kernel.org (It is not SoC FPGA repogitory)

$ git clone http://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git

4.5 Confirm branch inside repository

Move to directory and show branch list.

$ cd linux-stable/
$ git branch -r

4.6 Checkout Kernel 4.5

From Kernel 4.6, USB driver is changed. It caused performance issue of USB 2.0. Select 4.5 for this demo.

$ git checkout -b test_branch origin/linux-4.5.y

4.7 Remove Commit for network driver

Kernel 4.5 has a network issue, so need to remove specific comit for network driver as follows:

$ git revert 88f8b1bb41c6208f81b6a480244533ded7b59493

4.8 Clone Intel Provided Linux Kernel

To modify a vanilla kernel for SoC FPGA, clone intel provided linux kernel.

$ cd ~/vino
$ git clone https://github.com/altera-opensource/linux-socfpga

4.9 Confirm Branch Inside Intel Provided Linux Kernel

This is same as above, mode to directory and confirm a list of branch.

$ cd linux-socfpga/
$ git branch -r

4.10 Checkout Kernel 4.9

$ git checkout -b test_branch origin/socfpga-4.9.78-ltsi

4.11 Merge SoC FPGA Modification to Vanilla Kernel

Return to work directory, and confirm difference of linux-stable/Drivers/video/fbdev & linux-socfpga/Drivers/video/fbdev directory.(This pase use Meld Diff Viewer for this purpose)

4.11.1Modify Makefile

Added CONFIG_FB_ALTERA... releted settings.

4.11.2 Modify Kconfig

Added FB_ALTERA_VIP... related settings.

4.11.3 Copy a source code for Frame Buffer Driver

Copy altvipfb*.c, altvipfb*.h files.

4.12 Set Compiler path to environment variable

$ cd ~/vino/linux-stable/
$ export CROSS_COMPILE=~/vino/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-
$ export ARCH=arm

4.13 Configure Linux Kernel for SoC FPGA

$ make socfpga_defconfig

4.14 Kernel setup from memuconfig

$ make menuconfig

4.14.1 Select Device Drivers

Hierarchy

Device Drivers --→
　[*] Multimedia support --→
      [*] Cameras/video grabbers support
      [*] Media USB Adapters --→
          <*> USB Video Class (UVC)
          [*] UVC input events device support
  Graphics support --→
    Frame buffer Devices --→
      <*> Altera VIP Frame Buffer II framebuffer support OF Device 

4.14.2 Add USB Camera Support

Select Multimedia support

Enable Cameras/video grabbers support.

Select Media USB Adapters.

Enable USB Video Class(UVC). (USB input events device support also enabled when check it)

4.14.3 Add Frame Buffer Driver

Select Graphics support.

Select Frame buffer Devices.

Enable Altera VIP Frame Buffer II framebuffer support OF Device.

4.14.3 Save and Exit

Select <Save>, and Exit from memuconfig typing ESC key several times.

4.15 Build Linux Kernel

$ make zImage

zImage is created to /home/<User>/vino/linux-stable/arch/arm/boot.

5. Create SD Card Image

5.1 Open terminal and move to work directory

$ cd ~/vino

5.2 Copy data for SD image in one directory

You need following data to create SD Card image

• preloader-mkpimage.bin
• u-boot.img
• soc_system.dtb
• u-boot.scr
• soc_system.rbf
• zImage (From 4. Create Linux Kernel)
• RootFS Directory (From 3. Create Root File System for OpenVINO)

FPGA hardware data and bootloader binary can be download form this link:

• sodia_ghrd_v18.1.1_vfb.zip

File	Location
preloader-mkpimage.bin	sodia_ghrd_v18.1.1_vfb\software\spl_bsp
u-boot.img	sodia_ghrd_v18.1.1_vfb\software\spl_bsp\uboot-socfpga
soc_system.dtb	sodia_ghrd_v18.1.1_vfb
u-boot.scr	sodia_ghrd_v18.1.1_vfb
soc_system.rbf	sodia_ghrd_v18.1.1_vfb\output_files

If you want to build a hardware data by yourself, please see appendix section.

5.3 Download Scpipt to create SD Card Image

You can create SD Card Image using linux command, but more easy to create it, Intel provided script for SD Card Image creation.

$ wget https://releases.rocketboards.org/2019.04/gsrd/tools/make_sdimage.py

5.4 Change a Permission of Script

$ chmod g+x make_sdimage.py

5.5 Execute SD Card Image Creation Script

$ sudo ./make_sdimage.py -f  -P preloader-mkpimage.bin,u-boot.img,num=3,format=raw,size=10M,type=A2  -P binary/*,num=2,format=ext3,size=13G  -P zImage,u-boot.scr,soc_system.rbf,soc_system.dtb,num=1,format=vfat,size=500M   -s 14G -n sodia_ncs2_vino.img

* You can specify a size of those partitions, and name of image file.

6. Programming SD Image to SD Card

6.1 Programming SD Image using Windows 7/10 Host

You can use Win32DiskImager to program SD Image.
The tool can be downloaded from here: http://sourceforge.net/projects/win32diskimager/files/latest/download

6.2 Programming SD Image using Linux Host

6.2.1 Confitm Disk Configuration

Confirm SD Card device name.

$ cat /proc/partitions

6.2.2 Write a Image to SD Card

$ sudo dd if=sodia_ncs2_vino.img of=/dev/sdd bs=1M

6.2.3 Execute Sync to flush the changes to the SD Card

$ sudo sync

7. Booting Sodia Board

7.1 Board Configuration and Connection

1. Power Input
2. Connect Ethernet cable to RJ-45
3. Connect USB Hub (Keyboard/Mouse/USB Camera/NCS2 are connected to USB Hub)
4. Connect USB cable for serial terminal. (115200-bps,8N1)
5. Connect DVI cable to Monitor
6. Insert SD card
7. MSEL
8. BSEL/CSEL
9. Power Switch

• MSEL (J7)
  You need to set MSEL is FPP mode for FPGA configuring from HPS. Please set MSEL as follows:

• BSEL & CSEL
  For SD Card Boot. You need to set BSEL & CSEL as follows:

7.2 Open Terminal Software

You can see command pronpt.

7.3 Update & Upgrade Root File System

$ sudo apt-get update

$ sudo apt-get -y upgrade

* Please note that those command are needed internet access, so you need to set proxy setting if you need.

7.4 Confirm Your Linux Information

7.4.1 Reboot

$ reboot

7.4.2 Confirm Distribution

$ lsb_release -a

7.4.3 Confirm Kernel Version

$ uname -a

7.4.4 Confrim GCC Version

$ gcc -v

7.4.5 Confirm Python Version

$ python --version
$ python3 --version

7.5 Install LXDE

7.5.1Install LXDE Package

$ sudo apt install -y task-lxde-desktop

7.5.2 Reboot

$ reboot

7.5.3 Login to Desktop

You can use following information to login.

• Login ID: linaro
• Password: linaro

7.6 Install Required Packages

7.6.1 Utility for USB Camara test

$ sudo apt install -y qv4l2

You can execute Qt V4L2 form menu.

You can see USB Camera stream.

7.6.2 Update cmake

$ sudo apt install -y cmake

Confirm cmake version

$ cmake --version

8 Intall OpenVINO Toolkit

8.1 Download OpenVINO Toolkit for Raspberry Pi.

$ cd Downloads/
$ wget https://download.01.org/opencv/2019/openvinotoolkit/R1/l_openvino_toolkit_raspbi_p_2019.1.094.tgz
$ tar -xf l_openvino_toolkit_raspbi_p_2019.1.094.tgz

8.2 Modify a script file for your SoC FPGA

$ cd inference_engine_vpu_arm/
$ cp bin/setupvars.sh bin/setupvars.sh.org
$ sed -i "s|<INSTALLDIR>|/home/linaro/Downloads/inference_engine_vpu_arm|" bin/setupvars.sh
$ sed -i "s/Raspbian/Debian/" bin/setupvars.sh
$ sed -i 's/$OS_VERSION = "9"/$OS_VERSION = "9.9"/' bin/setupvars.sh

* Please note that <INSTALLDIR> is depend on your system. Please set correct path.

Here is modified settings. (Just confirm)

8.3 Setup Environment Variable

$ source bin/setupvars.sh

8.4 Add USERS Group & USB Rules

$ sudo usermod -a -G users "$(whoami)" 
$ sh install_dependencies/install_NCS_udev_rules.sh

8.5 Build Sample Application

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-march=armv7-a" ../deployment_tools/inference_engine/samples

$ make -j2

If you want to build application independently, you can specify a application name as argument.

$ make -j2 object_detection_demo_ssd_async

8.6 Setting USB Rules

$ lsusb
$ cat /etc/udev/rules.d/97-myriad-usbboot.rules

$ sudo vi /etc/udev/rules.d/97-myriad-usbboot.rules

8.7 Reboot

$ sudo reboot

9 Execute OpenVINO Sample Application

9.1 Confirm Video Device

$ cd ~/Downloads/inference_engine_vpu_arm/build/
$ ls /dev/video*

9.2 Download Training Model for Face Detection

$ wget --no-check-certificate https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/face-detection-adas-0001/FP16/face-detection-adas-0001.bin

$ wget --no-check-certificate https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/face-detection-adas-0001/FP16/face-detection-adas-0001.xml

Pre-Trained Models are provided from following web site. Please download for your application.

https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/

9.3 Run Sample Application

$ source ../bin/setupvars.sh
$ ./armv7l/Release/object_detection_demo_ssd_async -m face-detection-adas-0001.xml -d MYRIAD -i /dev/video0

Appendix

Following sections are only needed if you would like to create hardware design or modify hardware design. Otherwise you can use prebuild images in section 5.

IP license is needed to recompile hardware image, please contact to your local sales.

10 Create Hardware Design

10.1 Launch Quartus Prime and Open Sodia Refelence Design(GHRD)

Refelence Design can be downloaded from this page.

https://rocketboards.org/foswiki/view/Documentation/MacnicaSodiaEvaluationBoard

10.2 Create PLL for DVI output

10.2.1 Select "PLL Intel FPGA IP" from IP Catalog

10.2.2 PLL Setting

Create 74.25MHz from 27MHz.

Parameter	Set Value
Device Speed Grade	-
PLL Mode	Integer-N PLL
Reference Clock Frequency	27.0 MHz
Operation Mode	direct
Enable locked output port	On
Enable physical output clock parameters	Off
Number of Clocks	1
Desired Frequency	74.25 MHz
Actual Frequency	74.250000 MHz
Phase Shift units	ps
Phase Shift	0 ps
Actual Phase Shift	0 ps
Duty Cycle	50 %
.. Other …	Default

10.2.2 Generate PLL IP

Click Finish and Generate IP. If you see following message, please select "Yes".

10.3 Launch Platform Designer and Add Modules

10.3.1 Clock Source

This clock source for Video Clock (74MHz)

Parameter	Set Value
Clock frequency	74250000 Hz
Clock frequency is known	On
Reset synchronous edges	None

10.3.2 Frame Buffer II

Use this module as frame reader. This module read a data from HPS SDRAM.

Parameter	Set Value
Maximum frame width	1280
Maximum frame height	720
Bits per color sample	8
Number of color planes	4
Color planes transmitted in parallel	On
Number of pixels in parallel	1
Interlace support	Off
Use separate clock for the Avalon-MM master interface(s)	On
Avalon-MM master(s) local ports width	256
FIFO depth Write	64
Av-MM burst target Write	32
FIFO depth Read	64
Av-MM burst target Read	32
Align read/write bursts on read boundaries	On
Maximum ancillary packets per frame	0
Maximum length ancillary packets in symbols	10
Frame buffer memory base address	0x00000000
Enable use of inter-buffer offset	Off
Inter-buffer offset	-
Prioritize Fmax over memory bandwidth	Off
Module is Frame Reader only	On
Module is Frame Writer only	Off
Frame dropping	On
Frame repeating	On
Delay length (frames)	1
Locked rate support	Off
Drop invalid frames	On
Drop/repeat user packets	Off
Run-time writer control	Off
Run-time reader control	On

10.3.3 Clocked Video Output II

Output Video Image as 1280x720@60p.

Parameter	Set Value
Image width/Active pixels	1280 pixels
Image height/Active lines	720 lines
Bits per pixel per color plane	8 bits
Number of color planes	4
Color plane transmission format	Parallel
Allow output of channels in sequence	Off
Number of pixels in parallel	1
Interlaced video	Off
Sync signals	On separate wires
Active picture line	0
Frame/Field 1: Ancillary packet insertion line	0
Embedded syncs only - Frame/Field 1: Horizontal blanking	0 pixels
Embedded syncs only - Frame/Field 1: Vertical blanking	0 lines
Separate syncs only - Frame/Field 1: Horizontal sync	46 pixels
Separate syncs only - Frame/Field 1: Horizontal front porch	110 pixels
Separate syncs only - Frame/Field 1: Horizontal back porch	220 pixels
Separate syncs only - Frame/Field 1: Vertical sync	5 lines
Separate syncs only - Frame/Field 1: Vertical front porch	5 lines
Separate syncs only - Frame/Field 1: Vertical back porch	20 lines
Interlaced and Field 0: F rising edge line	0
Interlaced and Field 0: F falling edge line	0
Interlaced and Field 0: Vertical blanking rising edge line	-
Interlaced and Field 0: Ancillary packet insertion line	-
Embedded syncs only - Field 0: Vertical blanking	-
Separate syncs only - Field 0: Vertical sync	-
Separate syncs only - Field 0: Vertical front porch	-
Separate syncs only - Field 0: Vertical back porch	-
Pixel FIFO size	1280 samples
FIFO level at which to start output	1279 pixels
Video in and out use the same clock	On
Use control port	Off
Generate synchronization outputs	-
Accept synchronization inputs	-
Set vco_clk_divider increment to pixels in parallel	-
Run-time configurable video modes	1 mode
Width of vid_std bus	1 bits
Low latency mode	0

10.3.4 Interrupt Latency CounterI

Modify IRQ_PORT_COUNT as 4.

10.3.5 Connecting modules on Platform Designer

Connectng modules as follows:

Please confirm following setting as well:

• Couduit of clk_1: clk_0,reset_0
• Base Address of alt_vip_cl_vfb_0: 0x0000_0000
• IRQ of al_tvip_cl_vfb_0: 3
• Conduit of alt_vip_cl_cvo_0: alt_vip_cl_cvo_0_clocked_video

10.3.6 Generate Platform Designer System

10.4 Confirm Quartus Prime Settings

Please ensure that vid_pll.v and soc_system.qip are listed in Settings Menu.

10.5 Modify Pin Assignment for DVI Output

10.5.1 Modify qsf file

Open soc_system.qsf and add following statement

set_location_assignment PIN_AA26 -to fpga_clk_27
set_location_assignment PIN_AC28 -to fpga_dvi_reset_n
set_location_assignment PIN_AC29 -to fpga_dvi_clk
set_location_assignment PIN_AA24 -to fpga_dvi_data[23]
set_location_assignment PIN_Y23 -to fpga_dvi_data[22]
set_location_assignment PIN_Y24 -to fpga_dvi_data[21]
set_location_assignment PIN_AD26 -to fpga_dvi_data[20]
set_location_assignment PIN_AC27 -to fpga_dvi_data[19]
set_location_assignment PIN_AH28 -to fpga_dvi_data[18]
set_location_assignment PIN_AH29 -to fpga_dvi_data[17]
set_location_assignment PIN_AJ29 -to fpga_dvi_data[16]
set_location_assignment PIN_AE26 -to fpga_dvi_data[15]
set_location_assignment PIN_AG27 -to fpga_dvi_data[14]
set_location_assignment PIN_AF30 -to fpga_dvi_data[13]
set_location_assignment PIN_AF29 -to fpga_dvi_data[12]
set_location_assignment PIN_AB26 -to fpga_dvi_data[11]
set_location_assignment PIN_AG28 -to fpga_dvi_data[10]
set_location_assignment PIN_AC25 -to fpga_dvi_data[9]
set_location_assignment PIN_V25 -to fpga_dvi_data[8]
set_location_assignment PIN_AB30 -to fpga_dvi_data[7]
set_location_assignment PIN_W25 -to fpga_dvi_data[6]
set_location_assignment PIN_AA30 -to fpga_dvi_data[5]
set_location_assignment PIN_AC30 -to fpga_dvi_data[4]
set_location_assignment PIN_AB28 -to fpga_dvi_data[3]
set_location_assignment PIN_AD29 -to fpga_dvi_data[2]
set_location_assignment PIN_AD30 -to fpga_dvi_data[1]
set_location_assignment PIN_AA28 -to fpga_dvi_data[0]
set_location_assignment PIN_W24 -to fpga_dvi_de
set_location_assignment PIN_AB25 -to fpga_dvi_hs
set_location_assignment PIN_V23 -to fpga_dvi_vs

10.5.1 Confirm Pin Setting from Assignment Editor

10.6 Modify TOP Module

Open ghrd_top.v and add following statement

    // DVI signals
    input  wire        fpga_clk_27,
    output wire        fpga_dvi_reset_n,
    output wire        fpga_dvi_clk,
    output wire [23:0] fpga_dvi_data,
    output wire        fpga_dvi_de,
    output wire        fpga_dvi_hs,
    output wire        fpga_dvi_vs, 

  .clk_0_clk                                    (vid_clk),      //                          clk_0.clk
  .reset_0_reset_n                              (pll_locked),   //                        reset_0.reset_n
  .alt_vip_cl_cvo_0_clocked_video_vid_data      (vid_data),     // alt_vip_cl_cvo_0_clocked_video.vid_data
  .alt_vip_cl_cvo_0_clocked_video_vid_datavalid (fpga_dvi_de),  //                               .vid_datavalid
  .alt_vip_cl_cvo_0_clocked_video_vid_v_sync    (fpga_dvi_hs),  //                               .vid_v_sync
  .alt_vip_cl_cvo_0_clocked_video_vid_h_sync    (fpga_dvi_vs),  //                               .vid_h_sync

  // DVI I/F
  wire              vid_clk;
  wire              pll_locked;
  wire  [31:0]      vid_data;

  vid_pll vid_pll_inst(
    .refclk         (fpga_clk_27) ,        // input  refclk_sig
    .rst            (!hps_fpga_reset_n) ,  // input  rst_sig
    .outclk_0       (vid_clk) ,            // output  outclk_0_sig
    .locked         (pll_locked)           // output  locked_sig
  );

  assign fpga_dvi_reset_n = pll_locked;
  assign fpga_dvi_clk     = vid_clk;
  assign fpga_dvi_data    = vid_data[23:0];

10.7 Modify sdc file

Open soc_system_timing.sdc and add following statement.

# 27MHz board input clock
create_clock -period 37.037 [get_ports fpga_clk_27]

# DVI I/F
derive_pll_clocks -create_base_clocks
create_generated_clock -name fpga_dvi_clk -source [get_nets {vid_pll_inst|vid_pll_inst|altera_pll_i|outclk_wire[0]}] -divide_by 1 -multiply_by 1 [get_ports {fpga_dvi_clk}]

set_output_delay -clock { fpga_dvi_clk } -rise -min 1.3 [get_ports {fpga_dvi_data[*] fpga_dvi_de fpga_dvi_hs fpga_dvi_vs}]
set_false_path -from [get_ports {fpga_dvi_reset_n}] -to *

10.8 Compile FPGA Design

Waiting several minutes to finish a compile.

10.9 Generate RBF File

10.9.1 Launch SoC EDS Command Shell

10.9.2 Generate RBF

$ quartus_cpf -c -o bitstream_compression=on soc_system.sof soc_system.rbf

11 Creating Bootloader(Preloader/U-Boot)

11.1 Launch BSP-Editor from SoC EDS Command Shell

$ bsp-editor &

Select File => New HPS BSP, then set a path for Preloader settings directory as <Project Directory>/hps_isw_handoff/soc_system_hps_0.

Confirm BOOT_FROM_SDMMC is selected, then click a Generate button.

11.2 Build Preloader

$ cd <Project Folder>/software/spl_bsp/
$ make

"preloader-mkpimage.bin" is generated in software/spl_bsp directory.

11.3 Build U-Boot

$ make uboot

"u-boot.img" is generated in software/spl_bsp/uboot-socfpga directory.

12 Create DeviceTree

12.1 Launch SoC EDS Comamnd Shell

12.2 Execute sopc2dts Command

$ sopc2dts --input soc_system.sopcinfo --output soc_system.dtb --type dtb --board soc_system_board_info.xml --board hps_common_board_info.xml --bridge-removal all --clocks

* Need to move hardware design top directory.

soc_system.dtb is generated in current directory.

13 Create U-Boot Script

13.1 Launch SoC EDS Command Shell

13.2 Create boot.script file

Open text editor and add following statement, then save as boot.script.

fatload mmc 0:1 $fpgadata soc_system.rbf; 
fpga load 0 $fpgadata $filesize;
setenv fdtimage soc_system.dtb;
run bridge_enable_handoff;
run mmcload;
run mmcboot;

13.3 Genete U-Boot Script

$ mkimage  -A arm -O linux -T script -C none -a 0 -e 0 -n "My script" -d boot.script u-boot.scr

"u-boot.scr" is generated in current directory.

14 Refelence Information