Video

Introduction

This project is a port of OpenCV running on Altera development board board (plus Bitec DVI HSMC) and Arrow SocKit board. The project is based on an infrastructure that includes X11 with the matchbox window manager which provides a lightweight graphical user interface with a simple X terminal and support for USB keyboard and mouse.Video output is handled by the FPGA image provided for Altera and SoCKit boards, through a custom frame buffer driver. Video input is (for now) handled via USB UVC.Kernel is based on the latest 3.10 with newest USB driver which allows connecting a USB hub with mixed full speed/high speed devices (earlier kernel builds won't work). The goal is to start from a software only port to finally get to a fully FPGA accelerated suite where individual kernels can be substituted with FPGA cores. ipTronix is currently developing a suite of IP blocks to ease implementation of hardware accelerated kernels which will range from full custom RTL to software programmed multiprocessor SIMD cores.

How to get the source and compile

1. clone the source from github - https://github.com/pnndra/altera-opencv/
2. checkout the dario-iptronix branch
3. compile the image and u-boot

#clone the source from github
git clone git://github.com/pnndra/altera-opencv/

#go into the 'altera-opencv' directory
cd altera-opencv

#checkout current branch
git checkout dario-iptronix

#setup build environment and variables
source ./iptronix-init

#build image and bootloader (kernel will be built by iptronix-image)
bitbake iptronix-image virtual/bootloader

How to run the image

Prerequisites

Since the kernel makes use of a custom frame buffer, in order to boot properly it's necessary to have the FPGA loaded before starting up the board ( SoCKit.sof or Altera.sof). at this time this has to be done manually through USB blaster; future builds will automatically load the FPGA from u-boot.Precompiled images can be downloaded from links below:

* Arrow SoCKit SD card image

* Altera Eval Board SD card image

Getting some image displayed

Once kernel is booted you'll see a terminal on the VGA output of SoCKit boards or on DVI output of BITEC DVI board connected to HSMC of Altera boards. In case you don't want to use video output it is also possible to remote X windows display to a PC. in order to do this you either need a linux based PC running X11 (any graphical desktop should work) or a windows based PC with either Cygwin/X or any other X11 server application such as moba Xterm. on the PC side, if you use Cygwin/X or linux you have to issue (prefix it with sudo if you don't have privileges under linux):

xhost +

Note that this will enable any other PC or Xwindows application running on another PC to be displayed on your machine so this may cause security issues. it is possible to allow only a specific target (the altera board) to display applications and in this case just replace + with the IP address of the target. note also that firewall may block incoming connections so make sure you have it either disabled or set up to allow the X11 server to receive incoming connections.

On the target board you need to inform the operating system it should send its output to the host running X11 Server. assuming your X11 server has IP address 192.168.1.1 enter the following instruction (of course replace the IP address with the right one):

export DISPLAY=192.168.1.1:0.0

Once this is done any graphical application will send its output through the network to the X11 server. in order to test it just try to run xeyes, which opens a window with two eyes which follow your mouse pointer.

What to look for

OpenCV sample programs are present in the /usr/bin directory and some sample image files are present in the /usr/share/opencv/samples directory, finally data for classifiers is found under /usr/share/opencv/data Note that in addition to OpenCV this image also includes MPlayer so it will be possible to playback multimedia files or preview UVC webcam through it.

In order to test the webcam use:

mplayer -tv driver=v4l2:width=320:height=240 tv://

Note that as of today UVC webcam support is still very preliminary and with higher resolutions than 320x240 the image may get corrupted (sometimes it gets corrupted also at 320x240... this is under investigation). An extract of a few very nice examples that can be run is:

Program	Description	Video
edge	edge detection through canny algorithm
morphology2	erode/dilate/open/close algorithms
fback	optical flow demo (requires camera)
retinaDemo	simulation of human retina behavior (requires camera)
laplace	laplacian kernels (requires camera)	FPGA acceleration demo with real time coefficient change via IDE
video_homography	track points of interest on a live image (requires camera)
camshiftdemo	track motion of a selected object (requires camera)
peopledetect	Find people in images (pedestrian detection)	SoC only people detection in real time (standard opencv, 1 fps) SoC only people detection in real time (iptronix optimizations,25 fps)