Embedded Visual Control

2012 (2st semester, Febr - June)

Code : 5HC99
Credits: 5 ECTS
Lecturers : Prof. dr. Henk Corporaal, Ir. Roel S. Pieters, MSc Zhenyu Ye, Ir Mark Wijtvliet
Tel. : +31-40-247 5195 / 3653 (secr.) 5462 (office)
Email:  R.S.Pieters at tue.nl; Z.Ye at tue.nl; M.Wijtvliet at tue.nl; H.Corporaal at tue.nl
Project location: Mechanical Engineering lab

News

Information on the course:

Description

This course aims at combining and understanding the vision, (robotic) control and embedded computation areas. For each discipline the student should become familiar with the main theories, mathematical formalisms and practical issues.
The course is project driven. Students have to perform advanced lab assignments, while studying in the mean time the required theoretical background. This year we will perform a project using the Parrot AR Drone Quadrocopter (see picture above).
This quadrocopter can be controled from your smartphone (eg. using your iPhone or Android).

This will be quite a challenging course, but the rewards will be high; you will learn a lot.
Only highly motivated students will be allowed to follow this course, with a maximum of 12 students.

Topics:

Apart from studying the required material on Vision, Control and Embedded Procesing Systems, the course will be largely based on project work. Students will be divided into groups of 2-3 people working one one or more lab assignments.
Possible assignments are in the area of:
  1. Global control: Let the Drone follow a trajectory, using camera based control. E.g. you may use homography based visual servo tracking (see below).
  2. Local control: Design or adapt the local control, using a combination of the available sensors and or vision, to make the Drone very stable.
  3. Combinations of Local and Global control.
About the Quadrocoptor:
The AR-Parrot quadrocoptor as platform for vision-based control is equiped with two cameras; front and pointing downwards. The global control of the quadrocoptor that will be designed consists of computing a 3D error (translation and rotation) by means of visual processing (SURF, homography) and feeding this back to the controller.
As visual servoing is by definition an error minimization method of control, each iteration the motion trajectory is a step-function. This gives a non-smooth motion profile which is undesirable. By computing a trajectory online the motion trajectory is designed and motion constraints (e.g., via-point trajectory, execution time, maximum velocity or acceleration) can be implemented.

In order to save battery life, it should be investigated if processing of images should be done onboard (ARM9) or on a remote pc, by streaming images (320 x 240 px, 16 bit color depth) via WiFi.

Course, Reading and Background Material

Below you'll find a lot of reading material. During the course we will instruct you what to read and study precisely, and what is background material.
Further material:

Code

You may use existing vision toolkits like OpenCV or ARToolkit. See below.
 The following are links to software for the AR Drone; note we did not test this fully yet !!

Slides

Slides as far as available will be made available during the course. All slides have to be studied (mandatory).
Check also our Wiki site for futher material and documentation.

Student presentations guidelines

At the end of this course each group has to give a final presentation and demonstration its project results.
Guidelines for the presentation will follow.

Projects

For the project we will use the Parror AR Drone Quadrocopter.
Prject details will be made available during the course.

Examination

The examination will be oral about the treated course theory, the lab report(s), and studied articles. You have give a detailed presentation about your project results, and give a demonstration.
Date: June 2012

Grading depends on your results on theory, lab exercises and your presentation.

Related material and other links

Interesting processor architectures:Blackfin Processor Block Diagram

Back to homepage of Henk Corporaal