CE316 and CE866: Computer Vision

Lectures and labs | Notes | Assignments | Further reading | Using Linux | Installing OpenCV

About the module

This module explores the algorithms and software that lets a computer 'understand' the content of images and videos.  This is far from being a solved problem and research into it is extraordinarily active, with the UK contributing some of the most important results globally.  The development of the discipline is so rapid that about half the content of the module couldn't have been taught as little as ten years ago.  Given what it is trying to do, you might think that vision is an area of artificial intelligence, placing it squarely in the realm of computer science.  While this is true to some extent it is far from the whole story, with researchers also having backgrounds in disciplines such as electronic engineering, mathematics, physics, psychology and medicine.  The contributions of the various disciplines will become more apparent as you learn about vision techniques.

In fact, this is not one module but two, for it is lectured simultaneously to final-year undergraduate students (CE316) and to postgraduate students (CE866).  Laboratory sessions are also joint between CE316 and CE866 but the assignments for the two modules are different; be sure you do the right ones!

The last decade has seen three major technological influences on computer vision. Firstly, digital imaging has transformed the capture of image and video data from something that pushed at the boundaries of real-time hardware and software into an everyday process.  The second influence is cheap data storage and processing, allowing the image and video data to be stored and manipulated in reasonable timescales.  The final influence, a consequence of having more CPU cycles to burn, is an expansion in the use of machine learning and an improvement in the learning algorithms available.

Machine learning has actually been employed in real-world vision systems for a long time, though in previous years this module stopped short of discussing it.  However, the last 2–3 years in particular have seen machine learning become much more widely used in vision applications, so this year there will be an introduction to it in the second half of the module.  To do that, some of the material taught in previous years will be skipped and some of it presented with a different emphasis, to make space for the new material and show how they work together.

Lectures and laboratories

Lectures Friday 15:00–17:00, LTB3 weeks 2–11
Laboratories Tuesday 12:00–14:00, CSEE Lab 1 weeks 3–11

CE316 CE866
The first lab session is the same for both CE316 and CE866

Lecture notes

The lecture notes are split up into individual topics in the following table and are presented in same the order as the lectures.  I shall capture images of the drawings etc that I make during the lectures and make them available here too — but that does not mean that you shouldn't make notes yourself: there are studies that show that students who take handwritten notes remember the material better than students who use computers or don't take notes at all — see the original article and a non-technical version.

The notes for a particular 'lecture' may not appear on the website until after it has taken place because there are occasions in which the session will involve you doing things rather than simply listening to me drone on.  Each lecture has an associated set of multi-choice questions so that you can check you picked up the most important points.

find out how much you already know quiz
Introduction to Computer Vision
The Human Visual System
(not examinable)
quiz
Getting to grips with OpenCV
[scribbles during the lecture]
quiz, Python code, C++ code
histogram worksheet, solutions
Evaluating vision systems
[scribbles during the lecture]
quiz
Convolution
[scribbles during the lecture]
quiz, histogram worksheets, solutions
Feature detection and description
[scribbles during the lecture]
quiz
Low-level vision
[scribbles during the lecture]
quiz
Stereo
[scribbles during the lecture]
quiz
faces
[scribbles during the lecture]
quiz
Getting started with machine learning
[scribbles during the lecture]
quiz, WISARD example
Neural networks
[scribbles during the lecture]
quiz
Concluding remarks

Further reading

In such a fast-moving subject area, printed textbooks are almost always out of date — and this is especially the case in the use of machine learning in computer vision.  A couple of widely-recommended texts are:

but, to be honest, it's best to ask questions in lectures and lab sessions if you are having trouble with the principles of the subject.

Regarding computer vision programming, there are about half a dozen books that describe earlier versions of OpenCV in the library, so looking through one of those might help you if you get stuck.  Other places to look are:

You might also be interested to look at some online resources:

Assignments

CE316 assignment #1
(due 10th November before noon)
CE866 assignment #1
(due 17th November before noon)
CE316 assignment #2
(due 8th December before noon)
CE866 assignment #2
(due 8th December before noon)

Using Linux

If your Linux skills need improvement, here are some places to look:

Installing OpenCV on your own machine

OpenCV and the Python environment described in lectures are part of the standard installation under Linux on the machines in CSEE's computer laboratories. Of course, you can also install OpenCV on your own computer, either under Unix (MacOS, Linux, etc) or Windows.  As OpenCV is being developed so quickly, there are significant differences between versions so programs written for v3 may well not work with v2 and vice versa.  It makes sense for you to install the same version on your machine as is used in CSEE's teaching labs, and the easiest way to find out the version you are using is to print it out via the Python interpreter:

$ python
>>> import cv2
>>>>>> print cv2.__version__
3.1.0
>>>

The version number printed out here is the one on the author's Mac, and the version in the software labs should be the same.

Remember that if you submit code for the assignments that does not run on the machines in Lab 1 under Linux, you will lose marks. You have been warned!

Under Linux, assuming you live in the Debian world (Debian, Ubuntu or derivatives), you can probably get by with the following commands to install OpenCV:

sudo apt-get install libopencv-dev python-opencv
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

The first of these installs the library and Python wrappers, while the second installs video codecs that may be needed.  Be sure to check that the version you install is compatible with the version in Lab 1.  I haven't done this recently myself, so if this is no longer correct please let me know and I'll update the recipe.  I'm also happy to add instructions for other distributions if anyone is crazy enough to live outside the Debian world.

Under MacOS, the route I took was to install MacPorts. Its instructions also tell you how to enable the compilers to be used from the command line (they are not by default, and it involves fiddling around with Xcode). With those steps done, you can simply use the commands:

sudo port install python27
sudo port install py27-numpy
sudo port select python27
sudo port install opencv +python27
Again, be sure to check that the version is compatible with that on the Lab 1 machines.  I know some Mac users prefer Homebrew, and Harrison Frith tells me that the following incantations work for El Capitan:

brew install python
brew tap homebrew/science
brew install opencv3 --with-contrib
brew install numpy
brew install jpeg libpng libtiff openexr
ln -s /usr/local/opt/opencv3/lib/python2.7/site-packages/cv2.so /usr/local/lib/python2.7/site-packages/
echo '#Homebrew' >> ~/.bash_profile
echo 'export PATH=/usr/local/bin:$PATH' >> ~/.bash_profile
source ~/.bash_profile

If you're an early adopter of Sierra, Harrison says you currently have to do the following instead because of a QT dependency:

brew install python
brew tap homebrew/science
brew install python jpeg libpng libtiff openexr 
brew install opencv3 --HEAD --with-contrib
brew link opencv3 --force
echo /usr/local/opt/opencv3/lib/python2.7/site-packages >> /usr/local/lib/python2.7/site-packages/opencv3.pth
echo '#Homebrew' >> ~/.bash_profile
echo 'export PATH=/usr/local/bin:$PATH' >> ~/.bash_profile
source ~/.bash_profile

Web pages maintained by Adrian F. Clark [contact]