EECS 331: Introduction to Computational Photography

Fall 2019 Tu-Thu 3:30-4:50pm - Professor Oliver (Ollie) Cossairt

Location: Tech LG66

 

 

Course Goals

To teach the fundamentals of modern camera architectures and give students hand-on experience acquiring, characterizing, and manipulating data captured using a modern camera platform. For example, students will learn how to estimate scene depth from a sequence of captured images.

 

Course Description

This course is the first in a two-part series that explores the emerging new field of Computational Photography. Computational photography combines ideas in computer vision, computer graphics, and image processing to overcome limitations in image quality such as resolution, dynamic range, and defocus/motion blur. This course will first cover the fundamentals of image sensing and modern cameras.  We will then continue to explore more advanced topics in computer vision. We will then use this as a basis to explore recent topics in computational photography such as motion/defocus deblurring cameras, light field cameras, and computational illumination.

 

This course will consist of six homework assignments and no midterm or final exam. We will provide a Nvidia Tegra tablet for each student in the course. Students will write programs that run on the phone to capture photos. Enrollment is limited to 30 students.

 

Prerequisites

EECS 211 and/or 230 or permission from instructor. Students should have experience with C/C++ and MATLAB programming. If you are interested, please contact the instructor to discuss!

 

Coursework and Grading

The course will consist of 6 homework assignments. Each assignment will consist of some camera programming and some image processing. The camera programming will be done in C/C++ and the image processing will be done using MATLAB. 

 

Grading will be based on a 100 point system. The homeworks will constitute the bulk of the course grade (90 points in total). Class attendance will constitute the other 10 points. Instructions for completing each assignment can be found at the following links:

 

HW1: Hello World Application (15 points)

HW2: Measuring Sensor Noise (15 points)

HW3: Flash/No Flash Photography (15 points)

HW4: HDR Imaging (15 points)

HW5: Depth From Focus (15 points)

HW6: Synthetic Aperture Imaging (15 points)

 

A discussion for each homework assignment has been created on Canvas. Please post all of your questions on the discussion board so that others may learn from your questions as well. Do not email the professor or TA directly with homework questions.

 

All Homeworks are to be submitted via Canvas by 11:59pm on the due date. Each student will be permitted ONE late submission for partial credit. Two points shall be docked from the submission for each 24-hour period. For instance, if the homework is due Tuesday at 11:59pm and it is submitted Wednesday between 12:00am and 11:59pm, 2 points will be docked. If the assignment is submitted on Thursday between 12:00am and 11:59pm, 4 points will be docked, and so on. Only ONE late assignment per student will be awarded partial credit. Any additional late assignments will receive no credit.

 

Course Syllabus

 

Tuesday 9/24/18	Introduction
Thursday 9/26/18	Image Formation
Tuesday 10/1/18	Image Sensing
Thursday 10/3/18	Image Processing I	HW1 Due
Tuesday 10/8/18	Image Processing II
Thursday 10/10/18	Flash and Lighting
Tuesday 10/15/18	No Class
Thursday 10/17/18	Radiometry	HW2 Due
Tuesday 10/22/18	HDR Imaging
Thursday 10/24/18	Photometric Stereo
Tuesday 10/29/18	Shape from Shading	HW3 Due
Thursday 10/31/18	Structured Light
Tuesday 11/5/18	No Class
Thursday 11/7/18	Depth from Focus
Tuesday 11/12/18	No Class	HW4 Due
Thursday 11/14/18	SIFT
Tuesday 11/19/18	Camera Calibration
Thursday 11/21/18	Stereo
Tuesday 11/26/18	Light Fields	HW5 Due
Thursday 11/28/18	Thanksgiving (No Class)
Tuesday 12/3/18	Light Transport
Thursday 12/5/18	Selected Topics	HW6 Due

 

Texts

Computational photography is a new and exciting field. No standard texts on this topic are available yet. Reading material and class slides will be will be available before each class. Optional texts include:

_       Forsyth and Ponce. Computer Vision: A Modern Approach. Pearson. 2002.

_       Richard Szeliski. Computer Vision: Algorithms and Applications. Springer. 2010.

_       Berthold K. P. Horn. Robot Vision. The MIT Press. 1986.

_       R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision, Cambridge Press, Cambridge, UK, 2000.

 

Course Instructor

Oliver (Ollie) Cossairt,

Office: Rm 3511, Seeley Mudd

Email: Ollie@eecs.northwestern.edu

Office Phone: (847) 491-0895.

Office Hours: Tuesday 2:30-3:30pm

 

Teaching Assistants

Florian Schiffers

Email: florian.schiffers@northwestern.edu

Office: Rm. 3406, Seeley Mudd

Office Hours: TBD

 

Useful Links

Similar Courses in Other Universities

_       Computational Photography (Gu, RIT)

_       Computational Photography SIGGRAPH Course (Raskar & Tumblin)

_       Computational Camera and Photography (Raskar, MIT)

_       Digital and Computational Photography (Durand & Freeman, MIT)

_       Computational Photography (Levoy & Wilburn, Stanford)

_       Computational Photography (Belhumeur, Columbia)

_       Computational Photography (Efros, CMU)

_       Computational Photography (Essa, Georgia Tech)

_       Computational Photography (Fergus, NYU)

_       Computer Vision (Seitz, U of Washington)

_       Computer Vision (Zhang, U of Wisconsin)

_       Computer Vision (Snavely, Cornell)

_       Introduction to Visual Computing (Kutulakos, U of Toronto)

More Links

_       What is Computational Camera, Shree Nayar, Columbia

_       Columbia Projects, Shree Nayar, Peter Belhumeur

_       MIT Projects, Fredo Durand, William Freeman, Edward Adelson, Antonio Torralba, Raskar Ramesh

_       Stanford Projects, Marc Levoy and collaborators

_       USC Projects, Paul Debevec and collaborators

_       CMU Projects, Narasimhan, Efros

_       Jack Tumblin's 'Questions' for the field

_       Conferences: ICCP 2011, ICCP 2010, ICCP 2009, SIGGRAPH, SIGGRAPH Asia, CVPR, ICCV, ECCV, ...

Acknowledgement

Many of the course materials are modified from the excellent class notes of similar courses offered in other schools by Shree Nayar, Marc Levoy, Jinwei Gu, Fredo Durand, and others. The instructor is extremely thankful to the researchers for making their notes available online.