Homework 3 - Playing with PCA

Navigation Path : Home Page > Computer Vision Homeworks > Homework 3 - Playing with PCA

• Authors : Sasakthi Abeysinghe, Timothy Simpson
• Date: 04/10/2006

• Problem Specification <Click Here...>

• Source Code - Written on Matlab v7.0 (R14)
  • Homework3.m - A script file responsible for starting the application
  • AverageImageSet.m - Performs the averaging pre-processing task
  • FrameDifferences.m - Performs the Frame Differences per-processing task
  • GetError.m - Compares an image with its reconstruction and returns the error value
  • GetErrorPlot.m - Plots the error of an image and its reconstructions with multiple amounts of components
  • LoadImageSet.m - Loads a set of images using the timestamps
  • PerformPreProcessing.m - Performs pre-processing on the set of images.
  • ReconstructChannel.m - Reconstructs an image using the coefficients and base images
  • ReconstructImage.m - Reconstructs an image using the coefficients and base images
  • RemakeImage.m - Resizes and recreates image into a visual form.
  • VisualizePrincipleImage.m - Lets you visualize a principle component

 

• Tasks Completed
  • A method for reading a large amount of chronologically ordered / unordered set of images and bringing it to form where it can be analyzed using PCA
  • Pre-proessing tasks
  • Visualizing component images
  • Measuring the reconstruction error and plotting graphs.

Input Data and Results -

No Pre-Processing - Camera #4 (South Brookings)

Original Image
2 - Component Recreation Error : 13655758	4 - Component Recreation Error : 5673350	6 - Component Recreation Error : 3480905	8 - Component Recreation Error : 3252184	10 - Component Recreation Error : 2834519
20 - Component Recreation Error : 2054924	30 - Component Recreation Error : 1345218	40 - Component Recreation Error : 1082149	50 - Component Recreation Error: 736449	100 - Component Recreation Error : 0
Top Ten Principle Components
(1) Strength = 6.2242 This seems to be the mean image of the image set.	(2) Strength = 0.9026 This component seems to be capturing the night images as one component. This could be explained by the fact that the night images rarely change	(3) Strength = 0.4549 We are guessing that this is a component showing how much of the quadrangle is in shadow.	(4) Strength = 0.2570 This Principle component seems to be capturing the glare that hits the window where the webcam is placed.	(5) Strength = 0.1951 This component seems to have captured the movement of the top of the tree (which is usually invisible due to the glare). A cloud formation seems to have also been captured.
(6) Strength = 0.1358	(7) Strength = 0.1202	(8) Strength = 0.0800	(9) Strength = 0.0710	(10) Strength = 0.0684
Change of the top 3 coefficients as a function of time
We see that in the first 150 mins, all three principle components seem to be stable. This most probably can be explained by the fact that the first 150 mins of the images were taken during night hours, and therefore didnt record much change. However at the introduction of sunlight we see that the principle components change dramatically. The second principle component which clearly displays the characteristics of being an average of all the night images, becomes less prominent, and the third principle component which we guess is the level of how much the quadrangle is in shadow becomes more prominent. Looking closer at the third principle component we see that its prominence slowly deteriorates over time. This could be explained by gradual loss of the region in the shadow as the day progresses. Looking at the first principle componenent it seems as if it is the mean image of all the images used in the PCA process. This can be explained by the fact that the value of the 1st principle component changes rapidly only during the night/day change, but stays the same during the rest of the period. In summary we see that the principle components change slowly through time, but are not directly correlated to each other, we would say that they might actually be orthogonal to each other.

Pre-processing Step 1: Averaging every 3 frames

2- Component Recreation Error :13231951	4 - Component Recreation Error :4855639	6 - Component Recreation Error: 3597763	8 - Component Recreation Error: 2828747	10-Component Recreation Error:2626594

Pre-processing Step 3: Averaging every 12 frames

2- Component Recreation Error :13548853	4 - Component Recreation Error :5470517	6 - Component Recreation Error: 4862998	8 - Component Recreation Error: 3345365

Pre-processing Step 4: Getting the difference between 2 frames

2- Component Recreation Error :195118184	4 - Component Recreation Error :129281682	6 - Component Recreation Error: 104589407	8 - Component Recreation Error: 90068505	10 - Component Recreation Error : 81628237

Pre-processing Step 5: Getting the difference between 2 frames on averaged images over an hour

2- Component Recreation Error :294267457	4 - Component Recreation Error :300747716	6 - Component Recreation Error: 280788167

Analysis on the Pre-Processing steps

• Averaging every 3 frames -
  • For our webcam, frames now span 15 minutes. This helps to eliminate minor temporal changes such as people walking around, branches swaying in the wind, and random UFO sightings. Longer effects, such as window glare from sunlight, parked car, or sleeping people will be reflected in the principle components.
• Averaging every 12 frames -
  • For our webcam, frames now span one hour. This eliminates fairly large temporal changes such as cloud cover and glare from the sunrise/sunset. Averaging more frames would simply reduce the imprint of temporal changes in the principle components even further.
• Image difference between each frame
  • Finding the differences between frames will emphasize the temporal differences in an image set. For example, person who repeatedly paces back and forth in the scene will leave a very large imprint in the principle components, since his difference will be felt in every frame. The areas of the scene with the highest amount of activity (such as a swaying tree, or the walkways) will be reflected in the primary principle components. The general images are harder to reconstruct using these principle components, since they focus on motion and not continual scenery.
• Image difference between the averages of 12 frames -
  • This looks weird, but there's a good reason why. Since averaging tends to focus on the more permanent features of a scene, and differencing tends to focus on the more volatile aspects of the scene, the composition of the two techniques will produce undefined results. However, if we were going to try to explain exactly what was going on, we would say we are focusing on those things that change the most among things that don't change that often. In hindsight, this is probably a silly thing to focus on.

Analysis on the Singular Values vs. Reconstruction Error

There is a proportional relationship between the singular values s(i) and the sum of the reconstruction error over the set of images using i-1 principle components.  Mathematically the ith singular value will indicate proportionally how much the total reconstruction error should drop after reconstructing the image set with i prinicple components, since s(i) is the relative importance of the ith principle component.

As a guess, I would say that the singular values act as inverse derivatives towards the reconstruction error. For example, when the singular values experience a large drop in value, it is a safe assumption that reconstruction with the next principle component will not decrease the overall reconstruction error as much as the previous principle component.

Algorithms

Visualizing A Principle Component - (VisualizePrincipleImage)

1. MinValue = The minimum element value in the principle component image
2. MaxValue = The maximum element value in the principle component image
3. Image = Image - MinValue ... In this step we shift the values in the image to be greater than equal to 0
4. Image = Image * 255 / (MaxValue - MinValue) ... In this step we normalize the values in the image to be between 0 and 255
5. Reshape the image to fit the original image dimensions
6. Display the image

General idea on what the other procedures do

1. Images when loaded are converted into a column form, where all 3 RGB channels are packed on top of each other and a single column matrix is made.
2. All the loaded images are packed into a m x n matrix where m is the total amount of pixels per image, and n is the amount of images in the image set
3. Pre-processing tasks are done.. The following pre-processing tasks are supported
   1. No pre-processing
   2. Averaging every 3 images (every 15 mins in a chronological ordering)
   3. Averaging every 12 images (every 1 hour in a chronological ordering)
   4. Getting the difference between subsequent images
   5. Averaging every 12 images, and therafter getting the differences between each pair.
4. Run SVD on the pre-processed image set and get values for u, s and v
5. Thereafter perform image reconstruction / error analysis tasks.