Homework 1 - Eye Finder

Navigation Path : Home Page > Computer Vision Homeworks > Homework 1 - Eye Finder

• Problem Specification <Click Here...>

• Source Code - Written on Matlab v7.0 (R14)
  • homework1.m - Script file starting the process <Click to download..>
  • loadFindEyeAndSave.m - Function responsible for loading and saving the images <Click to download..>
  • findEye.m - Function responsible for the actual eye-finding algorithm <Click to download..>
  • Eye.jpg - A template eye which is used in the eye detection process <Click to download..>

Question 1

• Name : Sasakthi Senanayaka Abeysinghe
• Date: 01/27/2006

Question 2

Image Name	Original Image	Detection Results
Crying Eyes
Catherine Zeta Jones
Jackie Chan
Jim Carrey

Question 3

• Algorithm
  1. sourceImage = Load original image from file, and convert it into grayscale and normalize the value range to be between 0 and 1
  2. template = Load template eye from file and convert it into grayscale and normalize the value range to be between 0 and 1
  3. w = Vary the width of the eye from 10 pixels to 25% of the width of the sourceImage
     1. eyeMask = Resize the template eye so that the width is w.
     2. x = Vary the x coordinate from 1 to the width of the sourceImage less the width of the eyeMask
        1. y = Vary the y coordinate from 1 to the height of the sourceImage less the height of the eyeMask
           1. imageMask = A section of the sourceImage starting at coordinates (x, y) which is the same size as the eyeMask.
           2. Run an SSD scoring function based on the square of the differences between the eyeMask and the imageMask, and store the result.
              1. ssd(y,x,w) = sum of all elements ( (eyeMask - imageMask)^2 ) / Normalization factor
              2. The normalization factor is used to normalize the ssd score to a value between 0 and 1
  4. threshold = Find the minimum value found using the SSD scoring function, and add a heuristic constant of 0.015 (The constant is added to ensure that more than 1 eye is found per image.. The constant was determined through experimentation)
  5. Find all locations in the SSD result matrix which have values less than the threshold and mark those locations as potential eyes in the destinationImage (The eye size marked is influenced by the eye size used when calculating the SSD score)

Question 4

• Analysis
• The fundamental idea behind the algorithm is to compare a template eye and blocks in an image and to assign each block a score between 0 and 1 based on their similarity. A score of 0 means that they are identical, while a score of 1 means that they are very different. Therefore we see that the strength of the algorithm is determined by the generality of the template eye being used. Although resizing and gray scaling is done to generalize the template eye to an extent it does not cover all different shapes of eyes, and dosen't cover the situation where the eye can be fully or partially closed.
• Template Eye:
• Strengths:
  • Looking at the examples, we see that the algorithm is strongest when the eyes in the image are as close to the template eye as possible. So looking at the template eye which was used in this run, we can say that the algorithm works best in the following conditions:
    • The eyes in the picture are open. (The sclera (white of the eye) and the iris (color of the eye) is visible).
    • The color of the iris is as dark as possible.
    • The shape of the eye matches the shape as the eye above.
    • The faces in the picture are vertical, and not at an angle.
  • Looking at Examples 2, 3 and 4 we can see the following shortcomings of the algorithm:
    1. Black letters in a white background were taken to be eyes.. This is due to the fact that the algorithm confused the White - Black - White pattern in the background to be the same as the White - Black - White pattern in the eye.
    2. Eyes of different shape (Jackie Chan's eyes) are not properly identified, therefore a non-optimal score was generated, resulting in many 'false-eye' hits on other image locations.
    3. The algorithm fails when the face is not vertical (Jim Carey's eyes). This too results in a non-optimal score, thus having many 'false-eye' hits on other image locations.
• Solving Shortcomings
  • The first shortcoming arose mostly due to the fact that the template eye is a picture of the eye and not the skin around it.
    • Therefore one approach of solving this problem we could introduce a larger picture for the template eye, which contains more skin from the face. However a fine balance should be found between the amount of eye pixels and the amount of skin pixels in the template eye, because otherwise the algorithm could end up throwing 'false-eye' hits whenever it sees a skin-like texture, even when an eye is not present. Finding this 'perfect' ratio needs to be done experimentally and therefore would take a considerable amount of time
    • Another approach is to use a template eye which has 2 matrices. The 1st matrix is a pixel map containing the average pixel color of many images of eyes. The 2nd matrix contains values based on their standard deviation. Using these two matrices we can define a scoring function which gives a higher priority to pixels which have a low standard deviation, and thereby capture the general characteristics of an eye, and use that in the detection process. However this requires a large amount of images to be used as the 'training' data set, and therefore would require a significant time for the data gathering and training operations. However, implementing this change would not take too much time.
  • The 2nd shortcoming arose due to the different shapes of the eyes.
  • This can be solved by using multiple template eyes for each type of eye. The algorithm could be run on each one of these template eyes, and the lowest SSD scores could be used to locate the positions of the eyes.
  • The 3rd shortcoming arose due to the orientation of the image.
    • This can be solved by adding a loop to the outermost layer of the algorithm which rotates the template eye at fixed angular intervals and runs the scoring function on all these orientations. After all the orientations are scored, the locations with the lowest scores (within a reasonable threshold) can be chosen as the eye locations. Although implementing this wouldn't take too much time, it would add a very large delay to the runtime of the algorithm. Therefore it was assumed that a pre-processing step is done on each image to correct its orientation before it is given to the algorithm.