CSE 730 Homework 3: Computer Vision/Audition

Due Sunday 18 November 2006, 11:59 PM (electronic)

or Friday 16 November 2006, 4:30 PM (paper) in my mailbox (no late paper hws accepted)

10 point penalty for every hour late

1. Do problem 24.10 (page 900), which asks about the image on page 893, which also appears in the notes. Note the word "reasons" (plural) in the question. Consider that you may have multiple images across time. In particular, what assumptions do you have to make about the world/scene to draw conclusions?
2. Imagine instead of query-by-humming you wanted to put a camera on your ipod and use hand signals to indicate pitch heights. How could you use current query-by-humming technology to accomplish this? (For more details than what I gave you in class, see this link (Brian Smith, Cornell) and this link (Brian Pardo, Northwestern)) What difficulties might you run into?

   I expect an outline of how you would set up the experiment, how input would be transformed into something useful, how you might apply learning, and how you would evaluate the system. I expect roughly 1/2 to 1-page answer.

3. This problem is primarily a programming assignment. You can use any programming language you want, however, I am adding some extra instructions for those who would like to tackle this in matlab.
   There are three image files in this directory.
   • shapes.raw: 300x150
   • dreese.raw: 167x143
   • num.raw: 186x102
   These files are stored in 8-bit grayscale, row by row.

   The assignment: implement the edge detection algorithm described in class and in the book to detect pixels that are likely edges in the images. Using the Canny Edge Detector built in to Matlab gets no credit. ;-)

   Part I: smooth each of the images with a 2-D gaussian filter. For one of the images, try different sigmas and different filter lengths. Describe your results. Turn in the code and the smoothed images.

   Part II: implement a vertical line detector with a Gaussian derivative in the x-direction. Remember that:

   Gx(x,y)=G'(x) G(y)

   The easiest way to estimate G'(x) is to take the difference G'(x)=G(x)-G(x-1) and set G'(0) to 0. This isn't the most accurate estimate, but it will do for this purpose.

   Show the output of the detector on the three image files.

   Part III: implement a similar horizontal line detector, and show the output.

   Part IV: Compute the any-orientation edge detection, given by sqrt(Gx(x,y)^2+Gy(x,y)^2). Show the output on all three images.

   Part V: What happens if you threshold the output at a certain level? Choose one level and show the output on all three images with the same level. Discuss your findings.

   Extra credit: Implement the final stages of Canny edge detection algorithm, where you find line segments from the edgels.

   ---

   In order to make your life a little easier, I'm providing some basic matlab tools. Even if you choose to code in a different language, you may find it easier to display images with these functions. In order to view the images, you'll have to be connected to stdsun via an x-terminal (or via some sort of x-connection). Here's how to display the images. Comments begin with % in matlab. First,download all of the data and .m files from here. Then:

   matlab
   % in matlab now
   
   display_imagefile('dreese.raw',167,143);
   % displays image
   
   i=load_image('dreese.raw',167,143);
   % loads image into 143x167 matrix
   
   display_image(i);
   %rescales and displays the image in i
   
   write_image(filename,i);
   % writes out the image file
   

   You can also call the function image(i) directly, but remember that matlab expects all of the elements of the image to be between 0 and 64, so you can get some wacky results if you have pixels higher than that. To create a matlab script, just put some matlab commands into a file with a .m extension; you can then run it from the matlab command line. Try running ``demo'' (which calls demo.m and does 1-d convolution on an example). Here are some basic matlab operations which you might find useful.

   a=0;
   % set a to 0
   a
   %
   % returns:
   % a =
   %
   %     0
   %
   
   b=[1 2;3 4];
   % sets b to a 2x2 matrix, first row 1 2 second row 3 4
   
   b*b
   % matrix multiplication
   % ans =
   %
   %   7    10
   %  15    22
   
   b.*b
   % point-by-point multiplication
   % ans =
   %
   %   1     4
   %   9    16
   
   c=[2;3];
   % column vector
   c*c'
   % c': c transposed
   % cross-multiplication of column x row
   % ans =
   %
   %     4     6
   %     6     9
   

   By-the-by, if you leave off the semicolon, it will print the answer. Here are some basic matlab operations which you might find useful.

   g=gausswin(11,5)
   % create a Gaussian window of size 11, stddev=1/5
   g =
   
       0.0000
       0.0013
       0.0243
       0.1915
       0.6615
       1.0000
       0.6615
       0.1915
       0.0243
       0.0013
       0.0000
   
   % now, I like to normalize g so it sums to one
   
   g=g/sum(g)
   
   g =
   
       0.0000
       0.0005
       0.0088
       0.0695
       0.2399
       0.3627
       0.2399
       0.0695
       0.0088
       0.0005
       0.0000
   
   % plot out g to see what it looks like
   
   plot(g)
   
   % graph of g appears
   
   % create a 2-d gaussian
   
   g2=g*g'
   
   % plot the surface and give it pretty colors
   surf(g2)
   colormap(hsv)
   
   % to print out an image:
   print -dpsc filename.ps
   % this will create filename.ps, which you can print using lpr
   
   % you can also ``print out'' an image as a jpeg if you want to
   % view it with another viewer
   print -djpeg filename.jpg
   
   

   You can use ``help X'' to get help on any function. Some other functions you might want to know about:
   • conv(h,x) 1-d convolution with filter and signal
   • conv2(h,x) 2-d convolution with filter and signal
   • diff(x) gives the difference function I outlined above; for a column vector you'll want dx=[0;diff(x)]
   • size(x) gives the size of each dimension of x
   Have fun!

> submit c730aa lab3 hw3

For the code, you should turn in either

• An executable version of your program along with the source code.
• If your program is written in an interpreted language, such as perl, lisp or python, you can turn in just the source. In that case, make sure to use a standard file extension so that we can determine how to run your program.