Due: At the start of class, Thursday, April 6, 2017.
The purpose of this assignment is to understand and implement methods for shot boundary detection in a video sequence.
In this assignment we'll detect shot boundaries in 151 frames from a well known cartoon (google “Snow White and the Seven Dwarfs Heigh Ho Song”). Shot boundaries mark transitions between “shots.” A shot is a sequence of frames where the camera sees more or less the same scene. In this case, shot boundaries mark the transition from one dwarf to another.
We can identify shot boundaries in two different ways:
You are given skeleton code for the assignment, Shot.py
You are given 151 frames in a directory of .jpg files, each of size 90 by 120. You can get the images from colours.zip.
Manually find the shot boundaries. Record the last frame number of each shot (i.e., if the shot changes from frame 3 to 4, you should record 3 as a shot boundary). Note that the first and last frames in the entire sequence do not count as shot boundaries.
First, we will try k-means as a method to detect the shot boundaries. kmeans is a SciPy function that implements the k-means clustering algorithm -- look at its documentation to see how it works.
We are going to construct features for each frame using histograms of pixel values. The code to read the images and to create gray histograms is provided (see the function compute_gray_histograms).
Once we have the histograms for each frame, the task is to cluster them using k-means. We know, a priori, that there are 4 shots. Therefore, we use k=4 clusters. Once we have the cluster centers, we assign each frame's histogram to its associated cluster. Finally, we find frames where the cluster assignment changes, using the function cluster2boundaries provided. As a test, use the function get_boundaries_cost to evaluate how good this prediction is.
Another way to detect shot boundaries is to detect changes between successive frames. For this part of the assignment, use only the gray value images.
Add code to your Shot.py file to implement the following measures of change between successive frames:
Answer the following written questions:
Hand in a printed copy of your Shot.py file. This should include all the code added to complete parts 1–8. Include sufficient comments for others to easily understand what you have done. In addition, hand in the plots requested in parts 2, 4, and 5–8, and your answers to the questions in parts 9 and 10.
For this assignment, we are offering the option of electronic handin for the requested plots. Refer to the department's Handin instructions for information about submitting assignments electronically. The course account is cs425 and the assignment name is hw7.