Kyle helped once more with image collection by pointing me towards openlibrary.org, which has a huge collection of book cover images for download (on the order of 10s of thousands). These do not come with ratings, but even without the ratings they can be used for unsupervised exploration.
Wednesday, April 11, 2012
Monday, April 9, 2012
Image collection, pre-processing, and clustering
I was able to add 90 book covers from the genre of health and fitness to the dataset, so that I have 60-100 book covers from 4 genres, now. Kyle generously shared a collection of .png files of philosophy book covers, grabbed from a collection of pdfs, but there are issues with finding corresponding ratings from Amazon (social commentary?). Rotten Tomatoes does not rate books, BUT I came across Goodreads just yesterday, and it seems promising as an alternate source for ratings and cover images.
I've written a script to load the images into MATLAB using eval and imread. From here I can directly access the RGB color values for each of the images, which I will use for my first set of image features, color histograms. I've chosen color as a first step because its readily available, and worth exploring.
I ran across this related project by Dr. Sai Chaitanya Gaddam at Boston University, Judging a movie by its cover: A clustering analysis. The gist:
Note: he did not use the poster image itself for clustering.
Along the way, to overcome the issue of a 2D representation of 3D data, he came up with a neat visualization of the color properties of an image:
I've written a script to load the images into MATLAB using eval and imread. From here I can directly access the RGB color values for each of the images, which I will use for my first set of image features, color histograms. I've chosen color as a first step because its readily available, and worth exploring.
I ran across this related project by Dr. Sai Chaitanya Gaddam at Boston University, Judging a movie by its cover: A clustering analysis. The gist:
- use Netflix Prize data set of ~100 million ratings on 17770 movies from 480189 users
- find movie similarity matrix
- use k-means to find movie clusters
- find "average" movie poster images from exemplar images from each cluster
He came up with a few averaged poster images like the ones below:
Note: he did not use the poster image itself for clustering.
Along the way, to overcome the issue of a 2D representation of 3D data, he came up with a neat visualization of the color properties of an image:
As for using color histograms to capture similarities between pictures, a classic technique called Histogram Intersection comes from Swain and Ballard 1991, where, given two histograms with n bins each, their intersection is:
Goals for Wednesday include:
- Finish the dataset and upload everything to MATLAB
- Calculate color histogram intersections between images
Goals for the rest of the week include:
- Find something meaningful to do with intersection values - run k-means?
- Read more classification literature
Tuesday, April 3, 2012
Image Collection
I was initially optimistic of finding an automated way of harvesting book cover images from Amazon, but after a few attempts at using website copying software (httrack, SiteSucker), difficulty with the website format (the covers aren't even directly downloadable files..?) and an approaching deadline, I resorted to using screenshots. For future knowledge, I would still be very interested in learning an automated way of doing this.
Finding a good variety of ratings for books has also been difficult. Books can only be sorted from highest-rated to lowest-rated, and I was not able to access search results past 100 pages.
Anyway, I at least have a range of ratings from 3-5 stars. I've been saving them in PNG format to avoid loss of image quality. I use XnView, a free image-editing software that lets me crop and re-size many images at a time (available at http://www.xnview.com/en/index.html). So far I have 66 SciFi/Fantasy, 98 Romance, and 85 History, with more being added every day. For next week, I'll need to finish image collection and have a solid idea of what features I am going to extract and how. Some of my favorite book covers so far:
If you're curious, here's a link to the full project proposal.
Related Work - Paper Gestalt
I realized that when Professor Belongie asked me about related work on Monday, he had (probably) meant this (humorous) paper he sent me about using computer vision to determine the quality of CVPR paper submissions.
Very witty paper by "von Bearnensquash", can be found at http://vision.ucsd.edu/sites/default/files/gestalt.pdf. They used "standard computer vision features" (LUV histograms, HoG, and gradient magnitude) and AdaBoost classification and found that "good" paper features include brightly colored graph and math equations, and "bad" paper features include complicated tables and missing pages (illustrated below).
They found that allowing for a false positive rate of 15%, they could successfully reject half of the "bad" papers.
The problem I'm addressing is similar but there's an important distinction. They use the content of the thing itself to evaluate quality, so it is sensible for their to be a relationship, but book cover images are not necessarily related to the content of what I'm evaluating for quality (the book itself).
In any case, AdaBoost could be a good classification method to try as it is simple and doubles as a feature selection method. There is a nice overview at https://hpcrd.lbl.gov/~meza/projects/MachineLearning/EnsembleMethods/introBoosting.pdf
Very witty paper by "von Bearnensquash", can be found at http://vision.ucsd.edu/sites/default/files/gestalt.pdf. They used "standard computer vision features" (LUV histograms, HoG, and gradient magnitude) and AdaBoost classification and found that "good" paper features include brightly colored graph and math equations, and "bad" paper features include complicated tables and missing pages (illustrated below).
They found that allowing for a false positive rate of 15%, they could successfully reject half of the "bad" papers.
The problem I'm addressing is similar but there's an important distinction. They use the content of the thing itself to evaluate quality, so it is sensible for their to be a relationship, but book cover images are not necessarily related to the content of what I'm evaluating for quality (the book itself).
In any case, AdaBoost could be a good classification method to try as it is simple and doubles as a feature selection method. There is a nice overview at https://hpcrd.lbl.gov/~meza/projects/MachineLearning/EnsembleMethods/introBoosting.pdf
Subscribe to:
Posts (Atom)