Tip:
Highlight text to annotate it
X
My name is Mathias Lux and I'm gonna show you what we did for the ACM Multimedia Grand Challenge 2012.
This is joint work of me, Mario Taschwer and our colleague from Florida, Oge Marques.
Our main idea was to classify photos based on the intentions of the photographer.
I have prepared two use cases.
Imagine you are on holiday and you really want to remember a certain scene, because you know that the image will be forgotten as the years pass by.
So you take a photo, file it and preserve the great feeling of being at a beautiful place in holiday.
Another use case is that you are on the universities parking lot, which is huge. Typically you search the area for your car right after work.
To remember where you parked the car this day days you take a picture and file it to find the car in the evening.
What we did for the Grand Challenge is that we created a classifier for photos.
We focused on the intentions of the photographers, and trained the classifier to separate photos that have been taken to preserve a good feeling ...
from those that were taken for other reasons.
We created an application that takes the photos, investigates the tags assigned to the photo, ...
and then decides upon whether the photo has been taken to preserve a good feeling ... or not.
We tested several approaches with different image features like metadata, title and visual features.
But evaluation has shown that tags worked best for our application.
By the way the photo set used to train the classifier is the result of a large survey and is provided online for scientific use.
Our prototype application has a simple interface.
It does a text search on Flickr and classifies photos to find those taken to preserve a good feeling.
Basically you enter a search term and images are retrieved from Flickr.
The result set is limited to 36 images.
They are shown on a grid and - of course - one can scroll down and view them.
However, the main clue in the application is that it classifies the retrieved 36 results.
With the icon in the upper right border, which we call the "good feelings switch", you can focus on those photos that have been taken to preserve a good feeling. All other photos are toned down.
We also presented the data set used to train the classifier in the CrowdMM workshop. This presentation, the data set, the papers and the poster are also available online.
Thank you for your time!