MediaEval Multimedia Benchmark
***Task proposal submission deadline: 21 December 2013***
MediaEval is a benchmarking initiative dedicated to evaluating new algorithms for multimedia access and retrieval. It emphasizes the 'multi' in multimedia and focuses on human and social aspects of multimedia tasks.
MediaEval is calling for proposals for tasks from researchers in academia and industry to run in the 2014 benchmarking season.
The proposal should contain the following elements:
For the last two points, it is helpful to refer to last year's survey to see the format of the task description and the type of questions.
There is no particular length specification for the proposal, some tasks will require more explanation than others. However, proposals are easier to manage if they are concise: in general, they should not exceed two pages.
Please email your proposal (as a .pdf) to Martha Larson email@example.com and Gareth Jones firstname.lastname@example.org by December 21, 2013.
Task proposals are accepted on the basis of the existence of a community of task supporters (i.e., researchers who are interested and would plan to participate in the task). Support is determined using a survey, which is circulated widely to the multimedia research community at the beginning of the year (January 2014). Task decisions are made mid-February. Tasks must also be viable given the design of the task and the resources available to the task organization team.
We encourage task proposers to join forces with colleagues from other institutions and other projects to create an organizing team large enough to bear the burden of data set generation, results evaluation, and working notes paper review. Please contact Martha Larson email@example.com if you have questions about task organization or if you are interested in being connected up with other people with similar interests and who could join together to form a task organizer team.
MediaEval has been experiencing steady growth since it was founded in 2008 as a track called "VideoCLEF" within the CLEF benchmark campaign. In 2010, it became an independent benchmark and in 2012 it ran for the first time as a fully "bottom-up benchmark", meaning that it is organized for the community, by the community, independently of a "parent" project. The MediaEval benchmarking season culminates with the MediaEval workshop. Participants come together at the workshop to present and discuss their results, build collaborations, and develop future task editions or entirely new tasks. Past working notes proceedings of the workshop include:
Example tasks that have run in past years are:
We expect the MediaEval 2014 workshop to be held in October 2014 in Europe, possibly returning to the venue of the MediaEval 2013 workshop in Barcelona. For more information on the MediaEval Multimedia benchmark, please visit http://www.multimediaeval.org/ or contact Martha Larson firstname.lastname@example.org.