|
|
Forum for Information Retrieval Evaluation ( FIRE ) 17 - 19 December 2012 Indian Statistical Institute, Kolkata |
News
Springer will publish the post-proceedings of FIRE 2010 & 2011 as an LNCS proceedings.
ACM TALIP Vol 9, Issue 3 & Issue 4. Special issue on ILIR.
Adhoc Retrieval
The FIRE ad hoc task is similar to the TREC ad hoc task. Its objective is to evaluate the effectiveness of retrieval systems in retrieving accurate and complete ranked lists of documents in response to fifty one-time information needs. The FIRE 2011 ad hoc task focuses specifically on South Asian languages. This is the third year of the FIRE Adhoc Task. New participants are welcome!
- We are supplementing the FIRE 2008 / 2010 Bengali Anandabazar Patrika (ABP) corpus, which had had news articles of the years spanning 2004 to 2007, with a longer span ranging from 2001 to 2010.
- The BDNews 24 collection covers news in English and Bengali from 2006 - 2010.
- Mono-lingual retrieval
- Bengali
- Gujarati
- Hindi
- Marathi
- Odia
- Tamil
- Cross-lingual retrieval
- Queries : Bengali / English / Gujarati / Hindi / Marathi / Odia / Tamil / Telegu
- Documents : Bengali / English / Gujarati / Hindi / Marathi / Odia / Tamil
The Bengali and Hindi topics will also be transliterated and made available in Roman script. All participants are encouraged to submit runs using these queries as well.
All participants should periodically check this website for announcements and updates.
All participants should subscribe to the track mailing list to participate in discussions and be informed of any late-breaking announcements join the FIRE discussion forum.
Each submission file should contain 1000 documents per topic, ranked 0-999, in the usual TREC / CLEF submission format, i.e. each line in the file should have the following fields:
<Query id> Q0 <DOCNO> <RANK> <SIMILARITY> <Run-ID>
Participants will need to submit a gzipped file containing retrieval results in the above format.
All participants are required to submit at least one run that uses only the title and description fields (no narrative) of the topics. There is no upper limit on the number of submitted runs. However, please assign a priority to each of your submissions. Runs will be included in the pooling process in order of priority.
