SMS-Based FAQ Retrieval

FIRE 2012 Task

Important Dates
Joint Task Coordinators
IBM Research

Run Submission Guidelines

1. You can submit upto 3 runs per subtask.

2. Each run will have its own file with the following file name format : emailaddress$subtaskName$RunNo.txt
where - emailaddress is the Email address used for registration
subTaskNames could be one of the following:
a) "eng-mono" - For the English Monolingual Retrieval Task
b) "hin-mono" - For the Hindi Monolingual Retrieval Task
c) "mal-mono" - For the Malayalam Monolingual Retrieval Task
d) "eng-multi" - For the English Multi-lingual Retrieval Task
e) "hin-multi" - For the Hindi Multi-lingual Retrieval Task
f) "mal-multi" - For the Malayalam Multi-lingual Retrieval Task
g) "cross" - For the cross lingual retrieval task
runNo : Integer between 1 and 3.

For example, the first run submission by user : for the Malayalam multilingual run would be:$mal-multi$1.txt

3. In the retrieval tasks, you may submit upto 5 matches per SMS query along with the scores between 0 and 1, where 1 being the highest.

(a) The format of the run submission file is as follows (FAQs to be sorted in decreasing order of scores - left to right).


(b) In case there are less than 5 matches :

The entry would stop at the appropriate count, For example, if there are only 3 matches - the row ends at the third match.
(c) In case a query returns no matches, the entry should be as follows:
(Comma separated and entry as NULL)

Please use the FIRE 2012 SMS Task Submission System to upload your runs.

Please prepare a flat zip/tgz archive containing only your run submissions and upload it it in the submission system. Only one gz/tgz/zip archive needs to be submitted.

The upload system refers to the submission as a "paper" but we would like to clarify you do not need to upload any pdf/write up at this stage and you should use the upload mechanism to submit only your zip/tgz archive containing your task runs.

All participating teams also need to submit a working note that outlines the approach followed by them. The Working note is due on November 28, 2012.

The post-proceedings will be published in Lecture Notes in Computer Science, Springer. Authors will be invited to submit a revised and expanded version of their working note to be published in the post-proceedings.