CROHME 2014 Tasks
CROHME 2014 will have three different tasks, for which data and tools are provided.
Task 1: Isolated Symbol Recognition
There are a large number of math symbols, making math symbol classification a challenging. In the last CROHME competition, there were 101 math symbol classes. In a handwritten expression, symbols need to be both segmented and classified, forcing classifiers to consider mis-segmented (i.e. invalid) symbols. In the competition, we will compare classification systems two ways: 1) over valid symbols, and 2) over both valid symbols and mis-segmented (invalid) symbols. Mis-segmented symbols should be assigned to a reject class.
Recognizing mathematical expressions without an accurate symbol classifier is very difficult. Processes for locating symbols (i.e. segmenting) and identifying relationships between symbols (i.e. parsing) need to consider symbol identities. While classification is also dependent upon segmentation and parsing decisions, isolated symbol classification is the logical first step in constructing a handwritten math recognizer. This is because the classifier output space is fixed, while segmentation and parsing involve search. A classifier can be incorporated into objective functions for search algorithms used in segmentation and parsing, so that classifier outputs can help drive the search towards interpretations that include valid symbol hypotheses.
Given the importance of accurate symbol classification in constructing handwritten math recognition systems, a second aim of Task 1 is to encourage the creation of open-source solutions that include feature extraction and recognition for handwritten math symbols. Thus, both open-source and proprietary (closed-source) systems will be accepted, but we will highlight open-source solutions in the competition results.
Click here to know further details about the Task-1.
Task 2: Mathematical Expression Recognition
Task 2 considers recognition of entire handwritten expressions, requiring systems to identify symbol locations and identities, along with the spatial layout of symbols (e.g. adjacent, superscript, etc.). Datasets in the previous competition were challenging, as shown by the competition results. The training set includes over 8,000 expressions. We will re-use this training set, with the same expression grammar (Part IV, from CROHME 2013). A new test set will be constructed for evaluating and ranking participating systems.
As with Task 1, both open source and proprietary systems are encouraged to participate, but we will highlight open-source solutions in the competition results.
Click here to know further details about the Task-2.
Task 3: Matrix Recognition
This will be a new, experimental task for CROHME 2014. An expression with matrices can be considered as an expression with nested sub-expressions arranged in a tabular layout (i.e. in rows and columns). Thus to recognize expressions with matrices, systems must detect cells, their arrangement, and their contents. As in the main competition (Task 2), we need to consider symbol segmentation, recognition and layout, but where the segmentation and layout of matrices and matrix cells must also be determined.
Evaluation and visualization tools capable of handling the multiple levels of interpretation structure involved will be provided. We hope that this will encourage participants to create new systems, and to discuss how best to evaluate systems for Task 3.
Click here to know further details about the Task-3.