Supplemental Material for the CL SI on Modality and Negation
This page provides supplemental material for the article Speculation and Negation: Rules, Rankers, and the Role of Syntax by E. Velldal, L. Øvrelid, J. Read and S. Oepen, 2012. The article appears in the Special Issue on Modality and Negation in the Journal of Computational Linguistics. Abstract and full text (pdf/html) are available through MIT Press.
Evaluation Software
When evaluating scope resolution for speculation cues we use the Java scorer provided by the CoNLL-2010 Shared Task organizers for Task 2. The scorer is available from the Shared Task webpage, as is a detailed description of the metric it implements. For negation, however, we use a slightly modified version supporting negation-type cues and scopes. The modified jar file can be downloaded here. It requires Java 1.6 or higher and can be run like;
java -jar neg_scorer.jar gold.xml test.xml.
System Output
The file velldal11.tgz bundles the system output for the final model configuration described in the paper, viz. the SVM-based filtered cue classifier coupled with the combination of dependency-based heuristics and discriminative constituent ranking for scope resolution. The output is provided as XML using the BioScope DTD, and the following files are included:
bse_speculation.xml: Speculation predictions for the CoNLL-2010 Shared Task evaluation databsa_negation.xml: Negation predictions for the BioScope abstracts (10-fold cross-validation)bsp_negation.xml: Negation predictions for the BioScope full papers (held-out)bsr_negation.xml: Negation predictions for the BioScope clinical reports (held-out)