A New Benchmark for Automatic Essay Scoring in Portuguese

Automatic Essay Scoring promises to scale up student feedback of written input, considerably improving learning. Resources for Automatic Essay Scoring in Portuguese are however scarce, not publicly available or contain inaccuracies that degrade performance. Moreover, they lack data provenance and a richer annotation and analysis. In this work we mitigate those issues by presenting a new benchmark for the task in Brazilian Portuguese. We accomplish that by downloading a collection of publicly available essays from websites that simulate University Entrance Exams, making both processed and raw data available, having a subset of the essays graded by expert annotators to assess the quality and difficulty of the task, and carrying out an extensive empirical analysis of state-of-the-art predictors considering multiple evaluation criteria.

Conference paper