Extracting a Representation from Text for Semantic Analysis Page: 241
The following text was automatically extracted from the image on this page using optical character recognition software:
Extracting a Representation from Text for Semantic Analysis
Rodney D. Nielsen',2, Wayne Ward',2, James H. Martin', and Martha Palmer'
1 Center for Computational Language and Education Research, University of Colorado, Boulder
2 BoulderLanguage Technologies, 2960 Center Green Ct., Boulder, CO 80301
Rodney.Nielsen, Wayne.Ward, James.Martin, Martha.Palmer@Colorado.edu
We present a novel fine-grained semantic rep-
resentation of text and an approach to con-
structing it. This representation is largely
extractable by today's technologies and facili-
tates more detailed semantic analysis. We dis-
cuss the requirements driving the
representation, suggest how it might be of
value in the automated tutoring domain, and
provide evidence of its validity.
This paper presents a new semantic representation
intended to allow more detailed assessment of stu-
dent responses to questions from an intelligent tu-
toring system (ITS). Assessment within current
ITSs generally provides little more than an indica-
tion that the student's response expressed the target
knowledge or it did not. Furthermore, virtually all
ITSs are developed in a very domain-specific way,
with each new question requiring the handcrafting
of new semantic extraction frames, parsers, logic
representations, or knowledge-based ontologies
(c.f., Jordan et al., 2004). This is also true of re-
search in the area of scoring constructed response
questions (e.g., Leacock, 2004).
The goal of the representation described here is
to facilitate domain-independent assessment of
student responses to questions in the context of a
known reference answer and to perform this as-
sessment at a level of detail that will enable more
effective ITS dialog. We have two key criteria for
this representation: 1) it must be at a level that fa-
cilitates detailed assessment of the learner's under-
standing, indicating exactly where and in what
manner the answer did not meet expectations and
2) the representation and assessment should be
learnable by an automated system - they should
not require the handcrafting of domain-specific
representations of any kind.
Rather than have a single expressed versus un-
expressed assessment of the reference answer as a
whole, we instead break the reference answer
down into what we consider to be approximately
its lowest level compositional facets. This roughly
translates to the set of triples composed of labeled
(typed) dependencies in a dependency parse of the
reference answer. Breaking the reference answer
down into fine-grained facets permits a more fo-
cused assessment of the student's response, but a
simple yes or no entailment at the facet level still
lacks semantic expressiveness with regard to the
relation between the student's answer and the facet
in question, (e.g., did the student contradict the
facet or completely fail to address it?) Therefore, it
is also necessary to break the annotation labels into
finer levels in order to specify more clearly the
relationship between the student's answer and the
reference answer facet. The emphasis of this paper
is on this fine-grained facet-based representation -
considerations in defining it, the process of extract-
ing it, and the benefit of using it.
2 Representing the Target Knowledge
We acquired grade 3-6 responses to 287 questions
from the Assessing Science Knowledge (ASK)
project (Lawrence Hall of Science, 2006). The re-
sponses, which range in length from moderately
short verb phrases to several sentences, cover all
16 diverse Full Option Science System teaching
and learning modules spanning life science, physi-
cal science, earth and space science, scientific rea-
soning, and technology. We generated a corpus by
transcribing a random sample (approx. 15400) of
the students' handwritten responses.
Proceedings of ACL-08: HLL; Short Papers (Companion Volume), pages 241-244,
Columbus, Ohio, USA, June 2008. 02008 Association for Computational Linguistics
Here’s what’s next.
This paper can be searched. Note: Results may vary based on the legibility of text within the document.
Tools / Downloads
Get a copy of this page or view the extracted text.
Citing and Sharing
Basic information for referencing this web page. We also provide extended guidance on usage rights, references, copying or embedding.
Reference the current page of this Paper.
Nielsen, Rodney D.; Ward, Wayne; Martin, James H. & Palmer, Martha. Extracting a Representation from Text for Semantic Analysis, paper, June 2008; Stroudsburg, Pennsylvania. (https://digital.library.unt.edu/ark:/67531/metadc1042597/m1/1/: accessed March 22, 2019), University of North Texas Libraries, Digital Library, https://digital.library.unt.edu; crediting UNT College of Engineering.