Co-training and Self-training for Word Sense Disambiguation Page: 1
The following text was automatically extracted from the image on this page using optical character recognition software:
Co-training and Self-training for Word Sense Disambiguation
Department of Computer Science and Engineering
University of North Texas
This paper investigates the application of co-
training and self-training to word sense disam-
biguation. Optimal and empirical parameter se-
lection methods for co-training and self-training
are investigated, with various degrees of error
reduction. A new method that combines co-
training with majority voting is introduced, with
the effect of smoothing the bootstrapping learn-
ing curves, and improving the average perfor-
The task of word sense disambiguation consists in assign-
ing the most appropriate meaning to a polysemous word
within a given context. Most of the efforts in solving
this problem were concentrated so far towards supervised
learning, where each sense tagged occurrence of a par-
ticular word is transformed into a feature vector, which
is then used in an automatic learning process. While
these algorithms usually achieve the best performance, as
compared to their unsupervised or knowledge-based al-
ternatives, there is an important shortcoming associated
with these methods: their applicability is limited only to
those words for which sense tagged data is available, and
their accuracy is strongly connected to the amount of la-
beled data available at hand. In this paper, we investigate
methods for building sense classifiers when only relatively
few annotated examples are available. We explore boot-
strapping methods using co-training and self-training, and
evaluate their performance on the SENSEVAL-2 nouns.
We show that classifiers built for different words have dif-
ferent behavior during the bootstrapping process. If the
right parameters for co-training and self-training can be
identified (growth size, pool size, and number of itera-
tions, as explained later in the paper), an average error
reduction of 25.5% is achieved, with similar performance
observed for both co-training and self-training. However,
with empirical settings, the error reduction is significantly
smaller, with a 9.8% error rate reduction achieved for a
new method that combines co-training with majority vot-
We first overview the general approach of bootstrap-
ping for natural language learning using co-training and
self-training. We then introduce the problem of super-
vised word sense disambiguation, and define several local
and topical basic classifiers. We investigate the applica-
bility of co-training and self-training to supervised word
sense disambiguation, starting with these basic classifiers,
and perform comparative evaluations of optimal and em-
pirical bootstrapping parameter settings.
2 Co-training and Self-training for Natural
Co-training and self-training are bootstrapping methods
that aim to improve the performance of a supervised learn-
ing algorithm by incorporating large amounts of unlabeled
data into the training data set.
Starting with a set of labeled data, co-training algorithms
(Blum and Mitchell, 1998) attempt to increase the amount
of annotated data using some (large) amounts of unlabeled
data. Shortly, co-training algorithms work by generating
several classifiers trained on the input labeled data, which
are then used to tag new unlabeled data. From this newly
annotated data, the most confident predictions are sought,
and subsequently added to the set of labeled data. The
process may continue for several iterations.
In natural language learning, co-training was applied
to statistical parsing (Sarkar, 2001), reference resolution
(Ng and Cardie, 2003), part of speech tagging (Clark et
al., 2003), and others, and was generally found to bring
improvement over the case when no additional unlabeled
data are used.
Here’s what’s next.
This paper can be searched. Note: Results may vary based on the legibility of text within the document.
Tools / Downloads
Get a copy of this page or view the extracted text.
Citing and Sharing
Basic information for referencing this web page. We also provide extended guidance on usage rights, references, copying or embedding.
Reference the current page of this Paper.
Mihalcea, Rada, 1974-. Co-training and Self-training for Word Sense Disambiguation, paper, May 2004; (https://digital.library.unt.edu/ark:/67531/metadc30955/m1/1/?rotate=270: accessed June 16, 2019), University of North Texas Libraries, Digital Library, https://digital.library.unt.edu; crediting UNT College of Engineering.