Text Mining for Automatic Image Tagging Page: 647

                                
Text Mining for Automatic Image Tagging

Chee Wee Leong and Rada Mihalcea and Samer Hassan
Department of Computer Science and Engineering
University of North Texas
cheeweeleong@my.unt.edu, rada@cs.unt.edu, samer@unt.edu

Abstract
This paper introduces several extractive
approaches for automatic image tagging,
relying exclusively on information mined
from texts. Through evaluations on two
datasets, we show that our methods ex-
ceed competitive baselines by a large mar-
gin, and compare favorably with the state-
of-the-art that uses both textual and image
features.
1 Introduction
With continuously increasing amounts of images
available on the Web and elsewhere, it is impor-
tant to find methods to annotate and organize im-
age databases in meaningful ways. Tagging im-
ages with words describing their content can con-
tribute to faster and more effective image search
and classification. In fact, a large number of ap-
plications, including the image search feature of
current search engines (e.g., Yahoo!, Google) or
the various sites providing picture storage services
(e.g., Flickr, Picasa) rely exclusively on the tags
associated with an image in order to search for rel-
evant images for a given query.
However, the task of developing accurate and
robust automatic image annotation models entails
daunting challenges. First, the availability of large
and correctly annotated image databases is cru-
cial for the training and testing of new annotation
models. Although a number of image databases
have emerged to serve as evaluation benchmarks
for different applications, including image anno-
tation (Duygulu et al., 2002), content-based im-
age retrieval (Li and Wang, 2008) and cross
language information retrieval (Grubinger et al.,
2006), such databases are almost exclusively cre-
ated by manual labeling of keywords, requiring
significant human effort and time. The content of
these image databases is often restricted only to a

few domains, such as medical and natural photo
scenes (Grubinger et al., 2006), and specific ob-
jects like cars, airplanes, or buildings (Fergus et
al., 2003). For obvious practical reasons, it is im-
portant to develop models trained and evaluated
on more realistic and diverse image collections.
The second challenge concerns the extraction
of useful image and text features for the construc-
tion of reliable annotation models. Most tradi-
tional approaches relied on the extraction of image
colors and textures (Li and Wang, 2008), or the
identification of similar image regions clustered as
blobs (Duygulu et al., 2002) to derive correlations
between image features and annotation keywords.
In comparison, there are only a few efforts that
leverage on the multitude of resources available
for natural language processing to derive robust
linguistic-based image annotation models. One
of the earliest efforts involved the use of captions
for face recognition in photographs through the
construction of a specific lexicon that integrates
linguistic and photographic information (Srihari
and Burhans, 1994). More recently, several ap-
proaches have proposed the use of WordNet as
a knowledge-base to improve content-based im-
age annotation models, either by removing noisy
keywords through semantic clustering (Jin et al.,
2005) or by inducing a hierarchical classification
of candidate labels (Srikanth et al., 2005).
In this paper, we explore the use of several natu-
ral language resources to construct image annota-
tion models that are capable of automatically tag-
ging images from unrestricted domains with good
accuracy. Unlike traditional image annotation
methodologies that generate tags using image-
based features, we propose to extract them in a
manner analogous to keyword extraction. Given a
target image and its surrounding text, we extract
those words and phrases that are most likely to
represent meaningful tags. More importantly, we
547

Coling 2010: Poster Volume, pages 647-655,
Beijing, August 2010