UNT at ImageCLEF 2010: CLIR for Wikipedia Images Page: 4
The following text was automatically extracted from the image on this page using optical character recognition software:
4 Results and Analysis
We submitted three official runs and have an unofficial manual run
* untaTxEn: This run uses automatic query construction using the portion of the
* untaTxFr: This run uses automatic query construction using the original
French portion of the text which was translated to English using Google
Translation. (This can be considered a standard CLIR scenario)
* untManlEn: This run uses the final version of the queries created by our first
user for each of the 70 topics.
* untMan2En: This run uses the final version of the queries created by our
second user for each of the 70 topics.
Table 1 shows the retrieval performance of each of our runs. As expected the
manual runs had the best performance on P@5, P@ 10 and P@20 retrieved documents
which is consistent with the procedures that the users followed to build their queries.
The best MAP for our official runs was the automatically generated English run.
However, the manual run generated by the second user did perform better than all our
runs. Our manual run 1 achieved a pretty high performance in terms of P@5, P@10
and P@20. When we compared this run with all runs of other participant teams is
among the top 10 runs in these measures. However, if we use MAP the best scoring of
our official runs is the Automatic English run followed by our unofficial manual run
Table 1 Performance of Official and Unofficial Runs
untaTxEn untaTxFr untManlEn untMan2En
#ret 55647 58476 11779 20255
#Relev 17660 17660 17660 17660
#relret 7840 7641 4584 5768
Avg-P 0.2251 0.22 0.2064 0.2349
exact-P 0.3025 0.2855 0.2603 0.3002
P@5 0.4857 0.46 0.6314 0.6171
P@10 0.4314 0.4229 0.5886 0.5914
P@20 0.3871 0.3986 0.5021 0.5521
Comparing the runs using a standard Recall-Precision graph gives a better picture
of the performance of the manual and automatic runs (see Figure 1). The manual runs
in general perform better on the early R-P levels (0-0.2) while the automatic runs
perform better on the higher levels of recall. This seems to be correlated to the
amount of images retrieved which basically indicate that the manual queries are
Here’s what’s next.
This paper can be searched. Note: Results may vary based on the legibility of text within the document.
Tools / Downloads
Get a copy of this page or view the extracted text.
Citing and Sharing
Basic information for referencing this web page. We also provide extended guidance on usage rights, references, copying or embedding.
Reference the current page of this Paper.
Ruiz, Miguel E.; Chen, Jiangping; Pasupathy, Karthikeyan; Chin, Pok & Knudson, Ryan. UNT at ImageCLEF 2010: CLIR for Wikipedia Images, paper, September 2010; (https://digital.library.unt.edu/ark:/67531/metadc96836/m1/4/: accessed May 20, 2019), University of North Texas Libraries, Digital Library, https://digital.library.unt.edu; crediting UNT College of Information.