Automatic Semantic Subject Indexing of Web Documents in Highly Inflected Languages

Presented at: 8th Extended Semantic Web Conference (ESWC2011)

by Reetta Sinkkilä, Osma Suominen, Eero Hyvönen

Structured semantic metadata about unstructured web documents can be created using automatic subject indexing methods, avoiding laborious manual indexing. A succesful automatic subject indexing tool for the web should work with texts in multiple languages and be independent of the domain of discourse of the documents and controlled vocabularies. However, analyzing text written in a highly inflected language requires word form normalization that goes beyond rule-based stemming algorithms. We have tested the state-of-the art automatic indexing tool Maui on Finnish texts using three stemming and lemmatization algorithms and tested it with documents and vocabularies of different domains. Both of the lemmatization algorithms we tested performed significantly better than a rule-based stemmer, and the subject indexing quality was found to be comparable to that of human indexers.

Keywords: automatic annotation, controlled vocabulary, inflected languages, subject indexing

Resource URI on the dog food server:

Explore this resource elsewhere: