Metrics for evaluating translation memory software

Abstract

Translation memory (TM) tools help human translators recycle portions of their previous work by storing previously translated material. In conventional TM tools, the aligned texts are divided into sentence-level source and target translation units for storage in the database. Each sentence of a new source text is compared with the units stored in the database, and the tool proposes matches that are exact or similar. This is referred to as a sentence-based approach to search and retrieval. A different and more recently developed approach involves storing full source- and target-text pairs (known as bitexts) in the database and identifying identical character strings of any length. This is referred to as a character-string-within-a-bitext (CSB)-based approach to search and retrieval. Because the second approach is more recent, traditional techniques for evaluating TM tools do not take into account this fundamental difference. Therefore, the goal of this thesis is to design and develop a new evaluation methodology that can be used to compare the two approaches to search and retrieval fairly and systematically, first by defining "usefulness" as a measurable attribute, then by measuring the usefulness of the output of each approach in an identical translation context. (Abstract shortened by UMI.)