Evaluating the impact of MeSH (Medical Subject Headings) terms on different types of searchers. Issue 4 (July 2017)

• Record Type: Journal Article
• Authors: Liu, Ying-Hsang; Wacholder, Nina
• Is Part Of: Information processing & management. Volume 53:Issue 4(2017:Jul.)
• Page Start: 851
• Page End: 870
• Publication Date: 2017-07
• Subjects: Information retrieval evaluation -- Medical Subject Headings -- Controlled vocabulary -- Domain knowledge -- User characteristics; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2017.03.004
• Languages: English
• ISSNs: 0306-4573

Abstract

Highlights: The usefulness of controlled vocabularies, exemplified by Medical Subject Headings (MeSH), was evaluated by a controlled user experiment. MeSH terms were most useful for domain experts in terms of precision measure. Domain knowledge was correlated with the precision score, whereas search training correlated with the recall score. This study demonstrated the feasibility of re-using a test collection originally created for evaluating the effectiveness of retrieval techniques for a controlled user experiment. Abstract: To what extent do MeSH terms improve search effectiveness for different kinds of users? We observed four different kinds of information seekers using an experimental information retrieval system: (1) search novices; (2) domain experts; (3) search experts and (4) medical librarians. Participants searched using either a version of the system in which MeSH terms were displayed or another version in which they had to formulate their own terms. The information needs were a subset of the relatively difficult topics originally created for the Text REtrieval Conference (TREC). Effectiveness of retrieval was based on the relevance judgments provided by TREC. The results of the study provide experimental evidence of the usefulness of MeSH terms and further identify the significant relationship between the user characteristics of domain knowledge and search training and the search performance in an interactive search environment.