WHAT to MEASURE in INTERACTIVE MULTIMEDIA IR EVALUATION?
Presentation paper, Nancy, October 1997 By Peter Ingwersen, PhD Head of Department of IR Theory Royal School of Librarianship Department of IR Theory Denmark Systems performance during interactive IR processes is affected by at least five “agents” or variables taking part in those processes: 1) the content and quality of the information space; 2) the search engine algorithms; 3) the interface functionality and ergonomics; 4) the cognitive space of the user, that is, the human perceptions of the work task and information need, and the current knowledge state; 5) the external assessor’s cognitive space which assesses the “relevance” of the retrieved documents or information objects in relation to an information need as perceived by the assessor (Figure 1). In traditional non-interactive experiments the number of variables is reduced to four: information space; search algorithm(s); simulated, stable information need (a query); assessor’s perception/interpretation of relevance for that (pe
