<XML><RECORDS><RECORD><REFERENCE_TYPE>3</REFERENCE_TYPE><REFNUM>8433</REFNUM><AUTHORS><AUTHOR>Baillie,M.</AUTHOR><AUTHOR>Azzopardi,L.</AUTHOR><AUTHOR>Ruthven,I.</AUTHOR></AUTHORS><YEAR>2007</YEAR><TITLE>A Retrieval Evaluation Methodology for Incomplete Relevance Assessments</TITLE><PLACE_PUBLISHED>Proceedings of the 29th European Conference on Information Retrieval (ECIR 2007)</PLACE_PUBLISHED><PUBLISHER>N/A</PUBLISHER><LABEL>Baillie:2007:8433</LABEL><KEYWORDS><KEYWORD>Evaluation</KEYWORD></KEYWORDS<ABSTRACT>In this paper we a propose an extended methodology for laboratory based Information Retrieval evaluation under incomplete relevance assessments. This new protocol aims to identify potential uncertainty during system comparison that may result from incompleteness. We demonstrate how this methodology can lead towards a finer grained analysis of systems. This is advantageous, because the detection of uncertainty during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections.</ABSTRACT></RECORD></RECORDS></XML>