Text Document

Evaluating Expertise Recommendations

Fulltext URI

Document type

Additional Information



Journal Title

Journal ISSN

Volume Title


Association for Computing Machinery


Finding a person who has the expertise to solve a specific problem is an important application of recommender systems to a difficult organizational problem. Prior systems have made attempts to implement solutions to this problem, but few systems have undergone systematic user evaluation. This work describes a systematic evaluation of the Expertise Recommender (ER), a system that recommends people who are likely to have expertise in a specific problem. ER and the organizational context for which it was designed are described to provide a basis for understanding this evaluation. Prior to conducting the evaluation, a baseline experiment showed that people are relatively good at judging coworkers' expertise when given an appropriate context. This finding provides a way to demonstrate the effectiveness of ER by comparing ER's performance to ratings by coworkers. The evaluation, the design, and results are described in detail. The results suggest that the participants agree with the recommendations made by ER, and that ER significantly outperforms other expertise recommender systems when compared using similar metrics.


McDonald, David W. (2001): Evaluating Expertise Recommendations. Proceedings of the 2001 ACM International Conference on Supporting Group Work. DOI: 10.1145/500286.500319. New York, NY, USA: Association for Computing Machinery. pp. 214–223. Boulder, Colorado, USA