Item

Evaluating Expertise Recommendations

Thumbnail Image

Fulltext URI

Document type

Additional Information

Date

2001

Authors

Journal Title

Journal ISSN

Volume Title

Source

Proceedings of the 2001 ACM International Conference on Supporting Group Work

Publisher

Association for Computing Machinery

Abstract

Finding a person who has the expertise to solve a specific problem is an important application of recommender systems to a difficult organizational problem. Prior systems have made attempts to implement solutions to this problem, but few systems have undergone systematic user evaluation. This work describes a systematic evaluation of the Expertise Recommender (ER), a system that recommends people who are likely to have expertise in a specific problem. ER and the organizational context for which it was designed are described to provide a basis for understanding this evaluation. Prior to conducting the evaluation, a baseline experiment showed that people are relatively good at judging coworkers' expertise when given an appropriate context. This finding provides a way to demonstrate the effectiveness of ER by comparing ER's performance to ratings by coworkers. The evaluation, the design, and results are described in detail. The results suggest that the participants agree with the recommendations made by ER, and that ER significantly outperforms other expertise recommender systems when compared using similar metrics.

Description

McDonald, David W. (2001): Evaluating Expertise Recommendations. Proceedings of the 2001 ACM International Conference on Supporting Group Work. DOI: 10.1145/500286.500319. New York, NY, USA: Association for Computing Machinery. pp. 214–223. Boulder, Colorado, USA

Keywords

expertise location, computer-supported cooperative work, CSCW, user evaluation, recommendation systems

Citation

DOI

10.1145/500286.500319

URI

URI

https://dl.eusset.eu/handle/20.500.12015/4785

Collections

GROUP 2001

Endorsement

Review

Supplemented By

Referenced By

Number of citations to item: 31

  • J. Grudin (2006): Enterprise Knowledge Management and Emerging Technologies, In: Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06), doi:10.1109/hicss.2006.156
  • Mark S. Ackerman, Juri Dachtera, Volkmar Pipek, Volker Wulf (2013): Sharing Knowledge and Expertise: The CSCW View of Knowledge Management, In: Computer Supported Cooperative Work (CSCW) 4-6(22), doi:10.1007/s10606-013-9192-8
  • Romain Robbes, David Rothlisberger (2013): Using developer interaction data to compare expertise metrics, In: 2013 10th Working Conference on Mining Software Repositories (MSR), doi:10.1109/msr.2013.6624041
  • R Hübscher (2010): Assigning Students to Groups Using General and Context-Specific Criteria, In: IEEE Transactions on Learning Technologies 3(3), doi:10.1109/tlt.2010.17
  • Adam Perer, Ido Guy, Erel Uziel, Inbal Ronen, Michal Jacovi (2011): Visual social network analytics for relationship discovery in the enterprise, In: 2011 IEEE Conference on Visual Analytics Science and Technology (VAST), doi:10.1109/vast.2011.6102443
  • Muhammad Aslam, A.M. Martinez Enriquez, D. Decouchant, Imaz G. Escalada (2008): Proximity and Intelligent Assistance for Web Cooperative Production, In: 2008 Second International Symposium on Intelligent Information Technology Application, doi:10.1109/iita.2008.579
  • Sandeep Yadav, Shashank Srivastav (2024): Making Use of Coverage to Assess the Research Paper Recommender System, In: 2024 1st International Conference on Advances in Computing, Communication and Networking (ICAC2N), doi:10.1109/icac2n63387.2024.10895344
  • Guoping Hu, Jingjing Liu, Hang Li, Yunbo Cao, Jian-Yun Nie, Jianfeng Gao (2006): A Supervised Learning Approach to Entity Search, In: Lecture Notes in Computer Science, doi:10.1007/11880592_5
  • A. Perer, I. Guy, E. Uziel, I. Ronen, M. Jacovi (2013): The Longitudinal Use of SaNDVis: Visual Social Network Analytics in the Enterprise, In: IEEE Transactions on Visualization and Computer Graphics 7(19), doi:10.1109/tvcg.2012.322
  • Ali Firdaus, Deris Stiawan, Samsuryadi (2023): Validity of Measurable Expertise on Publication Recognition, In: 2023 10th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), doi:10.1109/eecsi59885.2023.10295653
  • Jeffrey W. Treem, Paul M. Leonardi (2016): Recognizing Expertise, In: Communication Research 2(44), doi:10.1177/0093650215605154
  • David W. McDonald (2003): Recommending collaboration with social networks, In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, doi:10.1145/642611.642714
  • Heiko Spallek, Titus Schleyer, Brian S. Butler (2008): Good Partners are Hard to Find: The Search for and Selection of Collaborators in the Health Sciences, In: 2008 IEEE Fourth International Conference on eScience, doi:10.1109/escience.2008.105
  • Pornpit Wongthongtham, Elizabeth Chang, Ahmed A. Aseeri (2000): Towards Social Network Based Approach for Software Engineering Ontology Sharing and Evolution, In: Lecture Notes in Computer Science, doi:10.1007/978-3-540-76890-6_51
  • Tim Reichling, Michael Veith (2000): Expertise Sharing in a Heterogeneous Organizational Environment, In: ECSCW 2005, doi:10.1007/1-4020-4023-7_17
  • John Anvik, Gail C. Murphy (2007): Determining Implementation Expertise from Bug Reports, In: Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007), doi:10.1109/msr.2007.7
  • Christoph Beckmann, Tom Gross (2011): AGReMo, In: Proceedings of the 29th Annual European Conference on Cognitive Ergonomics, doi:10.1145/2074712.2074747
  • Wayne G. Lutters (2004): Identifying expertise markers in routine work artifacts: exploring micronote taking behavior, In: Computers in Human Behavior 2(20), doi:10.1016/j.chb.2003.10.014
  • A. Becks, T. Reichling, V. Wulf (2003): Supporting collaborative learning by matching human actors, In: 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the, doi:10.1109/hicss.2003.1173695
  • Jonathan Grudin, Steven Poltrock (2012): Taxonomy and Theory in Computer Supported Cooperative Work, In: The Oxford Handbook of Organizational Psychology, Volume 2, doi:10.1093/oxfordhb/9780199928286.013.0040
  • Leysia Palen, Sarah Vieweg, Kenneth Mark Anderson (2011): Supporting “Everyday Analysts” in Safety- and Time-Critical Situations, In: The Information Society 1(27), doi:10.1080/01972243.2011.534370
  • Titus Schleyer, Heiko Spallek, Brian S Butler, Sushmita Subramanian, Daniel Weiss, M Louisa Poythress, Phijarana Rattanathikun, Gregory Mueller (2008): Facebook for Scientists: Requirements and Services for Optimizing How Scientific Collaborations Are Established, In: Journal of Medical Internet Research 3(10), doi:10.2196/jmir.1047
  • Shawn Minto, Gail C. Murphy (2007): Recommending Emergent Teams, In: Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007), doi:10.1109/msr.2007.27
  • John Anvik, Gail C. Murphy (2011): Reducing the effort of bug report triage, In: ACM Transactions on Software Engineering and Methodology 3(20), doi:10.1145/2000791.2000794
  • Omayma Husain, Naomie Salim, Rose Alinda Alias, Samah Abdelsalam, Alzubair Hassan (2019): Expert Finding Systems: A Systematic Review, In: Applied Sciences 20(9), doi:10.3390/app9204250
  • Michael B. Twidale (2005): Over the Shoulder Learning: Supporting Brief Informal Learning, In: Computer Supported Cooperative Work (CSCW) 6(14), doi:10.1007/s10606-005-9007-7
  • Manuela I. Martín-Vicente, Alberto Gil-Solla, Manuel Ramos-Cabrer, Yolanda Blanco-Fernández, Martín López-Nores (2012): Semantic inference of user’s reputation and expertise to improve collaborative recommendations, In: Expert Systems with Applications 9(39), doi:10.1016/j.eswa.2012.01.162
  • Christian Murphy, Swapneel Sheth, Gail Kaiser, Lauren Wilcox (2008): genSpace: Exploring social networking metaphors for knowledge sharing and scientific collaborative work, In: 2008 23rd IEEE/ACM International Conference on Automated Software Engineering - Workshops, doi:10.1109/asew.2008.4686308
  • Yutaro KASHIWA, Masao OHIRA (2020): A Release-Aware Bug Triaging Method Considering Developers' Bug-Fixing Loads, In: IEICE Transactions on Information and Systems 2(E103.D), doi:10.1587/transinf.2019edp7152
  • Meikuan Huang, Joshua Barbour, Chunke Su, Noshir Contractor (2013): Why do group members provide information to digital knowledge repositories? a multilevel application of transactive memory theory, In: Journal of the American Society for Information Science and Technology 3(64), doi:10.1002/asi.22805
  • Md Ahasanuzzaman, Gustavo A. Oliva, Ahmed E. Hassan (2023): Using knowledge units of programming languages to recommend reviewers for pull requests: an empirical study, In: Empirical Software Engineering 1(29), doi:10.1007/s10664-023-10421-9
Please note: Providing information about citations is only possible thanks to to the open metadata APIs provided by crossref.org and opencitations.net. These lists may be incomplete due to unavailable citation data.source: opencitations.net, crossref.org