Evaluating Expertise Recommendations
Loading...
Fulltext URI
Document type
Additional Information
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Association for Computing Machinery
Abstract
Finding a person who has the expertise to solve a specific problem is an important application of recommender systems to a difficult organizational problem. Prior systems have made attempts to implement solutions to this problem, but few systems have undergone systematic user evaluation. This work describes a systematic evaluation of the Expertise Recommender (ER), a system that recommends people who are likely to have expertise in a specific problem. ER and the organizational context for which it was designed are described to provide a basis for understanding this evaluation. Prior to conducting the evaluation, a baseline experiment showed that people are relatively good at judging coworkers' expertise when given an appropriate context. This finding provides a way to demonstrate the effectiveness of ER by comparing ER's performance to ratings by coworkers. The evaluation, the design, and results are described in detail. The results suggest that the participants agree with the recommendations made by ER, and that ER significantly outperforms other expertise recommender systems when compared using similar metrics.
Description
Keywords
expertise location, computer-supported cooperative work, CSCW, user evaluation, recommendation systems
Citation
URI
Collections
Endorsement
Review
Supplemented By
Referenced By
Number of citations to item: 28
- Christoph Beckmann, Tom Gross (2011): AGReMo, In: Proceedings of the 29th Annual European Conference on Cognitive Ergonomics, doi:10.1145/2074712.2074747
- J. Grudin (2006): Enterprise Knowledge Management and Emerging Technologies, In: Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06), doi:10.1109/hicss.2006.156
- Heiko Spallek, Titus Schleyer, Brian S. Butler (2008): Good Partners are Hard to Find: The Search for and Selection of Collaborators in the Health Sciences, In: 2008 IEEE Fourth International Conference on eScience, doi:10.1109/escience.2008.105
- Yutaro KASHIWA, Masao OHIRA (2020): A Release-Aware Bug Triaging Method Considering Developers' Bug-Fixing Loads, In: IEICE Transactions on Information and Systems 2(E103.D), doi:10.1587/transinf.2019edp7152
- Jeffrey W. Treem, Paul M. Leonardi (2016): Recognizing Expertise, In: Communication Research 2(44), doi:10.1177/0093650215605154
- Omayma Husain, Naomie Salim, Rose Alinda Alias, Samah Abdelsalam, Alzubair Hassan (2019): Expert Finding Systems: A Systematic Review, In: Applied Sciences 20(9), doi:10.3390/app9204250
- Guoping Hu, Jingjing Liu, Hang Li, Yunbo Cao, Jian-Yun Nie, Jianfeng Gao (2006): A Supervised Learning Approach to Entity Search, In: Lecture Notes in Computer Science, doi:10.1007/11880592_5
- David W. McDonald (2003): Recommending collaboration with social networks, In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, doi:10.1145/642611.642714
- Titus Schleyer, Heiko Spallek, Brian S Butler, Sushmita Subramanian, Daniel Weiss, M Louisa Poythress, Phijarana Rattanathikun, Gregory Mueller (2008): Facebook for Scientists: Requirements and Services for Optimizing How Scientific Collaborations Are Established, In: Journal of Medical Internet Research 3(10), doi:10.2196/jmir.1047
- Christian Murphy, Swapneel Sheth, Gail Kaiser, Lauren Wilcox (2008): genSpace: Exploring social networking metaphors for knowledge sharing and scientific collaborative work, In: 2008 23rd IEEE/ACM International Conference on Automated Software Engineering - Workshops, doi:10.1109/asew.2008.4686308
- Leysia Palen, Sarah Vieweg, Kenneth Mark Anderson (2011): Supporting “Everyday Analysts” in Safety- and Time-Critical Situations, In: The Information Society 1(27), doi:10.1080/01972243.2011.534370
- John Anvik, Gail C. Murphy (2011): Reducing the effort of bug report triage, In: ACM Transactions on Software Engineering and Methodology 3(20), doi:10.1145/2000791.2000794
- Manuela I. Martín-Vicente, Alberto Gil-Solla, Manuel Ramos-Cabrer, Yolanda Blanco-Fernández, Martín López-Nores (2012): Semantic inference of user’s reputation and expertise to improve collaborative recommendations, In: Expert Systems with Applications 9(39), doi:10.1016/j.eswa.2012.01.162
- Pornpit Wongthongtham, Elizabeth Chang, Ahmed A. Aseeri (2000): Towards Social Network Based Approach for Software Engineering Ontology Sharing and Evolution, In: Lecture Notes in Computer Science, doi:10.1007/978-3-540-76890-6_51
- Shawn Minto, Gail C. Murphy (2007): Recommending Emergent Teams, In: Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007), doi:10.1109/msr.2007.27
- Jonathan Grudin, Steven Poltrock (2012): Taxonomy and Theory in Computer Supported Cooperative Work, In: The Oxford Handbook of Organizational Psychology, Volume 2, doi:10.1093/oxfordhb/9780199928286.013.0040
- Muhammad Aslam, A.M. Martinez Enriquez, D. Decouchant, Imaz G. Escalada (2008): Proximity and Intelligent Assistance for Web Cooperative Production, In: 2008 Second International Symposium on Intelligent Information Technology Application, doi:10.1109/iita.2008.579
- John Anvik, Gail C. Murphy (2007): Determining Implementation Expertise from Bug Reports, In: Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007), doi:10.1109/msr.2007.7
- Adam Perer, Ido Guy, Erel Uziel, Inbal Ronen, Michal Jacovi (2011): Visual social network analytics for relationship discovery in the enterprise, In: 2011 IEEE Conference on Visual Analytics Science and Technology (VAST), doi:10.1109/vast.2011.6102443
- Michael B. Twidale (2005): Over the Shoulder Learning: Supporting Brief Informal Learning, In: Computer Supported Cooperative Work (CSCW) 6(14), doi:10.1007/s10606-005-9007-7
- Md Ahasanuzzaman, Gustavo A. Oliva, Ahmed E. Hassan (2023): Using knowledge units of programming languages to recommend reviewers for pull requests: an empirical study, In: Empirical Software Engineering 1(29), doi:10.1007/s10664-023-10421-9
- Romain Robbes, David Rothlisberger (2013): Using developer interaction data to compare expertise metrics, In: 2013 10th Working Conference on Mining Software Repositories (MSR), doi:10.1109/msr.2013.6624041
- Meikuan Huang, Joshua Barbour, Chunke Su, Noshir Contractor (2013): Why do group members provide information to digital knowledge repositories? a multilevel application of transactive memory theory, In: Journal of the American Society for Information Science and Technology 3(64), doi:10.1002/asi.22805
- Mark S. Ackerman, Juri Dachtera, Volkmar Pipek, Volker Wulf (2013): Sharing Knowledge and Expertise: The CSCW View of Knowledge Management, In: Computer Supported Cooperative Work (CSCW) 4-6(22), doi:10.1007/s10606-013-9192-8
- A. Becks, T. Reichling, V. Wulf (2003): Supporting collaborative learning by matching human actors, In: 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the, doi:10.1109/hicss.2003.1173695
- Wayne G. Lutters (2004): Identifying expertise markers in routine work artifacts: exploring micronote taking behavior, In: Computers in Human Behavior 2(20), doi:10.1016/j.chb.2003.10.014
- Tim Reichling, Michael Veith (2000): Expertise Sharing in a Heterogeneous Organizational Environment, In: ECSCW 2005, doi:10.1007/1-4020-4023-7_17
- Ali Firdaus, Deris Stiawan, Samsuryadi (2023): Validity of Measurable Expertise on Publication Recognition, In: 2023 10th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), doi:10.1109/eecsi59885.2023.10295653