Authors with most documents
Browse
Recent Submissions
- Conference PaperThe Problem of Majority Voting in Crowdsourcing with Binary Classes(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Salminen, Joni; Kamel, Ahmed Mohamed; Jung, Soon-Gyo; Jansen, BernardWhen there are two classes, a majority vote can always be obtained with three labelers. Researchers can utilize this property to obtain a false sense of confidence in their ground truth labels. We demonstrate such a case with 3000 crowdsourced labels for an online hate dataset. Evaluating with percentage agreement, Gwet’s AC1, and Krippendorff’s alpha, results show that using more raters teases out the hidden nuances in raters’ preferences. We show that full agreement among the raters monotonically decreases from three raters (28.4%) to nine raters (19.5%). Ten raters have a higher agreement than any other number of raters, which supports the idea of increasing the number of raters for subjective labeling tasks. Nevertheless, while beneficial, increasing the number of raters cannot be considered as a fundamental solution to the issue of agreement in subjective crowdsourcing tasks, as even with ten raters, there is a non- negligible number of ties (4.11%). We suggest having a small sample of the data labeled by five or more raters to evaluate the stability of agreement among the raters.
- Conference PaperA Survey of Digital Working Conditions of Danish Knowledge Workers(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Nouwens, Midas; Nylandsted Klokmose, ClemensWe present a representative survey of the digital working conditions of 466 Danish knowledge workers. We provide data on 1) the hardware and software they use to accomplish their main job activities, 2) the strategies they use to personalise their software, and 3) their digital competences. Our results show that the average Danish knowledge worker primarily uses a laptop and a smartphone to accomplish their work; they use an average of four software applications, mostly developed by large US corporations; they infrequently personalise their software using built-in settings and rarely personalise using plugins, scripts, or reprogramming; they are most capable in using collaboration and communication tools, feel more comfortable formatting other worker’s digital content than creating their own, and are confident they can solve most technical issues. These results put into question the relevance of the long-standing Personal Computing dream envisioned by HCI pioneers, highlights the tensions between software applications and the digital sovereignty of the European continent, and emphasise the importance of including digital tools in our conceptualisation and regulation of working conditions.
- Conference PaperTowards “Explorable” AI: Learning from ML Developers’ Sensemaking Practices(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Wolf, Christine T.In this note, we report on a qualitative design study in the field of machine learning (ML) and in particular on the sensemaking practices of ML developers as they interact with the interface of a novel adversarial AI method. This paper makes contributions to discourses on interpretable or explainable AI (XAI) systems through an empirical understanding of ML developers’ sensemaking practices. These findings make salient the concept of “explorability” as an alternative design metaphor for interactive AI systems – instead of a focus on explainability or interpretability as fixed qualities of AI systems, explorability focuses on emergent meanings and ways in which they might be enabled or constrained through practice.
- Conference PaperMaking online participatory design work: Understanding the digital ecologies of older adults(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Cerna, Katerina; Müller, ClaudiaParticipatory design (PD) is a meaningful approach to involve older adults into design; however, currently we lack understanding how to do such work online. In our paper, we report from a study where we organized 19 PD workshops online with older adults. We argue that to do so in a meaningful way, a mutually shaped understanding of older adults’ digital ecologies is at the core of organizing such PD processes. We present an empirical account of how digital ecologies of our older participants have become an issue to tackle in the online PD workshops. Further, we provide a solution, a mapping technique, and report from our efforts to evaluate it, that should help to overcome the situation when digital ecologies become a problem in PD online.
- Conference PaperIs a GIF Worth a Thousand Words? Understanding the Use of Dynamic Graphical Illustrations for Procedural Knowledge Sharing on wikiHow(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Zhu, Qingxiaoyang; Wang, Hao-ChuanInformational presentations in Graphics Interchange Format (GIF), have been commonly used to convey emotional, cultural, and non-verbal cues in computer-mediated communication. However, the usage and impact of animated GIFs in the sharing and consumption of procedural knowledge, such as how-to instructions, remains unclear. In this paper, we leverage an online collaborative procedural knowledge-sharing platform – wikiHow to investigate the roles of GIFs in the construction and editing of How-To tutorials and how multimodal tutorials impact learners’ perception and learning from the shared expositions. Through data analytics of archived editing histories, article content and user ratings of wikiHow pages, we found that tutorials consisting of multiple modalities, including animated GIFs and images in addition to text, in general introduced more edits and more textual content compared with text-only articles. When learners learned from these wikiHow tutorials, the tutorials with rich modalities also received higher usefulness evaluation from learners, and accumulated more success stories from the learners in following the tutorials to perform procedural tasks. We discuss the implications for future research on multimodal factors for collaborative procedural knowledge sharing.
- Conference PaperDesigning a Web Application for Simple and Collaborative Video Annotation That Meets Teaching Routines and Educational Requirements(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Klug, Daniel; Schlote, ElkeVideo annotation and analysis is an important activity for teaching with and about audiovisual media artifacts because it helps students to learn how to identify textual and formal connections in media products. But school teachers lack adequate tools for video annotation and analysis in media education that are easy-to-use, integrate into established teaching organization, and support quick collaborative work. To address these challenges, we followed a design-based research approach and conducted qualitative interviews with teachers to develop TRAVIS GO, a web application for simple and collaborative video annotation. TRAVIS GO allows for quick and easy use within established teaching settings. The web application provides basic analytical features in an adaptable work space. Key didactic features include tagging and commenting on posts, sharing and exporting projects, and working in live collaboration. Teachers can create assignments according to grade level, learning subject, and class size. Our work contributes further insights for the CSCW community about how to implement user demands into developing educational tools.
- Conference Paper“Should We Meet IRL?": Gauging Matches in Virtual Reality(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Yeleswarapu, Tejaswini; Nair, Pranav; Rangaswamy, NimmiVirtual Reality has evolved as a powerful, embedded and immersive technology medium to transform dating experiences. However, there is no rigorous CSCW research examining ‘dating’ in VR, despite social interaction being a serious topic of exploration. We aim to push the CSCW discourse on social interaction further by analyzing the dynamics of romantic reciprocality in a fully immersive VR application. Through a qualitative study of 30 participants in 15 pairs, we examine a customizable VR application ‘RecRoom’ as a dating technology medium to analyze how dimensions of interaction - including but not limited to voice, haptics and spatiality - influence dynamics of dating experiences. We employ Tinder as a contrasting chat based medium to situate and deepen our learnings about dating in VR. Our study finds VR allowing users to efficiently and effectively ‘gauge’ matches resulting in well informed decisions to meet (or not) virtual partners ’IRL’ or in real life than existing chat based mediums like Tinder. We believe this leads to improved experience of first dates.
- Conference PaperPlanning for Inclusive Design Workshops: Fostering Collaboration between People with and without Visual Impairment(Proceedings of 19th European Conference on Computer-Supported Cooperative Work, 2021) Bittenbinder, Sven; Pinatti de Carvalho, Aparecido Fabiano; Krapp, Eva; Müller, Claudia; Wulf, VolkerCarrying out successful design workshops can be a challenging task. This can turn even more difficult, if one attempts to engage in more inclusive design workshops, where a broad range of user profiles are covered. If some of these profiles refer to people with impairments, things can get even more complicated. Furthermore, there are also associated challenges when trying to carry out something that is usually implemented as a face-to-face activity in an online format. This exploratory paper introduces a discussion on a few lessons learned from organising design workshops including both people with and without visual impairments. It also outlines our response to the situation created by the COVID-19 pandemic, which prevented us to engage in face-to-face design workshops. Based on feedback received from participants of a first in-person design workshop organised within one of our projects and on informal interviews carried out mainly over the phone to discuss ways to enhance the collaboration between people with and without visual impairments during such activities, we go on to introduce some relevant aspects that should be taken into consideration when planning inclusive design workshops. This is a preliminary contribution, meant to raise discussions on technology-mediated inclusive participatory design initiatives to further inform the development of a solid methodological contribution to CSCW.