Browsing by Subject "crowdsourcing"

1 - 14 of 14
  • No Thumbnail Available
    Conference Paper
    Centralized Crowdsourcing in Disaster Management: Findings and Implications
    (Proceedings of the 8th International Conference on Communities and Technologies, 2017) Auferbauer, D.; Tellioğlu, Hilda; Lewkowicz, Myriam; Rohde, Markus; Mulder, Ingrid; Schuler, Douglas
    Computer supported cooperative work (CSCW) has become an important aspect in crisis and disaster management. Volunteers undertaking relief efforts in affected areas are increasingly using information and communication technologies to coordinate their work. Relief organizations are recognizing this trend and have started to adapt new communication channels to interact with citizens. In this paper, we describe the crowdtasking approach, a centralized form of crowdsourcing for crisis and disaster management. We present a prototype implementation of the approach and report on our findings from the system's first field trial. We conclude by discussing implications of this approach for CSCW and community building in crisis and disaster management. Lastly, we give an outlook on future research based on our experience with crowdtasking.
  • No Thumbnail Available
    Conference Paper
    Enabling Uneven Task Difficulty in Micro-Task Crowdsourcing
    (Proceedings of the 2018 ACM International Conference on Supporting Group Work, 2018) Jiang, Yu; Sun, Yuling; Yang, Jing; Lin, Xin; He, Liang
    In micro-task crowdsourcing markets such as Amazon's Mechanical Turk, how to obtain high quality result without exceeding the limited budgets is one main challenge. The existing theory and practice of crowdsourcing suggests that uneven task difficulty plays a crucial role to task quality. Yet, it lacks a clear identifying method to task difficulty, which hinders effective and efficient execution of micro-task crowdsourcing. This paper explores the notion of task difficulty and its influence to crowdsourcing, and presents a difficulty-based crowdsourcing method to optimize the crowdsourcing process. We firstly identify task difficulty feature based on a local estimation method in the real crowdsourcing context, followed by proposing an optimization method to improve the accuracy of results, while reducing the overall cost. We conduct a series of experimental studies to evaluate our method, which show that our difficulty-based crowdsourcing method can accurately identify the task difficulty feature, improve the quality of task performance and reduce the cost significantly, and thus demonstrate the effectiveness of task difficulty as task modeling property.
  • No Thumbnail Available
    Journal Article
    Good Enough is Good Enough: Overcoming Disaster Response Organizations’ Slow Social Media Data Adoption
    (Computer Supported Cooperative Work (CSCW): Vol. 23, No. 4-6, 41974) Tapia, Andrea H.; Moore, Kathleen
    Organizations that respond to disasters hold unreasonable standards for data arising from technology-enabled citizen contributions. This has strong negative potential for the ability of these responding organizations to incorporate these data into appropriate decision points. We argue that the landscape of the use of social media data in crisis response is varied, with pockets of use and acceptance among organizations. In this paper we present findings from interviews conducted with representatives from large international disaster response organizations concerning their use of social media data in crisis response. We found that emergency responders already operate with less than reliable, or “good enough,” information in offline practice, and that social media data are useful to responders, but only in specific crisis situations. Also, responders do use social media, but only within their known community and extended network. This shows that trust first begins with people and not data. Lastly, we demonstrate the barriers used by responding organizations have gone beyond discussions of trustworthiness and data quality to that of more operational issues.
  • No Thumbnail Available
    Journal Article
    Journalists as Crowdsourcerers: Responding to Crisis by Reporting with a Crowd
    (Computer Supported Cooperative Work (CSCW): Vol. 23, No. 4-6, 41974) Dailey, Dharma; Starbird, Kate
    Widespread adoption of new information communication technologies (ICTs) is disrupting traditional models of news production and distribution. In this rapidly changing media landscape, the role of the journalist is evolving. Our research examines how professional journalists within a rural community impacted by Hurricane Irene successfully negotiated a new role for themselves, transforming their journalistic practice to serve in a new capacity as leaders of an online volunteer community. We describe an emergent organization of media professionals, citizen journalists, online volunteers, and collaborating journalistic institutions that provided real-time event coverage. In this rural context, where communications infrastructure is relatively uneven, this ad hoc effort bridged gaps in ICT infrastructure to unite its audience. In this paper, we introduce a new perspective for characterizing these information-sharing activities: the “human powered mesh network” extends the concept of a mesh network to include human actors in the movement of information. Our analysis shows how journalists played a key role in this network, and facilitated the movement of information to those who needed it. These findings also note a contrast between how HCI researchers are designing crowdsourcing platforms for news production and how crowdsourcing efforts are forming during disaster events, suggesting an alternative approach to designing for emergent collaborations in this context.
  • No Thumbnail Available
    Text Document
    Location-Based Crowdsourcing of Hyperlocal News: Dimensions of Participation Preferences
    (Proceedings of the 2012 ACM International Conference on Supporting Group Work, 2012) Väätäjä, Heli; Vainio, Teija; Sirkkunen, Esa
    We studied the mobile users' preferences and concerns of using location-based assignments (LBA) and geotagging in crowdsourced news making. First, nine readers who had submitted reader's photos were interviewed about their perceptions of LBA and geotagging scenarios. Second, a quasi-experiment in field conditions was carried out with nineteen participants. After completing four LBA tasks with a mobile phone, participants were interviewed on their perceptions and asked to complete a questionnaire on their preferences for receiving LBA and usage of geotags. Findings indicate that the perceived benefits of LBA and geotagging are greater than the perceived risks. The task type, temporal context, preciseness of location query, proximity to the reporting location, parallel tasks, social context and incentives affected the participation preferences. We propose a framework for participation preferences to support further studies in location-based crowdsourcing and in the development of crowdsourcing processes and systems.
  • No Thumbnail Available
    Conference Paper
    The Morphing Organization: Rethinking Groupwork Systems in the Era of Crowdwork
    (Proceedings of the 2014 ACM International Conference on Supporting Group Work, 2014) Anya, Obinna; Carletti, Laura; Coughlan, Tim; Hansson, Karin; Liu, Sophia B.
    Web 2.0 has provided organizations remarkable opportunities to improve productivity, gain competitive advantage, and increase participation by engaging a crowd to accomplish tasks at scale. However, establishing and integrating crowd-based systems into organizations is still an open question. The systems and the collaborative processes they enable appear diametrically in dissonance with the norms and culture of collaboration and knowledge sharing in traditional organizations. They require mechanisms for articulation of work, coordination, cooperation, and knowledge co-creation that are fundamentally different from those in current groupwork systems and processes. Building on two workshops hosted at ACM CSCW 2014, we will explore questions such as: How does the shift in organizational work from a closed system with known individuals, to an open and crowd model that requires engagement with an undefined network of people, affect how we conceptualize groupwork? What are the implications for the design of groupwork systems? What can the crowdsourcing research community learn from groupwork systems, or conversely what can groupwork researchers learn from crowdsourcing? How do cultures, motivations, ownership and representation fit into these systems? This workshop will bring together researchers and practitioners in crowdsourcing, social computing, collaborative technologies, organizational science, and workplace research, to discuss the future of groupwork systems in the era of crowdwork with the goal of articulating an agenda for future research.
  • No Thumbnail Available
    Conference Paper
    Pay-per-Question: Towards Targeted Q&A with Payments
    (Proceedings of the 2018 ACM International Conference on Supporting Group Work, 2018) Jan, Steve T.K.; Wang, Chun; Zhang, Qing; Wang, Gang
    Online question and answer (Q&A) services are facing key challenges to motivate domain experts to provide quick and high-quality answers. Recent systems seek to engage real-world experts by allowing them to set a price on their answers. This leads to a targeted" Q&A model where users to ask questions to a target expert by paying the price. In this paper, we perform a case study on two emerging targeted Q&A systems Fenda (China) and Whale (US) to understand how monetary incentives affect user behavior. By analyzing a large dataset of 220K questions (worth 1 million USD), we find that payments indeed enable quick answers from experts, but also drive certain users to game the system for profits. In addition, this model requires users (experts) to proactively adjust their price to make profits. People who are unwilling to lower their prices are likely to hurt their income and engagement over time.
  • No Thumbnail Available
    Journal Article
    Rating Working Conditions on Digital Labor Platforms
    (Computer Supported Cooperative Work (CSCW): Vol. 28, No. 5, 2019) Harmon, Ellie; Silberman, M. Six
    The relations between technology, work organization, worker power, workers’ rights, and workers’ experience of work have long been central concerns of CSCW. European CSCW research, especially, has a tradition of close collaboration with workers and trade unionists in which researchers aim to develop technologies and work processes that increase workplace democracy. This paper contributes a practitioner perspective on this theme in a new context: the (sometimes global) labor markets enabled by digital labor platforms. Specifically, the paper describes a method for rating working conditions on digital labor platforms (e.g., Amazon Mechanical Turk, Uber) developed within a trade union setting. Preliminary results have been made public on a website that is referred to by workers, platform operators, journalists, researchers, and policy makers. This paper describes this technical project in the context of broader cross-sectoral efforts to safeguard worker rights and build worker power in digital labor platforms. Not a traditional research paper, this article instead takes the form of a case study documenting the process of incorporating a human-centered computing perspective into contemporary trade union activities and communicating a practitioner’s perspective on how CSCW research and computational artifacts can come to matter outside of the academy. The paper shows how practical applications can benefit from the work of CSCW researchers, while illustrating some practical constraints of the trade union context. The paper also offers some practical contributions for researchers studying digital platform workers’ experiences and rights: the artifacts and processes developed in the course of the work.
  • No Thumbnail Available
    Text Document
    Readers' Motivations to Participate in Hyperlocal News Content Creation
    (Proceedings of the 2012 ACM International Conference on Supporting Group Work, 2012) Väätäjä, Heli
    Readers are increasingly participating to news content creation by submitting user-generated content (UGC). We studied the participation motivations of active readers who send photo content to a hyperlocal news publisher. The first results based on an online questionnaire indicate that fun, the opportunity to get a monetary reward and informing others of local issues are the strongest motivators. In addition, participation to the news making activity and self-expression are important motivations. Those who intentionally planned and searched for topics to report with photos, reported more often the opportunity for extra income and development as a photographer as participation motivations than those, who captured photos when a good topic came about.
  • No Thumbnail Available
    Conference Paper
    Supporting High-Stakes Investigations with Expert-Led Crowdsourcing
    (Companion Proceedings of the 2023 ACM International Conference on Supporting Group Work, 2023) Venkatagiri, Sukrit
    My dissertation introduces the concept of expert-led crowdsourcing (ELC) to support expert investigators who increasingly face limits on their time and attention. ELC combines experts’ domain knowledge and experience with the speed and scale of crowds. I study ELC in two investigative domains: journalism and law enforcement. Through four studies, I show: 1) how novice crowds can effectively augment expert investigators’ work practice; 2) the ethical tensions in conducting an ELC investigation for real-world, sensitive investigations; 3) how capture-the-flag competitions increase inter-team collaboration in ELC investigations; and 4) how different teamwork structures affect intra-team collaboration in ELC investigations.
  • No Thumbnail Available
    Conference Paper
    Turk-Life in India
    (Proceedings of the 2014 ACM International Conference on Supporting Group Work, 2014) Gupta, Neha; Martin, David; Hanrahan, Benjamin V.; O'Neill, Jacki
    Previous studies on Amazon Mechanical Turk (AMT), the most well-known marketplace for microtasks, show that the largest population of workers on AMT is U.S. based, while the second largest is based in India. In this paper, we present insights from an ethnographic study conducted in India to introduce some of these workers or Turkers" -- who they are, how they work and what turking means to them. We examine the work they do to maintain their reputations and their work-life balance. In doing this, we illustrate how AMT's design practically impacts on turk-work. Understanding the "lived work" of crowdwork is a valuable first step for technology design."
  • No Thumbnail Available
    Journal Article
    Turking in a Global Labour Market
    (Computer Supported Cooperative Work (CSCW): Vol. 25, No. 1, 42401) Martin, David; O’Neill, Jacki; Gupta, Neha; Hanrahan, Benjamin V.
    This paper examines how working in the global labour market of Amazon Mechanical Turk (AMT) impacts upon and is understood by two different groups of workers. To do this we report on two qualitative studies; one of US and another of Indian crowdworkers (or ‘Turkers’) that we analysed from an ethnomethodological orientation. Our data is naturalistic and comes from a variety of sources—interviews, observations and forum posts—where Turkers describe their work, and their understandings of that work and of the transnational market they work in. We compare and contrast their situations, their reflections on the marketplace and its global reach, and we take a look at their understandings of one another. Our focus is on ‘the work to making turking work’ (Martin et al. 2014 ). That is, the work that turkers do to organise and make sense of their work as they operate in the AMT marketplace, such that they can do so effectively. AMT is a technologically mediated marketplace—that is the distribution, completion and payment of work is done online, almost completely through the AMT platform. The design of the platform has important consequences for how Turkers experience and understand the market (including its global or transnational nature). We discuss how our findings relate to a variety of CSCW issues and provide an initial examination of how they relate to globalisation both as a mundanely experienced phenomenon and as a topic of academic interest. We finish the article by drawing on our own experiences in research and design to look at how technology can be used to intervene in a market like this to try to address imbalances in power and agency between employers and workers.
  • No Thumbnail Available
    Conference Paper
    Understanding and Designing Sociotechnical Systems to Support the Impression Management Practices of Online Freelance Workers
    (Companion Proceedings of the 2020 ACM International Conference on Supporting Group Work, 2020) Foong, Eureka
    A growing number of freelancers worldwide are involved in online, project-based knowledge work. Compared to employees in organizations, freelancers face serious challenges securing work and mitigate this by constructing favorable impressions on peers and clients. Sociotechnical systems present new opportunities and challenges to support impression management and two factors that influence it: 1) audience understanding and 2) gender role constraints. The goal of my dissertation is to understand and design sociotechnical systems that support freelancers' impression management. First, I will study and design online feedback exchange (OFE) systems that can help freelancers better understand their audiences by providing feedback on projects and portfolios. Second, I will investigate how gender role constraints influence freelancers' pricing behavior in online labor marketplaces. My research will contribute a novel OFE system for improving the quality of freelancers' portfolios and knowledge of gender differences in freelancers' pricing behavior to guide the design of sociotechnical systems that better support this emerging workforce.
  • No Thumbnail Available
    Text Document
    Workflow Transparency in a Microtask Marketplace
    (Proceedings of the 2012 ACM International Conference on Supporting Group Work, 2012) Kinnaird, Peter; Dabbish, Laura; Kiesler, Sara
    Interdependent tasks in Mechanical Turk (MTurk) can be managed efficiently with a workflow, a sequence of tasks through which work passes to its completion. We ask if workers should be informed about the workflow, which we call workflow transparency. Transparency could motivate workers or induce social loafing. We describe three experiments to determine the effects of workflow transparency in MTurk. We compared a text description of the workflow, a visualization of the workflow, and the combination of text and visualization with a control condition giving no workflow information. Workflow transparency marginally increased volunteerism on a charity identification task (experiment 1) and significantly increased volunteerism and quality on a business identification task (experiment 2). Results were weaker with a less experienced worker sample (experiment 3). We suggest further research on the design of workflow information to increase workers' motivation.