Simulations at Work —a Framework for Configuring Simulation Fidelity with Training Objectives Abstract This study aims to provide framework for considering fidelity in the design of simulator training. Simulator fidelity is often characterised as the level of physical and visual similarity with real work settings, and the importance of simulator fidelity in the creation of learning activities has been extensively debated. Based on a selected literature review and fieldwork on ship simulator training, this study provides a conceptual framework for fidelity requirements in simulator training. This framework is applied to an empirical example from a case of ship simulator training. The study identifies three types of simulator fidelity that might be useful from a trainer’s perspective. By introducing a framework of technical, psychological and interactional fidelity and linking these concepts to different levels of training and targeted learning outcomes, the study demonstrates how the fidelity of the simulation relates to the level of expertise targeted in training. The framework adds to the body of knowledge on simulator training by providing guidelines for the different ways in which simulators can increase professional expertise, without separating the learning activity from cooperative work performance.

Crowd Dynamics: Conflicts, Contradictions, and Community in Crowdsourcing

Rating Working Conditions on Digital Labor Platforms Abstract The relations between technology, work organization, worker power, workers’ rights, and workers’ experience of work have long been central concerns of CSCW. European CSCW research, especially, has a tradition of close collaboration with workers and trade unionists in which researchers aim to develop technologies and work processes that increase workplace democracy. This paper contributes a practitioner perspective on this theme in a new context: the (sometimes global) labor markets enabled by digital labor platforms. Specifically, the paper describes a method for rating working conditions on digital labor platforms (e.g., Amazon Mechanical Turk, Uber) developed within a trade union setting. Preliminary results have been made public on a website that is referred to by workers, platform operators, journalists, researchers, and policy makers. This paper describes this technical project in the context of broader cross-sectoral efforts to safeguard worker rights and build worker power in digital labor platforms. Not a traditional research paper, this article instead takes the form of a case study documenting the process of incorporating a human-centered computing perspective into contemporary trade union activities and communicating a practitioner’s perspective on how CSCW research and computational artifacts can come to matter outside of the academy. The paper shows how practical applications can benefit from the work of CSCW researchers, while illustrating some practical constraints of the trade union context. The paper also offers some practical contributions for researchers studying digital platform workers’ experiences and rights: the artifacts and processes developed in the course of the work.

The Evolution of Power and Standard Wikidata Editors: Comparing Editing Behavior over Time to Predict Lifespan and Volume of Edits Abstract Knowledge bases are becoming a key asset leveraged for various types of applications on the Web, from search engines presenting ‘entity cards’ as the result of a query, to the use of structured data of knowledge bases to empower virtual personal assistants. Wikidata is an open general-interest knowledge base that is collaboratively developed and maintained by a community of thousands of volunteers. One of the major challenges faced in such a crowdsourcing project is to attain a high level of editor engagement. In order to intervene and encourage editors to be more committed to editing Wikidata, it is important to be able to predict at an early stage, whether an editor will or not become an engaged editor. In this paper, we investigate this problem and study the evolution that editors with different levels of engagement exhibit in their editing behaviour over time. We measure an editor’s engagement in terms of (i) the volume of edits provided by the editor and (ii) their lifespan (i.e. the length of time for which an editor is present at Wikidata). The large-scale longitudinal data analysis that we perform covers Wikidata edits over almost 4 years. We monitor evolution in a session-by-session- and monthly-basis, observing the way the participation, the volume and the diversity of edits done by Wikidata editors change. Using the findings in our exploratory analysis, we define and implement prediction models that use the multiple evolution indicators.

Examining Community Dynamics of Civic Crowdfunding Participation Abstract Over the past decade, crowdfunding has emerged as a legitimate, albeit niche, resource for public service delivery. Predicated on utilizing the resources of the crowd to address public issues, civic crowdfunding has the potential to offer citizens a greater role in service delivery and community development. This study investigates community dynamics and their potential impact on project success in jurisdictions proposing civic crowdfunding proposals. The results highlight the dynamics and characteristics of communities where project proposals are likely to find funding success. The results further highlight several potential opportunities for future research to better understand how and why these projects truly work.

Investigating the Amazon Mechanical Turk Market Through Tool Design Abstract We developed TurkBench to better understand the work of crowdworkers on the Amazon Mechanical Turk (AMT) marketplace. While we aimed to reduce the amount of invisible, unpaid work that these crowdworkers performed, we also probed the day-to-day practices of crowdworkers. Through this probe we encountered a number of previously unreported difficulties that are representative of the difficulties that crowdworkers face in both building their own tools and working on AMT. In this article, our contributions are insights into 1) a number of breakdowns that are occurring on AMT and 2) how the AMT platform is being appropriated in ways that, at the same time, mitigate some breakdowns while exacerbating others. The breakdowns that we specifically discuss in this paper, are the increasing velocity of the market (good HITs are grabbed within seconds), the high amount of flexibility that requesters can and do exercise in specifying their HITs, and the difficulty crowdworkers had in navigating the market due to the large amount of variation in how HITs were constructed by requesters. When the velocity of the market is combined with a poor search interface, a large amount in variation in how HITs are constructed, and little infrastructural support for workers, the resulting work environment can be frustrating and difficult to thrive in.

Citizen Representation in City Government-Driven Crowdsourcing Abstract This article examines the citizen representativeness of crowdsourcing achieved through 311 systems—the non-emergency and quality of life service request reporting systems used by local governments. Based on surveys of San Francisco residents conducted in 2011, 2013, and 2015, our findings suggest that no systematic biases exist in participation rates across a range of socio-economic indicators. In addition, the findings provide evidence that participation may be responding positively to the city’s responsiveness, thus creating a self-reinforcing process that benefits an increasingly diverse and representative body of users. This inquiry builds on earlier studies of Boston and San Francisco that show that 311 systems did not bias response to traditionally disadvantaged groups (lower socioeconomic status or racial/ethnic minorities) at the demand level nor from high-volume users.

Crowd Anatomy Beyond the Good and Bad: Behavioral Traces for Crowd Worker Modeling and Pre-selection Abstract The suitability of crowdsourcing to solve a variety of problems has been investigated widely. Yet, there is still a lack of understanding about the distinct behavior and performance of workers within microtasks. In this paper, we first introduce a fine-grained data-driven worker typology based on different dimensions and derived from behavioral traces of workers. Next, we propose and evaluate novel models of crowd worker behavior and show the benefits of behavior-based worker pre-selection using machine learning models. We also study the effect of task complexity on worker behavior. Finally, we evaluate our novel typology-based worker pre-selection method in image transcription and information finding tasks involving crowd workers completing 1,800 HITs. Our proposed method for worker pre-selection leads to a higher quality of results when compared to the standard practice of using qualification or pre-screening tests. For image transcription tasks our method resulted in an accuracy increase of nearly 7% over the baseline and of almost 10% in information finding tasks, without a significant difference in task completion time. Our findings have important implications for crowdsourcing systems where a worker’s behavioral type is unknown prior to participation in a task. We highlight the potential of leveraging worker types to identify and aid those workers who require further training to improve their performance. Having proposed a powerful automated mechanism to detect worker types, we reflect on promoting fairness, trust and transparency in microtask crowdsourcing platforms.

Capitalizing Relationships: Modes of Participation in Crowdsourcing Abstract While crowds online are increasingly used for data gathering and problem solving, the relationships and structures within these processes remain largely unexamined. For understanding the usage of crowdsourcing and to design appropriate technologies and processes, it is important to understand how different tools support relationships in these contexts. Based on an extensive literature review of existing crowdsourcing tools and practices, we contribute with the development of a typology of alienation in crowdsourcing by using Marx’s theory of alienation. The theory serves as a lens to compare and contrast a number of currently available tools for crowdsourcing, focusing on how relationships between participants are supported and capitalized within the tool. We show how different types of crowdsourcing practices can be described in terms of alienation where the producer, the producers, the consumers, and products are connected in different modes of participation. This systematical application of Marx theory of alienation provides a way to compare the technical support for social relationships in a number of platforms used for crowdsourcing.

Infrastructuring and Participatory Design: Exploring Infrastructural Inversion as Analytic, Empirical and Generative Abstract The participatory design of CSCW systems increasingly embraces activities of reconfiguring the use of existing interconnected systems in addition to developing and implementing new. In this article, we refer to such activities of changing and improving collaboration through the means of existing information infrastructures as infrastructuring. We investigate a relational perspective on infrastructuring and provide an overview and a detailed account of a local infrastructuring process by tracing the concrete relations that emerged. The elusive quality of information infrastructures as being invisible is analyzed through the notion of infrastructural inversion. Infrastructural inversion is the gestalt switch of shifting attention from the activities invisibly supported by an infrastructure to the activities that enable the infrastructure to function and meet desired needs for collaborative support. Initially, infrastructural inversion was conceived as a conceptual-analytic notion, but recent research has also positioned it as an empirical-ethnographic and generative-designerly resource. In this study, we rely on all of these stances and contribute to the generative-designerly position. We explain the notion of infrastructural inversion and describe how it is distinct from the CSCW concept of articulation work. The context of the analysis includes a participatory design project that sought to reduce patients’ fasting time prior to surgical operations by improving the interdepartmental coordination at a hospital. The project revealed the webs of relations and interdependencies in which fasting time is inscribed at the local level as well as regionally, nationally, and beyond. We pursue the relations, trace their connectedness across multiple scopes, and show how the process alternated between empirical and analytic activities of exploring relations and design-oriented activities of reaching closure. Our analysis shows that the notion of infrastructural inversion can enrich participatory design: Infrastructural inversion embraces the exploratory activities of tracing relations, while the design agenda drove the need for reaching closure. We conclude by discussing lessons learned for infrastructuring and for participatory design that engages with infrastructuring.