Testing in the Wild: The Social and Organisational Dimensions of Real World Practice
Loading...
Fulltext URI
Document type
Text/Journal Article
Additional Information
Journal Title
Journal ISSN
Volume Title
Publisher
Springer
Abstract
Testing is a key part of any systems engineering project. There is an extensive literature on testing, but very little that focuses on how testing is carried out in real-world circumstances. This is partly because current practices are often seen as unsophisticated and ineffective. We believe that by investigating and characterising the real-world work of testing we can help question why such ‘bad practices’ occur and how improvements might be made. We also argue that the testing literature is too focused on technological issues when many of the problems, and indeed strengths, have as much do with work and organisation. In this paper we use empirical examples from four systems engineering projects to demonstrate how and in what ways testing is a cooperative activity. In particular we demonstrate the ways in which testing is situated within organisational work and satisfices organisational and marketplace demands.
Description
Keywords
dependability, ethnography, ethnomethodology, organisational issues, software development, systems testing, work practices
Citation
URI
Collections
Endorsement
Review
Supplemented By
Referenced By
Number of citations to item: 62
- Kristian Wiklund, Sigrid Eldh, Daniel Sundmark, Kristina Lundqvist (2017): Impediments for software test automation: A systematic literature review, In: Software Testing, Verification and Reliability 8(27), doi:10.1002/stvr.1639
- Nils Wild, Horst Lichter, Peter Kehren (2020): Test Automation Challenges for Application Landscape Frameworks, In: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), doi:10.1109/icstw50294.2020.00059
- Dusica Marijan (2012): A Review of Two Experiences from Applying Model Based Testing in Practice, In: 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops, doi:10.1109/issrew.2012.78
- Emelie Engström, Per Runeson (2010): A Qualitative Survey of Regression Testing Practices, In: Lecture Notes in Computer Science, doi:10.1007/978-3-642-13792-1_3
- Raphael Pham, Stephan Kiesling, Olga Liskin, Leif Singer, Kurt Schneider (2014): Enablers, inhibitors, and perceptions of testing in novice software teams, In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, doi:10.1145/2635868.2635925
- Hina Shah, Mary Jean Harrold, Nancy J. Nersessian (2011): Studying the Influence of Culture on Outsourced, Offshored Software-Testing Practice: An Ethnographic Approach, In: 2011 IEEE Sixth International Conference on Global Software Engineering Workshop, doi:10.1109/icgse-w.2011.27
- Per Runeson, Emelie Engström (2012): Regression Testing in Software Product Line Engineering, In: Advances in Computers, doi:10.1016/b978-0-12-396535-6.00007-7
- Mujtaba Hassan, Muzammil Hussain, Salah-u-din Ayubi, Maham Irfan (2019): A Policy Recommendations Framework To Resolve Global Software Development Issues, In: 2019 International Conference on Innovative Computing (ICIC), doi:10.1109/icic48496.2019.8966719
- Tanjila Kanij, Robert Merkel, John Grundy (2015): An Empirical Investigation of Personality Traits of Software Testers, In: 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering, doi:10.1109/chase.2015.7
- Moritz Beller, Georgios Gousios, Annibale Panichella, Andy Zaidman (2015): When, how, and why developers (do not) test in their IDEs, In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, doi:10.1145/2786805.2786843
- Juha Itkonen, Mika V. Mantyla, Casper Lassenius (2013): The Role of the Tester's Knowledge in Exploratory Software Testing, In: IEEE Transactions on Software Engineering 5(39), doi:10.1109/tse.2012.55
- Pradeep Waychal, Luiz Fernando Capretz, Jingdong Jia, Daniel Varona, Yadira Lizama (2021): Practitioners’ Testimonials about Software Testing, In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), doi:10.1109/saner50967.2021.00070
- Anca Deak (2014): What Characterizes a Good Software Tester? – A Survey in Four Norwegian Companies, In: Lecture Notes in Computer Science, doi:10.1007/978-3-662-44857-1_11
- Julia Prior (2013): A sense of working there, In: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, doi:10.1145/2541016.2541089
- Liane Colonna (2023): The AI Act’s Research Exemption: A Mechanism for Regulatory Arbitrage?, In: YSEC Yearbook of Socio-Economic Constitutions, doi:10.1007/16495_2023_59
- Kristian Wiklund, Monika Wiklund (2018): The Next Level of Test Automation: What About the Users?, In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), doi:10.1109/icstw.2018.00045
- Bruce R. Maxim, Marouane Kessentini (2016): An introduction to modern software quality assurance, In: Software Quality Assurance, doi:10.1016/b978-0-12-802301-3.00002-8
- Jalal Mahmud, Allen Cypher, Eben Haber, Tessa Lau (2012): Design and industrial evaluation of a tool supporting semi-automated website testing, In: Software Testing, Verification and Reliability 1(24), doi:10.1002/stvr.1484
- Tabata Pérez Rentería y Hernández, Nicola Marsden (2021): Offshore Software Testing in the Automotive Industry, In: Research Anthology on Recent Trends, Tools, and Implications of Computer Programming, doi:10.4018/978-1-7998-3016-0.ch070
- Ashley D Radomski, Lori Wozney, Patrick McGrath, Anna Huguet, Lisa Hartling, Michele P Dyson, Kathryn Bennett, Amanda S Newton (2019): Design and Delivery Features That May Improve the Use of Internet-Based Cognitive Behavioral Therapy for Children and Adolescents With Anxiety: A Realist Literature Synthesis With a Persuasive Systems Design Perspective, In: Journal of Medical Internet Research 2(21), doi:10.2196/11128
- Luiz Fernando Capretz, Saad Harous, Ali Bou Nassif (2022): What UAE Software Students Think About Software Testing: A Replicated Study, In: Advances in Intelligent Systems and Computing, doi:10.1007/978-3-031-16684-6_7
- Hina Shah, Mary Jean Harrold, Saurabh Sinha (2014): Global software testing under deadline pressure: Vendor-side experiences, In: Information and Software Technology 1(56), doi:10.1016/j.infsof.2013.04.005
- Hina Shah, Saurabh Sinha, Mary Jean Harrold (2011): Outsourced, Offshored Software-Testing Practice: Vendor-Side Experiences, In: 2011 IEEE Sixth International Conference on Global Software Engineering, doi:10.1109/icgse.2011.32
- Degif Teka, Yvonne Dittrich, Mesfin Kifle, Carmelo Ardito, Rosa Lanzilotti (2017): User Involvement and Usability Evaluation in Ethiopian Software Organizations, In: THE ELECTRONIC JOURNAL OF INFORMATION SYSTEMS IN DEVELOPING COUNTRIES 1(83), doi:10.1002/j.1681-4835.2017.tb00616.x
- Tabata Pérez Rentería y Hernández, Nicola Marsden (2017): Offshore Software Testing in the Automotive Industry, In: International Journal of Information Technology Project Management 4(8), doi:10.4018/ijitpm.2017100101
- Claire Hill, Tessa Reardon, Lucy Taylor, Cathy Creswell (2022): Online Support and Intervention for Child Anxiety (OSI): Development and Usability Testing, In: JMIR Formative Research 4(6), doi:10.2196/29846
- Moritz Beller, Georgios Gousios, Annibale Panichella, Sebastian Proksch, Sven Amann, Andy Zaidman (2019): Developer Testing in the IDE: Patterns, Beliefs, and Behavior, In: IEEE Transactions on Software Engineering 3(45), doi:10.1109/tse.2017.2776152
- Tone Bratteteig, Ina Wagner (2016): Unpacking the Notion of Participation in Participatory Design, In: Computer Supported Cooperative Work (CSCW) 6(25), doi:10.1007/s10606-016-9259-4
- Anca Deak, Tor Stalhane (2013): Organization of Testing Activities in Norwegian Software Companies, In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, doi:10.1109/icstw.2013.18
- Ilkka Tervonen, Antti Haapalahti, Lasse Harjumaa, Jouni Simila (2013): Outsourcing Software Testing: A Case Study in the Oulu Area, In: 2013 13th International Conference on Quality Software, doi:10.1109/qsic.2013.53
- Fabio Massacci (2010): Infringo ergo sum, In: Proceedings of the FSE/SDP workshop on Future of software engineering research, doi:10.1145/1882362.1882411
- Vitalis Ndume, Yaw Nkansah-Gyekye, Jesuk Ko (2014): A web service approach for integrating e-Health Data in a low bandwidth environment, In: Proceedings of the 2nd Pan African International Conference on Science, Computing and Telecommunications (PACT 2014), doi:10.1109/scat.2014.7055133
- Mika V. Mäntylä, Juha Itkonen (2014): How are software defects found? The role of implicit defect detection, individual responsibility, documents, and knowledge, In: Information and Software Technology 12(56), doi:10.1016/j.infsof.2013.12.005
- Simone Romano, Davide Fucci, Giuseppe Scanniello, Burak Turhan, Natalia Juristo (2017): Findings from a multi-method study on test-driven development, In: Information and Software Technology, doi:10.1016/j.infsof.2017.03.010
- Dave Cliff, Linda Northrop (2012): The Global Financial Markets: An Ultra-Large-Scale Systems Perspective, In: Lecture Notes in Computer Science, doi:10.1007/978-3-642-34059-8_2
- Yadira Lizama, Daniel Varona, Pradeep Waychal, Luiz Fernando Capretz (2019): The Unpopularity of the Software Tester Role Among Software Practitioners: A Case Study, In: Springer Series in Reliability Engineering, doi:10.1007/978-3-030-36518-9_7
- Claire Hill, Cathy Creswell, Sarah Vigerland, Maaike H. Nauta, Sonja March, Caroline Donovan, Lidewij Wolters, Susan H. Spence, Jennifer L. Martin, Lori Wozney, Lauren McLellan, Leonie Kreuze, Karen Gould, Maral Jolstedt, Martina Nord, Jennifer L. Hudson, Elisabeth Utens, Jeroen Ruwaard, Casper Albers, Muniya Khanna, Anne Marie Albano, Eva Serlachius, Stefan Hrastinski, Philip C. Kendall (2018): Navigating the development and dissemination of internet cognitive behavioral therapy (iCBT) for anxiety disorders in children and young people: A consensus statement with recommendations from the #iCBTLorentz Workshop Group, In: Internet Interventions, doi:10.1016/j.invent.2018.02.002
- Edward Meade, Emma O’Keeffe, Niall Lyons, Dean Lynch, Murat Yilmaz, Ulas Gulec, Rory V. O’Connor, Paul M. Clarke (2019): The Changing Role of the Software Engineer, In: Communications in Computer and Information Science, doi:10.1007/978-3-030-28005-5_53
- Eric Monteiro, Neil Pollock, Ole Hanseth, Robin Williams (2012): From Artefacts to Infrastructures, In: Computer Supported Cooperative Work (CSCW) 4-6(22), doi:10.1007/s10606-012-9167-1
- Louise Petersen Matjeka, Dag Svanæs, Alf Inge Wang (2022): Turning Eight Family Homes Into Interactive, Pervasive Playgrounds During the COVID-19 Pandemic Lockdown, In: Frontiers in Computer Science, doi:10.3389/fcomp.2022.822337
- Anca Deak (2012): Understanding Socio-technical Factors Influencing Testers in Software Development Organizations, In: 2012 IEEE 36th Annual Computer Software and Applications Conference, doi:10.1109/compsac.2012.103
- Fredrik Asplund (2019): Exploratory testing: Do contextual factors influence software fault identification?, In: Information and Software Technology, doi:10.1016/j.infsof.2018.11.003
- Samir Passi, Phoebe Sengers (2020): Making data science systems work, In: Big Data & Society 2(7), doi:10.1177/2053951720939605
- Xin Xia, David Lo, Pavneet Singh Kochhar, Zhenchang Xing, Xinyu Wang, Shanping Li (2015): Experience report: An industrial experience report on test outsourcing practices, In: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), doi:10.1109/issre.2015.7381830
- Helen Sharp, Yvonne Dittrich, Cleidson R. B. de Souza (2016): The Role of Ethnographic Studies in Empirical Software Engineering, In: IEEE Transactions on Software Engineering 8(42), doi:10.1109/tse.2016.2519887
- Nasir Mehmood Minhas, Kai Petersen, Jürgen Börstler, Krzysztof Wnuk (2020): Regression testing for large-scale embedded software development – Exploring the state of practice, In: Information and Software Technology, doi:10.1016/j.infsof.2019.106254
- Anca Deak, Tor Stålhane, Guttorm Sindre (2016): Challenges and strategies for motivating software testing personnel, In: Information and Software Technology, doi:10.1016/j.infsof.2016.01.002
- Hina Shah, Mary Jean Harrold (2013): Culture and Testing: What is the Relationship?, In: 2013 IEEE 8th International Conference on Global Software Engineering, doi:10.1109/icgse.2013.15
- Claire Hill, Tessa Reardon, Lucy Taylor, Cathy Creswell (2021): Online Support and Intervention for Child Anxiety (OSI): Development and Usability Testing (Preprint), doi:10.2196/preprints.29846
- Ronnie Edson de Souza Santos, Cleyton Vanut Cordeiro de Magalhaes, Jorge da Silva Correia-Neto, Fabio Queda Bueno da Silva, Luiz Fernando Capretz, Rodrigo Souza (2017): Would You Like to Motivate Software Testers? Ask Them How, In: 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), doi:10.1109/esem.2017.16
- Hina Shah, Mary Jean Harrold (2010): Studying human and social aspects of testing in a service-based software company, In: Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering, doi:10.1145/1833310.1833327
- Josh Tenenberg, Wolff-Michael Roth, David Socha (2015): From I-Awareness to We-Awareness in CSCW, In: Computer Supported Cooperative Work (CSCW) 4-5(25), doi:10.1007/s10606-014-9215-0
- Muhammad Shah Jahan, Muhammad Talha Riaz, Kashif, Muhammad Abbas (2019): Software Testing Practices in IT Industry of Pakistan, In: Proceedings of the 6th Conference on the Engineering of Computer Based Systems, doi:10.1145/3352700.3352724
- John Fowler, Mark Zachry (2024): VizCare: Using UCD Techniques to Impact the Child Welfare System, In: 2024 IEEE International Professional Communication Conference (ProComm), doi:10.1109/procomm61427.2024.00029
- Anca Deak (2013): Understanding the Influence of Social and Technical Factors Testers in Software Organizations, In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, doi:10.1109/icst.2013.76
- Liane Colonna, Stockholm Law Submitter (2023): The AI Act’s Research Exemption: A Mechanism for Regulatory Arbitrage?, In: SSRN Electronic Journal, doi:10.2139/ssrn.4575971
- Stefan Mohacsi, Rudolf Ramler (2018): Why Software Testing Fails: Common Pitfalls Observed in a Critical Smart Metering Project, In: Lecture Notes in Business Information Processing, doi:10.1007/978-3-030-05767-1_6
- Nilam Kaushik, Mazeiar Salehie, Ladan Tahvildari, Sen Li, Mark Moore (2011): Dynamic Prioritization in Regression Testing, In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, doi:10.1109/icstw.2011.68
- Tanjila Kanij, Robert Merkel, John Grundy (2013): An empirical study of the effects of personality on software testing, In: 2013 26th International Conference on Software Engineering Education and Training (CSEE&T), doi:10.1109/cseet.2013.6595255
- Matt Osborne, Richard Hawkins, Mark Nicholson, Rob Alexander (2024): Understanding safety engineering practice: Comparing safety engineering practice as desired, as required, and as observed, In: Safety Science, doi:10.1016/j.ssci.2024.106424
- Mika V. Mäntylä, Juha Itkonen, Joonas Iivonen (2011): Who tested my software? Testing as an organizationally cross-cutting activity, In: Software Quality Journal 1(20), doi:10.1007/s11219-011-9157-4
- Götz Hoeppe (2021): Encoding Collective Knowledge, Instructing Data Reusers: The Collaborative Fixation of a Digital Scientific Data Set, In: Computer Supported Cooperative Work (CSCW) 4(30), doi:10.1007/s10606-021-09407-2