Testing in the Wild: The Social and Organisational Dimensions of Real World Practice

dc.contributor.authorRooksby, John
dc.contributor.authorRouncefield, Mark
dc.contributor.authorSommerville, Ian
dc.date.accessioned2020-06-06T13:07:31Z
dc.date.available2020-06-06T13:07:31Z
dc.date.issued2009
dc.date.issued2009
dc.description.abstractTesting is a key part of any systems engineering project. There is an extensive literature on testing, but very little that focuses on how testing is carried out in real-world circumstances. This is partly because current practices are often seen as unsophisticated and ineffective. We believe that by investigating and characterising the real-world work of testing we can help question why such ‘bad practices’ occur and how improvements might be made. We also argue that the testing literature is too focused on technological issues when many of the problems, and indeed strengths, have as much do with work and organisation. In this paper we use empirical examples from four systems engineering projects to demonstrate how and in what ways testing is a cooperative activity. In particular we demonstrate the ways in which testing is situated within organisational work and satisfices organisational and marketplace demands.de
dc.identifier.doi10.1007/s10606-009-9098-7
dc.identifier.pissn1573-7551
dc.identifier.urihttp://dx.doi.org/10.1007/s10606-009-9098-7
dc.identifier.urihttps://dl.eusset.eu/handle/20.500.12015/3995
dc.publisherSpringer
dc.relation.ispartofComputer Supported Cooperative Work (CSCW): Vol. 18
dc.relation.ispartofseriesComputer Supported Cooperative Work (CSCW)
dc.subjectdependability
dc.subjectethnography
dc.subjectethnomethodology
dc.subjectorganisational issues
dc.subjectsoftware development
dc.subjectsystems testing
dc.subjectwork practices
dc.titleTesting in the Wild: The Social and Organisational Dimensions of Real World Practicede
dc.typeText/Journal Article
gi.citation.endPage580
gi.citation.startPage559
gi.citations.count62
gi.citations.elementKristian Wiklund, Sigrid Eldh, Daniel Sundmark, Kristina Lundqvist (2017): Impediments for software test automation: A systematic literature review, In: Software Testing, Verification and Reliability 8(27), doi:10.1002/stvr.1639
gi.citations.elementNils Wild, Horst Lichter, Peter Kehren (2020): Test Automation Challenges for Application Landscape Frameworks, In: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), doi:10.1109/icstw50294.2020.00059
gi.citations.elementDusica Marijan (2012): A Review of Two Experiences from Applying Model Based Testing in Practice, In: 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops, doi:10.1109/issrew.2012.78
gi.citations.elementEmelie Engström, Per Runeson (2010): A Qualitative Survey of Regression Testing Practices, In: Lecture Notes in Computer Science, doi:10.1007/978-3-642-13792-1_3
gi.citations.elementRaphael Pham, Stephan Kiesling, Olga Liskin, Leif Singer, Kurt Schneider (2014): Enablers, inhibitors, and perceptions of testing in novice software teams, In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, doi:10.1145/2635868.2635925
gi.citations.elementHina Shah, Mary Jean Harrold, Nancy J. Nersessian (2011): Studying the Influence of Culture on Outsourced, Offshored Software-Testing Practice: An Ethnographic Approach, In: 2011 IEEE Sixth International Conference on Global Software Engineering Workshop, doi:10.1109/icgse-w.2011.27
gi.citations.elementPer Runeson, Emelie Engström (2012): Regression Testing in Software Product Line Engineering, In: Advances in Computers, doi:10.1016/b978-0-12-396535-6.00007-7
gi.citations.elementMujtaba Hassan, Muzammil Hussain, Salah-u-din Ayubi, Maham Irfan (2019): A Policy Recommendations Framework To Resolve Global Software Development Issues, In: 2019 International Conference on Innovative Computing (ICIC), doi:10.1109/icic48496.2019.8966719
gi.citations.elementTanjila Kanij, Robert Merkel, John Grundy (2015): An Empirical Investigation of Personality Traits of Software Testers, In: 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering, doi:10.1109/chase.2015.7
gi.citations.elementMoritz Beller, Georgios Gousios, Annibale Panichella, Andy Zaidman (2015): When, how, and why developers (do not) test in their IDEs, In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, doi:10.1145/2786805.2786843
gi.citations.elementJuha Itkonen, Mika V. Mantyla, Casper Lassenius (2013): The Role of the Tester's Knowledge in Exploratory Software Testing, In: IEEE Transactions on Software Engineering 5(39), doi:10.1109/tse.2012.55
gi.citations.elementPradeep Waychal, Luiz Fernando Capretz, Jingdong Jia, Daniel Varona, Yadira Lizama (2021): Practitioners’ Testimonials about Software Testing, In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), doi:10.1109/saner50967.2021.00070
gi.citations.elementAnca Deak (2014): What Characterizes a Good Software Tester? – A Survey in Four Norwegian Companies, In: Lecture Notes in Computer Science, doi:10.1007/978-3-662-44857-1_11
gi.citations.elementJulia Prior (2013): A sense of working there, In: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, doi:10.1145/2541016.2541089
gi.citations.elementLiane Colonna (2023): The AI Act’s Research Exemption: A Mechanism for Regulatory Arbitrage?, In: YSEC Yearbook of Socio-Economic Constitutions, doi:10.1007/16495_2023_59
gi.citations.elementKristian Wiklund, Monika Wiklund (2018): The Next Level of Test Automation: What About the Users?, In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), doi:10.1109/icstw.2018.00045
gi.citations.elementBruce R. Maxim, Marouane Kessentini (2016): An introduction to modern software quality assurance, In: Software Quality Assurance, doi:10.1016/b978-0-12-802301-3.00002-8
gi.citations.elementJalal Mahmud, Allen Cypher, Eben Haber, Tessa Lau (2012): Design and industrial evaluation of a tool supporting semi-automated website testing, In: Software Testing, Verification and Reliability 1(24), doi:10.1002/stvr.1484
gi.citations.elementTabata Pérez Rentería y Hernández, Nicola Marsden (2021): Offshore Software Testing in the Automotive Industry, In: Research Anthology on Recent Trends, Tools, and Implications of Computer Programming, doi:10.4018/978-1-7998-3016-0.ch070
gi.citations.elementAshley D Radomski, Lori Wozney, Patrick McGrath, Anna Huguet, Lisa Hartling, Michele P Dyson, Kathryn Bennett, Amanda S Newton (2019): Design and Delivery Features That May Improve the Use of Internet-Based Cognitive Behavioral Therapy for Children and Adolescents With Anxiety: A Realist Literature Synthesis With a Persuasive Systems Design Perspective, In: Journal of Medical Internet Research 2(21), doi:10.2196/11128
gi.citations.elementLuiz Fernando Capretz, Saad Harous, Ali Bou Nassif (2022): What UAE Software Students Think About Software Testing: A Replicated Study, In: Advances in Intelligent Systems and Computing, doi:10.1007/978-3-031-16684-6_7
gi.citations.elementHina Shah, Mary Jean Harrold, Saurabh Sinha (2014): Global software testing under deadline pressure: Vendor-side experiences, In: Information and Software Technology 1(56), doi:10.1016/j.infsof.2013.04.005
gi.citations.elementHina Shah, Saurabh Sinha, Mary Jean Harrold (2011): Outsourced, Offshored Software-Testing Practice: Vendor-Side Experiences, In: 2011 IEEE Sixth International Conference on Global Software Engineering, doi:10.1109/icgse.2011.32
gi.citations.elementDegif Teka, Yvonne Dittrich, Mesfin Kifle, Carmelo Ardito, Rosa Lanzilotti (2017): User Involvement and Usability Evaluation in Ethiopian Software Organizations, In: THE ELECTRONIC JOURNAL OF INFORMATION SYSTEMS IN DEVELOPING COUNTRIES 1(83), doi:10.1002/j.1681-4835.2017.tb00616.x
gi.citations.elementTabata Pérez Rentería y Hernández, Nicola Marsden (2017): Offshore Software Testing in the Automotive Industry, In: International Journal of Information Technology Project Management 4(8), doi:10.4018/ijitpm.2017100101
gi.citations.elementClaire Hill, Tessa Reardon, Lucy Taylor, Cathy Creswell (2022): Online Support and Intervention for Child Anxiety (OSI): Development and Usability Testing, In: JMIR Formative Research 4(6), doi:10.2196/29846
gi.citations.elementMoritz Beller, Georgios Gousios, Annibale Panichella, Sebastian Proksch, Sven Amann, Andy Zaidman (2019): Developer Testing in the IDE: Patterns, Beliefs, and Behavior, In: IEEE Transactions on Software Engineering 3(45), doi:10.1109/tse.2017.2776152
gi.citations.elementTone Bratteteig, Ina Wagner (2016): Unpacking the Notion of Participation in Participatory Design, In: Computer Supported Cooperative Work (CSCW) 6(25), doi:10.1007/s10606-016-9259-4
gi.citations.elementAnca Deak, Tor Stalhane (2013): Organization of Testing Activities in Norwegian Software Companies, In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, doi:10.1109/icstw.2013.18
gi.citations.elementIlkka Tervonen, Antti Haapalahti, Lasse Harjumaa, Jouni Simila (2013): Outsourcing Software Testing: A Case Study in the Oulu Area, In: 2013 13th International Conference on Quality Software, doi:10.1109/qsic.2013.53
gi.citations.elementFabio Massacci (2010): Infringo ergo sum, In: Proceedings of the FSE/SDP workshop on Future of software engineering research, doi:10.1145/1882362.1882411
gi.citations.elementVitalis Ndume, Yaw Nkansah-Gyekye, Jesuk Ko (2014): A web service approach for integrating e-Health Data in a low bandwidth environment, In: Proceedings of the 2nd Pan African International Conference on Science, Computing and Telecommunications (PACT 2014), doi:10.1109/scat.2014.7055133
gi.citations.elementMika V. Mäntylä, Juha Itkonen (2014): How are software defects found? The role of implicit defect detection, individual responsibility, documents, and knowledge, In: Information and Software Technology 12(56), doi:10.1016/j.infsof.2013.12.005
gi.citations.elementSimone Romano, Davide Fucci, Giuseppe Scanniello, Burak Turhan, Natalia Juristo (2017): Findings from a multi-method study on test-driven development, In: Information and Software Technology, doi:10.1016/j.infsof.2017.03.010
gi.citations.elementDave Cliff, Linda Northrop (2012): The Global Financial Markets: An Ultra-Large-Scale Systems Perspective, In: Lecture Notes in Computer Science, doi:10.1007/978-3-642-34059-8_2
gi.citations.elementYadira Lizama, Daniel Varona, Pradeep Waychal, Luiz Fernando Capretz (2019): The Unpopularity of the Software Tester Role Among Software Practitioners: A Case Study, In: Springer Series in Reliability Engineering, doi:10.1007/978-3-030-36518-9_7
gi.citations.elementClaire Hill, Cathy Creswell, Sarah Vigerland, Maaike H. Nauta, Sonja March, Caroline Donovan, Lidewij Wolters, Susan H. Spence, Jennifer L. Martin, Lori Wozney, Lauren McLellan, Leonie Kreuze, Karen Gould, Maral Jolstedt, Martina Nord, Jennifer L. Hudson, Elisabeth Utens, Jeroen Ruwaard, Casper Albers, Muniya Khanna, Anne Marie Albano, Eva Serlachius, Stefan Hrastinski, Philip C. Kendall (2018): Navigating the development and dissemination of internet cognitive behavioral therapy (iCBT) for anxiety disorders in children and young people: A consensus statement with recommendations from the #iCBTLorentz Workshop Group, In: Internet Interventions, doi:10.1016/j.invent.2018.02.002
gi.citations.elementEdward Meade, Emma O’Keeffe, Niall Lyons, Dean Lynch, Murat Yilmaz, Ulas Gulec, Rory V. O’Connor, Paul M. Clarke (2019): The Changing Role of the Software Engineer, In: Communications in Computer and Information Science, doi:10.1007/978-3-030-28005-5_53
gi.citations.elementEric Monteiro, Neil Pollock, Ole Hanseth, Robin Williams (2012): From Artefacts to Infrastructures, In: Computer Supported Cooperative Work (CSCW) 4-6(22), doi:10.1007/s10606-012-9167-1
gi.citations.elementLouise Petersen Matjeka, Dag Svanæs, Alf Inge Wang (2022): Turning Eight Family Homes Into Interactive, Pervasive Playgrounds During the COVID-19 Pandemic Lockdown, In: Frontiers in Computer Science, doi:10.3389/fcomp.2022.822337
gi.citations.elementAnca Deak (2012): Understanding Socio-technical Factors Influencing Testers in Software Development Organizations, In: 2012 IEEE 36th Annual Computer Software and Applications Conference, doi:10.1109/compsac.2012.103
gi.citations.elementFredrik Asplund (2019): Exploratory testing: Do contextual factors influence software fault identification?, In: Information and Software Technology, doi:10.1016/j.infsof.2018.11.003
gi.citations.elementSamir Passi, Phoebe Sengers (2020): Making data science systems work, In: Big Data & Society 2(7), doi:10.1177/2053951720939605
gi.citations.elementXin Xia, David Lo, Pavneet Singh Kochhar, Zhenchang Xing, Xinyu Wang, Shanping Li (2015): Experience report: An industrial experience report on test outsourcing practices, In: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), doi:10.1109/issre.2015.7381830
gi.citations.elementHelen Sharp, Yvonne Dittrich, Cleidson R. B. de Souza (2016): The Role of Ethnographic Studies in Empirical Software Engineering, In: IEEE Transactions on Software Engineering 8(42), doi:10.1109/tse.2016.2519887
gi.citations.elementNasir Mehmood Minhas, Kai Petersen, Jürgen Börstler, Krzysztof Wnuk (2020): Regression testing for large-scale embedded software development – Exploring the state of practice, In: Information and Software Technology, doi:10.1016/j.infsof.2019.106254
gi.citations.elementAnca Deak, Tor Stålhane, Guttorm Sindre (2016): Challenges and strategies for motivating software testing personnel, In: Information and Software Technology, doi:10.1016/j.infsof.2016.01.002
gi.citations.elementHina Shah, Mary Jean Harrold (2013): Culture and Testing: What is the Relationship?, In: 2013 IEEE 8th International Conference on Global Software Engineering, doi:10.1109/icgse.2013.15
gi.citations.elementClaire Hill, Tessa Reardon, Lucy Taylor, Cathy Creswell (2021): Online Support and Intervention for Child Anxiety (OSI): Development and Usability Testing (Preprint), doi:10.2196/preprints.29846
gi.citations.elementRonnie Edson de Souza Santos, Cleyton Vanut Cordeiro de Magalhaes, Jorge da Silva Correia-Neto, Fabio Queda Bueno da Silva, Luiz Fernando Capretz, Rodrigo Souza (2017): Would You Like to Motivate Software Testers? Ask Them How, In: 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), doi:10.1109/esem.2017.16
gi.citations.elementHina Shah, Mary Jean Harrold (2010): Studying human and social aspects of testing in a service-based software company, In: Proceedings of the 2010 ICSE Workshop on Cooperative and Human Aspects of Software Engineering, doi:10.1145/1833310.1833327
gi.citations.elementJosh Tenenberg, Wolff-Michael Roth, David Socha (2015): From I-Awareness to We-Awareness in CSCW, In: Computer Supported Cooperative Work (CSCW) 4-5(25), doi:10.1007/s10606-014-9215-0
gi.citations.elementMuhammad Shah Jahan, Muhammad Talha Riaz, Kashif, Muhammad Abbas (2019): Software Testing Practices in IT Industry of Pakistan, In: Proceedings of the 6th Conference on the Engineering of Computer Based Systems, doi:10.1145/3352700.3352724
gi.citations.elementJohn Fowler, Mark Zachry (2024): VizCare: Using UCD Techniques to Impact the Child Welfare System, In: 2024 IEEE International Professional Communication Conference (ProComm), doi:10.1109/procomm61427.2024.00029
gi.citations.elementAnca Deak (2013): Understanding the Influence of Social and Technical Factors Testers in Software Organizations, In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, doi:10.1109/icst.2013.76
gi.citations.elementLiane Colonna, Stockholm Law Submitter (2023): The AI Act’s Research Exemption: A Mechanism for Regulatory Arbitrage?, In: SSRN Electronic Journal, doi:10.2139/ssrn.4575971
gi.citations.elementStefan Mohacsi, Rudolf Ramler (2018): Why Software Testing Fails: Common Pitfalls Observed in a Critical Smart Metering Project, In: Lecture Notes in Business Information Processing, doi:10.1007/978-3-030-05767-1_6
gi.citations.elementNilam Kaushik, Mazeiar Salehie, Ladan Tahvildari, Sen Li, Mark Moore (2011): Dynamic Prioritization in Regression Testing, In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, doi:10.1109/icstw.2011.68
gi.citations.elementTanjila Kanij, Robert Merkel, John Grundy (2013): An empirical study of the effects of personality on software testing, In: 2013 26th International Conference on Software Engineering Education and Training (CSEE&T), doi:10.1109/cseet.2013.6595255
gi.citations.elementMatt Osborne, Richard Hawkins, Mark Nicholson, Rob Alexander (2024): Understanding safety engineering practice: Comparing safety engineering practice as desired, as required, and as observed, In: Safety Science, doi:10.1016/j.ssci.2024.106424
gi.citations.elementMika V. Mäntylä, Juha Itkonen, Joonas Iivonen (2011): Who tested my software? Testing as an organizationally cross-cutting activity, In: Software Quality Journal 1(20), doi:10.1007/s11219-011-9157-4
gi.citations.elementGötz Hoeppe (2021): Encoding Collective Knowledge, Instructing Data Reusers: The Collaborative Fixation of a Digital Scientific Data Set, In: Computer Supported Cooperative Work (CSCW) 4(30), doi:10.1007/s10606-021-09407-2

Files