Metrics for the Evaluation of Team Productivity Factors in Agile Software Development: A Systematic Literature Mapping
Abstract
The productivity of agile teams represents a crucial aspect for software development companies, since it directly influences their performance and efficiency, fundamental aspects for the constant search for improvements. Different studies show that team productivity in agile software development (ASD) is an abstract concept, its meaning is still being studied, and is under construction. On the other hand, its measurement is oriented towards establishing different factors of various types that generate an impact within the team, the organization, and even the market. The objective of this study was to identify the metrics used to evaluate eleven factors classified in the categories of meaning, impact, and flexibility related to team productivity in the field of ASD. This analysis was performed using systematic literature mapping (SLM) directed according to Petersen's protocol. As a result, 37 primary investigations were identified in the IEEE Xplore DL and Scopus bibliographic databases. The findings of this research include the definition of the chosen factors, the determination of productivity metrics for each of them, and an attempt at mathematical formalization for some of these metrics. Furthermore, it was noted that 91.8 % of the studies refer to productivity factors associated with the category of meaning and impact, while there are few studies that detail factors related to the category of flexibility.
References
Pragma. “Agilismo: clave para la transformación digital.” academiapragma.com. Accessed: Jan. 07, 2021. [Online]. Available: https://www.pragma.com.co/academia/conceptos/agilismo-clave-para-la-transformacion-digital
H. C. Beh, Y. Y. Jusoh, R. Abdullah, and S. Hassan, “Dimensions in Measuring Performance of Agile Software Development Projects: A Literature Review,” in 2022 Applied Informatics International Conference (AiIC), Serdang, Malaysia, 2022, pp. 83-87. https://doi.org/10.1109/AiIC54368.2022.9914025
L. M. Restrepo-Tamayo, G. P. Gasca-Hurtado, S. Galeano Ospino, and L. Machuca-Villegas, “Transferencia de conocimiento en equipos de desarrollo de software usando gamificación: un cuasiexperimento,” Ingeniare. Revista chilena de ingeniería, vol. 30. no. 4 pp. 705–718, Dec. 2022. http://dx.doi.org/10.4067/S0718-33052022000400705
B. Binboga, and C. A. Gumussoy, “Factors Affecting Agile Software Project Success,” IEEE Access, Apr. 2024. https://doi.org/10.1109/ACCESS.2024.3384410
B. Portelli, B. Behlendorf, and T. O’Reilly, “The 17th State of Agile Report,” Digital.ai, Raleigh, USA, Rep. 17th. Accessed: Nov. 02, 2020. [Online]. Available: https://stateofagile.com
M. G. S. Ortega, G. B. Soto, R. R. P. Cinco, and L.-F. Rodríguez, “Framework to analyze the relationship between agile software engineers’ emotions and their productivity,” in 2021 10th International Conference on Software Process Improvement (CIMPS), Torreón, Coahuila, Mexico, 2021, pp. 66–69. https://ieeexplore.ieee.org/document/9652780
M. Á. Maldonado Arango, “Factores que afectan la productividad en equipos Scrum analizados con pensamiento sistémico,” M.S. thesis, Universidad Nacional de Colombia, Medellín, Colombia, 2017. [Online]. Available: https://repositorio.unal.edu.co/handle/unal/59778
C. de O. Melo, D. S. Cruzes, F. Kon, and R. Conradi, “Interpretative case studies on agile team productivity and management,” Inf. Softw. Technol., vol. 55, no. 2, pp. 412–427, Feb. 2013. https://doi.org/10.1016/j.infsof.2012.09.004
A. Hernández-López, “Medidas de productividad en los proyectos de desarrollo de software: una aproximación por puestos de trabajo,” Ph.D. thesis, Universidad Carlos III de Madrid, España, 2014. https://dialnet.unirioja.es/servlet/tesis?codigo=8609
C. Stylianou, and A. S. Andreou, “Investigating the Impact of Developer Productivity, Task Interdependence Type and Communication Overhead in a Multi-Objective Optimization Approach for Software Project Planning,” Adv. Eng. Softw., vol. 98, pp. 79–96, Aug. 2016. https://doi.org/10.1016/j.advengsoft.2016.04.001
F. Tavakoli, and T. J. Gandomani, “A Novel Team Productivity Model for XP Teams,” J. Cases Inf. Technol., vol. 20, no. 4, pp. 93–109, 2018. https://doi.org/10.4018/JCIT.2018100106
C. de O. Melo, and F. Kon, “Empirical Evaluation of Agile Practices Impact on Team Productivity BT - Agile Processes in Software Engineering and Extreme Programming,” in Lecture Notes in Business Information Processing, A. Sillitti, O. Hazzan, E. Bache, and X. Albaladejo, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 322–323. https://doi.org/10.1007/978-3-642-20677-1_26
V. A. Bollati, G. Gaona, P. B. Lima, and L. Cuenca Pletsch, “Software Development Teams: Factors Influencing their Productivity,” in 21st LACCEI International Multi-Conference for Engineering, Education, and Technology: Leadership in Education and Innovation in Engineering in the Framework of Global Transformations: Integration and Alliances for Integral Development, Buenos Aires, Argentina, Jul. 2023, Paper 1561. https://doi.org/10.18687/laccei2023.1.1.1561
M. Guerrero-Calvache, and G. Hernández, “Team productivity in agile software development: A systematic mapping study,” in Communications in Computer and Information Science, H. Florez, H. Gomez, Eds., Switzerland, Cham: Springer International Publishing, 2022, pp. 455–471. https://doi.org/10.1007/978-3-031-19647-8_32
M. Guerrero-Calvache, and G. Hernández, “Team Productivity factors in Agile Software Development: An Exploratory Survey with Practitioners BT - Applied Informatics,” in Communications in Computer and Information Science, H. Florez, and M. Leon, Eds., Switzerland, Cham: Springer Nature Switzerland, 2024, pp. 261–276. https://doi.org/10.1007/978-3-031-46813-1_18
M. Turić, S. Čelar, and S. Dragičević, “Productivity Factors in Agile Software Development Projects,” in Proceedings of the 34th DAAAM International Symposium, Vienna, Austria, 2023, pp. 0004–0008. https://doi.org/10.2507/34th.daaam.proceedings.001
I. Fatema, and K. Sakib, “Using Qualitative System Dynamics in the Development of an Agile Teamwork Productivity Model,” International Journal on Advances in Software, vol. 11, no. 12, pp. 170–185, Jul. 2018. https://www.researchgate.net/publication/326412693_Using_Qualitative_System_Dynamics_in_the_Development_of_an_Agile_Teamwork_Productivity_Model
E. D. Canedo, and G. A. Santos, “Factors Affecting Software Development Productivity: An empirical study,” in Proceedings of the XXXIII Brazilian Symposium on Software Engineering, New York, NY, USA: Association for Computing Machinery, 2019, pp. 307–316. https://doi.org/10.1145/3350768.3352491
J. Iqbal, M. Omar, and A. Yasin, “Defining Teamwork Productivity Factors in Agile Software Development,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 12, no. 3, p. 1160, May. 2022. https://doi.org/10.18517/ijaseit.12.3.13648
L. Machuca-Villegas, G. P. Gasca-Hurtado, S. M. Puente, L. Marcela, and R. Tamayo, “Factores sociales y humanos que influyen en la productividad del desarrollo de software: Medición de la percepción,” RISTI: Revista Ibérica de Sistemas e Tecnologias de Informação, no. E41, pp. 488-502, Dec. 2021. https://dialnet.unirioja.es/servlet/articulo?codigo=8610713
L. Machuca-Villegas, and G. P. Gasca-Hurtado, “Towards a Social and Human Factor Classification Related to Productivity in Software Development Teams,” in Trends and Applications in Software Engineering Advances in Intelligent Systems and Computing, J. Mejia, M. Muñoz, Á. Rocha, and A. J. Calvo-Manzano, Eds., Switzerland, Cham: Springer International Publishing, 2020, pp. 36–50. https://doi.org/10.1007/978-3-030-33547-2_4
L. Machuca-Villegas, G. P. Gasca-Hurtado, L. M. Restrepo Tamayo, and S. Morillo Puente, “Social and Human Factor Classification of Influence in Productivity in Software Development Teams,” in Communications in Computer and Information Science, M. Yilmaz, J. Niemann, P. Clarke, R. Messnarz, Eds., Switzerland, Cham: Springer International Publishing, 2020, pp. 717–729. https://doi.org/10.1007/978-3-030-56441-4_54
F. Fagerholm, M. Ikonen, P. Kettunen, J. Münch, V. Roto, and P. Abrahamsson, “How Do Software Developers Experience Team Performance in Lean and Agile Environments?,” in Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, New York, NY, USA: Association for Computing Machinery, 2014, pp. 1-10. https://doi.org/10.1145/2601248.2601285
L. Machuca-Villegas, G. P. Gasca-Hurtado, S. Morillo Puente, and L. M. Restrepo Tamayo, “An Instrument for Measuring Perception about Social and Human Factors that Influence Software Development Productivity,” JUCS - Journal of Universal Computer Science, vol. 27, no. 2, pp. 111–134, Feb. 2021. https://doi.org/10.3897/jucs.65102
K. Petersen, S. Vakkalanka, and L. Kuzniarz “Guidelines for conducting systematic mapping studies in software engineering,” Information and Software Technology, vol. 64, pp. 1–18, Aug. 2015. https://doi.org/10.1016/j.infsof.2015.03.007
H. Klein, and S. Canditt, “Using Opinion Polls to Help Measure Business Impact in Agile Development,” in Proceedings of the 1st International Workshop on Business Impact of Process Improvements (BiPi '08), New York, NY, USA: Association for Computing Machinery, 2008, pp. 25–32. https://doi.org/10.1145/1370837.1370843
E.-M. Ahmed, and A. Sidky, “25 percent Ahead of Schedule and just at “Step 2” of the SAMI,” in 2009 Agile Conference, Chicago, IL, USA, 2009, pp. 162-169. https://doi.org/10.1109/AGILE.2009.63
C. Ertaban, E. Sarikaya, and S. Bagriyanik, “Agile Performance Indicators for Team Performance Evaluation in a Corporate Environment,” in Proceedings of the 19th International Conference on Agile Software Development: Companion, New York, NY, USA: Association for Computing Machinery, 2018, pp. 1-3. https://doi.org/10.1145/3234152.3234156
A. O. Gadelsied, T. M. M. Elhassan, W. M. Mohamed, and H. M. Abushama, “Assessment method for Scrum culture within the Development team,” in 2020 International Conference on Computer, Control, Electrical, and Electronics Engineering (ICCCEEE), Khartoum, Sudan, 2021, pp. 1-6. https://doi.org/10.1109/ICCCEEE49695.2021.9429608
N. Abbas, A. M. Gravell, and G. B. Wills, “The Impact of Organization, Project and Governance Variables on Software Quality and Project Success,” in 2010 Agile Conference, Nashville, TN, USA, 2010, pp. 77-86. https://doi.org/10.1109/AGILE.2010.16
L. Brezocnik, and C. Majer, “Product Evaluation Through Contractor and In-House Metrics,” in Proceedings of the SQAMIA 2016: 5th Workshop of Software Quality, Analysis, Monitoring, Improvement, and Applications, Budapest, Hungary, Aug. 2016. https://ceur-ws.org/Vol-1677/paper02.pdf
Y. Dubinsky, and O. Hazzan, “The role of a project-based capstone course,” in Proceedings. 27th International Conference on Software Engineering, St. Louis, MO, USA, 2005, pp. 645-646. https://doi.org/10.1109/ICSE.2005.1553628
I. Signoretti, S. Marczak, L. Salerno, A. d. Lara, and R. Bastos, "Boosting Agile by Using User-Centered Design and Lean Startup: A Case Study of the Adoption of the Combined Approach in Software Development,” in 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Porto de Galinhas, Brazil, 2019, pp. 1-6. https://doi.org/10.1109/ESEM.2019.8870154
B. Martens, and J. Franke, “Identifying Agile Roles in Software Engineering Projects using Repository and Work-Tracking Data,” in 2022 International Conference on Data and Software Engineering (ICoDSE), Denpasar, Indonesia, 2022, pp. 83-88. https://doi.org/10.1109/ICoDSE56892.2022.9971901
D. Wahyudin, M. Heindl, B. Eckhard, A. Schatten, and S. Biffl, “In-Time Role-Specific Notification as Formal Means to Balance Agile Practices in Global Software Development Settings,” in Balancing Agility and Formalism in Software Engineering, B. Meyer, J. R. Nawrocki, and B. Walter, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 208-222. https://doi.org/10.1007/978-3-540-85279-7_17
S. D. Vishnubhotla, E. Mendes, and L. Lundberg, “Designing a Capability-Centric Web Tool to Support Agile Team Composition and Task Allocation: A Work in Progress,” in 2018 IEEE/ACM 11th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), Gothenburg, Sweden, 2018, pp. 41-44. https://ieeexplore.ieee.org/document/8445534
A. Agarwal, N. K. Garg, and A. Jain, “Quality assurance for Product development using Agile,” in 2014 International Conference on Reliability Optimization and Information Technology (ICROIT), Faridabad, India, 2014, pp. 44-47. https://doi.org/10.1109/ICROIT.2014.6798281
S. Čelar, M. Turić, and L. Vicković, “Method for personal capability assessment in agile teams using personal points,” in 2014 22nd Telecommunications Forum Telfor (TELFOR), Belgrade, Serbia, 2014, pp. 1134-1137. https://doi.org/10.1109/TELFOR.2014.7034607
E. Mendes, D. Viana, S. D. Vishnubhotla, and L. Lundberg, “Realising Individual and Team Capability in Agile Software Development: A Qualitative Investigation,” in 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Prague, Czech Republic, 2018, pp. 183–190. https://doi.org/10.1109/SEAA.2018.00037
S. Downey, and J. Sutherland, “Scrum Metrics for Hyperproductive Teams: How They Fly like Fighter Aircraft,” in 2013 46th Hawaii International Conference on System Sciences, Wailea, HI, USA, 2013, pp. 4870-4878. https://doi.org/10.1109/HICSS.2013.471
E. Scott, K. N. Charkie, and D. Pfahl, “Productivity, Turnover, and Team Stability of Agile Teams in Open-Source Software Projects,” in 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Portoroz, Slovenia, 2020, pp. 124-131. https://doi.org/10.1109/SEAA51224.2020.00029
K. M. Bumbary, “Using Velocity, Acceleration, and Jerk to Manage Agile Schedule Risk,” 2016 International Conference on Information Systems Engineering (ICISE), Los Angeles, CA, USA, 2016, pp. 73-80. https://doi.org/10.1109/ICISE.2016.21
F. Almeida, and P. Carneiro, “Performance metrics in scrum software engineering companies,” International Journal of Agile Systems and Management, vol. 14, no. 2, pp. 205-2023, Sep. 2021. https://doi.org/10.1504/IJASM.2021.10041601
S. Vazifeh-Noshafagh, V. Hajipour, S. Jalali, D. Di Caprio, and F. J. Santos-Arteaga, “Maturing the Scrum Framework for Software Projects Portfolio Management: A Case Study-Oriented Methodology,” IEEE Access, vol. 10, pp. 123283–123300, Nov. 2022. https://doi.org/10.1109/ACCESS.2022.3224447
S. L. Ramírez-Mora, and H. Oktaba, “Team Maturity in Agile Software Development: The Impact on Productivity,” in 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), Madrid, Spain, 2018, pp. 732-736. https://doi.org/10.1109/ICSME.2018.00091
L. Fitzpatrick, “The Role of Emotional Intelligence in Agile Project Management,” in 2022 Portland International Conference on Management of Engineering and Technology (PICMET), Portland, OR, USA, 2022, pp. 1-9. https://doi.org/10.23919/PICMET53225.2022.9882795
A. Alhubaishy, and L. Benedicenti, “Affect and Affective Trust in Agile Requirements Engineering,” in 2018 1st International Workshop on Affective Computing for Requirements Engineering (AffectRE), Banff, AB, Canada, 2018, pp. 9-12. https://doi.org/10.1109/AffectRE.2018.00007
A. Mashmool, S. Khosravi, J. H. Joloudari, I. Inayat, T. J. Gandomani, and A. Mosavi, “A Statistical Model to Assess the Team’s Productivity in Agile Software Teams,” in 2021 IEEE 4th International Conference and Workshop Óbuda on Electrical and Power Engineering (CANDO-EPE), Budapest, Hungary, 2021, pp. 11-18. https://doi.org/10.1109/CANDO-EPE54223.2021.9667902
W. Alsaqaf, M. Daneva, and R. Wieringa, “Agile Quality Requirements Engineering Challenges: First Results from a Case Study” in 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Toronto, ON, Canada, 2017, pp. 454-459. https://doi.org/10.1109/ESEM.2017.61
W. M. Farid, and F. J. Mitropoulos, “NORMATIC: A visual tool for modeling Non-Functional Requirements in agile processes,” in 2012 Proceedings of IEEE Southeastcon, Orlando, FL, USA, 2012, pp. 1-8. https://doi.org/10.1109/SECon.2012.6196989
B. Kumar, U. K. Tiwari, D. C. Dobhal, and H. S. Negi, “User Story Clustering using K-Means Algorithm in Agile Requirement Engineering,” in 2022 International Conference on Computational Intelligence and Sustainable Engineering Solutions (CISES), Greater Noida, India, 2022, pp. 1-5. https://doi.org/10.1109/CISES54857.2022.9844390
M. Oriol et al., “Data-driven and tool-supported elicitation of quality requirements in agile companies,” Software Quality Journal, vol. 28, no. 3, pp. 931–963, Sep. 2020. https://doi.org/10.1007/s11219-020-09509-y
I. Wentzlaff, “Establishing a Requirements Baseline by Functional Size Measurement Patterns,” in CEUR Joint Proceedings of the REFSQ 2017 Co-Located Events: 1st International Workshop on Requirements Prioritization and Enactment, vol. 1796, Feb. 2017. https://ceur-ws.org/Vol-1796/priore-paper-1.pdf
M. Silva et al., “A Comparative Analysis of Agile Teamwork Quality Models,” in 2021 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Hvar, Croatia, 2021, pp. 1-6. https://doi.org/10.23919/SoftCOM52868.2021.9559062
A. Poth, M. Kottke, and A. Riel, “Evaluation of Agile Teamwork Quality,” in Agile Processes in Software Engineering and Extreme Programming -- Workshops, M. Paasivaara and P. Kruchten, Eds., Switzerland, Cham: Springer International Publishing, 2020, pp. 101–110. https://doi.org/10.1007/978-3-030-58858-8_11
A. Poth, M. Kottke, and A. Riel, “Agile Teamwork Quality in the Context of Agile Transformations - A Case Study in Large-Scaling Environments,” in Systems, Software and Services Process Improvement, M. Yilmaz, J. Niemann, P. Clarke, and R. Messnarz, Eds., Switzerland, Cham: Springer International Publishing, 2020, pp. 232–243. https://doi.org/10.1007/978-3-030-56441-4_17
A. Freire, M. Perkusich, R. Saraiva, H. Almeida, and A. Perkusich, “A Bayesian networks-based approach to assess and improve the teamwork quality of agile teams,” Inf Softw Technol., vol. 100, pp. 119–132, Aug. 2018. https://doi.org/10.1016/j.infsof.2018.04.004
M. Silva, A. Freire, M. Perkusich, K. Gorgônio, H. Almeida, and A. Perkusich, “On the Influence of Different Perspectives on Evaluating the Teamwork Quality in the Context of Agile Software Development,” in Proceedings of the XXXIV Brazilian Symposium on Software Engineering, New York, NY, USA: Association for Computing Machinery, 2020, pp. 1-10. https://doi.org/10.1145/3422392.3422397
S. Coyle, and J. Barata, “Socially-constructed metrics for Agile Quality: An Action Research Study,” in Australasian Conference on Information Systems, Wollongong, Australia, 2016. https://aisel.aisnet.org/acis2016/70
C. Matthies, “Agile Process Improvement in Retrospectives,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), Montreal, QC, Canada, 2019, pp. 150-152. https://doi.org/10.1109/ICSE-Companion.2019.00063
F. Kortum, J. Klünder, O. Karras, W. Brunotte, and K. Schneider, “Which Information Help agile Teams the Most? An Experience Report on the Problems and Needs,” in 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Portoroz, Slovenia, 2020, pp. 306-313. https://doi.org/10.1109/SEAA51224.2020.00058
L. Gren, “Learning More from Crossing Levels: Investigating Agility at Three Levels of the Organization,” in 2017 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 2017, pp. 1035-1038. https://doi.org/10.1109/CSCI.2017.179
E. A. Arteaga Castillo, “Factores y estrategias basadas en software libre que propician el aprendizaje organizacional en el área de tecnología de la Universidad de Nariño,” M.S. tesis, Universidad Autónoma de Bucaramanga, 2018. [Online]. Available: https://repository.unab.edu.co/handle/20.500.12749/3428
Downloads
Copyright (c) 2024 TecnoLógicas
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Altmetric
Funding data
