Adequacy Checking of Personal Software Development Effort Estimation Models Based upon Fuzzy Logic: A Replicated Experiment

Comprobación de la Adecuación de Modelos de Estimación del Esfuerzo de Desarrollo de Software Personal Basados en Lógica Difusa: Un Experimento Replicado

Cuauhtémoc López Martín¹, Cornelio Yáñez Márquez², Agustín Gutiérrez Tornés³ and Edgardo Felipe Riverón⁴

^1,2,4 Center for Computing Research, National Polytechnic Institute; P.O. 07738, México, D.F., E–mails: cuauhtemoc@sagitario.cic.ipn.mx, cyanez@cic.ipn.mx, edgardo@cic.ipn.mx

³ Systems Coordinator, Banamex, México, D.F.; ITESM, México, D.F., E–mail: agustin.tornes@itesm.mx

Article received on March 09, 2006
Accepted on July 10, 2007

Abstract

There are two main stages for using an estimation model (1) it must be determined whether the model is adequate to describe the observed (actual) data, that is, the model adequacy checking or verification; if it resulted adequate then (2) the estimation model is validated in its environment using new data. This paper is related to the first step. An investigation aimed to compare personal Fuzzy Logic Systems (FLS) with linear regression is presented. These FLS are derived from a replicated experiment using a sample integrated by ten developers. This experiment is based on both a common process and inside of a controlled environment. In six of ten cases the multiple range tests for Magnitude of Relative Error (MRE) by technique show that fuzzy logic is slightly better than linear regression. These results show that a FLS could be use as an alternative for the software development effort estimation at personal level.

Keywords: Software development effort estimation, Fuzzy logic, Linear Regression, Magnitude of Relative Error.

Resumen

Existen dos fases principales en el uso de un modelo de estimación: (1) se debe determinar si el modelo es adecuado para describir los datos observados (reales), eso es, la comprobación de la adecuación del modelo o verificación del mismo; si éste resultara adecuado, entonces (2) el modelo de estimación se valida en su ambiente usando datos nuevos. Este artículo está relacionado con la primera etapa. Se presenta una investigación dirigida a la comparación de Sistemas de Lógica Difusa (SLD) personales. Estos SLD se derivan a partir de un experimento replicado con base en una muestra de diez desarrolladores, así como en un proceso de desarrollo común dentro de un entorno controlado. En seis de los diez casos, las pruebas de rango múltiple de la Magnitud del Error Relativo (MER) por técnica, muestran que la lógica difusa es ligeramente mejor que la regresión simple. Estos resultados muestran que un SLD podría ser utilizado como alternativa para la estimación del esfuerzo de desarrollo de software a nivel personal.

Palabras clave: Estimación del esfuerzo de desarrollo de software, Lógica difusa, Regresión lineal, Magnitud del error relativo.

DESCARGAR ARTÍCULO EN FORMATO PDF

Acknowledgements

We would like to thank Center for Computing Research, National Polytechnic Institute, Mexico as well as CONACYT. Moreover, to Development Team which is working for Federal Commission of Electricity at Guadalajara, Jalisco, México as well as the bachelor students of Computational Systems Engineering of the University del Valle de Atemajac (UNIVA), Guadalajara.

References

1. Ahmed M. A, Saliu M. O. and AlGhamdi J., "Adaptive fuzzy logic–based framework for software development effort prediction", Information and Software Technology, Vol. 47, No. 1, 2005, pp. 31–48.        [ Links ]

2. Boehm B. W. and Fairley R. E., "Software Estimation Perspectives", IEEE Software November 2000, pp 22–26.        [ Links ]

3. Boehm B., Abts Ch. and Chulani S., "Software Development Cost Estimation Approaches – A Survey". Chulani Ph. D. Report 1998.        [ Links ]

4. Boehm B., Software Engineering Economics, Prentice Hall, 1981.        [ Links ]

5. Boehm B. COCOMO II. Prentice Hall. 2000.        [ Links ]

6. Braz, M. and Vergilio S., "Using Fuzzy Theory for Effort Estimation of Object–Oriented Software", Proceedings of the 16^th IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2004.        [ Links ]

7. Briand L.C., Langley T. and Wieczorek L, "A replicated Assessment and Comparison of Common Software Cost Modeling Techniques", IEEE International Conference on Software Engineering (ICSE), 2000, Limerick, Ireland.        [ Links ]

8. Briand L.C., Emam K.E., Surmann D. and Wieczorek I., "An Assessment and Comparison of Common Software Cost Estimation Modeling Techniques". ISERN–1998–27.        [ Links ]

9. Brooks F. P. Jr., "Three Great Challenges for Half–Century–Old Computer Science", Journal of the ACM, Vol. 50, No. 1, January 2003, pp. 25–26.        [ Links ]

10. Burguess C. J. and Lefley M. Can genetic programming improve software effort estimation? A comparative evaluation. Information and Software Technology. Elsevier. 2001.        [ Links ]

11. Gray A. R. and MacDonell S. G., "Applications of Fuzzy Logic to Software Metric Models for Development Effort Estimation", Proceedings of NAFIPS 1997.        [ Links ]

12. Höst M and Wohlin C., "A subjective effort estimation experiment", Information and Software Technology, Elsevier, 1997.        [ Links ]

13. Huang X., Ren J. and Capretz L.F., "A Neuro–Fuzzy Tool for Software Estimation", Proceedings of the 20th IEEE International Conference on Software Maintenance, 2004.        [ Links ]

14. Humphrey W. The Personal Software Process, Technical Report CMU/SEI–2000–022.        [ Links ]

15. Humphrey W. A Discipline for Software Engineering, Addison Wesley, 1995.        [ Links ]

16. Idri, A., Abran, A. and Khoshgoftaar T. M., "Computational Intelligence in Empirical Software Engineering", First USA–Morocco Workshop on Information Technology, ENSIAS, Rabat, 2003.        [ Links ]

17. Idri A., Khoshgoftaar T. M., Abran A. Can Neural Networks be Easily Interpreted in Software Cost Estimation. World Congress on Computational Intelligence. Hawai. 2002a.        [ Links ]

18. Idri, A., Abran, A. and Khoshgoftaar T., "Estimating Software Project Effort by Analogy Based on Linguistic Values", Proceedings of the Eight IEEE Symposium on Software Metrics (METRIC), 2002b.        [ Links ]

19. Idri, A., Abran, A. and Khoshgoftaar, T., "Fuzzy Analogy: a New Approach for Software Cost Estimation", International Workshop on Software Measurement (IWSM'01), Montréal, Québec, Canada, August 28–29, 2001.        [ Links ]

20. Jorgensen M., Kirkeboen G., Sjoberg D., Anda B. and Brathall L., "Human Judgement in Effort Estimation of Software Projects", International Conference on Software Engineering, Limerick, Ireland, 2000.        [ Links ]

21. Kadoda G., Cartwright M., Chen L., Shepperd M. Experiences Using Case–Based Reasoning to Predict Software Project Effort. Proceedings of the EASE Conference Keele, UK. 2000.        [ Links ]

22. Kitchenham B. A., Pfleeger S. L., Pickard L. M., Jones P. W., Hoaglin D. C., Emam K. E. and Rosenberg J., "Preliminary Guidelines for Empirical Research in Software Engineering", IEEE Transactions on Software Engineering, Vol. 28, No. 8, August 2002.        [ Links ]

23. Kok P., Kitchen han B.A, Kirakowski J. The MERMAID Approach to Software Cost Estimation. Proceedings ESPRIT Technical week. 1990.        [ Links ]

24. Lind D. A., Mason R. D., Marchal W. G., Basic Statistics, McGraw Hill, 2000.        [ Links ]

25. López–Martín Cuauhtémoc, Leboeuf J., Yáñez Cornelio and Agustín Gutiérrez T., "Software Development Effort Estimation Using Fuzzy Logic: A Case Study", Encuentro Internacional de Ciencias de la Computación, IEEE Computer Society Press, September, 2005, pp. 113–120.        [ Links ]

26. MacDonell S. G. and Gray A. R., "Alternatives to Regression Models for Estimating Software Projects", Proceedings of the IFPUG Fall Conference, 1996.        [ Links ]

27. MacDonell S. G., "Software source code sizing using fuzzy logic modelling". Elsevier Science, 2003.        [ Links ]

28. Mendes E., Mosley N. and Watson L, "A Comparison of Case–Based Reasoning Approaches to Web Hypermedia project Cost Estimation", ACM, 2002.        [ Links ]

29. Musflek P., Pedrycz W., Succi G. and Reformat M., "Software Cost Estimation with Fuzzy Models", Applied Computing Review, Vol. 8, No. 2, 2000, pages 24–29.        [ Links ]

30. Park R. E., "Software Size Measurement: A Framework for Counting Source Statements", SEI, Carnegie Mellon University, September 1992.        [ Links ]

31. Schofield C., "Non–Algorithmic Effort Estimation Techniques", ESERG, TR98–01.        [ Links ]

32. Seaman C. B., "Qualitative Methods in Empirical Studies of Software Engineering", IEEE Transactions on Software Engineering, Vol. 25, No. 4, July/August, 1999        [ Links ]

33. Secretaría de Economia, "Programa para el Desarrollo de la Industria del Software", June 2002.        [ Links ]

34. Srinivasan K., Fisher D. Machine Learning Approaches to Estimating Software Development Effort. IEEE Transactions on Software Engineering, Vol. 21, No. 2, 1995.        [ Links ]

35. Weiss N.A. Introductory Statistics. Addison Wesley. Fifth Edition. 1999.        [ Links ]

36. Zhiwei Xu Z. and Khoshgoftaar T. M., "Identification of fuzzy models of software cost estimation". Elsevier Fuzzy Sets and Systems. Volume 145, Issue 1, 1 July 2004, pp.141–163.        [ Links ]

Appendix

The following lists include the identifiers, names, programming language, and their job/university of developers:

a) Development Team, Federal Commission of Electricity from Guadalajara, Director's email: omar.delacruz@cfe.gob.mx. A: (Alatorre Carranza N., C) B: (De la Cruz Preciado O., Pascal); C: (Flores Gómez C., COBOL); D: (Galindo Gauna R., C); E: (García Ramos M., C); F: (Guerra Martínez A., Pascal); G: (Guzmán Martínez A., C); H: ( Hernández Hernández P., COBOL) I: (Hernández Ramos A., COBOL); J: (Partida Menchuca L., COBOL).

b) Bachelor Students, University del Valle de Atemajac (UNIVA), Guadalajara, http://www.univa.mx/, Director's email: martin.rodriguez@univa.mx. K: (Becerril Ramírez J., Delphi); L: (Herrera Rábago F., Pascal); M: (Navarro Rodríguez J., C); N: (Santana Ruelas J., C) O: (Vargas Mora D., JAVA).