A Parameter Free BBN Discriminant Function for Optimum Model Complexity versus Goodness of Data Fitting

 

M. Naeem*¹ and S. Asghar²

 

¹ Department of Computer Sciences, Faculty of Computing Mohammad Ali Jinnah, University Islamabad Pakistan. *naeems.naeem@gmail.com

² COMSATS Institute of Information Technology, Islamabad Pakistan.

 

ABSTRACT

Bayesian Belief Network (BBN) is an appealing classification model for learning causal and noncausal dependencies among a set of query variables. It is a challenging task to learning BBN structure from observational data because of pool of large number of candidate network structures. In this study, we have addressed the issue of goodness of data fitting versus model complexity. While doing so, we have proposed discriminant function which is non-parametric, free of implicit assumptions but delivering better classification accuracy in structure learning. The contribution in this study is twofold, first contribution (discriminant function) is in BBN structure learning and second contribution is for Decision Stump classifier. While designing the novel discriminant function, we analyzed the underlying relationship between the characteristics of data and accuracy of decision stump classifier. We introduced a meta characteristic measure AMfDS (herein known as Affinity Metric for Decision Stump) which is quite useful in prediction of classification accuracy of Decision Stump. AMfDS requires a single scan of the dataset.

Keywords: machine learning, Bayesian network, decision stump, K2, data characterization.

 

DESCARGAR ARTÍCULO EN FORMATO PDF

 

References

[1] W.J. Wang and Q. Xu, "A Bayesian Combination Forecasting Model for Retail Supply Chain Coordination," J. Appl. Res. Technol., vol. 12, no. 2.         [ Links ]

[2] H.H. Aviles-Arriaga et al., "A comparison of dynamic naive bayesian classifiers and hidden markov models for gesture recognition," J. Appl. Res. Technol., vol. 9, no. 1, pp. 81-102, 2011.         [ Links ]

[3] M.M. Haji et al., "Current Transformer Saturation Detection Using Gaussian Mixture Models," J. Appl. Res. Technol., vol. 11, pp. 79-87, 2013.         [ Links ]

[4] D. Heckerman et al., "Learning Bayesian networks: The combination of knowledge and statistical data," Mach. Learn., vol. 20, no. 3, pp. 197-243, 1995.         [ Links ]

[5] D. Heckerman, "A tutorial on learning with Bayesian networks," in Innovations in Bayesian Networks, Springer, 2008, pp. 33-82.         [ Links ]

[6] M. Naeem and S. Asghar, "A Novel Feature Selection Technique for Feature Order Sensitive Classifiers," An. Ser. Inform. Ann. Comput. Sci. Ser., vol. 11, no. 1, pp. 31-38, Dec. 2013.         [ Links ]

[7] S.P. Brooks and B.J.T. Morgan, "Optimization using simulated annealing," The Statistician, pp. 241-257, 1995.         [ Links ]

[8] S. Kirkpatrick, "Optimization by simulated annealing: Quantitative studies," J. Stat. Phys., vol. 34, no. 5-6, pp. 975-986, 1984.         [ Links ]

[9] G.F. Cooper and E. Herskovits, "A Bayesian method for the induction of probabilistic networks from data," Mach. Learn., vol. 9, no. 4, pp. 309-347, 1992.         [ Links ]

[10] T.D. Nielsen and F.V. Jensen, Bayesian networks and decision graphs. Springer, 2009.         [ Links ]

[11] W. Buntine, "Learning classification trees," Stat. Comput., vol. 2, no. 2, pp. 63-73, 1992.         [ Links ]

[12] H. Akaike, "A new look at the statistical model identification," Autom. Control IEEE Trans. On, vol. 19, no. 6, pp. 716-723, 1974.         [ Links ]

[13] J. Skilling, Maximum entropy and Bayesian methods, vol. 45. Springer, 1989.         [ Links ]

[14] W. Lam and F. Bacchus, "Learning Bayesian belief networks: An approach based on the MDL principle," Comput. Intell., vol. 10, no. 3, pp. 269-293, 1994.         [ Links ]

[15] J. Suzuki, "Learning Bayesian belief networks based on the MDL principle: An efficient algorithm using the branch and bound technique," IEICE Trans. Inf. Syst., vol. 82, no. 2, pp. 356-367, 1999.         [ Links ]

[16] A.M. Carvalho et al., "Discriminative learning of Bayesian networks via factorized conditional log-likelihood," J. Mach. Learn. Res., vol. 12, pp. 21812210, 2011.         [ Links ]

[17] H. Bozdogan, "Model selection and Akaike's information criterion (AIC): The general theory and its analytical extensions," Psychometrika, vol. 52, no. 3, pp. 345-370, 1987.         [ Links ]

[18] M. Pelikan et al., Scalable optimization via probabilistic modeling. Springer, 2006.         [ Links ]

[19] M. Pelikan, Hierarchical Bayesian optimization algorithm. Springer, 2005.         [ Links ]

[20] E.S. Correa and J. L. Shapiro, "Model complexity vs. performance in the Bayesian optimization algorithm," in Parallel Problem Solving from Nature-PPSN IX, Springer, 2006, pp. 998-1007.         [ Links ]

[21] G. Schwarz, "Estimating the dimension of a model," Ann. Stat., vol. 6, no. 2, pp. 461-464, 1978.         [ Links ]

[22] N. Friedman and M. Goldszmidt, "Learning Bayesian networks with local structure," in Learning in graphical models, Springer, 1998, pp. 421-459.         [ Links ]

[23] D.M. Chickering et al., "Large-sample learning of Bayesian networks is NP-hard," J. Mach. Learn. Res., vol. 5, pp. 1287-1330, 2004.         [ Links ]

[24] L. Yang and J. Lee, "Bayesian Belief Network-based approach for diagnostics and prognostics of semiconductor manufacturing systems," Robot. Comput.-Integr. Manuf., vol. 28, no. 1, pp. 66-74, 2012.         [ Links ]

[25] W. Iba and P. Langley, "Induction of One-Level Decision Trees.," 1992, pp. 233-240.         [ Links ]

[26] M. Naeem and S. Asghar, "An Information Theoretic Scoring Function in Belief Network," Int. Arab J. Inf. Technol., vol. 11, no. 5, pp. 1-10, 2014        [ Links ]

[27] M. Naeem and S. Asghar, "A novel mutual dependence measure in structure learning," J. Natl. Sci. Found. Sri Lanka, vol. 41, no. 3, pp. 203-208, 2013.         [ Links ]

[28] D.H. Wolpert, "On bias plus variance," Neural Comput., vol. 9, no. 6, pp. 1211-1243, 1997.         [ Links ]

[29] S. Ali and K.A. Smith, "On learning algorithm selection for classification," Appl. Soft Comput., vol. 6, no. 2, pp. 119-138, 2006.         [ Links ]

[30] Datafit, "DataFit curve fitting (nonlinear regression) and data plotting software, and DataFitX ActiveX curve fitting engine." 2013.         [ Links ]