banner
Home > Open Journal Systems

A Perspective of Conventional and Bio-inspired Optimization Techniques in Maximum Likelihood Parameter Estimation

Yongzhong Lu, Min Zhou, Shiping Chen, David Levy, Jicheng You

Abstract


Maximum likelihood estimation is a method of estimating the parameters of a statistical model in statistics. It has been widely used in a good many multi-disciplines such as econometrics, data modelling in nuclear and particle physics, and geographical satellite image classification, and so forth. Over the past decade, although many conventional numerical approximation approaches have been most successfully developed to solve the problems of maximum likelihood parameter estimation, bio-inspired optimization techniques have shown promising performance and gained an incredible recognition as an attractive solution to such problems. This review paper attempts to offer a comprehensive perspective of conventional and bio-inspired optimization techniques in maximum likelihood parameter estimation so as to highlight the challenges and key issues and encourage the researches for further progress.

Keywords


maximum likelihood estimation; bio-inspired optimization; differential evolution; swarm intelligence-based algorithm; genetic algorithm; particle swarm optimization; ant colony optimization.

Full Text:

PDF

References


1. Aguirregariria, V., A hybrid genetic algorithm for the maximum likelihood estimation of models with multiple equilibria: A first report, New Mathematics and Natural Computation 1 (2) (2005) 295-303.

2. An, ., Aksoy, S., Ankan, O., Maximum likelihood estimation of Gaussian mixture models using stochastic search, Pattern Recognition 45 (7) (2012) 2804-2816.

3. Augustyniak, M., Maximum likelihood estimation of the Markov-switching GARCH model, Computational Statistics and Data Analysis 76 (2) (2014) 61-75.

4. Baghishani, H., Mohammadzadeh, M., A data cloning algorithm for computing maximum likelihood estimates in spatial generalized linear mixed models, Computational Statistics and Data Analysis 55 (4) (2011) 1748-1759.

5. Baltagi, B. H., Bresson, G., Maximum likelihood estimation and Lagrange multiplier tests for panel seemingly unrelated regressions with spatial lag and spatial errors: An application to hedonic housing prices in Paris, Journal of Urban Economics 69 (1) (2011) 24-42.

6. Chang, J.-C., Chen, C.-Y., Maximum likelihood DOA estimation with sensor position perturbation using particle swarm optimization, Advances in Intelligent Systems and Applications: Smart Innovation, Systems and Technologies 20 (1), Springer-Verlag, Berlin, Heidelberg, 2013, pp. 187-197.

7. Denoeux, T., Maximum likelihood estimation from fuzzy data using the EM algorithm, Fuzzy Sets and Systems 183 (1) (2011) 72-91.

8. Dosiek, L., Pierre, J. W., Follum, J., A recursive maximum likelihood estimator for the online estimation of electromechanical modes with error bounds, IEEE Transactions on Power Systems 28 (1) (2013) 441-451.

9. Dutta, P., Arnaud, E., Prados, E., Saujot, M., Calibration of an integrated land-use and transportation model using maximum likelihood estimation, IEEE Transactions on Computers 63 (1) (2014) 167-178.

10. El-Kafafy, M., Peeters, B., Guillaume, P., Troyer, T. D., Constrained maximum likelihood modal parameter identification applied to structural dynamics, Mechanical Systems and Signal Processing 72-73 (2016) 567-589.

11. Firmino, P. R. A., Mattos Netob, P. S. G. d., Ferreira, T. A. E., Correcting and combining time series forecasters, Neural Networks 50 (2014) 1-11.

12. Haryanto, A., Hong, K.-S., Maximum likelihood identification of Wiener-Hammerstein models, Mechanical Systems and Signal Processing 41 (1-2) (2013) 54-70.

13. Hirai, S., Yamanishi, K., Efficient computation of normalized maximum likelihood codes for Gaussian mixture models with its applications to clustering, IEEE Transactions on Information Theory 59 (11) (2013) 7718-7727.

14. Hu, C., Liu, Q., Online identification for hypersonic vehicle using recursive maximum likelihood method based on interior-point algorithm, in: 25th Chinese Control and Decision Conference, Guiyang, China, IEEE, 2013, pp. 1862-1867.

15. Galimberti, G., Soffritti, G., A multivariate linear regression analysis using finite mixtures of t distributions, Computational Statistics and Data Analysis 71 (2014) 138-150.

16. Giacomina, P. A. S., Hemerlyb, E. M., Pedrycz, W., A probabilistic approach for designing nonlinear optimal robust tracking controllers for unmanned aerial vehicles, Applied Soft Computing 34 (2015) 26-38.

17. Gonzlez, M., Minuesa, C., Puerto, I. d., Maximum likelihood estimation and expectation-maximization algorithm for controlled branching processes, Computational Statistics and Data Analysis 93 (2016) 209-227.

18. Jelonek, T. M., Reilly, J. P., Maximum likelihood estimation for direction of arrival using a nonlinear optimising neural network, in: International Joint Conference on Neural Networks, IEEE, San Diego, 1990, pp. 253-258.

19. Li, J., Ding, F., Maximum likelihood stochastic gradient estimation for Hammerstein systems with colored noise based on the key term separation technique, Computers and Mathematics with Applications 62 (11) (2011) 4170-4177.

20. Li, J., Ding, F., Yang, G., Maximum likelihood least squares identification method for input nonlinear finite impulse response moving average systems, Mathematical and Computer Modelling 55 (3-4) (2012) 442-450.

21. Liu, Z.-M., Huang, Z.-T., Zhou, Y.-Y., An efficient maximum likelihood method for direction-of-arrival estimation via sparse Bayesian learning, IEEE Transactions on Wireless Communications 11 (10) (2012) 1-11.

22. Martin, R., Hanb, Z., A semiparametric scale-mixture regression model and predictive recursion maximum likelihood, Computational Statistics and Data Analysis 94 (2016) 75-85.

23. Mirikitani, D., Nikolaev, N., Nonlinear maximum likelihood estimation of electricity spot prices using recurrent neural networks, Neural Computing and Applications 20 (1) (2011) 79-89.

24. Mohsin, M., Kazianka, H., Pilz, J., Likelihood and objective Bayesian modeling of acidity and major ions in rain fall using a bivariate pseudo-Gamma distribution, Computers & Geosciences 54 (2013) 269-278.

25. Ogasawara, H., Asymptotic expansions for the estimators of Lagrange multipliers and associated parameters by the maximum likelihood and weighted score methods, Journal of Multivariate Analysis 147 (2016) 20-37.

26. Pan, J., Ma, W.-K., Jalden, J., MIMO detection by Lagrangian dual maximum-likelihood relaxation: reinterpreting regularized lattice decoding, IEEE Transactions on Signal Processing 62 (2) (2014) 511-524.

27. Pence, B.L., Fathy, H.K., Stein, J.L., Recursive maximum likelihood parameter estimation for state space systems using polynomial chaos theory, Automatica 47 (11) (2011) 2420-2424.

28. Poladian, L., A GA for maximum likelihood hylogenetic inference using neighbour-joining as a genotype to phenotype mapping, in: Proceedings of the 7th annual conference on Genetic and evolutionary computation, New York, USA, ACM, 2005, pp. 415-422.

29. Reed, H. M., Nicholsb, J. M., Earls, C. J., A modified differential evolution algorithm for damage identification in submerged shell structures, Mechanical Systems and Signal Processing 39(1-2) (2013) 396-408.

30. Risholm, P., Janoos, F., Norton, I., Golby, A. J., Wells III, W. M., Bayesian characterization of uncertainty in intra-subject non-rigid registration, Medical Image Analysis 17 (2013) 538-555.

31. Skaug, H. J., Yu, J., A flexible and automated likelihood based framework for inference in stochastic volatility models, 76 (2014) 642-654.

32. Stathakis, E., Jalden, J., Rasmussen, L. K., Skoglund, M., Uniformly improving maximum-likelihood SNR estimation of known signals in Gaussian channels, IEEE Transactions on Signal Processing 62 (1) (2014) 156-167.

33. Stoica, P., Babu, P., Maximum-likelihood nonparametric estimation of smooth spectra from irregularly sampled data, IEEE Transactions on Signal Processing 59 (12) (2011) 5746-5758.

34. Sun, Q., Lim, C.-C., Liu, L., Maximum likelihood state estimation for Markov jump systems with uncertain mode-dependent delays, Journal of the Franklin Institute 353 (2016) 594-614.

35. Tay, D., Poh, C. L., Kitney, R. I., An evolutionary data-conscious artificial immune recognition system, in: Genetic and evolutionary computation conference, Amsterdam, Netherland, ACM, 2013, pp. 1101-1108.

36. Tay, D., Poh, C. L., Kitney, R. I., A novel neural-inspired learning algorithm with application to clinical risk prediction, Journal of Biomedical Informatics, 54 (2015) 305-314.

37. Wang, D. Q., Chu, Y. Y., Yang, G. W., Ding, F., Auxiliary model-based recursive generalized least squares parameter estimation for Hammerstein OEAR systems, Mathematical and Computer Modelling 52 (1-2) (2010) 309-317.

38. Wang, X., Sun, W., Wu Z., Yang H., Wang Q., Color image segmentation using PDTDFB domain hidden Markov tree model, Applied Soft Computing 29 (2015) 138-152.

39. Wang, Z., Luo, J.-A., Zhang, X.-P., A novel location-penalized maximum likelihood estimator for bearing-only target localization, IEEE Transactions on Signal Processing 60 (12) (2012) 6166-6181.

40. Wen, C.-C., Chen, Y.-H., Nonparametric maximum likelihood analysis of clustered current status data with the gamma-frailty Cox model, Computational Statistics and Data Analysis 55 (2) (2011) 1053-1060.

41. Xu, B., Lu, M., Ren, Y., Zhua, P., Shi, J., Cheng, D., Multi-task ant system for multi-object parameter estimation and its application in cell tracking, Applied Soft Computing 35 (2015) 449-469.

42. Yao, K., Yu, D., Deng, L., Gong, Y., A fast maximum likelihood nonlinear feature transformation method for GMM-HMM speaker adaptation, Neurocomputing 128 (2014) 145-152.

43. Zhang, Z., Lin, J., Shi, Y., Application of artificial bee colony algorithm to maximum likelihood DOA estimation, Journal of Bionic Engineering 10 (2013) 100-109




DOI: https://doi.org/10.32629/jai.v1i2.28

Refbacks

  • There are currently no refbacks.


Copyright (c) 2018 Yongzhong Lu

License URL: https://creativecommons.org/licenses/by-nc/4.0