一類目標(biāo)函數(shù)的逆向構(gòu)造
INVERSE CONSTRUCTING OF A SET OF OBJECTIVE FUNCTIONS
-
摘要: 面向解決真實(shí)世界問題的神經(jīng)應(yīng)用需求,本文提出了一種構(gòu)造目標(biāo)函數(shù)的逆向方法,即將目標(biāo)函數(shù)的構(gòu)造任務(wù)轉(zhuǎn)化為誤差信號的設(shè)計(jì)。應(yīng)用這一方法,我們構(gòu)造出了一類的目標(biāo)函數(shù),它不僅可以解除均方誤差(MSE)函數(shù)的假飽和狀態(tài),從而縮短了網(wǎng)絡(luò)的訓(xùn)練時間,而且能夠克服相對熵函數(shù)帶來的過度適應(yīng)性問題,從而提高了網(wǎng)絡(luò)的泛化能力。
-
關(guān)鍵詞:
- 目標(biāo)函數(shù); 逆向構(gòu)造; 誤差信號; MSE; 相對熵
Abstract: To meet the requirements with large-scale neural networks for real-world applications, an inverse way of constructing objective functions was proposed in this paper, which translates the task of constructing objective functions into the design of error signals. Followed this way, a set of objective functions has been given as examples to eliminate the false saturation in Mean Squared Error (MSE) and overspecialization in Cross Entropy (CE). The verification of its power was also made by the comparison with MSE and CE in the tasks of estimating the scaled likelihood for the Hidden Markov Models' states in the Hybrid HMM/ANN models, and showed consistent advantages with the theoretical expectations. -
Richard M D, Lippmann R P. Neural network classifiers estimate Bayesian a posteriori probabilities[J].Neural Computation.1991, 3:461-483[2]Gish H. A probabilistic approach to the understanding and training of neural network classifiers,Proc. IEEE Intl Conf. on ASSP, New Mexico, USA: 1990, 1361-1364.[3]Haykin S. Neural Netwoks: Comprehensive Foundation. New York: Macmillan College Publishing Company, 1994, 696. ［4］Karayiannis N B. Accelerating the training of feedforward neural networks using generalized Hebbian rules for initializing the internal representations. IEEE Trans. on Neural Networks,1996, 7(2): 419-425.[4]Morgan N, Bourlard H. Continuous speech recognition: An introduction to the hybrid HMM/connectionist approach. IEEE Signal Processing Magazine, 1995, 12(3): 25-42.[5]杜利民,候自強(qiáng).自動語音識別中的人工神經(jīng)網(wǎng)絡(luò)方法.物理學(xué)進(jìn)展,1996,(9).[6]焦李成.神經(jīng)網(wǎng)絡(luò)系統(tǒng)理論,西安:西安電子科技大學(xué)出版社,1992,26-41. -
計(jì)量
- 文章訪問數(shù): 2231
- HTML全文瀏覽量: 85
- PDF下載量: 523
- 被引次數(shù): 0
下載:
