一级黄色片免费播放|中国黄色视频播放片|日本三级a|可以直接考播黄片影视免费一级毛片

高級(jí)搜索

留言板

尊敬的讀者、作者、審稿人, 關(guān)于本刊的投稿、審稿、編輯和出版的任何問(wèn)題, 您可以本頁(yè)添加留言。我們將盡快給您答復(fù)。謝謝您的支持!

姓名
郵箱
手機(jī)號(hào)碼
標(biāo)題
留言內(nèi)容
驗(yàn)證碼

一種基于N-gram模型和機(jī)器學(xué)習(xí)的漢語(yǔ)分詞算法

吳應(yīng)良,  韋崗,  李海洲

吳應(yīng)良, 韋崗, 李海洲. 一種基于N-gram模型和機(jī)器學(xué)習(xí)的漢語(yǔ)分詞算法[J]. 電子與信息學(xué)報(bào), 2001, 23(11): 1148-1153.
引用本文: 吳應(yīng)良, 韋崗, 李海洲. 一種基于N-gram模型和機(jī)器學(xué)習(xí)的漢語(yǔ)分詞算法[J]. 電子與信息學(xué)報(bào), 2001, 23(11): 1148-1153.
Wu Yingliang, Wei Gang, Li Haizhou. A WORD SEGMENTATION ALGORITHM FOR CHINESE LANGUAGE BASED ON N-GRAM MODELS AND MACHINE LEARNING[J]. Journal of Electronics & Information Technology, 2001, 23(11): 1148-1153.
Citation: Wu Yingliang, Wei Gang, Li Haizhou. A WORD SEGMENTATION ALGORITHM FOR CHINESE LANGUAGE BASED ON N-GRAM MODELS AND MACHINE LEARNING[J]. Journal of Electronics & Information Technology, 2001, 23(11): 1148-1153.

一種基于N-gram模型和機(jī)器學(xué)習(xí)的漢語(yǔ)分詞算法

A WORD SEGMENTATION ALGORITHM FOR CHINESE LANGUAGE BASED ON N-GRAM MODELS AND MACHINE LEARNING

  • 摘要: 漢語(yǔ)的自動(dòng)分詞,是計(jì)算機(jī)中文信息處理領(lǐng)域中一個(gè)基礎(chǔ)而困難的課題。該文提出了一種將漢語(yǔ)文本句子切分成詞的新方法,這種方法以N-gram模型為基礎(chǔ),并結(jié)合有效的Viterbi搜索算法來(lái)實(shí)現(xiàn)漢語(yǔ)句子的切詞。由于采用了基于機(jī)器學(xué)習(xí)的自組詞算法,無(wú)需人工編制領(lǐng)域詞典。該文還討論了評(píng)價(jià)分詞算法的兩個(gè)定量指標(biāo),即查準(zhǔn)率和查全率的定義,在此基礎(chǔ)上,用封閉語(yǔ)料庫(kù)和開(kāi)放語(yǔ)料庫(kù)對(duì)該文提出的漢語(yǔ)分詞模型進(jìn)行了實(shí)驗(yàn)測(cè)試,表明該模型和算法具有較高的查準(zhǔn)率和查全率。
    Abstract: Automatic word segmentation for the Chinese language is a fundamental and difficult problem in the field of computer Chinese language information processing. This paper presents a new method for segmenting the input Chinese language text sentence into words, which consists of a character-based N-gram model and an efficient Viterbi search algorithm. In addition, two performance evaluation ration targets, i.e. Recall and Precision for word segmentation algorithm are discussed, The effectiveness has been confirmed by evaluation experiments using the closed texts and open texts corpus.
    • HTML全文
  • 梁南元,漢語(yǔ)計(jì)算機(jī)自動(dòng)分詞知識(shí),中文信息學(xué)報(bào),1989,4(2),29-33.[2]王德春,應(yīng)用語(yǔ)言學(xué)概論,上海,上海外語(yǔ)教育出版社,1997年12月第1版,88-120.[3]E. Charniak, C. Hendrickson, N. Jacoboson, M. Perkowitz, Equations for part-of speech tagging,AAAI-93, 1993, 784 789.[4]K. Church, A stochastic parts program and noun phrase parser for unrestricted text, ANLP-88,1998, 136-143.[5]S. Sakai, Morphological category bigram: A single language model for both spoken language and text, ISSD-93, 1993, 97-90.[6]M. Yamamoto, A re-estimation method for stochastic language modeling from ambigous obser-vations, in Proceeding of WVLC-96, California, 1996, 155-167.[7]趙以寶, 孫圣和, 一種基于單字統(tǒng)計(jì)二元文法的自組詞音字轉(zhuǎn)換算法,電子學(xué)報(bào), 1998, 26(10), 55-58.[8]F. Jelinek, Self-Organized Language Modeling for Speech Recognition, IBM Research Report,IBM T, J. Watson Research Center, 1985. Reprinted in Reading in Speech Recognition, Waibel,A., and Lee, K-F. (Eds.), Morgan Kaufann Publishers, 1990, 450-506.[9]S.M. Katz, Estimation of probailities from sparse data for the language model component ofspeech recognizer, IEEE Trans. on Acousttics, Speech, and Signal Processing, 1987, ASSP-35(3),400-401.[10]R. Rosenfeld, The CMU statistical language modeling toolkit and its use in the 1994 ARPA CSR evaluation, In the Proc. of ARPA Spoken Language Systems Technology Workshop, Washington, 1995, 47-50.
  • 加載中
計(jì)量
  • 文章訪問(wèn)數(shù):  4162
  • HTML全文瀏覽量:  180
  • PDF下載量:  1293
  • 被引次數(shù): 0
出版歷程
  • 收稿日期:  1999-09-29
  • 修回日期:  2000-04-06
  • 刊出日期:  2001-11-19

目錄

    /

    返回文章
    返回