國家教育研究院

　　為了因應大型評量的需要，自動化的口語評量系統為目前評量發展的新趨勢。自 動化的評量系統在電機領域已經應用很廣，但在教育領域應用的較為有限，本研 究建構自動化的口語評量系統，進行語音辨識系統建置。樣本為在研究院上課的 校長儲訓班學員，演說內容為教學、校務等相關議題。本研究提出了多模態計算 架構來建置自動即席演講評分系統，結合了候用校長培訓計畫的音訊與視訊資料， 並改善演算法達到更佳的二元分類辨識結果，目前已完成兩種自動評分的方法， 第一是僅選擇前20%和後20%的資料，讓機器來學習評分，第二則使用全部的資 料。 研究顯示，第一種方法，機器學習的效果較好，評出來的分數和人工評分 者的相關性較高。未來在增進系統的辨識率方面，可透過加入文本資料、處理音 視訊錄製過程中的雜訊、收集更多演講樣本，評分者的評分等，以提高自動化系 統的準確率。

　　對教育政策之建議: 本研究建構自動化的口語評量系統，國內外學者對於自動化 口語評量工具的研究與討論較為侷限，本研究成果能補齊過去研究不足之處，其 開發歷程與研究成果皆能提供未來研究進一步參考。

　　Automatic scoring system is the current trend for large scale assessment. Automatic scoring system has applied widely in electronic engineering field, but still quite limited in education field. In this work, we employ behavioral signal processing (BSP)-based methodology to develop a computational framework that can automate the scoring process of pre-service school principals’ oral presentations given at the yearly training program. Using the audio-video feature extraction approach with session-level representation techniques based on bag-of-word and Fisher-vector encoding to characterize each candidate principal’s multimodal behavior during an impromptu speech examination.Two approaches have been used, first is to choose the top and bottom 20% of the total samples and further labeling them as high and low scoring oral presentations. Second approach is using the entire database. The first approach has higher agreement level. For the future study, including the lexical content, exploring more approach to handle the varying noise structures, collecting more samples and rater could be possibly increase the accuracy rate of this system. Contributions: This study construct a automatic scoring system. There is quite limited studies and discussions related to this field in Taiwan. The results of this study can be provided as preliminary studies for performance assessment for teacher certification or recruitment assessments.