運用大型語言模型分析國小資優生獨立研究報告的參考文獻特徵

Abstract

本研究旨在探討人工智慧大型語言模型在獨立研究參考文獻書目整理中的應用效果，並瞭解國小資優生獨立研究參考文獻的特徵。研究利用科展比對工具系統進行3個直轄市1,627篇獨立研究報告進行文本重複性比對。運用GPT-4 整理7,022筆國小資優生獨立研究報告的參考文獻，進行參考文獻特徵計量分析。研究結果如下： 一、GPT-4在進行參考文獻整理任務時，平均的F1分數可達0.9，具有較高的準確性。 二、國小資優生獨立研究報告之間的文本重複性不高，但學生沒有良好的文內引用習慣，可能會造成不當引用的誤會。 三、不同年度的獨立研究報告在參考文獻特徵顯示出參考資料來源受到時間的流行影響，目前最主要的參考文獻來源為網路資源。 四、各學科領域的獨立研究報告在參考文獻使用上呈現出差異性。人文類與設計與創作類的獨立研究報告使用最多參考文獻，並偏好使用書籍，自然類與數學類的獨立研究報告常使用科展報告。 五、得獎與未得獎的獨立研究報告在參考文獻的數量和來源上存在差異。前者參考文獻數量較多，且集中在書籍、期刊等具有權威性的資料來源。得獎報告的參考文獻使用會影響未來獨立研究的參考文獻使用。This study investigates the effectiveness of large language models in organizing bibliographies for independent research and explores the characteristics of references used by gifted elementary school students in their independent studies.Methods: The study utilized the Comparison System for NTSEC to conduct text duplication comparisons on 1,627 independent research reports from three cities. GPT-4 was then employed to extract bibliographic data from textual paragraphs in these reports, aiming to analyze the characteristics of references used by gifted ele-mentary students in their independent studies.Results: 1. GPT-4 achieved an average F1 score of 0.9 in extracting bibliographic data from textual paragraphs, demonstrating high accuracy in reference pro-cessing. 2. The text duplication rate among the reports was low, but there was a lack of proper in-text citation habits, potentially leading to citation misunderstand-ings. 3. The choice of reference sources was influenced by contemporary trends, with online resources being the primary source. 4. Reference usage varied across different academic disciplines. Humanities and creative design reports used the most references, primarily books, while natu-ral sciences and mathematics reports frequently used science fair reports.5. Award-winning reports used more references, mainly from authoritative sources like books and journals, potentially influencing reference choices in future independent study.