機器學習應用於生涯興趣評量之探索性研究

Applying Machine Learning to Career Interest Assessment: An Exploratory Study

以往應用機器學習（machine learning, ML）於心理特徵評量時，幾乎都以量表自評分數為標註，少數以專家衡鑑為標註，缺乏用相同資料進行雙標註的研究，雙標註研究能讓研究者比較量表標註與專家標註的結果，並從中分析與發現一些新知識。本研究即擬在後現代生涯諮商諮詢情景設定下，以生涯興趣為研究標的，探索ML應用於學習專家衡鑑、或應用於逼近量表分數的表現。本研究以1007位大學生為訓練集，訓練了專家標註與量表標註兩種ML模型，並以250位樣本的測試集分析模式品質與效度表現。研究結果顯示，將ML應用於逼近量表分數，所得模式品質較低（r = .26），聚斂與區辨效度皆不佳，缺乏效標關連效度證據，品質離實用門檻尚有不短的距離；將ML應用於學習專家衡鑑，所得模式品質有不錯表現（r = .60），在聚斂與區辨效度的表現尚佳，效標關連效度表現雖未達顯著，但仍有一定程度的正向效果，而且品質大抵是接近實用門檻的。實務應用時建議依循生涯諮商諮詢原本作法，前段可用ML模式協助初步衡鑑生涯興趣，後段視案主需要實施系統性量表測量，區別分析顯示前後環節都有增益效果。最後，本研究亦討論了專家衡鑑與量表自評低相關現象對ML研究的可能意涵與限制。

Machine learning (ML) has been applied to psychological characteristic assessment, but most studies have used self-reported scores from scales as annotations, while only a few have used expert ratings as labels. There is a lack of studies using the same data for dual annotating, which can allow researchers to compare the results of self-reported and expert-rated annotation models and to discover new knowledge from the results. This study aims to explore the application of ML on career interests in the context of postmodern career counseling and consultation. The performance of ML applied to learning expert ratings or ML applied to approximating self-reported scale scores are explored. Using a training set of 1007 university students, two ML models were trained with annotations from experts and scales. The model quality and validity performance were analyzed using a test set of 250 samples. Results revealed that applying ML to approximate scale scores yielded only low model quality (r = .26), with poor convergence and discriminant validity. No evidence of criterion-related validity. The model quality was still far from the practical threshold. In contrast, applying ML to learn expert assessments showed moderate model performance (r = .60) with satisfactory convergence and discriminant validity. Although criterion-related validity did not reach significance, but there was a certain degree of positive effect. The model quality was generally close to the practical threshold. In practical application, it is suggested to follow the original practice of career counseling and consultation. In the first stage, ML models can be used to assist in the initial assessment of career interests. In the second stage, systematic scale measurement can be implemented according to the needs of the client. The discriminant analysis showed that there was an incremental effect in both stages. Finally, this study also discussed the possible implications and limitations of the low correlation between expert ratings and self-reported ratings for ML research.