Background: The growing need for objective and accurate evaluation in Taekwondo poomsae competitions has highlighted the limitations of subjective human judgment. Objectives: This study aims to develop an automated scoring framework using camera-based pose estimation and advanced neural networks to improve the consistency and accuracy of poomsae evaluation. Design: Comparative analysis of neural network architectures on a large-scale dataset of poomsae movements. Methods: A dataset of 902,306 labeled frames, captured from 48 participants performing 62 distinct movements using synchronized multi-view cameras, was analyzed. Five neural networks (HNN, 1D CNN, GCN, MLP, SANN) were implemented and evaluated using accuracy, precision, recall, and F1-score. Results: The HNN demonstrated superior performance with an F1-score of 0.78 in classifying Taekwondo poomsae postures. The 1D CNN followed with an F1-score of 0.76, while GCN, MLP, and SANN achieved F1-scores of 0.74, 0.70, and 0.66, respectively. The HNN's hierarchical feature extraction approach proved effective in capturing the complex spatial and temporal patterns inherent in poomsae movements. Conclusion: Hierarchical Neural Networks outperform other architectures in poomsae classification, establishing a foundation for objective and scalable scoring systems in competitive settings.

Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment. The mean of the overall population correlation between automated and human scoring in essay writing was .78. The overall common d effect size was 0.001. Results from this meta-analysis indicated a strong relationship with no discrepancies between automated and human scoring. Both the I2 and Q values suggested that the population correlation values studied seemed to be heterogeneous, in contrast to homogenous d effect sizes. Therefore, it is necessary to investigate the sources of the between-studies variations for r correlations. Practical implications for ways of reporting results of automatic-scoring systems research and limitations of the study are also discussed.

This study investigated the feasibility of adopting an automatic scoring system (ASS) in a domestic English-speaking education context. Scope, test items, assessment criteria, scoring methods, and reporting strategies of six overseas English-speaking tests utilizing ASSs were examined. Moreover, a comparative analysis was conducted to identify disparities between ASS-based and non-ASS-based speaking tests. Findings were: 1) some ASS-based tests utilized ASS technology throughout the assessment, while others adopted a hybrid scoring system involving human raters; 2) compared to non-ASS-based tests, ASS-based tests used more test items targeting low-level skills such as sound and forms but fewer test items targeting conversation and discourse level skills; 3) pronunciation, fluency, and vocabulary were widely employed as evaluation criteria with sparse use of organization, content, and task completion in most ASS-based tests; 4) differences were minimal in assessment criteria application and score calculation between ASS-based and non-ASS-based tests; and 5) some ASS-based tests provided criteria-specific results and feedback with total scores and proficiency levels.