This study investigated rater effects in the ICNALE Global Rating Archives (GRA), which contain ratings from 160 raters on 140 speeches and 140 essays across 10 rating criteria. After cleaning the raw dataset, two separate three-facet partial credit models were fitted to the speech and essay datasets to examine rater consistency, and a bias analysis was conducted to investigate the relationship between raters’ prior rating experience and severity. The results indicated that six raters in the speech ratings exhibited overly inconsistent rating patterns, but no rater in the essay ratings did. It was also found that raters’ prior rating experience was significantly associated with the complexity, involvement, and accuracy criteria in the speech ratings and with the complexity criterion in the essay ratings. However, the corresponding effect sizes for these interactions were trivial, indicating limited practical impact. The educational implication is that rigorous rater training could enhance rater consistency among both pre-service and less-experienced in-service English teachers in speaking and writing assessment.

1. INTRODUCTION
2. LITERATURE REVIEW
    2.1. ICNALE Studies Using CEFR Levels as a Proficiency Framework
    2.2. Rater Effects in Rater-mediated Performance Assessment
3. METHODOLOGY
    3.1. Corpus Data and Rater Training
    3.2. MFRM Analysis
4. RESULTS
    4.1. Rater Consistency for Speeches and Essays
    4.2. Effects of Prior Rating Experience
5. DISCUSSION
    5.1. Rater Consistency for Speeches and Essays
    5.2. Effects of Prior Rating Experience
6. CONCLUSION
REFERENCES
APPENDIX

• Hyunwoo Kim(Associate Research Fellow, Korea Institute for Curriculum and Evaluation, 8 Gyohak-ro, Deoksan-eup, Jincheon-gun, Chungcheongbuk-do 27873, Republic of Korea)