This exploratory study investigates rater severity, internal consistency, and rater-related interaction effects in a graph-based writing test for English placement purposes. A total of 101 ESL test-takers completed two graph-based writing tasks, and three trained raters evaluated the performances using a five-point analytic scoring rubric. Multi-faceted Rasch Measurement was employed, and the results indicated that, although the raters demonstrated acceptable levels of internal consistency, they differed significantly in overall severity, suggesting that they were not fully interchangeable. Interaction analyses further revealed no unexpected severity or leniency toward specific test-takers. However, differential rater functioning emerged regarding organization, with two raters exhibiting opposing severity tendencies. These findings suggest that rater training may enhance internal consistency but does not fully eliminate differences in rating severity. They further highlight the need for targeted, criterion-focused calibration for organization in graph-based writing tasks, and scoring rubric refinement to support valid score interpretations and fair placement decisions.

1. INTRODUCTION
2. LITERATURE REVIEW
    2.1. Multi-faceted Rasch Measurement (MFRM)
    2.2. Studies on Rater Variability in Performance Assessments
    2.3. Studies Investigating Graph-based Writing Tasks
3. METHODOLOGY
    3.1. Participants
    3.2. Materials
    3.3. Procedures
    3.4. Data Analysis
4. RESULTS AND DISCUSSION
    4.1. Descriptive Statistics
    4.2. Global Model Fit
    4.3. Calibration of Test-takers, Raters, and Scoring Criteria
    4.4 Overall Severities and Internal Self-consistency of the Raters (RQ 1)
    4.5. Test-taker-by-rater Interaction and Rater-by-criteria Interaction (RQ 2)
5. CONCLUSION
REFERENCES
APPENDIX A 

• YunDeok Choi(Assistant Professor, Department of English Education, Chungnam National University; 99 Daehak-ro, Yuseong-gu, Daejeon 34134, South Korea)