Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment. The mean of the overall population correlation between automated and human scoring in essay writing was .78. The overall common d effect size was 0.001. Results from this meta-analysis indicated a strong relationship with no discrepancies between automated and human scoring. Both the I2 and Q values suggested that the population correlation values studied seemed to be heterogeneous, in contrast to homogenous d effect sizes. Therefore, it is necessary to investigate the sources of the between-studies variations for r correlations. Practical implications for ways of reporting results of automatic-scoring systems research and limitations of the study are also discussed.

1. INTRODUCTION
2. REVIEW OF LITERATURE
    2.1. Automated Scoring Systems
    2.2. Inter-Rater Agreement and discrepancy
    2.3. Meta-Analysis Models
3. METHODOLOGY
    3.1. Data Gathering
    3.2. Data Cleansing
    3.3. Data Analysis
4. RESULTS
    4.1. Description of Research Sources
    4.2. Heterogeneity Tests
    4.3. Overall Means of the Correlation and d
    4.4. Consistency Across Studies
    4.5. Influence of the Numbers of Point on the Scale
5. DISSCUSSION AND CONCLUSION

• Jiyeo Yun(Elementary School Teacher, Jeongdong Elementary School, Gyeongsangnamdo Office of Education, 25, Daegok 1 gil, Sacheon-si, Gyeongsangdam-do 52523, Korea)