논문 상세보기
pp.79-90
Written examination for driver’s license certification plays a critical role in promoting road safety by assessing the applicants' understanding of traffic laws and safe driving practices. However, concerns have emerged regarding structural biases in multiple-choice question (MCQ) formats, such as disproportionate answer placement and leading linguistic cues, which may allow test-takers to guess the correct answers without substantive legal knowledge. To address these problems, this paper proposes a prompt-driven evaluation framework that integrates structural item analysis with response simulations using a large language model (LLM). First, we conducted a quantitative analysis of 1,000 items to assess formal biases in the answer positions and option lengths. Subsequently, GPT-based simulations were performed under four distinct prompt conditions: (1) safety-oriented reasoning without access to legal knowledge, (2) safety-oriented reasoning with random choices for knowledge-based questions, (3) performance-oriented reasoning using all available knowledge, and (4) a random-guessing baseline model to simulate non-inferential choice behavior. The results revealed notable variations in item difficulty and prompt sensitivity, particularly when safety-related keywords influence answer selection, irrespective of legal accuracy. The proposed framework enables a pretest diagnosis of potential biases in the MCQ design and provides a practical tool for enhancing the fairness and validity of traffic law assessments. By improving the quality control of item banks, this approach contributes to the development of more reliable knowledge-based testing systems that better support public road safety.
1. 서론
1.1. 연구배경
1.2. 연구목적
2. 선행연구 검토
2.1. 운전면허 학과시험 제도의 변화와 평가 한계
2.2. 문항 구성 및 언어 기반 정답 유도 가능성
2.3. 객관식 문항의 형식적 편향 연구
2.4. LLM의 사용자 맞춤형 반응성과 평가도구 활용 연구
2.5. LLM 프롬프트 기반 문제은행 품질 진단의 가능성
2.6. 운전면허 학과시험과 교통안전 간 상관관계
3. 연구 방법론
3.1. 분석 대상 및 문항 데이터 구성
3.2. 문항 형식 분석 기준
3.3. LLM 기반 시뮬레이션 설계
3.4 시뮬레이션 결과 분석 방법
4. 결과 및 시사점
4.1. 문항 형식 분석 결과
4.2. LLM 시뮬레이션 결과 분석
4.3. 종합 분석 및 시사점
5. 결론
REFERENCES
