In this research, a new Test and Evaluation (T&E) procedure for defense AI systems is proposed to fill the existing gap in established methodologies. This proposed concept incorporates a data-based performance evaluation, allowing for independent assessment of AI model efficacy. It then follows with an on-site T&E using the actual AI system. The performance evaluation approach adopts the project promotion framework from the defense acquisition system, outlining 10 steps for R&D projects and 9 steps for procurement projects. This procedure was crafted after examining AI system testing standards and guidelines from both domestic and international civilian sectors. The validity of each step in the procedure was confirmed using real-world data. This study's findings aim to offer insightful guidance in defense T&E, particularly in developing robust T&E procedures for defense AI systems.

1. 서 론
2. AI 시스템 시험 관련 문헌연구
3. 국방 인공지능(AI) 체계 시험평가 개념
4. 연구개발사업에서 데이터 기반 성능평가 절차
4. 구매사업에서 데이터 기반 성능평가 절차
5. 결론 및 향후 연구방향
    5.1 결론
    5.2 향후 연구방향
References

• 이용복(국방대학교 국방과학학과) | Yong-Bok Lee (Korea National Defense University) Corresponding author
• 최민우(국방대학교 국방과학학과) | Min-Woo Choi (Korea National Defense University)
• 이민호(국방대학교 국방과학학과) | Min-ho Lee (Korea National Defense University)