In this research, a new Test and Evaluation (T&E) procedure for defense AI systems is proposed to fill the existing gap in established methodologies. This proposed concept incorporates a data-based performance evaluation, allowing for independent assessment of AI model efficacy. It then follows with an on-site T&E using the actual AI system. The performance evaluation approach adopts the project promotion framework from the defense acquisition system, outlining 10 steps for R&D projects and 9 steps for procurement projects. This procedure was crafted after examining AI system testing standards and guidelines from both domestic and international civilian sectors. The validity of each step in the procedure was confirmed using real-world data. This study's findings aim to offer insightful guidance in defense T&E, particularly in developing robust T&E procedures for defense AI systems.

In the defense acquisition, a company’s goal is to maximize profits, and the government’s goal is to allocate budgets efficiently. Each year, the government estimates the ratio of indirect cost sector to defense companies, and estimates the ratio to be applied when calculating cost of the defense articles next year. The defense industry environment is changing rapidly, due to the increasing trend of defense acquisition budgets, the advancement of weapon systems, the effects of the 4th industrial revolution, and so on. As a result, the cost structure of defense companies is being diversifying. The purpose of this study is to find an alternative that can enhance the rationality of the current methodology for estimating the ratio of indirect cost sector of defense companies. To do this, we conducted data analysis using the R language on the cost data of defense companies over the past six years in the Defense Integrated Cost System. First, cluster analysis was conducted on the cost characteristics of defense companies. Then, we conducted a regression analysis of the relationship between direct and indirect costs for each cluster to see how much it reflects the cost structure of defense companies in direct labor cost-based indirect cost rate estimates. Lastly a new ratio prediction model based on regularized regression analysis was developed, applied to each cluster, and analyzed to compare performance with existing prediction models. According to the results of the study, it is necessary to estimate the indirect cost ratio based on the cost character group of defense companies, and the direct labor cost based indirect cost ratio estimation partially reflects the cost structure of defense companies. In addition, the current indirect cost ratio prediction method has a larger error than the new model.

지식정보사회의 등장은 개인과 기업, 또는 공공기관 및 국가에게 부여된 목표와 성과를 달성하기 위해 유용한 정보를 획득, 가공하여 이를 활용할 수 있는 환경과 능력을 갖추기 위한 IT 기반의 지식경영 시스템 구축을 조직성과 향상의 핵심요인으로 인식하게 하였다. 이러한 환경의 변화는 국방획득사업을 관리하는 관련조직들에게도 전문성, 기밀성, 정보접근의 제한성이라는 국방획득사업의 특성과 권위성, 보수성, 비밀성, 정보보호의 의식 등의 국방조직의 특징을 극복하기 위한 수단으로 기존의 아날로그적 방식으로 관리 및 유지되던 사업 관리의 Know-How나 중요 지식 정보들을 IT기반으로 구축된 지식경영 시스템을 이용한 시스템 구축 및 체계적 관리의 필요성을 인식하게 하였으며, IT기반의 지식경영 시스템의 도입 및 정착을 진행하게 하였다. 이에 본 연구는 국방획득사업의 성과를 향상시킬 수 있다면, 지식경영의 핵심요인 중 하나인 지식경영 시스템 역시 국방획득사업의 성과를 향상시키는데 기여한다는 실증적 연구결과를 제안하였다.

The purpose of this paper is to suggest the methodology for the establishment of operational concept for speed-up of defense robot and improvement direction of the defense acquisition system for the defense robot. In order to achieve this goal, the current defense acquisition system was analyzed into long-term planning, mid-term programming, and project execution stages. And I suggest the methodology for the establishment of operational concept for speed-up of defense robot and direction of development of the defense robot acquisition system considering the characteristics of the robot in terms of core technologies of robot, robot ecosystem and effectiveness-based-robot-design, respectively. Based on the methodology for establishment of the operational concept of defense robot and development direction of the defense acquisition system presented in this study, it will be possible to design efficiently the defense robot in the future.