Reinforcement learning (RL) is successfully applied to various engineering fields. RL is generally used for structural control cases to develop the control algorithms. On the other hand, a machine learning (ML) is adopted in various research to make automated structural design model for reinforced concrete (RC) beam members. In this case, ML models are developed to produce results that are as similar to those of training data as possible. The ML model developed in this way is difficult to produce better results than the training data. However, in reinforcement learning, an agent learns to make decisions by interacting with an environment. Therefore, the RL agent can find better design solution than the training data. In the structural design process (environment), the action of RL agent represent design variables of RC beam. Because the number of design variables of RC beam section is many, multi-agent DQN (Deep Q-Network) was used in this study to effectively find the optimal design solution. Among various versions of DQN, Double Q-Learning (DDQN) that not only improves accuracy in estimating the action-values but also improves the policy learned was used in this study. American Concrete Institute (318) was selected as the design codes for optimal structural design of RC beam and it was used to train the RL model without any hand-labeled dataset. Six agents of DDQN provides actions for beam with, beam depth, bottom rebar size, number of bottom rebar, top rebar size, and shear stirrup size, respectively. Six agents of DDQN were trained for 5,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases that is not used for training. Based on this study, it can be seen that the multi-agent RL algorithm can provide successfully structural design results of doubly reinforced beam.

MES(manufacturing execution system) plays a critical role in improving production efficiency by managing operations across the entire manufacturing system. Conventional manufacturing systems employ a centralized control structure, which has limitations in terms of the flexibility, scalability and reconfigurability of the manufacturing system. Agent-based manufacturing systems, on the other hand, are better suited to dynamic environments due to their inherent high autonomy and reconfigurability. In this study, we propose an agent-based MES and present its collaboration model between agents along with a data structure. The agent-based MES consists of three types of core agents: WIPAgent, PAgent(processing agent), and MHAgent(material handling agent). The entire manufacturing execution process operates through collaboration among these core agents, and all collaboration is carried out through autonomous interactions between the agents. In particular, the order-by-order dispatching process and the WIP(work-in-process) routing process are represented as respective collaboration models to facilitate understanding and analyzing the processes. In addition, we define data specifications required for MES implementation and operation, and their respective structures and relationships. Moreover, we build a prototype system employing a simulation model of an exemplary shop-floor as a simulation test bed. The framework proposed in this study can be used as a basis for building an automated operating system in a distributed environment.

Recently, in the manufacturing industry, changes in various environmental conditions and constraints appear rapidly. At this time, a dispatching system that allocates work to resources at an appropriate time plays an important role in improving the speed or quality of production. In general, a rule-based static dispatching method has been widely used. However, this static approach to a dynamic production environment with uncertainty leads to several challenges, including decreased productivity, delayed delivery, and lower operating rates, etc. Therefore, a dynamic dispatching method is needed to address these challenges. This study aims to develop a reinforcement learning-based dynamic dispatching system, in which dispatching agents learn optimal dispatching rules for given environmental states. The state space represents various information such as WIP(work-in-process) and inventory levels, order status, machine status, and process status. A dispatching agent selects an optimal dispatching rule that considers multiple objectives of minimizing total tardiness and minimizing the number of setups at the same time. In particular, this study targets a multi-area manufacturing system consisting of a flow-shop area and a cellular-shop area. Thus, in addition to the dispatching agent that manages inputs to the flow-shop, a dispatching agent that manages transfers from the flow-shop to the cellular-shop is also developed. These two agents interact closely with each other. In this study, an agent-based dispatching system is developed and the performance is verified by comparing the system proposed in this study with the existing static dispatching method.

In the manufacturing industry, dispatching systems play a crucial role in enhancing production efficiency and optimizing production volume. However, in dynamic production environments, conventional static dispatching methods struggle to adapt to various environmental conditions and constraints, leading to problems such as reduced production volume, delays, and resource wastage. Therefore, there is a need for dynamic dispatching methods that can quickly adapt to changes in the environment. In this study, we aim to develop an agent-based model that considers dynamic situations through interaction between agents. Additionally, we intend to utilize the Q-learning algorithm, which possesses the characteristics of temporal difference (TD) learning, to automatically update and adapt to dynamic situations. This means that Q-learning can effectively consider dynamic environments by sensitively responding to changes in the state space and selecting optimal dispatching rules accordingly. The state space includes information such as inventory and work-in-process levels, order fulfilment status, and machine status, which are used to select the optimal dispatching rules. Furthermore, we aim to minimize total tardiness and the number of setup changes using reinforcement learning. Finally, we will develop a dynamic dispatching system using Q-learning and compare its performance with conventional static dispatching methods.

The threat of North Korea's long-range firepower is recognized as a typical asymmetric threat, and South Korea is prioritizing the development of a Korean-style missile defense system to defend against it. To address this, previous research modeled North Korean long-range artillery attacks as a Markov Decision Process (MDP) and used Approximate Dynamic Programming as an algorithm for missile defense, but due to its limitations, there is an intention to apply deep reinforcement learning techniques that incorporate deep learning. In this paper, we aim to develop a missile defense system algorithm by applying a modified DQN with multi-agent-based deep reinforcement learning techniques. Through this, we have researched to ensure an efficient missile defense system can be implemented considering the style of attacks in recent wars, such as how effectively it can respond to enemy missile attacks, and have proven that the results learned through deep reinforcement learning show superior outcomes.

실제세계에서 데이터 수집의 비용과 한계를 고려할 때, 시뮬레이션 생성 환경은 데이터 생성 과 다양한 시도에 있어 효율적인 대안이다. 이 연구에서는 Unity ML Agent를 로그라이크 장 르에 적합한 강화학습 모델로 구현하였다. 간단한 게임에Agent를 이식하고, 이 Agent가 적을 인식하고 대응하는 과정을 코드로 작성하였다. 초기 모델은 조준사격의 한계를 보였으나 RayPerceptionSensor-Component2D를 통해 Agent의 센서 정보를 직접 제공함으로써, Agent가 적을 감지하고 조준 사격을 하는 능력을 관찰할 수 있었다. 결과적으로, 개선된 모델 은 평균3.81배 향상된 성능을 보여주었으며, 이는 Unity ML Agent가 로그라이크 장르에서 강화학습을 통한 데이터 수집이 가능함을 입증한다.

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.