Anomaly detection of Machine Learning such as PCA anomaly detection and CNN image classification has been focused on cross-sectional data. In this paper, two approaches has been suggested to apply ML techniques for identifying the failure time of big time series data. PCA anomaly detection to identify time rows as normal or abnormal was suggested by converting subjects identification problem to time domain. CNN image classification was suggested to identify the failure time by re-structuring of time series data, which computed the correlation matrix of one minute data and converted to tiff image format. Also, LASSO, one of feature selection methods, was applied to select the most affecting variables which could identify the failure status. For the empirical study, time series data was collected in seconds from a power generator of 214 components for 25 minutes including 20 minutes before the failure time. The failure time was predicted and detected 9 minutes 17 seconds before the failure time by PCA anomaly detection, but was not detected by the combination of LASSO and PCA because the target variable was binary variable which was assigned on the base of the failure time. CNN image classification with the train data of 10 normal status image and 5 failure status images detected just one minute before.

This paper describes the development of algorithm for direct data transmission between Raw VLBI Data Buffer (RVDB) and Huge Capacity Data Server (HCDS) operated in Korea-Japan Correlation Center (KJCC). The transmitted data is the VLBI observation data, which is recorded at each radio telescope site, and the data transmitting rate is varying from 1 Gbps, in usual case, upto 8 Gbps. The developed algorithm for data transmission enables the direct data transmission between RVDB and HCDS through 10 Gbps optical network using VLBI Data Interchange Format (VDIF). Proposed method adopts the conventional UDP/IP protocol, but in order to prevent the loss of data during data transmission, the packet error monitoring and data re-transmission functions are newly designed. The VDIF specification and VDIFCP (VDIF Control Protocol) are used for the direct data transmission between RVDB and HCDS. To validate the developed algorithm for data transmission, we conducted the data transmission from RVDB to HCDS, and compared to the transmitted data with the original data bit by bit. We confirmed that the transmitted data is identical to the original data without any loss and it has been recovered well even if there were some packet losses.

데이터는 살아 움직이고 있다. 10년 전만 해도 10GB 정도의 데이터라면 대용량 데 이터라고 불리던 시절이 있었다. 하지만 지금은 10TB보다 큰 데이터베이스도 흔하다. 결국, 대용량 데이터베이스(VLDB)의 시대가 개막된 것이다. VLDB로 변한 데이터베이스에는 백업, 복구, 관리와 같은 문제점이 있지만 그 중에 서도 성능 문제를 빼놓을 수 없다. 데이터베이스에 많은 데이터가 있고 그렇게 많은 데이터 중에서 필요한 몇 건의 데이터만 추출하는 것이 쉬운 일이 아니다. 과거에는 데이터가 적었기 때문에 이러한 것이 큰 문제가 아니었지만 이제는 VLDB가 되면서 성능 최적화는 일상적이고도 중요한 이슈가 되었다. 따라서 VLDB가 된 데이터베이스 나 VLDB로 변하고 있는 데이터베이스에서 성능 관리를 하고 최적화할 수 있는 전문 기술이 필요하다.

In this paper, we introduce the development of the large storage system in order to record the observed space radio signal in the Korean VLBI Network(KVN) with high-speed. The KVN is the Very Long Baseline Interferometery(VLBI) to observe the birth of star, the structure of space by constructing radio telescope with diameter 21m at the Seoul, Ulsan, Jeju from 2001 to 2007 years. To do this, Korea Astronomy Observatory joined the international consortium for developing the high-speed large storage system(Mark 5), which is developed by MIT Haystack observatory. The Mark 5 system based on hard disk has to record up to 1 Gbps the observed space radio signal. The main features of Mark 5 system are as follows; First it is able to directly record the input data to the hard disk without PC1(Peripheral Component Interconnect) internal bus, and the second, it has two hard disk banks, which are able to hot-swap ATA/IDE type very cheap up to 1 Gbps recording and playback. The third is that it follows the international VLBI standard interface hardware(VSI-H). Therefore it can be connect directly the VSI-H type system at the input/output. Finally it also supports e- VLBI(Electronic-VLBI) through the standard Gigabits Ethernet connection.