The deep space orbit determination software (DSODS) is a part of a flight dynamic subsystem (FDS) for the Korean Pathfinder Lunar Orbiter (KPLO), a lunar exploration mission expected to launch after 2018. The DSODS consists of several sub modules, of which the orbit determination (OD) module employs a weighted least squares algorithm for estimating the parameters related to the motion and the tracking system of the spacecraft, and subroutines for performance improvement and detailed analysis of the orbit solution. In this research, DSODS is demonstrated and validated at lunar orbit at an altitude of 100 km using actual Lunar Prospector tracking data. A set of a priori states are generated, and the robustness of DSODS to the a priori error is confirmed by the NASA planetary data system (PDS) orbit solutions. Furthermore, the accuracy of the orbit solutions is determined by solution comparison and overlap analysis as about tens of meters. Through these analyses, the ability of the DSODS to provide proper orbit solutions for the KPLO are proved.