In this study, nitric acid oxidation with varied treatment temperature and time was conducted on the surfaces of polyacrylonitrile- based ultrahigh modulus carbon fibers. Scanning electron microscopy, X-ray photoelectron spectroscopy, Raman spectroscopy and surface tension/dynamic contact angle instruments were used to investigate changes in surface topography and chemical functionality before and after surface treatment. Results showed that the nitric acid oxidation of ultrahigh modulus carbon fibers resulted in decreases in the values of the crystallite thickness Lc and graphitization degree. Meanwhile, increased treating temperature and time made the decreases more obviously. The surfaces of ultrahigh modulus carbon fibers became much more activity and functionality after surface oxidation, e.g., the total surface energy of oxidized samples at 80 °C for 1 h increased by 27.7% compared with untreated fibers. Effects of surface nitric acid oxidation on the mechanical properties of ultrahigh modulus carbon fibers and its reinforced epoxy composites were also researched. Significant decreases happened to the tensile modulus of fibers due to decreased Lc value after the nitric acid oxidation. However, surface treatment had little effect on the tensile strength even as the treating temperature and processing time increased. The highest interfacial shear strength of ultrahigh modulus carbon fibers/epoxy composites increased by 25.7% after the nitric acid oxidation. In the final, surface oxidative mechanism of ultrahigh modulus carbon fibers in the nitric acid oxidation was studied. Different trends of the tensile strength and tensile modulus of fibers in the nitric acid oxidation resulted from the typical skin–core structure.