Carbon fibers (CFs) with different tensile moduli of 280–384 GPa were applied to investigate the relationship between crystalline structure and compressive failure. The carbon chemical structure and crystalline structure were studied by Raman, highresolution transmission electron microscopy (HRTEM) and X-ray diffraction (XRD). The correlation between compressive strength and crystalline structure was investigated. The results showed that the transition point between medium and high tensile modulus was around 310 GPa, and within the range of medium modulus, the compressive strength of CFs improved with the increase of tensile modulus, and the compressive strength also improved with the increase of crystal thickness Lc, crystal width La, and crystal plane orientation; In the high modulus range, the correlation law was opposite, which was mainly influenced by the grain boundary structure. CFs with tensile modulus lower than 310 GPa exhibited bucking and kinking fracture under compressive loading, while shear fracture was observed for CFs with tensile modulus higher than 310 GPa.