A thorough knowledge and understanding of the structure–property relationship between thermal conductivity and C-fiber morphology is important to estimate the behavior of carbon fiber components, especially under thermal loading. In this paper, the thermal conductivities of different carbon fibers with varying tensile modulus were analyzed perpendicular and parallel to the fiber direction. Besides the measurement of carbon fiber reinforced polymers, we also measured the thermal conductivity of single carbon fibers directly. The measurements clearly proved that the thermal conductivity increased with the tensile modulus both in fiber and perpendicular direction. The increase is most pronounced in fiber direction. We ascribed the increase in tensile modules and thermal conductivity to increasing anisotropy resulting from the orientation of graphitic domains and microvoids.