Graphene fiber is considered as a potential material for wearable applications owing to its lightness, flexibility, and high electrical conductivity. After the graphene oxide (GO) solution in the liquid crystal state is assembled into GO fiber through wet spinning, the reduced graphene oxide (rGO) fiber is obtained through a reduction process. In order to further improve the electrical conductivity, herein, we report N, P, and S doped rGO fibers through a facile vacuum diffusion process. The precursors of heteroatoms such as melamine, red phosphorus, and sulfur powders were used through a vacuum diffusion process. The resulting N, P, and S doped rGO fibers with atomic% of 6.52, 4.43 and 2.06% achieved the higher electrical conductivities compared to that of rGO fiber while preserving the fibrious morphology. In particular, N doped rGO fiber achieved the highest conductivity of 1.11 × 104 S m−1, which is 2.44 times greater than that of pristine rGO fiber. The heteroatom doping of rGO fiber through a vacuum diffusion process is facile to improve the electrical conductivity while maintaining the original structure.