Low thermal conductivity carbon fibers from polyacrylonitrile (PAN) are currently being explored as an alternative for traditional rayon-based carbon fibers with a thermal conductivity of 4 W/m K. Compared to multiple component electrospinning, this research demonstrated another feasible way to make low thermal conductivity carbon fibrous material by electrospinning PAN followed by carbonization and alkali activation. The effects of activation condition on microstructure, pore formation, and thermal conductivity of the resultant carbon nanofibrous material were investigated. The processing-structure-thermal conductivity relationship was revealed and mechanism of thermal conductivity reduction was discussed. The overall thermal conductivity of the prepared carbon nanofibrous material is a result of combined effects from factors of carbon structure and number of pores rather than volume of pores or specific surface area. The activated carbon nanofibrous materials showed thermal conductivity as low as 0.12 W/m K, which is a reduction of ~ 99% when compared to that of solid carbon film and a reduction of ~ 95% when compared to that of carbon nanofibrous material before activation.