Segregated composites, where fillers are selectively placed at the matrix interface to form a segregated filler network, are attracting attention because they can provide excellent conductive properties at low filler content. In this study, the anisotropic enhancement in thermal conductivity of composites was discovered due to the unique structure of the segregated network. The segregated composites were produced using a typical mechanical mixing of matrix pellets and the internal structure was precisely analyzed using three-dimensional non-destructive analysis. The segregated composites slightly improved in the through-plane thermal conductivity, but the in-plane thermal conductivity increased rapidly, showing the anisotropic thermal conductivity. The maximum improvement in the in-plane thermal conductivity of the segregated composites increased by 112.5 (at 7 wt% graphene nanoplatelet) and 71.4% (at 10 wt% multi-walled carbon nanotube), respectively, compared to that of the random composites filled with the same amount of filler. On the other hand, the electrical conductivity of the segregated composites was isotropic due to the difference in the transport mechanisms of electrons and phonons. The anisotropic thermal conductivity developed by the segregated network was helpful in inducing effective heat dissipation of commercial smartphone logic boards.