Smart materials capable of changing their characteristics in response to stimuli such as light, heat, pH, and electric and magnetic fields are promising for application to flexible electronics, soft robotics, and biomedicine. Compared with conventional rigid materials, these materials are typically composed of soft materials that improve the biocompatibility and allow for large and dynamic deformations in response to external environmental stimuli. Among them, smart magnetic materials are attracting immense attention owing to their fast response, remote actuation, and wide penetration range under various conditions. In this review, we report the material design and fabrication of smart magnetic materials. Furthermore, we focus on recent advances in their typical applications, namely, soft magnetic actuators, sensors for self-assembly, object manipulation, shape transformation, multimodal robot actuation, and tactile sensing.