The need for high-performance environmental remediation has increased due to the environment’s ongoing degradation in the form of significant growth in industrialization and urbanization. Therefore, the toxic heavy metals can easily enter into environmental as well as foods and thus the search of clean water for drinking, household and irrigation purposes is of crucial importance. To meet this challenge, microelectrodes are flexible, low-cost and easier for fabrication has become the strong role in the detection of heavy metals with high sensitivity towards higher adsorption of heavy metals from contaminated water. To improve the sensitivity of the microelectrodes, carbon-based microelectrodes decorated with nanomaterials have been explored for the detection of metal ions thereby their presence in trace levels can be estimated. The aim of the present review is to summarize the recent developments in carbon-based microelectrodes for the electrochemical determination of heavy metals. It is followed by the various nanomaterials decorated on the carbon microelectrodes for detection of heavy metals was systematically discussed. Finally, the application and the future perspectives in the development of smart electrochemical sensing is provided. This short review will provide the useful information for the recent development in microelectrodes and also guide the pathway for the detection of heavy metals.