This study was conducted to re-establish the withdrawal time (WT) of ivermectin (IVM) in goats in accordance with the implementation of the positive list system (PLS). Thirty-four healthy goats were topically administered IVM at doses of 0.5 mg/kg body weight (BW) (IVM-1, n = 16) or 1.0 mg/kg BW (IVM-2, n = 16) as a single treatment. Tissue samples were collected from four goats at 3, 7, 14, and 28 days post-administration. Residual concentrations of IVM in edible tissues were determined using LC-MS/MS based on a previously validated analytical method. In the IVM-1 group, IVM concentrations in all edible tissues were below the limit of quantification (LOQ) by day 3 post-administration. In the IVM-2 group, IVM concentrations in muscle, liver, kidney, and fat were below the LOQ by days 7, 3, 3, and 7 post-administration, respectively. WTs were estimated in accordance with the European Medicines Agency guideline on the determination of withdrawal periods. The calculated WTs for IVM-1 and IVM-2 were 12 and 19 days, respectively. In conclusion, the estimated WT of IVM in edible goat tissues was shorter than the currently recommended withdrawal period of 28 days.