This study aimed to estimate the effects of replacing Mushroom By-Product (MBP) with Tofu By-Product (TBP) on the chemical composition, microbes, and rumen fermentation indices of Fermented Diets (FDs). The basal diet was formulated using MBP, TBP, rice bran, molasses, and inoculants. The MBP in the basal diet was replaced with TBP at 0, 5, and 10% on Dry Matter (DM) basis for the experimental diets. The experimental diets were fermented at 39°C for 144 h. Chemical composition, pH, microbes, and rumen fermentation indices of the FDs were analyzed. With increasing TBP replacement, crude protein content of FDs increased (L, P < 0.001), whereas crude ash content decreased (L, P = 0.002). Lactic acid bacteria and Bacillus subtilis contents in the TBP-replaced FDs were higher than those in the control (P < 0.05), whereas pH level and mold count were lower (P < 0.05). With increasing TBP replacement, in vitro rumen digestibility of DM (L, P = 0.053) and neutral detergent fiber (L, P = 0.024) increased, wheres rumen pH changed (P = 0.026) quadratically. Rumen total volatile fatty acid (L, P = 0.001) and iso-butyrate contents (Q, P = 0.003) increased with increasing TBP replacement. In conclusion, this study indicates that the replacement of MBP with TBP could improve the quality of FD.