Changes in the earth's climate depend on changes in the net sunlight reaching us. The net depends on the sun's output and earth's reflectance, or albedo. Here we develop the limits on the changes in the sun's output in historical times based on the physics of the origin of solar cycle changes. Many have suggested that the sun's output could have been 0.5% less during the Maunder minimum, whereas the variation over the solar cycle is only about 0.1%. The frequencies of solar oscillations (f- and p-modes) evolve through the solar cycle, and provide the most exact measure of the cycle-dependent changes in the sun. But precisely what are they probing? The changes in the sun's output, structure and oscillation frequencies are driven by some combination of changes in the magnetic field, thermal structure and velocity field. It has been unclear what is the precise combination of the three. One way or another, this thorny issue rests on an understanding of the response of the solar structure to increased magnetic field, but this is complicated. Thus, we do not understand the origin of the sun's irradiance increase with increasing magnetic activity. Until recently, it seemed that an unphysically large magnetic field change was required to account for the frequency evolution during the cycle. However, the problem seems to have been solved (Dziembowski, Goode & Schou 2001) using f-mode data on size variations of the sun. From this and the work of Dziembowski & Goode (2003), we suggest that in historical times the sun couldn't be much dimmer than it is at activity minimum.