We conduct a quantitative analysis of the star formation history (SFH) of the Milky Way's (MW) bulge by exploiting the constraining power of its stellar [Fe/H] and [Mg/Fe] distribution functions. Using Apache Point Observatory Galactic Evolution Experiment survey data, we confirm the previously established bimodal [Mg/Fe]-[Fe/H] distribution within 3 kpc of the inner Galaxy. To fit the chemical bimodal distribution, we use a simple but flexible star formation framework, which assumes two distinct stages of gas accretion and star formation, and systematically evaluate a wide multidimensional parameter space. We find that the data favour a three-phase SFH that consists of an initial starburst, followed by a rapid star formation quenching episode, and a lengthy, quiescent secular evolution phase. The metal-poor, high-α bulge stars ([Fe/H] < 0.0 and [Mg/Fe] > 0.15) are formed rapidly (<2 Gyr) during the early starburst. The density gap between the high- and low-α sequences is due to the quenching process. The metal-rich, low-α population ([Fe/H] > 0.0 and [Mg/Fe] < 0.15) then accumulates gradually through inefficient star formation during the secular phase. This is qualitatively consistent with the early SFH of the inner disc. Given this scenario, a notable fraction of young stars (age <5 Gyr) is expected to persist in the bulge. Combined with extragalactic observations, these results suggest that a rapid star formation quenching process is responsible for bimodal distributions in both the MW's stellar populations and in the general galaxy population and thus plays a critical role in galaxy evolution.