Ultrashort pulses of microwave radiation have been produced in a dielectric-lined Cherenkov free-electron maser (FEM) amplifier. An intense initial seed pulse, due to coherent spontaneous emission (CSE), arises at the leading edge of the electron pulse. There is evidence to show that 3-4 cycle spikes are produced through the amplification of these seed pulses. A strong dependence of the start-up power on the rise time of the electron pulse has been found. The experimental results are verified by a theoretical analysis. Our study shows that amplification in a FEM amplifier is always initiated by CSE arising from the edge of the electron pulse when the rise time is comparable to the electromagnetic wave period.