The best-designed voltage regulator in the world is for nothing if power supply noise couples to the output signal directly.
That seems to be what's happening here. It's the current though the transformer secondaries which is inducing the noise I see. The shape of the noise pulse (spike up, spike down, 60 Hz cycle) shows the origin. I got careless and simply placed the transformers too close to the amplifier circuit.