I tried your patch but did not encounter this behavior when switching between Mono and Poly. Were you using the standalone synth built into Syntorial or the standalone plugin? VST or AU?
This was my attempt using Tal-NoiseMaker. I actually started my attempt using a different synth, but wasn’t getting it close enough to the sound as the other synth didn’t have controls over the pulse width.