It's not the belt per se that makes it noisy. It's the dry clutch. The plates rattle around and with no oil to dampen the noise, that's what you hear. I personally do mind the sound and have thought about doing it, but my clutch is set-up perfect for my current motor, so I really don't want to mess with it.
oops, didn't see QuickHD already posted the same thing.