....Perhaps it is because when one is in an environment free of outside distractions such as music they come face to face with something that they don't want to really face up to... themselves.
That assumes, and I can understand why people would think this way, that extraneous sights and sounds are most likely to get in the way of the practice of Aikido and the seeking of enlightenment. Just for an exercise however, and despite the risks you mention, try a thought experiment. Start with the thought that there may be something external to nage and uke that could help improve their aikido and their seeking after enlightenment. I've read that OSensei believed sound vibrations can connect us with heaven. Is absolute silence the only way we will ever get a chance to hear the sounds of heaven? On the other hand, maybe the sacred music of the universe can be heard even with the radio on and the right music playing. Just a thought.