Actually, Saito-shihan once said that O-sensei made him wear a hakama upon the first day of traning. Saito-shihan was too poor to purchase one so he tore the fabric off an old couch, had it sewn into the hakama and dyed it black. He went on to say that the dye came off on the dojo mats!
Since when did the hakama, which is nothing more than apprpriate attire for the Japanese male way back when, become a symbol of presitge and inclusioin??? Who is the one who sai "only black belt can wear this"? Its like someone telling me that I cannot wear my pants!!!
The first 4 years of my training was with the AAA. They do not allow anyone under shodan to waer the hakama. It has its pros and cons.