As is the answer with most questions, the answer to this would be, "It depends."
My current dojo has people with dan rankings wearing a black belt and those who have yet to reach shodan (1st degree black belt) wearing white belts. We allow pretty much anyone to wear a hakama, although people usually wait until after their first test (5th kyu).
However, I've also trained at places where they wear colored belts in the adult system. Most of these places wear a white belt at 6th and 5th kyu, a blue belt at 4th and 3rd kyu, and a brown belt at 2nd and 1st kyu. I've also been to dojo where they had even more colors reminiscent of my days in karate (yellow, orange, purple, blue, green, red, brown)...
Some dojo reserve hakama at shodan, others somewhere below that, and others allowing people to wear hakama from the beginning. Other aikido dojo (eg Yoshinkan) hardly ever wear a hakama -- even the teachers -- outside of a large, public demonstration.
To top it all off, I've also heard of organizations wherein people wear different colored hakama according to their rank (including a white hakama near the beginning of their training).
So, in my experience at least, it all really depends on the organization and the dojo on what type of "belt" system a dojo uses...
Hope that helps,