Here in Ireland, most people start wearing a hakama between second kyu and shodan. If a hakama is an ego trip for somebody, they have an ego, and giving hakama to everybody won't stop that. However, here the hakama serves the very useful purpose of letting beginners know who they can learn most from, and even for visiting instructors to see who they can best demonstrate with.
Also, in clubs where the hakama isn't worn by beginners, it begins to carry with it certain responsibilities. The wearer is sending out the message that he can take good ukemi, and you can take this into account in your training with them.
However I'd have to agree with mikey when he said:
"It is probably best to not give the matter too much weight, and when in Rome do as the Romans."