I agree that comparing is something that we naturally do. I'm in a dojo with no rank system and no colored belts. All i know is i've been learning for 2 years and so-and-so has for .. years etc. But at the end of the day we do discuss and make "ranks".
We'd make comments such as
"Oh, A is getting really good.", "B's flow is much better now.", or "C seems to be having trouble in ... area."
So i think we unconsciously know who's better in what area and who's not, even if we don't know what rank we are.
But on the question whether rank really matters, well unfortunately we live in the world where we need money to eat and need a system to run an organization. So, imo depends on how you want your aikido to grow.