Or I can steal theirs.
Lynn is spot on with this answer. Who was O-Sensei telling this to? It was a bunch of young deshi, early in their training. What did O-Sensei do? He looked them in the eyes and stole their spirit...
A correct understanding of this idea, in my opinion, is that you make the decision of whether you look him in the eye or not based on your assessment of threat. If I encounter a huge guy with jail tattoos and multiple pierced body parts, someone who is able to create fear in me, I will disconnect from him by going soft focus and not let him use the projection of his intention via his eyes to catch my mind. But if I look at the attacker and feel that I am, in fact, the more skilled one, the one who has less to fear, then I will do the exact opposite. I will look him in the eye and project my spirit into his mind.
Being able to do this can be a from of conflict resolution in itself. I have had a couple of times when the look was enough to have a potential attacker back off and go away. But it is a technique that must be used properly. It is not an aggressive technique in that it mustn't have aggressive emotion in it or it can cause a potential alpha dog type to feel he has to attack in order to maintain position or honor. It should be devoid of aggression but absolutely without fear. If you do not think that in a given situation you can achieve that, then don't look at his eyes.
It is my opinion that Aikido practice is about communication. So i teach my students to look at the eyes. After all, how do you feel about communication with someone who won't make eye contact? We have all sorts of associations with "meeting the eyes" and most are positive. So I teach that since Aikido is about "connection", we might as well connect this way as well in our practice. I'd like my students to be able to "steal the other guys spirit" and the won't develop that by looking away. But I make clear that, in a real self defense confrontation, my use of this technique would be totally dependent on my confidence that I would be better at it than the other fellow. Otherwise I wouldn't.