Aikido isn't violent, I am. So if I eliminate "I," there is no one to become violent. This is one of the basic purposes of zen meditation.
The Buddhists recommend certain types of "medicine" for certain ailments. For anger, which leads to violence, they recommend meditation.
At the moment of confrontation in a street fight, all emotions interfere with clear thinking. This includes love, hate, anger, etc. Remaining calm and clear allows one to remain in the ultimate state of readiness.
I've been in countless confrontations where a potential attacker is enraged. He is waiting for and expects a reaction from me. When none comes, he starts to become confused, he babbles, he doesn't know how to respond. Like a brief, but violent thunderstorm, he starts to blow himself out. The confrontation ends because the equation is not 1 plus 1, but 1 plus 0. He, therefore, has nothing to fight. What I appear to be and what I really am are two different things.
If and when he attacks, he's easily thrown because by not being in control of his emotions, he's easily unbalanced.
Even if he did attack, say with a knife, the same principle would apply. I would remain calm and clear — no "I," no fear — a void. Even if I broke his arm in self-defense, I would be calm, clear, and not violent.
An onlooker may say the action was "violent," but my essential nature would not be violent, because "I" would not exist. "I" simply interferes with my clarity of mind, so therefore, there is no use for it.
P.S. In normal conversation, "I" is simply a manner of speaking, but in fact it is an illusion and in no way describes true reality. Until a person discovers their essence, they can never really know what this reality is. In the meantime, as far as I know, zen meditation is the best course of action.