A physical attack requires a physical response. Words are just words, they bother us because we allow them too. They pose no physical threat. .
While I understand the distinction you're making, words CAN cause physical threat.
A kid (or even adult) bullied, threatened and stressed out (even without "being hit") can suffer physically.
The stress alone from an/constant emotional attacks can cause people to stop eating, stomach pains, throw up, even actual muscle pain.
One always has the option of walking away but sometimes even that won't protect someone from the physical effects of an emotional attack.
I think you're right about them hurting us when we allow it. Control that and presto.