It may not be OK to attack for the cited reasons, but it does happen. "What the FZZK you lookin' at M***** F*****!!!"
exactly what happend to me. years ago i was walking in a small town, didn't pay attention to things around me. i was looking down. i happened to look up as this man approach. he started to scream at me for looking at him and acted agressively. i circled a bit wider and kept my distance but sill looked at him. he walked off and still screamed loudly. learned a lesson on situational awareness.