I love this old story:
Bandits were roaming the countryside. Everyone was afraid and went into hiding. Except one man who went about his day. When cornered by the bandits, they ask if he knew who they were. he said yes, he knew. The leader said, "Do you know I can run you through without flinching?" The man replied, "Do you know i can be run through without flinching?" The bandits left.
One of you must be afraid. Get off the line (don't take it personally), and let it be them.
IMHO, O'Sensei didn't have to be intimidated because of his skill, fearless.