Maybe I'm off base. Maybe I'm not aiki/harmonious, but here's my opinion:
Your body and mind are the only things you truly own. You have the right to protect them. You have the right to refuse training with someone you think is dangerous.
If I were in this situation, I wouldn't attend classes if he instructed. If he was a student in the same class, I wouldn't partner with him at all. I also would not feel bad about out. Would I be showing bias against him- Your darn right! But my opinion would change if his behaviors changed.