You are within your bounds to refuse to train with anyone who physically hurts you, or disrespects you.
However, I've dealt with the good old 5th kyu shihan syndrome. I know men who have the same complaint, so it breaches gender. But I understand how it is hard for a woman, being a woman. You have to pop your hips a little more and take ukemi faster. There are always be low ranked males waltzing in who will make you prove your rank to them. :/
I dealt with one male week 1 student(who no longer trains). I bowed to work with him, he refused me and said "I don't want to hit or throw a woman!" For a split second I was gonna accept his refusal, but then I got mad and told him "What makes you think you could pull off either in your first week?" I made him train.
In my region woman can wear hakama at an earlier stage than men. Men have to wait until black belt. I always feel bad for 1st kyu men at seminars, being told how to do something by 5th kyu shihan.
Hi Maggie - it's frustrating because the time this man takes to talk nonsense, is less time we have to train. You are absolutely right about Ukemi. When I train with sempai, I never complain...They generally know what kind of falls I'm better at, or worse at. They don't try to hurt me and I respect their bodies as well. It's always the new students who come in rough and rugged and full of bravado...I did too, when I first started. I didn't know any better and I got offended if someone tried to tell me. It wasn't until I trained regularly and seriously that I began to understand what people tried to tell me. Thankfully, they were gracious to me and accepted my apologies when I presented them.