You're in a tough spot. You put your life on hold, more or less, always there for everybody, and now they're uncomfortable that you want something for yourself. No one to make their sandwiches anymore. Fold their laundry. Wash the bathroom.
1) we would be apart
2) you would apparently be greatly enjoying yourself outside of my presence (meaning you enjoy my absence)
3) The comparison angle; how do I compare to the dojo guys, can I compete, can I show that I can protect the family
4) Traditional roles; (similar to # 3) I should be enough to protect you, and I certainly don't need you to protect me.
Even if all you get out of aikido is increased fitness, it is ultimately a benefit to them.
The above sounds just about right on!! I think maybe especially #2 and #3.
(note: I do still do the housework and the laundry, just not as efficiently as I used to) LOL I need to delegate more