Well, I'll toss in my 2 cents and say I've trained with sensei who didn't want you going anywhere else. That's fine depending on what you're doing. I find it odd in the context of Aikido, however, as it is such a big, wide, fluttering tent of things. And I can only think of a handful of people out there where I'd even want to do such a thing in that world.
In the world of koryu arts, well, that's a different issue with different factors.
I will also say the ones I've met who were adamant about "go no where else" were the ones who were most, um, how to say it, haunted by self-doubt themselves. I enjoy cross training. I enjoy getting out and learning new things. I do, however, recognize that if you're a teacher as well you need to be able to tread softly on issues of style specific methods and approaches, if for no other reason for the sake of the students.
But all that said I'm still somewhat baffled when I hear of people doing that. If you feel the need to go outside, well, it is your life and your training. If your concern with your sensei's position on the topic overrides that, well, that's a decision for you to make and not random strangers on the internet.
So I'm not sure how anyone else is going to help you on this one.
Best of luck.