the reason I take aikido lessons is to learn how to self defense.
I think this is a big part of your problem. The teaching in a traditional aikido dojo is not oriented toward modern street self-defense. Therefore, there's a disconnect between the aikido you are trying to perform and the aikido your instructor is teaching. In a sense, you and your partners are practicing different martial arts. No wonder you're frustrated.