Excellent Question and Point of Interest
I think this is an excellent question you have asked. It's something that I'm pretty sure has crossed the minds of all aikidoka at one point or another concerning what constitutes martial arts as well as what consitutes self-defense.
My own opinion on the matter depends highly on the issue of what you're looking to do with your martial skills. Sure, every technique that is praticed in aikido, tai chi, tong soo do, or judo can be seen as self-defense. Yet self-defense often implies the rationalization that you're learning certain technique to prepare solely for the worst case senario (a fight). If in fact you are training for that type of situation then according to nearly all high ranking martial artists and their predecessors, you are missing the point completely.
That point being that we are training in martial arts with the understanding that we should never have to use them. And that through our training and our practice we come to understand that it is not the application of techniques on other people that martial artists primarily train for, but the ability to reframe tough situations, as well as harmonize themselves through certain movements which is truly unique and special.
To reframe this idea a bit more, O Sensei is quoted as having said "Aikido is musogi (cleansing of the body/soul)." It is my personal feeling that is the goal of nearly all martial arts, not to flood the mind with thoughts of emminent conflict, but to clarify it through training so we never reach the point of phsycial conflict.