Well, could be that it's because you are playing a friendly game with your friends. Let me explain what I mean. Many beginers (and I don't know if you are one or not) are taught shihonage in a nice controlled way in which uke's arm is taken straight back against them. This protects uke's arm and gives nage a chance to learn proper position. It also allows uke to take a nice easy backfall. Now, watch some old film of O'Sensei or Kanai, or the other shihan doing "at speed" shihonage. They are turning their hips and taking the arms sideways and back. If uke doesn't "get ahead" of the throw (which requires a high level of skill and knowledge of what's going on) their arm will at least be badly wrenched or perhaps dislocated.
Are you willing to dislocate your friend's arm?
Also, in judo or wrestling you start in a clinch. That's not where aikido starts. In fact that is a very unfavorable place to start for the aikidoka. Are you willing to punch your friend HARD in the face or at least deliver a punch with such vigor that they THINK you will punch them? That is one way to "lead" or divert their attention from the clinch and give you an opening to use a technique.
So, if you aren't willing to use aikido techniques fully, then in a friendly game of "let's wrestle" aikido may not work. Of course even if you are on the ground, and you put a good nikkyo on them, it will stop the match
My two cents