I am not a little person ( 6'00" and 230 lbs), but one of the guys I regularly practice in strong as an ox and can/does stop me cold if I try to muscle my way through a technique. So I regularly seek him out on the mat.
One of my other favorite partners in less than half my size. If I muscle my way through technique, I send her flying. Since we are friends and unscheduled air travel is rude in Aikido, I must practice smooth and clean technique.
If I always practiced against like sized people, I would only learn how to defeat myself