Maybe I am thinking of fearing hurting someone differently than most...
To me, there is the knowledge that done incorrectly harm can be done (driving drunk is stupid, road rage is stupid) but with that the responsibility to drive correctly and sober. I have a lot more control on the mat than I do in my car (in some ways another good anology as I drive a sports car in an area filled with SUVs).
OTOH, I don't, to my knowledge, throw hard (to me different from 'train hard'), I try to give no 'added value' to the attacker's energy---my goal is to use their force, not mine. Luckily, many with ukemi that could be greatly improved are beginners, who either do not put a lot of energy into an attack, or at least keep in mind the warning I give them of 'don't attack harder or faster than you can fall'. There's one large senior kyu in this group--- she's to me the equivalent of a car dangerously out of control on the road, but on the mat I can limit her damage much more easily. I leave her to the big blackbelts, or work with her when I know it's an easy fall, or just let her throw me.
At my first dojo, if a collision occured sensei took to task the two most senior students in each pair, regardless of who threw whom. I think this was good in getting people to better watch their surroundings. If you were uke, you kept SA (situational awareness) and didn't let your kohai partner toss you into someone. If I am working with a beginner, especially on a crowded mat, I try to use a corner, and throw to the outside, since they can occasionally land in an unexpected place, and sometimes get up slowly and with poor SA.
I try to maintain awareness of my partner, and what his body is telling me, at all times. If I feel the slack is out of his joint and he's fighting the technique (beginners and sankyo are the only time this has happened with me), I let him go. I figure if I had it correctly, he'd get hurt. If I had it wrong, it wasn't going to work at that point and a different approach was needed anyway.
If he's a beginner and I think he just didn't know he was in danger from the joint lock, rather than do the technique I offer to show him how to avoid pain in that technique, put him gently into it, tell him how to get the pain off (eg, 'face your center to mine and get it lower and closer' nikkyo), put more on, have him readjust a few times, then start again and have him slowly move in the wrong direction (the one usually found so naturally by beginners). I count it as my four turns, and a great investment in a partner who won't do something dangerous the next time.
I do atemi slowly at beginners (and the occasional heavy footed senior). The kind of 'space saver' atemi, that just keeps uke from moving into me, I don't worry about with anyone. Only really big guys run their faces into my hand, and I figure it can't hurt or they wouldn't do it so often.
So I just don't see how someone is going to get hurt at my hands. I've had large friends point out that big guys worry about it because of their size...maybe that is so, but I think if they keep in mind the little ukes are generally faster and more flexible, they'd realize how difficult it is to damage us. If I drove too fast, hogged the road, changed lanes erraticly, didn't pay attention...yes, I should be afraid of crashing my car. But if I go no faster than road conditions allow, allow for stupid moves on other drivers part, don't fight over the right of way, stay alert... well, many decades of driving and no crashes. I think my Aikido life will go about the same.