I started when I was 39 and I'm 49 now. My sensei is in his mid 50's and still takes ukemi. One thing I have noticed about students who are older than say about 45 is that they are more tentative about learning ukemi, which is, I think, natural. You might want to pay serious attention to the quality of the mats in the various dojo you visit. However, assuming you don't enroll in a dojo that has 30 year old straw tatami that are worn out, and the mats are good, you should not be too tentative while learning ukemi (falling). Being tentative will make you tight, which will make your ukemi very angular, which will casue more aches and pains. You should expect a few aches and pains as you learn to fall, but if you are being taught well, and are diligent about practicing ukemi, you should not experience injury. DON'T try and do what the 20 year old kids do. You'll see them take big falls and land hard sometimes, and they can do that. They won't like it when they are older though and may have to relearn ukemi.
I think fear of falling is the largest thing holding back new older students.
My two cents