Suwari waza is tough on the knees.
Back when my knees and body could do suwari waza, 5 minutes on a regular basis (even as little as once a week) made an enormous difference in how tough it was on my knees. The other thing that made a difference - and this is a complement to Cliff's suggestion about finding the muscles that are tight or weak - was learning at some point to keep my weight over my feet and moving my feet through the center point. Before that, I tended to swing my feet around and "lean" onto my knees. Both of those mistakes made suwari waza much more work and much harder on various points of my body (like knees).
Ikeda Sensei's video/DVD on suwari waza (titled "Za") is a good resource.