I know I sound like a broken record, but I know exactly what you're talking about. I was a walking wreck when I did nothing but run, stretch and do Aikido. I took a little over a year off and did nothing but work on fitness.
For endurance I used primarily interval protocols with calisthenics, running, stair climbing, etc... For strength, I worked on building basic strength and muscle mass in the big movements: pullups, pushups, rows, overhead press, dips, squats, and deadlift variants. I went back to Aikido and the stresses are all well within functioning parameters. No injuries, no problems, no limitations. I haven't been able to train as consistently as before for situational reasons, but I've done a few marathon seminars and trips that I couldn't have even dreamed about doing before, with no fallout other than a couple days of soreness.
Yoga is nice, but it's not a holistic preparation for dynamic athletic activity. From your description, it sounds like you are simply engaged in an activity that is too demanding for your state of fitness and conditioning. Take a half-year off and get in shape, then try again. Sartre and Camus are optional.