Don't translate energy as voodoo magical rainbow powers. Force and energy are also used in mundane ways.
Say, I go to do Iriminage. As I lead someone down, the person falls on in the fetal position and pulls my arm down. The throw I am supposed to be practicing is to follow someone standing back up. I can try to powerlift my partner, but I am not training for progressive resistance. I am training to feel where gravity, their own muscles, and their own intention wants to go. If I start to follow the resistance and feeling of force on force, I get worse. But, my partner can maybe give me some strength training.
Later on, we're doing free style. Someone is used to stuffing my practice by bearing down. But, now the rules don't apply to me. My partner pushes down, and I accelerate my partner's head at the ground. Or, it's on the street and I will give them a mouthful of my knees and then the pavement for dessert. In my experience, the partner who can be a nuisance during kata practice is constantly scared or getting injured in Randori. In one of the last seminars I went to, a big white belt moose was stuffing as many people as he could. He didn't know the difference between strong and brittle, and spent the last hour of the day with shoulder and neck injuries.
Make this about any other martial art. Someone wants to practice boxing with you, but all they do is kick you in the shin. Are you learning boxing? Or someone who wants to practice self defense stuff, and they are taking the role of someone who is trying to kill you, but they don't attack you - they run away and hide. How is your practice going? Are you getting ready for the guy who really does want to kill you in real life?
The first rule for practice from O Sensei said any of these techniques can kill, so be mindful and follow instructions.
Shioda Sensei had five specific ways for Uke to attack for Kokyu Doza - pushing, pulling, stiffening on the spot, crushing and closing, big circular lead. If someone is supposed to be practicing push, it doesn't help them to learn push if you pull instead. It's like trying to teach someone math by having them recite Chaucer. There are rules to the game in every sport and activity, and we have them too.