I'd say it's all inside you... although any force that your opponent provides can and will be used against him.
Well duh, it is called Aiki (in this context harmony/joining/fitting in with applied force/power/energy) and yeah you should use whatever force is given against the other person if you're good enough
That being said and more generally speaking to the topic of the thread, I think again a few things are being conflated - there's the body conditioning and skill building (including sensitivity to forces inside you as well as how forces outside you will be managed by ... the forces inside you) to develop the baseline internal strength, then there's the skill building of applying said things against a dumb force, then there's the skill building of applying said things against more intelligent forces. I generally subscribe to the theory that it takes a lot more conditioning and work than most people are generally willing to do so what comes is a lot of people thinking jin manipulation is the end-game working with bodies that are doing some local muscley version of it.
Kuzushi on contact is a combination of skill/conditioning/application. You have to do the work in the first two to enable the third, but you can "cheat" in your combination of all three to compensate for lacking in any single area. The question is how important is it to excel in all three or do you just care if the other person falls down by any means necessary?