If they give even a little energy (and real attacks always have energy) then it is very easy to move them. If you connect with Uke as if to do irimi Nage or Ikyo and move around him to the back in ura far enough this movement alone will be enough to break his balance.
Go ahead, try it. Find a person with training time equivalent to your own from the striking art of your choice and perform ikkyo or irimi nage against a tsuki attack. You may limit their initial attack to a straight punch to your choice of targets, but may not otherwise constrain their movement, including followup strikes, if any. (That is, you can't tell them to behave like an aikido-trained uke would.) Please report your experiences, with video.
Alternatively, find an equally competent judoka and do the same experiment from a basic shoulder grab.
I'm not saying you can't. I'm not saying you can. I don't know you. I just want to know what happens, and how you experience the connection between you and your partner.