View Full Version : Motion capture systems and martial arts

Please visit our sponsor:

06-02-2005, 12:34 PM
A client I work with will be shortly installing a motion capture system for their own use. I have free access to this facility whenever I need it (typical commerical joints charge US$1000 per hour or more).

Since I am a realtime 3D graphics geek by day and already spend a lot of time working within this realm, the idea of combining the job with my hobby is naturally quite an appealing thing. Specifically, one of my post-blackbelt activity interests is in the area of making some interactive 3D trainers/demo/something applications involving martial arts. Along with my teacher and whoever else I can con^H^H^H talk into helping out, I'll be doing a lot of recording of real techniques. Ideally, as part of this, I'd like to be able to illustrate the circular motion - it will be really easy to highlight given such data - just join the dots of each sample point with lines.

Apart from a few games, I've never seen this done before in the large scale. I'd thought I'd ask here to see if anyone has done motion capture work of martial arts techniques. Any information you can share would be greatly appreciated. As they haven't yet purchased the system, if you have specific recommendations for good systems that could handle the fast paced motions of martial arts (not necessarily just Aikido, but I would also like to do maybe Tai Chi and Hapkido too) at full speed, that I could feed to them ahead of time to influence their purchasing system, that would greatly appreciated.

Finally, if you have no idea what I was just talking about, but would be interested in some sort of interactive 3D "thing" that showed off Aikido or one of the other martial arts, please let me know what you'd be interested in seeing. This will be a long term project - I'm expecting 12+ months, but something that could benefit us all in the long run.

06-02-2005, 12:51 PM
Hi Justin,

You might want to contact the folks at http://www.aikido3d.com and http://www.adaptiveperception.com/ as they seem to have done some work in this field already.

Hope that helps,

-- Jun

06-02-2005, 01:16 PM
I had no idea that people were doing this. This is quite remarkable. I want the next generation of this technology to have more of an X-ray feature (only more with muscles) to highlight what is going on inside the body in an ideal situation. Like, how the muscles are lengthened and widened, how force is distributed, and how muscles / muscle groups work together to make that whip-like effect. Great project!


Adam Alexander
06-02-2005, 01:28 PM
I don't get it. Why would you want something like that? There's already videos available.

I'm not trying to discourage your pursuit, it's just that neither site really seems to explain why there doing it.

06-02-2005, 01:38 PM
Thanks Jun. Looks interesting both sites. A little different to what I have in mind, but they should have some good information. One of the things I have in mind is to combine this with the rigid body physics engine like used in games, so that you can play with parameters such as size and weight differences between uke and nage.

Jean, I believe the phrase "because we can" applies here :) At least in my case it is for personal interest, that I have access to such equipment for no cost, and it's already what I do for work. The basic difference between a video and these is that you can take any vantage point you like and turn on/off any feature you like. For example, tracing the footwork rather than the motion of the hands - just turn off the bodies and leave the foot prints.

The same question applies to books - why would anyone want to buy a book when the only way to learn is to do it? People like information in different ways. Pictures help, maybe a computer-based simulation will too. For example the Tae Kyan site in Korea is doing something very similar, teaching people the very old martial art there that contributed a lot to TKD and Hapkido kicking techniques.

06-02-2005, 01:49 PM
Actually, I think it would be very interesting. Sound like what the do during the Olympics with figure skating. Of course, I imagine hooking ourselves up and seeing what we are doing wrong compared against what is done right.

06-02-2005, 01:58 PM
Yes, the olympics stuff is a little different to this, but is close enough. You need a lot of computational horsepower to do that in realtime from video inputs. The more traditional way that I will be using is either via magnetic or optical (IR) sensor tracking. These require almost no computing grunt initially, just a bit of post-processing to remove jitter. The rest of the work is in assembling a large collection of techniques and then working out how to present it all. I might just go purchase a copy of that Aikido3D, as that seems like a good setup the more I read over it.

As for the comparison of DIY, that could be an interesting challenge! I've seen people hack things like webcams for simplistic motion capture - enough to control say a weapon in a FPS, but not much more than that. Certainly fullbody work would be quite a lot more difficult. In addtion, one must be careful comparing your techniques with others. The way I do a technique as a really tall guy would be different to those of a really short person.

Nick P.
06-02-2005, 02:08 PM
edit: did not see Justin's post.

On the one hand: COOL! VERY COOL!
On the other hand: Would you not rather be training?

I might just get the aikido3d one. Seems interesting enough, especially the "Turn on Uke & Nage’s hip radials to watch their centers interact." feature.

As to your comment, Jean, about videos; videos, no matter how well done, always seem a little flat. With this you can rotate around the action from any angle you wish. That might be very useful for some people...or a waste of $50. Time will tell.

Who knows, perhaps we are witnessing the birth of a fantastic learning tool (but of course there is no substitute for mat time).

06-02-2005, 02:10 PM
Isn't motion-capture what LucasFilm did with some of their CGI work (the battle droids movement for walking) and also if I recall correctly the animation for their Revenge of the Sith video? That's also what I think of when I hear "motion capture". Can you tell I'm a lay person when it comes to this?

I see what you mean about the comparison. It might not be very efficient or practical.

06-02-2005, 02:46 PM
We need a "diff" function where if I am having trouble throwing someone, we can record it. Then we can get some sempai of similar size to throw that same uke. We need a read out of things like, successful attempt had 20% more wrist rotation, was 5 inches closer just prior to the unbalancing, and the nage making the unsuccessful attempt had a different foot forward, and was 14 degrees more bend over from the sucessful nage...

It could be hooked up to a major database and the successful attempts on basic technique for various body types could be saved and accessed by the internet.


06-02-2005, 02:52 PM
Yes, that's precisely what it is. Basically, stick a bunch of white dots on a person and then capture their precise movements. Those movements can then be mapped onto another object with roughly the same physical shape (eg humanoid shape). For instance, we could substitute the human body with Marvin the Martian and the movements would look identical. With good tools you can even manipulate those same movements and extrapolate a tall thin person onto a short fat person and have it still look sane.

Mike Sigman
06-02-2005, 03:03 PM
Finally, if you have no idea what I was just talking about, but would be interested in some sort of interactive 3D "thing" that showed off Aikido or one of the other martial arts, please let me know what you'd be interested in seeing. This will be a long term project - I'm expecting 12+ months, but something that could benefit us all in the long run. Just to throw in a quick opinion, I don't think these types of video analyses are all that helpful. To see why I think you'd be wasting your time with Aikido, Tai Chi, and similar arts, take a look at the Gozo Shioda video called "Shingi Denju". What Shioda does in the last part (he's mostly in seiza) is the essence of Aikido's control and blending of forces.... yet all you can see is the results, not what he's actually doing. A motion study wouldn't have any motion to study since he rigs force paths with his mind, once he has a connection to uke. Showing the type of film analysis that you're suggesting of what Shioda is doing would show nothing. Hence, most of these 3D modelings tend to show "technique" and not the essence of Aikido, Tai Chi, etc. Not trying to rain on your parade, by any means, but I thought someone should note the problem with these sorts of analyses.



Adam Alexander
06-02-2005, 04:07 PM
Well, I don't know if I'd call them worthless (as seems did the previous post). However, I think the statement,"doing it because we can," is enough for me.

I think if you want serious investors, you may want to find more reasons than that (not that I'm a serious investor, just seems like those folks wouldn't drop a knot on something that doesn't look commercially promising).

06-03-2005, 12:34 AM
Jean, just to alay your fears: I'm not looking for investment. In fact, I'm not even looking to sell it. It would mostly be for my dojo and for anyone else that is interested in obtaining a copy. All of the data would be free for anyone to use within some limits of copyright, too. Mostly this post is to go scouring for technical information from fellow martial arts/computer geeks that may have already tried this out and have some recommendations. Our client won't have the system purchased and installed for another month or two, so I wouldn't be able to work on this before then (they also happen to be 1000 miles south of where I live :) ).

06-03-2005, 07:48 AM
I think it is a wonderful idea. Regardless of what mental pathways need to be created, you still need to have a form within which to set those up. Otherwise, people would just be making attackers fall from across the room. However, as I said before I think it would be interesting to have a mode where you can show what the forces are doing inside the bodies, and it would be an interesting extention to add where the mind pictures the paths to ground as well.

(Interpreting "aren't all that helpful" as 'seems worthless' didn't seem that passive-aggressive to me. I got the impression that it would be worthless to you too - what's the big deal?)


06-03-2005, 01:37 PM
Actually, I think motion capture would be a valuable tool in helping people learn techniques. Of course it can't capture internal movement or ki (can that, ki, really be 'captured'?). Neither does books, video, or mediums like this, but people find it useful enough to register and make posts and discuss such matters. I checked out the Aikido3D which features Donovan Waite, btw, and it looked very interesting. It's very similar to using video, but the motion capture feature allows you to see views that you normally don't get to see with a video. The program allows you to choose different views, angles, and i believe even close-ups. It just seems to be a step above regular videos and not as limited as regular videos or DVDs.

Mike Sigman
06-03-2005, 01:47 PM
Actually, I think motion capture would be a valuable tool in helping people learn techniques. Of course it can't capture internal movement or ki (can that, ki, really be 'captured'?). Hmmmm.... that's actually a good thought. It's too complicated to just assert a position, but *maybe* it's possible to do an overlay on top of the motion-study video which shows some representation of the forces involved. I'd have to think about it before I stuck my foot too far in my mouth, but it's not a bad thought process you've triggered, Anne Marie.

Justin, if you ever decide to go ahead with your project and you want to consider the possibility of an overlay type of idea, I'd be glad to try and formulate my thoughts and give them to you.

Mike Sigman

Gareth Hinds
06-08-2005, 06:56 AM
I've done a little work with mocap, though it was five years ago and things may've changed significantly. The obvious things I think you need to consider about the system are these:

If you are going to show both uke and nage, you need two rigs, or one rig with enough markers to distribute on two people. I've never worked with two person mocap, and I suspect it may be quite a bit more problematic than one person. For example, if you have 2 separate rigs, you have to sync them up. If you use an optical system, you will have some problems with one person occluding the other, and perhaps even points "jumping" from one skeleton to the other when they get occluded. If you use a wired system, the wires will be fairly restrictive of what you can do.

Those are the first things that spring to mind. Anyway, those groups who are already trying this should have good technical information for you, and if nothing else I bet it will be fun to experiment with.

07-22-2006, 06:53 AM
Gareth: I understand that we now have sensors capable of doing the same thing as the wired system, only wireless.

James Kelly
07-23-2006, 01:10 AM
A motion study wouldn't have any motion to study since he rigs force paths with his mind...
Ahh, but the mo cap studios out here in hollywood can capure that too. You should see this stuff.

07-23-2006, 03:14 AM
Hi Justin,

I have used 3D Studio Max and Lightwave at university. I have been interested in doing some kind of 3D character animation using aikido however it would be very difficult to keyframe animate the movements. My university has a motion capture facility so I may try it next semester.

What would be really cool is to have live action and 3D combined so you can have see through effects like in Romeo Must Die. I have seen this done on a few martial arts documentaries on the Discover Channel.