Mar 01, 2016
Matchmoving is the process of following one or more features in a piece of video over time. This technique is incredibly useful and is the first step to just about every motion design and visual effects shot involving footage. But as with every other facet of motion graphics, VFX, and animation, matchmoving is a huge umbrella, under which whole industries have specialized. The tools and techniques available for building new assets into live action footage are intensely varied: How do you know which route to take?
There are three main kinds of matchmoving.
2D matchmoving software treats your footage as flat; the tracker can’t perceive depth in the image, so it just does its best to follow position, scale, and rotation. Since we’re not worried about calculating depth, 2D matchmoving is generally a very fast process. And despite the lack of depth information, it’s not uncommon to see 2D matchmoving techniques used to create an illusion of depth by tracking several 2D points in an image, and using those points to transform or warp another 2D asset. For example, in our Little Mermaid spot for Amopé last summer, we used 2D matchmoving techniques (along with lots of rotoscoping; more on that soon!) to realistically place stock footage of an underwater coral reef into footage we shot of our model in a swimming pool.
3D matchmoving is the process of creating a virtual 3D camera in your scene that moves exactly the same way that your real camera did when you shot your footage. The software does this by tracking hundreds of points in your shot all at once, using techniques pretty similar to 2D matchmoving, and then some fancy math happens that I don’t understand, which allows the software to recreate an estimation of the depth in the footage virtually.
It’s rarely as fast or as easy as 2D matchmoving, and to do it right in a complex shot requires a lot of planning ahead when you go to shoot your scene, including the placement of tracking markers on your set during the shoot. But even though it’s a pain sometimes, this is an essential tool if you want to composite your motion graphics or animation work into live-action footage.
Motion capture (or performance capture) is a lot like 3D matchmoving, in that you’re measuring many features of a subject and how those features relate to each other over time. However in this case, we don’t care about what the camera’s doing—we’re trying to capture a performance.
This is how Andy Serkis becomes Gollum in Lord of the Rings. Everything from his feet all the way up to his facial expression gets recorded using sensors attached to his body and observed using multiple strategically-placed cameras, and can then be applied to a 3D model after the fact. The same technology can be extended to assist with 3D matchmoving as well; by strapping high-end sensors to cameras and props, data about the position and movement of those objects can be recorded and synchronized with live-action footage, taking out tons of computational guesswork and adding a potentially significant degree of accuracy.
Whether it’s nailing a hastily-built illustration of a cat to your supervisor’s hands, or precisely mapping a detailed facial expression onto a sophisticated 3D model, matchmoving in all of its forms is an essential tool in the motion designer’s toolkit. Your goal may be a seamless composite with existing footage, or a stylistic and organic look with no footage at all, but in either case it pays to get to know your matchmoving software on a first-name basis.
Want to learn how Genuine can help you with your next digital, creative, tech or media project? Get in touch with us at firstname.lastname@example.org.