Q: My green/blue/yellow/mauve screen footage I shot on DV/HDV looks like crap when I key it! What am I doing wrong?[]
A: Shooting on DV.
But seriously, DV is not a format renowned for it's keying abilities. There are, however, some things you can do in After Effects to help achieve a great key with it. (Note: This does not cover shooting techniques, just post-production solutions)
Start out by drawing a garbage matte to eliminate any area that is not being used. Basically any area where your subject is never at that point in the shot. Don't make garbage masks complicated and use as few of points as possible. Example:
The mask is drawn around the subject so that at no point during the shot any part of the subject will go outside of the mask
IMPORTANT: Don't use the native Color Key plug-in. It's horrible. Use Keylight by the Foundry (comes with the Pro version of AE now) or Ultimatte.
- Before you apply keylight select Effect>Channel>Channel Combiner and choose RGB to YUV (it's going to make your footage look crazy, but that's okay for right now).
- Next select Effect>Blur & Sharpen>Channel Blur and blur the Green and Blue channels just slightly (no more than to 2 or 3).
- Now add another instance of Channel Combiner below that and select YUV to RGB and it'll put your footage back to normal.
- Next you might want to consider using (if you have it) CW Smooth Screen to flatten out the green screen color in the shot and make it a little more even.
- Now apply keylight and use the color picker to select green closest to the subject not being keyed-out.
IMPORTANT: Adjust the Pre-Blur only if you have to. I know some will tell you to do this for DV footage, but I recommend only doing it as a last resort.
- From the View drop box select Screen Matte. This will show you in detail what is going on. You want everything that needs to vanish to be completely black and everything that needs to stay to be completely white.
- Under the screen matte tab use the Clip Black and Clip White (preferably in that order) to adjust the settings until it looks like I just described. Now apply the screen softness to smooth the edges, but not too much.
- After this, if you've still got some edge issues, try using CW Matte EZ Feather (if you've got it).
- Now add Effect>Matte>Simple Choker and choke the matte as needed to further clean up the edges. If you need to be more precise at this point use the regular Matte Choker plug-in.
You should have some pretty decent footage now. As ever, go back and tweak the settings even more if needed.
By this point you might find your 3 gigs of RAM getting eaten up, so I'd recommend pre-rendering just the keyed footage before you add your background plate and do your color matching. Pre-rendering will also allow you to choke the edges a little more for clean-up.
3rd Party Plug-ins used:
Keylight from the Foundry Composite Wizard from Red Giant