Adding Sign Language to a Video

When producing educational video for use on the web it is important to communicate clearly with your target audience. If your material is to be of use to the deaf community or viewers with hearing difficulties, there are a number of options: add subtitles or closed captions or an overlaid sign language interpreter who interprets spoken information in visual form. British Sign Language is the first language of over 70,000 people in the UK but many other sign languages exist internationally. This advice document looks at the practical aspects of adding a sign language interpreter to video delivered via the web.

Introduction

There are many ways to achieve our goal, and many pieces of equipment and software which might be of use. Consider the detail of what you'd like to create:

• Which video format should be chosen?
• What's an acceptable balance between file size and visual quality?
• Should the interpreter be partially overlaid onto the video footage or over to one side?
• Should a non-signed version of the same video material also be provided?

There are no universal answers to these questions, take a look around at what others have done before making your decisions. Some good general advice is provided by our sister advisory service JISC TechDis.

For this tutorial we've decided on the Flash Video format, which is widely used and well supported. We've decided to partially overlay our interpreter so viewers won't have to look back and forth between the video and a separate window containing our signer. This however, does result in a slightly more complex workflow that includes 'chromakeying' (the process of making part of a video transparent so it can be overlaid on another). This involved video production and subsequent post-production workflow means that adding a signer to video is often done in batches and in the following broad phases which are explored in more detail below:

• Pre-production phase: Prepare video to be interpreted
• Production phase: Record the interpreter against a green screen as they sign along to the video material
• Post-production phase: Add the image of the interpreter over the top of the original video material and encode for delivery via the web

We've selected an inexpensive set up for around £1,000 which offers us the ability to 'chromakey' our interpreter. Our kit list is:

• 1x Practica DVC high definition camcorder with a 2GB SD card, £200
• 2x red head video production light, £119 each
• 1x photofill photographer's light, £83
• Adobe After Effects 7, £233
• Chromakey fabric, £28

If overlaying of the signer isn't required (the videos will instead be shown side by side) the redhead lights and chromakey fabric are not required.

Pre-production phase

A carefully planned workflow can save both time and effort later on. Select only the videos which you know will be used once signing has been added. It is common to find closed captions or subtitles in addition to sign, so consider adding transcription into the pre-production phase. Make sure your signer has plenty of time to view all video footage to be interpreted before production begins.

Setting up for chromakeying is perhaps the most important thing to get right: we want to achieve an even green colour behind our signer. This even colour will later be removed in post-production to leave a transparent area behind our interpreter. Conversely, it's vital not to have any green actually on our signer: no green clothing (stick with a dark colour if possible) and no green light reflecting from the green screen and giving out signer a green hue.

The simplest way to achieve this is to illuminate the green screen evenly with two red head (800W) lights then place the signer at least three meters in front of the screen. A photo fill light can then be used to illuminate our interpreter (Fig 1.0). Don't forget to provide a monitor and speakers for the signer's use, the program to be interpreted can be played by the camera person when the signer is ready to begin. Placing the monitor close to the camera assures the signer's line of sight engages the audience (Fig 1.1).

Fig 1.0 Production setup

Figure 1.1 Diagram of production set up

Production phase

During shooting, pay particular attention to the camera's white balance. Remember we want our screen to be nothing but green and our signer to show no green at all. We offer more advice on using a camcorder in a more general sense. Mistakes are inevitable and it may take a number of tries before the camera operator and interpreter are both happy with the recording which has been made. Fostering an atmosphere where re-takes are permitted is likely, in the end, to be more productive than a highly pressured situation. Allow plenty of time to make the recording. We will also need to record audio along with the video image, use the microphone to record the soundtrack from your source video as it plays back. This will allow you to synch up the audio later on. This will be vital if the post-production person does not sign to prevent any 'slippage' between the signer and the signed video.

Post-production phase

We've elected to use Adobe After Effects 7 as our chromakeying software, but many other packages are available. After Effects is available for both Mac an PC. After Effects accepts most common video file formats, so no transcoding is likely to be required. Simple copy the clips from your camera onto the hard drive of your computer then import them into After Effects. Figure 1.2 shows the project window with all the files we will need for this project. They are: our original video 'Eco-tourism', the slip of our signer IMAG0003.mov, copied directly from the camcorders hard drive. After Effects is used to arrange and manipulate text, images, videos and audio and output as a new video file. It does this in work areas called 'compositions'. We'll now make a new composition.

Fig 1.2 After Effects Project window

Go to the menu bar at the top of the After Effects window and select 'Composition' then 'New Composition'. The new composition window will open (fig 1.3). We've selected a good all-round preset, PAL DV. Although this is higher resolution than is required for most web delivery, it does mean we could also use the clip at events or for teaching purposes. We offer more advice on selecting a suitable video file format. Once you've selected a preset, set the duration to match your source video and click OK. Don't worry if this is a 'guestimate', it can be changed later.

Fig 1.3 After Effects Composition window

Once this is done a new composition will appear in your project panel (fig 1.4), if you ever get lost, double click on this to go back to your work area. Both a monitor window and a timeline will open up for your new composition.

Fig 1.4 After Effects Project window with composition 'Comp 1' listed

Now we need to lay our signer video and also the original video program onto the composition timeline. A composition's timeline is a graphical representation of how a composition changes over time. If a graphical such as an image or a video item is above another item in the timeline it will obscure or cover over that item, so it's important which order we arrange our video in. Place your signer over your source video (see fig 1.5).

Fig 1.5 After Effects Timeline

Fig 1.6 shows what we have at this stage. Now comes the chromakeying - our goal is to render the green area of the signing video transparent. Fortunately After Effects comes with a great tool for doing just that 'Keylight'. Click on your signer layer in the timeline to select it, then go to the top menu and select 'effects', 'keying' then 'Keylight'.

Fig 1.6 Footage before chromakeying

Fig 1.7 shows the effects control panel, which will pop up when the effect is added to our signer. We have to tell Keylight the colour of to be removed. Next to 'Screen Colour' Click there is a small black block and a pipette icon, click on the pipette and then click on an area of your green screen in the monitor window. Your green areas should now be transparent and (if your original video was placed beneath your singer on the timeline) you should see both videos overlaid.

Fig 1.7 Keylight effects controls

Fig 1.8 shows what we have at this stage. Keylight has many refining options which can be a life saver if your green screen wasn't quite evenly lit.

Fig 1.8 Footage after chromakeying

Our next job is to arrange our videos. Click on your signer and small blue squares will appear in the corners of the video. These are resizing handles. Hold down shift and drag one of these inward to make the video smaller. We placed our signer in the lower right hand corner. Similarly we resized our source footage slightly so our signer wouldn't obscure too much of the image (fig 1.9).

Fig 1.9 videos after size adjustment

This does leave a border around the source video. This can be set to any solid colour by selecting 'Composition' from the top menu then 'Background color' and selecting a colour. Alternatively, you might wish to add another copy of your source video, beneath everything else. This can be re-coloured or blurred (see our example below) using other effects in After Effects.

Next we will synchronise our signer with the source footage. Fig 2.0 shows the playback controls. Clicking on the RAM Preview will allow you to hear the video's sound on playback. You should hear two versions of the same audio track: one is the original from your source video, the other (probably lower quality) version was recorded with your camcorder's mic onto the video you made of your signer. The easiest way to synchronise the videos is to select the signer clip in the timeline and slide it left and right until you hear only one audio track.

Fig 2.0 After Effects playback controls

Finally we need to export the now signed video as a new file. Select 'Composition' then 'Add to render queue' from the top menu. The Render Queue window will open (fig 2.1). This allows you to select from an extensive range of presets in both standard and high definition. We selected H.264 then the YouTube preset for a widescreen, standard definition video.

Fig 2.1 After Effects Render Queue

Conclusion

While this guide provides a technical overview of one way to add a signer to a video program, there are many other considerations. Such technical operations can add value to video resources but at are technically involved and should be guided by institutional policy. One guiding principle should be: is this what the end user wants or needs? JISC TechDis has advice about achieving an optimum balance while ensuring all teaching and learning materials are fit-for-purpose.

Sample of signing footage before chromakeying.
If you cannot see the video above, please use this link to download the video file (8.1MB)

Sample of completed video.
If you cannot see the video above, please use this link to download the video file (10.9MB)

Programme content 'Lesson Starters' used with the kind permission of Teachers' TV.

Published in: Creating

Tweet this!