SmartMouth™ Help Documentation

Version 1.1.1 (r2)

Overview

SmartMouth allows you to automatically lip sync characters in Flash. This document covers SmartMouth in detail.

Help Contents

•   Installation
•   User Interface
  • Phonemes
  • Modes
  • Actions
•   Using the Trial
•   How to Obtain a Serial Number
•   Activation
•   Updates
•   Tips & Best Practices
•   Frequently Asked Questions
  

Installation (top ^)

SmartMouth is installed using Adobe's free Extension Manager (EM) which is automatically installed along with Flash Professional. Unzip the package that you've downloaded from Ajar Productions. Inside the unzipped directory, you will find a file named SmartMouth.mxp. The simplest way to install SmartMouth is to double-click on this file. In most cases, this will automatically launch EM and begin the installation process. In some cases, double-clicking may not launch the correct version of EM. The EM version corresponds directly with the version of Flash. For instance, if you want to install SmartMouth into Flash CS5, make sure you are using EM CS5. You may need to launch EM separately and click the "Install" button. Read the license agreement and follow the instructions. After you've installed the extension, you may need to restart Flash.

User Interface (top ^)

SmartMouth can be accessed via the Commands menu in Flash Professional (Commands > Lip Sync with SmartMouth).

The following figure diagrams each aspect in the SmartMouth interface:

• A. Info Button. Clicking this button launches the Info window (used for activation, purchasing a license, and checking for updates).
• B. Help Button. Clicking this button launches the locally stored help documentation.
• C. Audio Layer. The audio layer selected determines which audio is analyzed. The list of available layers is generated automatically by SmartMouth.
• D. Mouth Layer The mouth layer selected determines which images are shown in the previews, as well as which layer the new keyframes will be applied to. The list of available layers is generated automatically by SmartMouth to include layers that have frames containing single symbol instances.
• E. Frame Range Controls. These controls allow you to set start and end frames for the audio analysis.
• F. Set End Frame to Last Frame Button. When clicked, this button sets the End Frame value to the final frame in the current scene.
• G. Set End Frame to End of Selection Button. This button resets the End Frame value to the last frame selected on the Timeline.
• H. Action. This setting determines the action that is taken when audio analysis is complete. See Actions below for more details.
• I. Mode. This setting determines the source of the mouth shapes based on common Flash lip syncing methods. See Modes below for more details.
• J. Limit to. This setting is primarily used with Symbols mode to limit the phoneme selectors to a single library folder. This can make selecting mouth shapes easier if your characters assets are in a single library folder.
• K. Phoneme/Viseme guides. These guides help you assign your mouth shapes. See Phonemes below for more details.
• L. Preview Images. These thumbnails display your mouth shapes for each phoneme.
• M. Frame Selectors. These inputs allow you control phoneme assignment. The type of frame selector will vary depending on the mode selected.
• N. Close Button. This will close the SmartMouth interface.
• O. Run Button. This button initiates the audio analysis. When the analysis is complete, the selected action will be taken.
• P. Advanced Button. This button opens the Advanced Settings Window, which allows you to control the frequency of the keyframes that SmartMouth renders (1s, 2s, 3s), as well as the loop settings (retain original, single frame, play once, loop).

Phonemes (top ^)

A phoneme is a fragment of audible speech. A viseme is the mouth shape that corresponds with a phoneme. The phonemes listed on the interface are abbreviated. The table below shows the full list of phonemes.

Abbreviated Phoneme	Full Letter List	Corresponding Examples
-	none	No vocal sound (or inaudible)
A	A, I	ah as in “cat,” a as in “say,” and i as in “kite”
O	O, U, (W), (R)	o as in “boat” and u as in “clue”
E	E, (I)	e as in “street” and eh as in “trek”
S	C, D, G, K, N, R, S, TH, Y,Z	s as in “stress” and t as in “tent”
L	L	l as in “lull”
M	M, B, P	m as in “might,” b as in “back,” and p as in “pass”
F	F, V	f and v as in “favor”

Modes (top ^)

The selected mode determines the source of the mouth shapes based on common Flash lip syncing methods. The methods/modes available include:

• Symbols. This mode is used when each mouth shape is stored in its own symbol. When keyframes are added or replaced, symbols are swapped. The position of the new mouth shape is determined by the position of the mouth shape on the previous frame.
• Frame #. This mode is used when the mouth shapes are on a single Timeline (with no frame labels). When keyframes are added or replaced, the new symbol instance's First Frame property is updated to correspond with the Frame Number that matches the assigned phoneme.
• Labels. This mode is used when the mouth shapes are on a single Timeline that contains frame labels. When keyframes are added or replaced, the new symbol instance's First Frame property is updated to correspond with the assigned phoneme.

Both Frame # and Labels modes will convert Movie Clip instances on the Stage to Graphic instances so that the First Frame can be set (Loop will always be set to Single Frame in these cases). These modes use the symbol instance present in the previous frame.

The mode(s) available will be determined by the contents of your file and/or the selected Mouth Layer.

Actions (top ^)

Actions determine what takes place when audio analysis is complete. Available actions include:

• Overwrite keyframes. This action overwrites any existing keyframes that correspond with changing phonemes.
• New Keyframes Only. This action respects any existing keyframes and only create and apply mouth shapes on new keyframes.
• Guide Layer Only. This action does not alter any symbol instances, but creates a guide layer with detected phonemes displayed as frame comments.
• Export to XML. This action has no effect on the Timeline, but allows you to save analyzed phoneme data as a simple XML file. The XML file can then be used to lip sync a character dynamically with ActionScript.

Many of the phoneme settings are unavailable when using Guide Layer Only and Export to XML actions because no phonemes are applied to the Timeline.

Using the Trial (top ^)

You are encouranged to download and try out an evaluation copy before purchasing a license. The trial limits SmartMouth to run no more than 3 times on a single document with no more than 60 frames rendered per attempt. Your trial version (evaluation copy) can be converted to a licensed copy by purchasing a serial number and activating your installation of SmartMouth. No additional installation is necessary.

How to Obtain a Serial Number (top ^)

You can purchase a licensed serial number by clicking the "Purchase a license" button in the Info window or by selecting "Purchase a license" from the right-click menu.

Activation (top ^)

You can activate a copy of SmartMouth via the Info window. Enter the license key you received via email in the "License Key" field and click the "Activate" button. If your copy has already been activated, the "Activate" button will not appear. Depending on the number of seats you purchased, you may be able to activate a single serial number multiple times. Each seat purchased corresponds to one installation of Flash Professional, whether on one machine or several.

Updates (top ^)

You can check for updates by opening the Info window and clicking the "Check Now" button. The field below will display any available updates (and an update link when applicable).

When a new version is available, you will be directed to a new download link. Any new version will be installed in the same manner as your first installation and will require that you re-accept the license agreement. Any activated serial number(s) will be retained upon installation of an update so long as the update is installed with the same version of Flash (and on the same machine).

Tips & Best Practices (top ^)

• Tip: Get quick access to several items from the right-click menu.
• For best results, a vocal track should be on its own layer and not mixed with music, sound effects, or other voices.
• For best results, make sure your imported audio files are reasonably loud.
• When automatically generating the Mouth Layer list, SmartMouth searches for layers containing one instance per frame. It is important that each keyframe only contains a single symbol instance when phonemes are applied as well.
• You can sync multiple symbols on a single layer in one pass (in Frame # and Labels modes), so long as each frame only contains one instance. This could be used for head turns and position changes. Make sure that each mouth symbol contains the same set of phonemes on the same frames.

Frequently Asked Questions (top ^)

Will SmartMouth work in other languages?

Yes. Nothing about the SmartMouth algorithm is language-specific. SmartMouth analyzes sound fragments. These fragments are common to most languages and require the same contortion of the face regardless. Keep in mind that the phonemes listed are based on English sounds.

How many frames can I process at a time with SmartMouth?

Flash Professional limits exported Timelines to 16,000 frames. It is recommended that you stay well below that for optimal functionality. This may require breaking your project into smaller files.

SmartMouth does a good job for most mouths, but sometimes makes mistakes. Do you have any tools to speed up the process of lip-syncing manually?

Yes, we do. Check out FrameSync or FrameScrub. Also, we are constantly seeking to improve the SmartMouth algorithm, so if you have recurring issues with certain sounds or audio files, please let us know at support@ajarproductions.com.

Does SmartMouth have only one mouth shape (i.e., the one shown in the tutorials) or are there a few options (e.g. fuller lips)? It possible for me to modify the mouth shapes myself?

The mouths used are completely up to you. The mouth shapes on the left side interface are merely shown as a guide. SmartMouth works with the mouth shapes that you have on Stage or in the Library. Live previews of your shapes are provided on the right side of the interface. Once you've run SmartMouth, your mouth shapes are still fully editable, as if you'd place them on the Timeline yourself. If you give the demo a try, we think you'll be pleased with the flexibility.

So SmartMouth works in the authoring tool, but what if I want to lip sync 'on the fly' and create something like a talking avatar program?

We are currently working on a product called SmartMouthRT to do just that.

 

<< Ajar Productions Main Page