【Tutorial】Use DeepMotion Mocap to Drive MMD Model

Related AI

Deepmotion

In this video

🎬 DeepMotion Mocap Animation: A Comprehensive Guide for MMD Models

DeepMotion is a cutting-edge tool for creating realistic motion capture (mocap) animations for MikuMikuDance (MMD) models. By transforming video input into detailed animations, it enables you to bring characters to life with precision. This guide will walk you through the process of using DeepMotion, from importing animations into Blender to tweaking and exporting them for use in Unity.

🎥 Creating Animation Using DeepMotion

1. Start on DeepMotion’s Website: Visit the DeepMotion website to begin. You can create animations from any video by uploading it manually or dragging and dropping it into the browser.
2. Customization Features: DeepMotion offers several options for fine-tuning your animation: Activate Physics Filter: For more realistic motion.Face Tracking and Hand Tracking: For detailed output.Upper Body Only Option: Select this if your video only includes the upper body.Motion Smoothing: Set this to 0.1 for optimal results.
3. Activate Physics Filter: For more realistic motion.
4. Face Tracking and Hand Tracking: For detailed output.
5. Upper Body Only Option: Select this if your video only includes the upper body.
6. Motion Smoothing: Set this to 0.1 for optimal results.
7. Generate and Download Animation: After setting your preferences, click 'Create' and 'Start Job' to generate your animation. Once completed, download the animation as an FBX file.

🛠️ Importing Animation into Blender

1. Import FBX File: Open Blender and import the FBX file containing your animation.
2. Adjust Shape Key Values: Use a Python script to adjust shape key values and rename them to match MMD formats. Since different MMD models may have varying shape key names, manual modification might be necessary.

🔧 Tweaking Shape Key Values

1. Python Script Functions: Adjust Shape Key Values: Find and adjust shape key values such as 'Jaw Open'.Rename Shape Keys: Match the shape key names to MMD formats.
2. Adjust Shape Key Values: Find and adjust shape key values such as 'Jaw Open'.
3. Rename Shape Keys: Match the shape key names to MMD formats.
4. Fine-Tuning: Create New Mapping: Copy and adjust MMD shape keys.Adjust Recorded Values: Take the difference between two values and multiply by a coefficient. For example, if the MMD mouth shape key maps to 'Jaw Open' and 'Mouth Close', adjust accordingly.
5. Create New Mapping: Copy and adjust MMD shape keys.
6. Adjust Recorded Values: Take the difference between two values and multiply by a coefficient. For example, if the MMD mouth shape key maps to 'Jaw Open' and 'Mouth Close', adjust accordingly.

🚀 Exporting and Importing into Unity

1. Export from Blender: Save the adjusted animation as an FBX file. You'll need two separate animations: Face Animation: For facial expressions.T-Pose Animation: For body movements.
2. Face Animation: For facial expressions.
3. T-Pose Animation: For body movements.
4. Import into Unity: Set Character to Humanoid: In Unity, configure your MMD character to humanoid.Create Timeline: Add a timeline game object.Add Animation Tracks: Body Animation: Add an animation track for body movements.Face Animation: Add an override track and drag the face animation onto it.
5. Set Character to Humanoid: In Unity, configure your MMD character to humanoid.
6. Create Timeline: Add a timeline game object.
7. Add Animation Tracks: Body Animation: Add an animation track for body movements.Face Animation: Add an override track and drag the face animation onto it.
8. Body Animation: Add an animation track for body movements.
9. Face Animation: Add an override track and drag the face animation onto it.
10. Adjust Face Mesh Name: Ensure that the name of the model’s face mesh matches the name recorded in the animation clip for proper function.

🏁 Final Comparison

After completing these steps, your MMD model should exhibit realistic mocap animations. Proper integration of body and face animations will result in a coherent and engaging performance.

Using DeepMotion, Blender, and Unity together allows you to create high-quality mocap animations for various applications, from gaming to film. By following this guide, you can effectively harness mocap technology to animate your MMD models and bring your creative projects to life.

🔍 Key Insights

• DeepMotion: Start with DeepMotion to create mocap animations from video.
• Blender: Import and adjust animations, including shape key values.
• Unity: Import animations, set up humanoid characters, and configure timelines.

Tags

DeepMotion