Skip to content

A system for generating animations from textual prompts using AI text-to-animation models and Unity game engine.

Notifications You must be signed in to change notification settings

CGVGroup/VH_Text2Anim_Generator

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

⚙️ Requirements

  • Unity 6000.0.28f1 installed.
  • Python installed (recommended >= 3.10).
  • CUDA-compatible NVIDIA GPU.
    A GPU with CUDA support is required to run the motion generation models.

🔧 Instructions

  1. Clone the repository Open your terminal in the desired folder and run:

    git clone https://github.com/ciroanni/MasterThesis
    cd MasterThesis
  2. Open the project in Unity (version Unity 6000.0.28f1)

  3. Open the "AI generation" tab. The AI generation interface will appear automatically once the project is loaded.

  4. Make sure to adjust the paths to match your local setup.

    Specifically, set the correct Python executable in the Python Path field. Example (on Windows):

    C:/Users/<user name>/AppData/Local/Programs/Python/Python310/python.exe
    

  5. Configure models. All commands and paths are located in config.json.

    • Do not modify the commands.
    • To use the models:
      • Clone the required models from my GitHub forks.
      • Follow each model's instructions in their repository to download the pretrained weights and datasets (T2M-GPT, MDM, and LADiff).
      • Then update the paths in config.json accordingly.
      • Note: running the models requires a CUDA-capable GPU.

    Supported models (so far):

    LADiff

    git clone -b master https://github.com/ciroanni/LADiff

    MDM

    git clone -b master https://github.com/ciroanni/motion-diffusion-model

    T2M-GPT

    git clone https://github.com/ciroanni/T2M-GPT

    If the folder "bvh2fbx" is empty or nonexistent after the clone command, run this (in the model folder):

    mkdir bvh2fbx #if there is no folder 
    git clone https://github.com/ciroanni/bvh2fbx

    If the folder "Motion" is empty or nonexistent after the clone command, run this (in the model folder):

    mkdir Motion #if there is no folder
    git clone https://github.com/ciroanni/Motion
  6. Now you can use the “AI generation” tab by selecting the model. The result (both in FBX and .anim) will be generated in the folder:

    Assets/Resources/results
    
  7. The SampleScene in

    Assets/Scenes
    

    is the scene used for the dataset videos used in the thesis work, and you can use it to visualize the results. In any case, the results are suitable for any humanoid rig.

About

A system for generating animations from textual prompts using AI text-to-animation models and Unity game engine.

Topics

Resources

Stars

Watchers

Forks

Languages

  • C# 99.2%
  • Python 0.8%