Skip to content

Conversation

rebeccazhang0707
Copy link

Description

Adds two place tasks and mimic tasks with Agibot A2D humanoid, using RMPFlow controller:

  • add agibot robot config in agibot.py and .usd asset
  • add motion_policy_configs and .urdf for rmpflow controller
  • add new task cfg: place_toy2box_rmp_rel_env_cfg, and place_upright_mug_rmp_rel_env_cfg
  • add new mimic task cfg: agibot_place_toy2box_mimic_env_cfg, and agibot_place_upright_mug_mimic_env_cfg
  • add new mimic task: in pick_place_mimic_env.py
  • add new subtasks in mdp.observations/terminations: object_grasped, object_placed_upright, object_a_is_into_b

Notes: This PR relies on PR (#3210) for RmpFlowAction support.

You can test the whole gr00t-mimic workflow by:

  1. Record Demos
./isaaclab.sh -p scripts/tools/record_demos.py \
  --dataset_file datasets/recorded_demos_agibot_right_arm_rel.hdf5 \
  --task Isaac-Place-Toy2Box-Agibot-Right-Arm-RmpFlow-v0 \
  --teleop_device spacemouse \
  --num_demos 1
  1. Replay Demos
./isaaclab.sh -p scripts/tools/replay_demos.py \
  --dataset_file datasets/recorded_demos_agibot_right_arm_rel.hdf5 \
  --task Isaac-Place-Toy2Box-Agibot-Right-Arm-RmpFlow-v0 \
  --num_envs 1
  1. Annotate Demos
./isaaclab.sh -p  scripts/imitation_learning/isaaclab_mimic/annotate_demos.py \
  --input_file datasets/recorded_demos_agibot_right_arm_rel.hdf5 \
  --output_file datasets/annotated_demos_agibot_right_arm_rel.hdf5 \
  --task Isaac-Place-Toy2Box-Agibot-Right-Arm-RmpFlow-Rel-Mimic-v0 \
  --auto 
  1. Generate Demos
./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
  --input_file datasets/annotated_demos_agibot_right_arm_rel.hdf5 \
  --output_file datasets/generated_demos_agibot_right_arm_rel.hdf5 \
  --task Isaac-Place-Toy2Box-Agibot-Right-Arm-RmpFlow-Rel-Mimic-v0 \
  --num_envs 16 \
  --generation_num_trials 10

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

- add surfacegripper actions
- add new task envs with UR10 + suction cups
- add galbot robot config
- add RMPFlowAction and RMPFlowActionCfg
- add motion_policy_configs for both galbot_left_arm_gripper and galbot_right_arm_suction
- add galbot stack_rmp_rel_env_cfg
- add galbot_stack_rmp_abs/rel_mimic_env
- add mdp.observations/terminations/events for galbot: support both suction_cup and parallel_gripper, get obs_in_base_frame
- fix a bug (eef_name) in franka_stack_ik_rel_mimic_env.py
- add new device_name in se3_spacemouse.py
…the RobotCfg back to default

- move surface_gripper_actions out of binary_joint_actions to a standalone file
- add RobotCfg in agibot.py
- place_toy2box: use Agibot right arm, and RmpFlow controller
- place_upright_mug: use Agibot left arm, and RmpFlow controller
- add agibot.urdf and motion_policy_configs
- add new subtasks:
    - object_grasped in observations.py
    - object_a_is_into_b and object_placed_upright in terminations.py
- add shared PickPlaceMimicEnv in pick_place_mimic_env.py for all pickplace tasks w/ single arm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant