Repository files navigation

1. Highlights
2. Getting Started
   • Simulation
   • Real-World
3. Checkpoints
4. TODO List
5. License
6. Citation.
7. Acknowledgment

• 🏆SOTA simulation performance Seer achieves state-of-the-art performance on simulation benchmarks CALVIN ABC-D and LIBERO-LONG.
• 💪Impressive Real-World performance Seer demonstrates strong effectiveness and generalization across diverse real-world downstream tasks.

We provide step-by-step guidance for running Seer in simulations and real-world experiments.Follow the specific instructions for a seamless setup.

• Installation
• Running Code

• Installation
• Running Code

For users aiming to train Seer from scratch or fine-tune it, we provide comprehensive instructions for environment setup, downstream task data preparation, training, and deployment.

• Installation
• Post-processing
• Fine-tuning & Scratch
• Inference

This section details the pre-training process of Seer in real-world experiments, including environment setup, dataset preparation, and training procedures. Downstream task processing and fine-tuning are covered inReal-World (Quick Training w & w/o pre-training).

• Installation
• Pre-processing
• Pre-training

Relevant checkpoints are available on thewebsite.

• Release real-world expriment code.
• Release CALVIN ABC-D experiment code (Seer).
• Release the evaluation code of Seer-Large on CALVIN ABC-D experiment.
• Release the training code of Seer-Large on CALVIN ABC-D experiment.
• Release LIBERO-LONG experiment code.
• Release simpleseer, a quick scratch training & deploying code.

All assets and code are under theApache 2.0 license unless specified otherwise.

If you find the project helpful for your research, please consider citing our paper:

@article{tian2024predictive,title={Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation},author={Tian, Yang and Yang, Sizhe and Zeng, Jia and Wang, Ping and Lin, Dahua and Dong, Hao and Pang, Jiangmiao},journal={arXiv preprint arXiv:2412.15109},year={2024}}

This project builds uponGR-1 andRoboflamingo. We thank these teams for their open-source contributions.