Embodied AI: A Comprehensive Guide to Algorithms, Robot Learning, and VLA Models

Embodied AI refers to intelligent systems that perceive and act through a physical body. These systems interact with their environment to gather information, understand problems, make decisions, and execute actions, resulting in intelligent and adaptive behaviors. This guide provides an entry point for newcomers to quickly grasp the main technologies involved in Embodied AI, understand their problem-solving capabilities, and gain direction for future in-depth exploration.

To build a strong foundation in Embodied AI, consider the following resources: * **Technical Roadmap:** YunlongDong's guide offers a foundational technical roadmap. * **Social Media:** Follow key accounts on platforms like WeChat (石麻日记, 机器之心, 新智元, 量子位, Xbot具身知识库, 具身智能之心, 自动驾驶之心, 3D视觉工坊, 将门创投, RLCN强化学习研究, CVHub) for insights and updates. * **AI Bloggers:** Explore lists of noteworthy AI bloggers on platforms like Zhihu. * **Robotics Labs:** Investigate summaries of robotics labs on Zhihu. * **Conferences and Journals:** Stay updated with high-quality publications in Science Robotics, TRO, IJRR, JFR, RSS, IROS, ICRA, ICCV, ECCV, ICML, CVPR, NIPS, ICLR, AAAI, and ACL. * **Stanford Robotics Introduction:** Access the Stanford Robotics Introduction website for comprehensive learning. * **Knowledge Bases:** Contribute to and utilize community-driven knowledge bases. * **Job Boards:** Explore job opportunities in Embodied AI. * **High-Impact Researchers:** Follow lists of influential researchers in the field. * **Communities:** Engage with communities like Lumina, DeepTimber, 宇树, Simulately, HuggingFace LeRobot, and K-scale labs.

This section covers essential algorithms and tools used in Embodied AI. * **Common Tools:** * **Point Cloud Downsampling:** Techniques like random, uniform, farthest point, and normal space downsampling are crucial for optimizing 3D applications. * **Eye-Hand Calibration:** Essential for determining the relative positions between cameras and robotic arms, categorized as eye-on-hand and eye-outside-hand. * **Vision Foundation Models:** * **CLIP:** Developed by OpenAI, CLIP calculates the similarity between images and language descriptions, with its intermediate visual features being highly beneficial for various downstream applications. * **DINO:** From Meta, DINO provides high-level visual features of images, aiding in the extraction of corresponding information. * **SAM (Segment Anything Model):** Also from Meta, SAM segments objects in images based on prompts or boxes. * **SAM2:** An upgraded version of SAM, capable of continuous object segmentation and tracking in videos. * **Grounding-DINO:** An image object detection framework developed by IDEA Research, useful for detecting target objects. * **OmDet-Turbo:** An open-source research project by OmAI Lab, offering open-vocabulary object detection (OVD) with high inference speed. * **Grounded-SAM:** Extends Grounding-DINO with segmentation capabilities, supporting detection and subsequent segmentation. * **FoundationPose:** A pose tracking model by Nvidia. * **Stable Diffusion:** A text-to-image model that can generate goal images and provide intermediate layer features for downstream applications. * **Depth Anything (v1 & v2):** Monocular depth estimation models from the University of Hong Kong and ByteDance. * **Point Transformer (v3):** A work on point cloud feature extraction. * **RDT-1B:** A foundational model for robotic bimanual manipulation from Tsinghua University. * **SigLIP:** Similar to CLIP, offering multimodal capabilities.

Robot Learning encompasses various techniques that enable robots to learn from experience and improve their performance. Key methods include: * **Model Predictive Control (MPC):** An advanced control strategy that uses a system's dynamic model to predict future behavior over a finite time horizon. MPC optimizes control inputs by solving an optimization problem to meet performance criteria and constraints. Resources include: * **Introductory Videos:** Model Predictive Control from the Huagong Robotics Laboratory. * **Theoretical Foundations:** Model predictive control: Theory and practice—A survey. * **Nonlinear MPC:** An Introduction to Nonlinear Model Predictive Control. * **Explicit MPC:** The explicit linear quadratic regulator for constrained systems. * **Robust MPC:** Predictive End-Effector Control of Manipulators on Moving Platforms Under Disturbance and Min-max feedback model predictive control for constrained linear systems. * **Learning-Based MPC:** Learning-Based Model Predictive Control for Safe Exploration and Confidence-Aware Object Capture for a Manipulator Subject to Floating-Base Disturbances. * **Reinforcement Learning (RL):** A learning paradigm where an agent learns to make decisions by interacting with an environment to maximize a reward signal. Resources include: * **Mathematical Principles:** Reinforcement Learning by Zhao Shiyu at Westlake University. * **Deep Reinforcement Learning Courses:** The Foundations of Deep RL in 6 Lectures, UC Berkeley CS285, and courses by Li Hongyi. * **Practical Implementation:** Gymnasium for hands-on experience. * **Imitation Learning:** A method where a robot learns by observing and imitating expert demonstrations. Resources include: * **Tutorials:** 《模仿学习简洁教程》 from Nanjing University LAMDA and Supervised Policy Learning for Real Robots, RSS 2024 Workshop.

Vision-Language-Action Models (VLA Models) integrate Vision-Language Models (VLMs) with robot control to generate robot actions directly from pre-trained VLMs. These models tokenize actions and fine-tune VLMs without requiring new architectures. * **Key Characteristics:** End-to-end, LLM/VLM backbones, pre-trained models. * **Categorization:** Model structure & size, pre-training & fine-tuning strategies, datasets, inputs & outputs, application scenarios. * **Resources:** * **Blogs:** 具身智能Vision-Language-Action的思考. * **Surveys:** A Survey on Vision-Language-Action Models for Embodied AI, 2024.11.28. * **Classic Works:** * **Autoregressive Models:** RT series (RT-1, RT-2, RT-Trajectory, AUTORT), RoboFlamingo, OpenVLA, TinyVLA, TraceVLA. * **Diffusion Models for Action Head:** Octo, π0, CogACT, Diffusion-VLA. * **3D Vision:** 3D-VLA, SpatialVLA. * **VLA-related:** FAST (π0), RLDG, BYO-VLA. * **Different Locomotion:** RDT-1B (bimanual), QUAR-VLA (quadruped), CoVLA (autonomous driving), Mobility-VLA (navigation), NaVILA (legged robot navigation). * **Dual-System Hierarchical VLA:** * Models like Hi-Robot and pi-0.5 use hierarchical architectures to mimic human rapid response and deep thinking mechanisms. * **Industrial-Grade VLA:** Figure: Helix, 智元：GO-1, Physical Intelligence, pi-0.5, Hi Robot, Nvidia: GROOT-N1, 灵初智能：Psi-R1, Google DeepMind: Gemini Robotics. * **Latest VLA Works:** SafeVLA, HybridVLA, DexVLA, DexGraspVLA, UP-VLA, CoT-VLA, UniAct.

Modern Embodied AI leverages the powerful information processing and generalization capabilities of Large Language Models (LLMs) for better robot planning. * **Resources:** * **Series:** Robotics+LLM系列通过大语言模型控制机器人. * **Wikis:** Embodied Agent wiki. * **Blogs:** Lilian Weng's AI Agent System Overview. * **Classic Works:** * **High-Level Strategy Generation:** PaLM-E, DO AS I CAN, NOT AS I SAY, Look Before You Leap, EmbodiedGPT. * **Unified Strategy Planning and Action Generation:** RT-2. * **Integration with Traditional Planners:** LLM+P, AutoTAMP, Text2Motion. * **Code as Policy:** Code as Policy, Instruction2Act. * **3D Visual Perception with LLMs:** VoxPoser, OmniManip. * **Multi-Robot Collaboration:** RoCo, Scalable-Multi-Robot.

Computer Vision plays a crucial role in enabling robots to perceive and understand their environment. Key areas include: * **2D Vision:** * **Classic Models:** CNN, ResNet, ViT, Swin Transformer. * **Generative Models:** Autoregressive models, Diffusion models. * **3D Vision:** * **Courses:** Andreas Geiger's 三维视觉导论, GAMES203 - 三维重建和理解. * **Classic Papers:** Diffusion Model for 2D/3D Generation, 3D生成相关论文-2024. * **4D Vision:** * **Video Understanding:** 开山之作, 论文串讲, LLM时代的视频理解综述. * **4D Generation:** Video Generation blog, 4D 生成的论文列表. * **Visual Prompting:** A method to guide large models with visual inputs. * **Affordance Grounding:** Locating interactive regions on objects. * **2D:** Cross-View-AG, AffordanceLLM. * **3D:** OpenAD, SceneFun3D.

This section covers the hardware and software tools essential for developing and deploying Embodied AI systems. * **Hardware:** * **Embedded Systems:** Platforms for running AI algorithms on robots. * **Mechanical Design:** Principles for designing robust and functional robot bodies. * **Robot System Design:** Integrating various components into a cohesive system. * **Sensors:** Devices for gathering environmental data (e.g., cameras, LiDAR). * **Tactile Sensing:** Technologies for enabling robots to feel and interact with objects. * **Software:** * **Simulators:** Tools for simulating robot environments and behaviors (e.g., MuJoCo, Isaac Lab, SAPIEN, Genesis). * **Benchmarks:** Standardized tasks for evaluating robot performance. * **Datasets:** Collections of data for training and testing AI models.

Explore curated lists of research papers to deepen your understanding of specific topics within Embodied AI: * **General Embodied AI:** Comprehensive lists covering various subfields. * **Specific Topics:** Lists focusing on areas like robot learning, computer vision, and multimodal models.

This guide provides a comprehensive overview of Embodied AI, covering essential resources, algorithms, and tools. By exploring these areas, newcomers can build a strong foundation and contribute to the advancement of this exciting field. The future of AI is embodied, and the journey starts here.