Transformer

arXiv 2603.20390v1

cs.CLTransformer具身智能

The Production of Meaning in the Processing of Natural Language

Understanding the fundamental mechanisms governing the production of meaning in natural language processing is critical for designing safe and engaging human-agent interactions. Research in cognitive science and social psychology has demonstrated that human semantic processing exhibits contextuality more consistent with quantum logical mechanisms than classical Boolean theories. Recent studies have found similar quantum-like behavioral signatures in large language models, including clear violations of the Bell inequality during interpretation of ambiguous expressions. This work explores the CHSH parameter across the inference parameter space of language models spanning four orders of magnitude in scale, cross-referencing findings with MMLU benchmarks, hallucination rates, and nonsense detection metrics to understand semantic contextuality in neural language systems.

arXiv 2603.20381v1

cs.CV自动驾驶CV

IndoorR2X: Indoor Robot-to-Everything Coordination with LLM-Driven Planning

IndoorR2X introduces a novel benchmark and simulation framework for LLM-driven multi-robot task planning in indoor environments. The system enables Robot-to-Everything communication by integrating observations from mobile robots and static IoT sensors like cameras to construct a global semantic state. This approach overcomes partial observability challenges that traditional R2R communication faces, reducing redundant exploration and enabling scalable scene understanding. The framework provides configurable simulation environments, sensor layouts, robot teams, and task suites for evaluating coordinated indoor robot operations.

IndoorR2X Authors

arXiv 2603.20182v1

cs.CV自动驾驶CV

The Robot's Inner Critic: Self-Refinement of Social Behaviors through VLM-based Replanning

This paper presents CRISP (Critique-and-Replan for Interactive Social Presence), an autonomous framework enabling robots to critique and replan their own social behaviors using a Vision-Language Model as a human-like social critic. The framework integrates joint and constraint extraction from robot description files, step-by-step behavior planning, low-level joint control code generation from visual information, VLM-based evaluation of social appropriateness, and iterative refinement through reward-based search. This approach enables robots to generate human-like, socially appropriate motions across various platforms with improved autonomy and naturalness.

Author 1 +1

arXiv 2603.20164v1

cs.RO端到端Transformer

AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning

This paper presents AGILE, an end-to-end workflow designed to address the challenges of transferring reinforcement learning policies from simulation to real humanoid robots. The framework standardizes the policy-development lifecycle through four key stages: interactive environment verification, reproducible training, unified evaluation, and descriptor-driven deployment. By mitigating common sim-to-real failure modes, AGILE enables systematic development of loco-manipulation skills for humanoid robots. The approach includes scenario-based tests and randomized rollouts under motion-quality diagnostics for automated regression testing and robustness assessment.

AGILE Team

arXiv 2603.20147v1

cs.CVCVTransformer

Morphology-Consistent Humanoid Interaction through Robot-Centric Video Synthesis

This paper presents Dream2Act, a robot-centric framework enabling zero-shot interaction for humanoid robots through generative video synthesis. By taking a third-person image of the robot and target object, the framework leverages video generation models to synthesize morphology-consistent motion for task completion. The approach employs a high-fidelity pose extraction system to recover physically feasible, robot-native joint trajectories from the synthesized videos. These trajectories are subsequently executed via a general-purpose whole-body controller, eliminating the need for extensive policy training or explicit motion retargeting that suffers from morphology gaps.

Dream2Act Team

arXiv 2603.19709v2

cs.CVCVTransformer

In-the-Wild Camouflage Attack on Vehicle Detectors through Controllable Image Editing

Deep neural networks have achieved remarkable success in computer vision but remain highly vulnerable to adversarial attacks. This paper proposes a new framework that formulates vehicle camouflage attacks as a conditional image-editing problem, exploring both image-level and scene-level camouflage generation strategies. The method fine-tunes a ControlNet to synthesize camouflaged vehicles directly on real images while enforcing vehicle structural fidelity, style consistency, and adversarial effectiveness through a unified objective. Experiments on COCO and LINZ datasets demonstrate that the approach achieves significantly stronger attack effectiveness with more than 38% AP50 decrease, while better preserving vehicle structure and improving human-perceived stealthiness compared to existing methods.

arXiv 2603.19456v1

cs.CL端到端Transformer

The Autonomy Tax: Defense Training Breaks LLM Agents

Large language model agents increasingly rely on external tools such as file operations, API calls, and database transactions to autonomously complete complex multi-step tasks. Practitioners deploy defense-trained models to protect against prompt injection attacks that manipulate agent behavior through malicious observations. This paper reveals a fundamental capability-alignment paradox where defense training designed to improve safety systematically destroys agent competence while failing to prevent sophisticated attacks. Evaluating defended models against undefended baselines across 97 agent tasks and 1,000 adversarial prompts, we uncover three systematic biases unique to multi-step agents including agent incompetence bias manifesting as immediate tool execution breakdown and cascade amplification bias causing early failures to propagate through retry loops. These findings demonstrate that current defense training approaches create significant trade-offs between safety and functional capability in autonomous LLM agents.

arXiv 2603.19423v1

cs.CVCVTransformer

Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

This paper addresses the spatial blindness problem in Multimodal Large Language Models by leveraging implicit 3D priors learned in video generation models. The authors propose VEGA-3D, a framework that repurposes pre-trained video diffusion models as latent world simulators to extract robust 3D structural priors and physical understanding. By extracting spatiotemporal features from intermediate noise levels and integrating them with semantic representations through token-level adaptive gated fusion, the method enriches MLLMs with dense geometric cues for improved scene understanding and geometric reasoning capabilities.

VEGA-3D Authors

arXiv 2603.19235v1

cs.CVCVTransformer

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

Vision-Language-Action (VLA) models integrate perception, language understanding, and motor control into unified architectures for robotic control. This study systematically investigates how VLAs process multimodal inputs to generate actions through activation injection, sparse autoencoders, and linear probes across six models (80M-7B parameters) using 394,000+ rollout episodes. The research reveals that the visual pathway is the dominant factor in action generation across all architectures, as injecting baseline activations into null-prompt episodes reproduces nearly identical behavior. Cross-task injection experiments demonstrate spatially bound motor programs tied to scene coordinates rather than abstract task representations. Language sensitivity is found to depend on task structure rather than model design: when visual context uniquely specifies the task, language is ignored, but when multiple goals share a scene, language becomes essential (achieving 94% accuracy in X-VLA libero_goal). These findings provide mechanistic insights into VLA operation and have implications for embodied AI systems requiring precise vision-language coordination for robotic manipulation.

Research Team

arXiv 2603.19233v1

cs.CVCVTransformer

OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation

This paper presents OmniVTA, a world-model-based visuo-tactile manipulation framework designed for contact-rich robotic manipulation tasks such as wiping and assembly. The work introduces OmniViTac, a large-scale dataset comprising 21,000+ trajectories across 86 tasks and 100+ objects with six physics-grounded interaction patterns. The framework integrates four tightly coupled modules including a self-supervised tactile encoder and a two-stream visuo-tactile world model for predicting contact dynamics. The research addresses limitations in existing methods by treating tactile signals actively to model contact dynamics and enable explicit closed-loop control, moving beyond passive observation approaches.

OmniVTA Authors

arXiv 2603.19201v2

cs.CVCVTransformer

Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models

Vision-Language-Action (VLA) models have emerged as a promising approach for general-purpose robot manipulation, combining visual perception, language understanding, and action planning in a unified framework. This work applies mechanistic interpretability techniques using Sparse Autoencoders (SAEs) to analyze hidden layer activations in VLA models, revealing sparse dictionary features that provide interpretable bases for model computation. The research discovers that most SAE features correspond to memorized sequences from training demonstrations, while some features represent interpretable, generalizable motion primitives and semantic properties. This analysis offers insights into VLA model generalizability and provides a framework for steering model behavior through identified interpretable features, advancing the understanding of embodied AI systems for robot manipulation tasks.

arXiv 2603.19183v1

cs.CVCVTransformer

MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model

Spatial reasoning is foundational for Vision-Language Models deployed as Vision-Language-Action agents in physical environments. This paper introduces MultihopSpatial, a benchmark designed for multi-hop and compositional spatial reasoning with complex queries across diverse spatial perspectives. The work proposes Acc@50IoU, a metric that simultaneously evaluates reasoning and visual grounding by requiring both answer selection and precise bounding box prediction. MultihopSpatial-Train provides a large-scale training corpus to foster spatial intelligence. Extensive evaluation of 37 state-of-the-art VLMs yields key insights into the capabilities and limitations of current models for robust VLA deployment in embodied AI scenarios.

MultihopSpatial Team

arXiv 2603.18892v1

cs.RO端到端Transformer

RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids

This paper presents RoboForge, a unified latent-driven framework for text-guided whole-body humanoid locomotion. The approach bridges natural language and physical robot execution through a retarget-free pipeline that couples motion generation and control bidirectionally. A Physical Plausibility Optimization module serves as the coupling interface, refining teacher-student distillation policies with plausibility-centric rewards to eliminate physical artifacts like floating, skating, and penetration. This work enables humanoid robots to execute natural language-directed motions with improved physical feasibility.

RoboForge Authors