Transformer

共 74 篇论文

cs.CL大语言模型Transformer

Causal Evidence that Language Models use Confidence to Drive Behavior

Metacognition, the ability to assess one's own cognitive performance, is a fundamental capability documented across species where internal confidence estimates guide adaptive behavior. This research investigates whether Large Language Models (LLMs) actively utilize confidence signals to regulate their behavior through a four-phase abstention paradigm. The study first establishes internal confidence estimates without abstention options, then reveals that LLMs apply implicit thresholds to these estimates when deciding whether to answer or abstain. Findings demonstrate that confidence serves as the dominant predictor of behavior, with effect sizes an order of magnitude larger than knowledge retrieval accessibility or semantic features. Causal evidence is provided through activation steering experiments, where manipulating internal confidence signals correspondingly shifts abstention rates, demonstrating a direct causal relationship between confidence estimation and behavioral regulation.

arXiv 2603.22161v1

cs.CV大语言模型CV

dynActivation: A Trainable Activation Family for Adaptive Nonlinearity

This paper introduces dynActivation, a per-layer trainable activation function that dynamically interpolates between base nonlinearities and linear paths using lightweight learned scalars. The proposed method is evaluated across vision tasks (CIFAR-10, MNIST) and language modeling tasks, demonstrating significant improvements in training efficiency (up to 54% faster) and performance. On CIFAR-10, dynActivation(Mish) achieves up to 14.02% improvement over static Mish, with 24% reduction in convergence time. In deep network scaling experiments (up to 75 layers), dynActivation maintains robust performance (95.3-99.3% accuracy) while ReLU collapses below 80%, demonstrating that adaptive nonlinearity linearization in deep layers enhances both training stability and final model quality.

dynActivation Authors

arXiv 2603.22154v1

cs.CV自动驾驶CV

OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation

This paper presents OpenEarth-Agent, the first tool-creation agent framework designed for open-environment Earth Observation. Unlike traditional tool-calling agents confined to closed environments with pre-defined tools, OpenEarth-Agent employs adaptive workflow planning and dynamic tool creation to generalize across diverse multi-source satellite data and heterogeneous remote sensing tasks. The framework integrates multi-stage tools with cross-domain knowledge bases, enabling robust execution throughout the entire Earth Observation pipeline. This approach represents a significant advancement in autonomous EO systems by moving beyond narrow, predefined toolkits to an open-ended, adaptive framework capable of handling unseen data modalities and novel tasks in land surface monitoring applications.

OpenEarth-Agent Team

arXiv 2603.22148v1

cs.CL大语言模型Transformer

Autoregressive vs. Masked Diffusion Language Models: A Controlled Comparison

This paper presents a controlled empirical comparison between autoregressive (AR) and masked diffusion (MDLM) language models trained on identical data (50M tokens from TinyStories), compute budget (20,000 steps, batch size 32, sequence length 512), and hardware (NVIDIA H100 80GB). The study reveals three key findings: both paradigms achieve comparable training throughput (~50K tokens/second), AR converges faster but overfits earlier while MDLM improves more gradually, and there exists a structural diversity-fluency trade-off with AR producing fluent but repetitive outputs and MDLM generating more diverse narratives.

Anonymous Authors

arXiv 2603.22075v1

cs.CL大语言模型端到端

On the Failure of Topic-Matched Contrast Baselines in Multi-Directional Refusal Abliteration

This paper investigates the effectiveness of topic-matched contrast baselines in multi-directional refusal abliteration for instruction-tuned language models. The research focuses on extracting refusal-mediating directions from the residual stream activation space of the Qwen 3.5 2B model using per-category matched prompt pairs, Self-Organizing Map extraction, and Singular Value Decomposition orthogonalization. The key finding reveals that while unmatched contrast successfully achieves complete refusal elimination across tested weight levels and layers, topic-matched contrast produces no functional refusal directions. This work highlights the importance of contrast baseline construction as a critical methodological consideration rather than an implementation detail in model abliteration research.

arXiv 2603.22061v1

cs.CL大语言模型端到端

Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch

This paper addresses the challenge of knowledge distillation between large language models with different tokenizers. The authors systematically analyze the attention mechanism of Dual-Space Knowledge Distillation with Cross-Model Attention (DSKD-CMA), revealing its strengths and limitations through token alignment probing and visualization. They propose DSKD-CMA-GA, a novel method leveraging Generative Adversarial learning to better align mismatched key-query distributions across models. Experimental results demonstrate consistent improvements in text generation quality as measured by ROUGE-L scores, offering a more transparent and effective approach to compressing large language models for efficient deployment.

arXiv 2603.22056v1

cs.CV自动驾驶CV

Future-Interactions-Aware Trajectory Prediction via Braid Theory

This paper presents a novel approach to multi-agent trajectory prediction for autonomous vehicles using braid theory as a fundamental mathematical framework. The method projects future trajectories of interacting agents into braids that precisely describe how trajectories cross over time, enabling exact representation of coordination modes between agents. By fully leveraging the expressivity of the braid representation, the approach conditions predicted trajectories on specific interaction patterns, moving beyond traditional heuristic-based behavior labeling methods. This results in improved joint prediction accuracy while maintaining computational efficiency compared to existing approaches that rely on extensive computational resources.

Anonymous Authors

arXiv 2603.22035v1

cs.CL大语言模型端到端

Retrieving Climate Change Disinformation by Narrative

This paper addresses the challenge of detecting climate disinformation narratives by re-framing traditional detection as a retrieval task. Rather than relying on fixed taxonomies that cannot accommodate emerging narratives, the approach ranks texts by alignment with a given narrative core message. The proposed SpecFi framework generates hypothetical documents to bridge abstract narrative descriptions with concrete textual instantiations, using community summaries from graph-based detection as few-shot examples for generation. The method achieves a MAP of 0.505 on the CARDS dataset without accessing narrative labels. Additionally, the paper introduces narrative variance, an embedding-based difficulty metric, and demonstrates its utility through partial correlation analysis for understanding narrative complexity in climate disinformation detection.

Anonymous Authors

arXiv 2603.22015v1

cs.AI大语言模型端到端

TREX: Trajectory Explanations for Multi-Objective Reinforcement Learning

This paper presents TREX, a Trajectory-based Explainability framework designed for Multi-Objective Reinforcement Learning (MORL). The work addresses the limitation that traditional Explainable Reinforcement Learning (XRL) methods are typically tailored for single scalar rewards and fail to provide explanations when agents optimize multiple conflicting objectives simultaneously. The proposed approach enables agents to explicitly reason about trade-offs between different objectives and generates interpretable explanations for the decision-making process behind objective trade-offs. By focusing on trajectory-level explanations, TREX provides insights into how agents navigate decision spaces when balancing competing objectives in complex real-world scenarios.

Multiple Authors

arXiv 2603.21988v1

cs.CL大语言模型端到端

SecureBreak: A Dataset Towards Safe and Secure Models

Large language models are becoming pervasive core components in many real-world applications. As a consequence, security alignment represents a critical requirement for their safe deployment. Although previous related works focused primarily on model architectures and alignment methodologies, these approaches alone cannot ensure the complete elimination of harmful generations. This concern is reinforced by the growing body of scientific literature showing that attacks, such as jailbreaking and prompt injection, can bypass existing security alignment mechanisms. As a consequence, additional security strategies are needed both to provide qualitative feedback on the robustness of the obtained security alignment at the training stage, and to create an ultimate defense layer to block unsafe outputs possibly produced by deployed models. To provide a contribution in this scenario, this paper introduces SecureBreak, a safety-oriented dataset designed to support the development of AI-driven security solutions for language models.

SecureBreak Team

arXiv 2603.21975v1

cs.AI大语言模型端到端

Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe

This paper presents a systematic empirical study on scaling Reinforcement Learning for Large Language Model agents in complex, multi-turn environments. Using TravelPlanner as a testbed, the authors decompose the agentic RL design space along five critical axes including reward shaping, model scaling, data composition, algorithm selection, and environmental stability. Their controlled experiments reveal key insights: reward and algorithm choices are scale-dependent with smaller models benefiting from staged rewards while larger models converge with simpler dense rewards, approximately 1K training samples with balanced difficulty mixture represents the optimal training budget, and environmental stability is crucial for preventing policy degradation. The work provides a practical recipe for developing autonomous LLM agents capable of long-horizon tool orchestration and planning.

TravelPlanner Research Team

arXiv 2603.21972v1

cs.CL大语言模型端到端

SLURP-TN: Resource for Tunisian Dialect Spoken Language Understanding

This paper introduces SLURP-TN, a comprehensive spoken language understanding resource specifically designed for the Tunisian dialect. The dataset comprises 4165 sentences recorded from 55 native speakers, totaling approximately 5 hours of acoustic material. By translating sentences from six SLURP domains, the authors address the critical gap in SLU resources for low-resource languages. The research develops baseline Automatic Speech Recognition and SLU models that leverage deep neural networks and pre-trained language models to extract semantic information from speech utterances in task-oriented dialogue systems. This work enables the Tunisian-speaking population to benefit from recent advances in natural language processing and speech recognition technology.

Elyadata Research Team

arXiv 2603.21940v1

cs.CV大语言模型CV

Guideline-grounded retrieval-augmented generation for ophthalmic clinical decision support

This paper presents Oph-Guid-RAG, a multimodal visual retrieval-augmented generation system designed for ophthalmology clinical decision support. The system treats guideline pages as independent evidence units and retrieves page images while preserving visual elements like tables and flowcharts. It implements a controllable retrieval framework with routing and filtering mechanisms to reduce noise while selectively introducing external evidence. The system combines query decomposition, rewriting, retrieval, reranking, and multimodal reasoning to provide traceable outputs with guideline references. Evaluated on HealthBench with doctor-based scoring, the approach significantly outperforms GPT-5.2 and GPT-5.4 on hard subsets, achieving +30.0% improvement in overall score and +10.4% to +24.4% gains in accuracy.

Oph-Guid-RAG Team

arXiv 2603.21925v1

cs.CL大语言模型端到端

P^2O: Joint Policy and Prompt Optimization

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs). However, vanilla RLVR suffers from inefficient exploration, particularly when confronting hard samples that yield near-zero success rates. In such scenarios, the reliance on sparse outcome rewards typically results in zero-advantage estimates, effectively starving the model of supervision signals despite the high informational value of these instances. To address this, we propose P^2O, a novel framework that synergizes Prompt Optimization with Policy Optimization. P^2O identifies hard samples during training iterations and leverages the GeneticPareto (GEPA) prompt optimization algorithm to evolve prompt templates that guide the model toward discovering successful trajectories.

arXiv 2603.21877v1

cs.CL大语言模型Transformer

Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization

Scaling laws for Large Language Models govern macroscopic resource allocation, yet translating them into precise Mixture-of-Experts (MoE) architectural configurations remains an open problem due to the combinatorially vast design space. This paper proposes a reusable framework for holistic MoE architectural optimization that addresses limitations in existing approaches. The authors demonstrate that FLOPs per token alone is insufficient for fair comparison of MoE models because varying computational densities across layer types can inflate parameters without proportional compute cost. They establish a joint constraint triad of FLOPs per token, active parameters, and total parameters, then reduce the 16-dimensional architectural search space to two sequential low-dimensional phases through algebraic constraints.

arXiv 2603.21862v1

cs.CL大语言模型Transformer

Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models

This paper investigates whether large language models demonstrate genuine moral reasoning capabilities or merely produce superficially convincing reasoning-like outputs. The study analyzes responses from 13 different LLMs across six classical moral dilemmas using Kohlberg's stages of moral development as an evaluation framework. Through an LLM-as-judge scoring pipeline validated across three judge models, over 600 responses were classified and analyzed. The research reveals a significant finding that LLM responses predominantly exhibit post-conventional reasoning patterns (Stages 5-6), which contradicts typical human developmental trajectories where such reasoning emerges later in moral development. This inversion suggests that alignment training may produce outputs that mimic advanced moral reasoning without the underlying developmental progression characteristic of human moral cognition.

arXiv 2603.21854v1

cs.CV自动驾驶CV

Multi-View Deformable Convolution Meets Visual Mamba for Coronary Artery Segmentation

This paper presents MDSVM-UNet, a novel two-stage framework for accurate coronary artery segmentation from CTA images. The proposed method integrates multi-view deformable convolution with Visual Mamba state space models to effectively capture long-range dependencies in vascular structures while maintaining linear computational complexity. This approach addresses challenges in segmenting complex multi-branching coronary arteries with severe class imbalance, enabling improved diagnosis and treatment planning for cardiovascular diseases. The synergy between deformable convolutions and Mamba architecture provides an efficient solution suitable for clinical deployment in resource-constrained environments.

arXiv 2603.21829v1

cs.AITransformer具身智能

The Presupposition Problem in Representation Genesis

Large language models represent the first systems to achieve high cognitive performance without clearly undergoing representation genesis - the transition from non-representing physical systems to content-sensitive behavior-guiding states. This paper examines the genesis question in LLMs, investigating which cognitive capacities are affected if genesis did not occur. The authors argue that major frameworks in philosophy of mind, including the Language of Thought hypothesis, teleosemantics, predictive processing, enactivism, and genetic phenomenology, share a common structural feature when addressing the genesis question, lacking the conceptual resources to fully explain LLM representation. The paper provides a theoretical analysis of how contemporary AI systems challenge traditional assumptions about representation and cognition.

arXiv 2603.21745v1

cs.CL自动驾驶端到端

MIND: Multi-agent Inference for Negotiation Dialogue in Travel Planning

This paper introduces MIND (Multi-agent Inference for Negotiation Dialogue), a framework for simulating realistic consensus-building among travelers with heterogeneous preferences in travel planning scenarios. Grounded in Theory of Mind (ToM), MIND incorporates a Strategic Appraisal phase that achieves 90.2% accuracy in inferring opponent willingness from linguistic nuances. The framework demonstrates significant improvements over traditional Multi-Agent Debate (MAD) approaches, with a 20.5% improvement in High-w Hit and 30.7% increase in Debate Hit-Rate. Qualitative evaluations using LLM-as-a-Judge confirm superior performance in Rationality (68.8%) and Fluency (72.4%), achieving an overall win rate of 68.3%. This work effectively models human negotiation dynamics through advanced language understanding and multi-agent reasoning.

arXiv 2603.21696v1

cs.CV自动驾驶CV

BiPreManip: Learning Affordance-Based Bimanual Preparatory Manipulation through Anticipatory Collaboration

This paper addresses bimanual robot manipulation tasks where coordinated actions between two robotic arms are required for complex object interactions. The work introduces a Collaborative Preparatory Manipulation framework that enables robots to perform sequential preparatory actions - such as pushing objects to accessible positions or lifting items - to facilitate subsequent goal-directed manipulations by the other arm. The proposed visual affordance-based approach first anticipates the final task objective and then generates appropriate preparatory manipulations, requiring deep understanding of object geometry, spatial relationships, and semantic properties. By learning from demonstrations and employing vision-based affordance recognition, the framework achieves effective bimanual coordination for tasks involving objects that are difficult to grasp directly.

BiPreManip Authors

arXiv 2603.21679v1

上一页第 2 / 4 页下一页