返回首页

大语言模型

共 53 篇论文

cs.SE大语言模型Transformer

Evaluating LLM-Based Test Generation Under Software Evolution

Large Language Models are increasingly being applied for automated unit test generation in software engineering. This paper presents a comprehensive empirical study evaluating the effectiveness of LLM-generated tests under program code evolution. Through a mutation-driven framework analyzing 22,374 program variants and 8 different LLMs, the researchers assess how generated tests respond to semantic-altering and semantic-preserving changes. The study reveals that while LLMs achieve strong baseline performance with 79% line coverage and 76% branch coverage on original programs, test quality degrades significantly during software evolution. This work provides critical insights into the limitations of current LLM-based testing approaches and highlights the need for more robust test generation methods that can adapt to code changes.

Research Team
26 days ago
arXiv 2603.23443v1
cs.CL大语言模型端到端

Off-Policy Value-Based Reinforcement Learning for Large Language Models

This paper addresses the critical challenge of improving data utilization efficiency in reinforcement learning for large language models. The authors propose ReVal, a novel Bellman-update-based method that enables off-policy learning through combining stepwise consistency signals with trajectory-level outcome verification. By supporting replay-buffer-based training, ReVal allows efficient reuse of past trajectories, significantly improving sample efficiency compared to on-policy approaches. Experimental results on mathematical reasoning benchmarks, particularly with DeepSeek-R1-Distill-1.5B, demonstrate that ReVal achieves faster convergence and superior final performance, with improvements of 2.7% on AIME24.

ReVal Team
26 days ago
arXiv 2603.23355v1
cs.CL大语言模型Transformer

MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation

Large language model (LLM)-based agents rely on memory mechanisms to reuse knowledge from past problem-solving experiences. Existing approaches typically construct memory in a per-agent manner, tightly coupling stored knowledge to a single model's reasoning style. In modern deployments with heterogeneous agents, a natural question arises: can a single memory system be shared across different models? We found that naively transferring memory between agents often degrades performance, as such memory entangles task-relevant knowledge with agent-specific biases. To address this challenge, we propose MemCollab, a collaborative memory framework that constructs agent-agnostic memory by contrasting reasoning trajectories generated by different agents on the same task.

MemCollab Authors
26 days ago
arXiv 2603.23234v1
cs.CV大语言模型端到端

End-to-End Training for Unified Tokenization and Latent Denoising

Latent diffusion models enable high-fidelity synthesis by operating in learned latent spaces. However, current approaches require complex multi-stage training where tokenizers must be trained separately before diffusion models. This paper proposes UNITE, an autoencoder architecture that unifies tokenization and latent diffusion through a single Generative Encoder with shared weights. The key innovation is treating tokenization and generation as the same latent inference problem under different conditioning regimes. The method introduces a single-stage training procedure that jointly optimizes both tokenization and generation tasks via two forward passes, enabling gradients to jointly shape the latent space for improved visual representation learning.

UNITE Authors
27 days ago
arXiv 2603.22283v1
cs.CV大语言模型

UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation

We present UniMotion, to our knowledge the first unified framework for simultaneous understanding and generation of human motion, natural language, and RGB images within a single architecture. Existing unified models handle only restricted modality subsets (e.g., Motion-Text or static Pose-Image) and predominantly rely on discrete tokenization, which introduces quantization errors and disrupts temporal continuity. UniMotion overcomes both limitations through a core principle: treating motion as a first-class continuous modality on equal footing with RGB. A novel Cross-Modal Aligned Motion VAE (CMA-VAE) and symmetric dual-path embedders construct parallel continuous pathways for Motion and RGB within a shared LLM backbone. To inject visual-semantic priors into motion representations without requiring images at inference, we propose Dual-Posterior KL Alignment (DPA), which distills a vision-fused encoder's richer posterior into the motion-only encoder. To address the cold-start problem -- where text supervision alone is too sparse to calibrate the newly introduced motion pathway -- we further propose Latent Reconstruction Alignment (LRA), a self-supervised pre-training strategy that uses dense motion latents as unambiguous conditions to co-calibrate the embedder, backbone, and flow head, establishing a stable motion-aware foundation for all downstream tasks. UniMotion achieves state-of-the-art performance across seven tasks spanning any-to-any understanding, generation, and editing among the three modalities, with especially strong advantages on cross-modal compositional tasks.

Ziyi Wang +4
27 days ago
arXiv 2603.22282v1
cs.CV大语言模型

ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model

Recent progress in latent world models (e.g., V-JEPA2) has shown promising capability in forecasting future world states from video observations. Nevertheless, dense prediction from a short observation window limits temporal context and can bias predictors toward local, low-level extrapolation, making it difficult to capture long-horizon semantics and reducing downstream utility. Vision--language models (VLMs), in contrast, provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames, but they are not ideal as standalone dense predictors due to compute-driven sparse sampling, a language-output bottleneck that compresses fine-grained interaction states into text-oriented representations, and a data-regime mismatch when adapting to small action-conditioned datasets. We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance via a dual-temporal pathway: a dense JEPA branch for fine-grained motion and interaction cues, and a uniformly sampled VLM \emph{thinker} branch with a larger temporal stride for knowledge-rich guidance. To transfer the VLM's progressive reasoning signals effectively, we introduce a hierarchical pyramid representation extraction module that aggregates multi-layer VLM representations into guidance features compatible with latent prediction. Experiments on hand-manipulation trajectory prediction show that our method outperforms both a strong VLM-only baseline and a JEPA-predictor baseline, and yields more robust long-horizon rollout behavior.

Haichao Zhang +7
27 days ago
arXiv 2603.22281v1
cs.CV大语言模型Transformer

3D-Layout-R1: Structured Reasoning for Language-Instructed Spatial Editing

Large Language Models and Vision Language Models demonstrate strong general reasoning capabilities but face challenges in spatial understanding and layout consistency for fine-grained visual editing tasks. This paper presents a Structured Reasoning framework that enables text-conditioned spatial layout editing through scene-graph reasoning. The system takes an input scene graph and natural-language instruction, then reasons over the graph structure to generate an updated scene graph satisfying the text condition while maintaining spatial coherence. By leveraging structured relational representations, the approach enhances both interpretability and control over spatial relationships. Evaluations on a text-guided layout editing benchmark covering sorting, spatial alignment, and room-editing tasks show that the training paradigm achieves an average 15% improvement in IoU and 25% reduction in center-distance error compared to Chain of Thought Fine-tuning baselines.

3D-Layout-R1 Authors
27 days ago
arXiv 2603.22279v1
cs.CV大语言模型CV

The Dual Mechanisms of Spatial Reasoning in Vision-Language Models

This paper investigates how vision-language models associate objects with their properties and spatial relations in multimodal tasks like image captioning and visual question answering. The research reveals that VLMs employ two concurrent mechanisms for spatial reasoning: the language model backbone represents content-independent spatial relations on visual tokens, while the vision encoder encodes object layouts that are directly utilized by the language model. The dominant spatial information originates from the vision encoder and is distributed globally across visual tokens, extending beyond object regions. These findings provide insights into the internal workings of multimodal models for spatial understanding.

Anonymous Authors
27 days ago
arXiv 2603.22278v1
cs.CL大语言模型Transformer

Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels

Weight-Decomposed Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction for more efficient LLM fine-tuning. However, the computational overhead of computing row-wise norms of the adapted weight matrices creates significant memory challenges, especially at high ranks and across hundreds of adapted modules. This work introduces a factored norm decomposition that eliminates dense matrix materialization by computing squared norms through base, cross, and Gram terms with O(d_out r + r^2) complexity. Additionally, fused Triton kernels combine the four-kernel DoRA composition into a single pass, achieving approximately 4x memory traffic reduction and numerical stability in near-unity rescaling regimes. These optimizations make high-rank DoRA feasible on common single-GPU setups for large language model adaptation.

DoRA Research Team
27 days ago
arXiv 2603.22276v1
cs.CL大语言模型

Greater accessibility can amplify discrimination in generative AI

Hundreds of millions of people rely on large language models (LLMs) for education, work, and even healthcare. Yet these models are known to reproduce and amplify social biases present in their training data. Moreover, text-based interfaces remain a barrier for many, for example, users with limited literacy, motor impairments, or mobile-only devices. Voice interaction promises to expand accessibility, but unlike text, speech carries identity cues that users cannot easily mask, raising concerns about whether accessibility gains may come at the cost of equitable treatment. Here we show that audio-enabled LLMs exhibit systematic gender discrimination, shifting responses toward gender-stereotyped adjectives and occupations solely on the basis of speaker voice, and amplifying bias beyond that observed in text-based interaction. Thus, voice interfaces do not merely extend text models to a new modality but introduce distinct bias mechanisms tied to paralinguistic cues. Complementary survey evidence ($n=1,000$) shows that infrequent chatbot users are most hesitant to undisclosed attribute inference and most likely to disengage when such practices are revealed. To demonstrate a potential mitigation strategy, we show that pitch manipulation can systematically regulate gender-discriminatory outputs. Overall, our findings reveal a critical tension in AI development: efforts to expand accessibility through voice interfaces simultaneously create new pathways for discrimination, demanding that fairness and accessibility be addressed in tandem.

Carolin Holtermann +5
27 days ago
arXiv 2603.22260v1
cs.CL大语言模型端到端

Confidence-Based Decoding is Provably Efficient for Diffusion Language Models

This paper addresses the decoding efficiency challenge in diffusion language models (DLMs), which have emerged as promising alternatives to autoregressive models for language generation. The research focuses on confidence-based decoding strategies that adaptively select and unmask tokens based on prediction confidence during the generation process. The authors develop a theoretical analysis framework specifically for an entropy sum-based strategy that continues unmasking tokens until cumulative entropy exceeds a threshold. This work provides the first formal theoretical guarantees demonstrating the efficiency of confidence-based decoding in diffusion-based text generation, offering insights into how adaptive token selection can improve sampling efficiency compared to traditional decoding approaches.

Anonymous Authors
27 days ago
arXiv 2603.22248v1
cs.CL大语言模型

MemDLM: Memory-Enhanced DLM Training

Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models, such as full-attention parallel decoding and flexible generation. However, they suffer from a notable train-inference mismatch: DLMs are trained with a static, single-step masked prediction objective, but deployed through a multi-step progressive denoising trajectory. We propose MemDLM (Memory-Enhanced DLM), which narrows this gap by embedding a simulated denoising process into training via Bi-level Optimization. An inner loop updates a set of fast weights, forming a Parametric Memory that captures the local trajectory experience of each sample, while an outer loop updates the base model conditioned on this memory. By offloading memorization pressure from token representations to parameters, MemDLM yields faster convergence and lower training loss. Moreover, the inner loop can be re-enabled at inference time as an adaptation step, yielding additional gains on long-context understanding. We find that, when activated at inference time, this Parametric Memory acts as an emergent in-weight retrieval mechanism, helping MemDLM further reduce token-level attention bottlenecks on challenging Needle-in-a-Haystack retrieval tasks. Code: https://github.com/JarvisPei/MemDLM.

Zehua Pei +6
27 days ago
arXiv 2603.22241v1
cs.CV大语言模型CV

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation

This paper presents SpatialReward, a novel verifiable reward model designed to evaluate fine-grained spatial relationships in text-to-image generation. The approach employs a multi-stage pipeline consisting of a Prompt Decomposer for extracting spatial metadata, expert detectors for visual grounding of object positions, and a vision-language model with chain-of-thought reasoning to assess complex spatial relations. By focusing on spatial layout evaluation rather than just semantic alignment, this work addresses a critical gap in current reward modeling approaches for generative AI systems.

Anonymous
27 days ago
arXiv 2603.22228v1
cs.CL大语言模型

Gumbel Distillation for Parallel Text Generation

The slow, sequential nature of autoregressive (AR) language models has driven the adoption of parallel decoding methods. However, these non-AR models often sacrifice generation quality as they struggle to model the complex joint distribution of token sequences. To narrow this performance gap, we introduce Gumbel Distillation, a novel distillation technique that enables parallel decoders to learn this distribution effectively. Our method leverages the Gumbel-Max trick to create a deterministic mapping from a latent Gumbel noise space to the output tokens of a high-performing AR teacher. As a model-agnostic technique, Gumbel Distillation seamlessly integrates with diverse parallel decoding architectures, including MDLM and BD3-LM. Experiments on LM1B and OpenWebText show that Gumbel Distillation substantially improves the generation quality of parallel language models, achieving a 30.0% improvement in MAUVE score and 10.5% in generative perplexity over MDLM trained on OpenWebText dataset. Code available at https://github.com/hxixixh/gumbel-distill.

Chi Zhang +3
27 days ago
arXiv 2603.22216v1
cs.CR大语言模型

Evaluating the Reliability and Fidelity of Automated Judgment Systems of Large Language Models

A Large Language Model (LLM) as judge evaluates the quality of victim Machine Learning (ML) models, specifically LLMs, by analyzing their outputs. An LLM as judge is the combination of one model and one specifically engineered judge prompt that contains the criteria for the analysis. The resulting automation of the analysis scales up the complex evaluation of the victim models' free-form text outputs by faster and more consistent judgments compared to human reviewers. Thus, quality and security assessments of LLMs can cover a wide range of the victim models' use cases. Being a comparably new technique, LLMs as judges lack a thorough investigation for their reliability and agreement to human judgment. Our work evaluates the applicability of LLMs as automated quality assessors of victim LLMs. We test the efficacy of 37 differently sized conversational LLMs in combination with 5 different judge prompts, the concept of a second-level judge, and 5 models fine-tuned for the task as assessors. As assessment objective, we curate datasets for eight different categories of judgment tasks and the corresponding ground-truth labels based on human assessments. Our empirical results show a high correlation of LLMs as judges with human assessments, when combined with a suitable prompt, in particular for GPT-4o, several open-source models with $\geqslant$ 32B parameters, and a few smaller models like Qwen2.5 14B.

Tom Biskupski +1
27 days ago
arXiv 2603.22214v1
cs.CL大语言模型Transformer

SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection

This paper addresses the challenge of incomplete knowledge coverage in large language models, particularly in specialized and data-scarce domains. The authors propose SPA (Scaling Prompt-engineered Augmentation), a method that uses carefully designed prompts to generate large-scale synthetic training data for knowledge injection. Through systematic comparisons, SPA demonstrates superior performance over strong baselines. The research identifies key limitations in existing approaches: RL-based methods suffer from diversity collapse at scale, while multi-stage prompting advantages disappear after careful tuning. These findings provide valuable insights for optimizing knowledge injection strategies in language models.

Anonymous
27 days ago
arXiv 2603.22213v1
cs.LG大语言模型

Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs

Multi-agent applications often execute complex tasks as multi-stage workflows, where each stage is an LLM call whose output becomes part of context for subsequent steps. Existing LLM serving systems largely assume homogeneous clusters with identical model replicas. This design overlooks the potential of heterogeneous deployments, where models of different sizes and capabilities enable finer trade-offs between latency and performance. However, heterogeneity introduces new challenges in scheduling across models with diverse throughput and performance. We present Chimera, a predictive scheduling system for multi-agent workflow serving on heterogeneous LLM clusters that jointly improves end-to-end latency and task performance. Chimera applies semantic routing to estimate per-model confidence scores for each request, predicts the total remaining output length of the workflow, and estimates per-model congestion using in-flight predicted token volumes for load balancing. We evaluate Chimera on representative agentic workflows for code generation and math reasoning using multiple heterogeneous LLM configurations. Across comparable settings, Chimera traces the best latency-performance frontier, reducing end-to-end latency by 1.2--2.4$\times$ and improving task performance by 8.0-9.5 percentage points on average over competitive baselines including vLLM.

Kangqi Ni +5
27 days ago
arXiv 2603.22206v1
cs.AI大语言模型端到端

CayleyPy-4: AI-Holography. Towards analogs of holographic string dualities for AI tasks

This paper presents the fourth installment of the CayleyPy project, applying AI methods to explore large graphs through a novel discrete version of holographic string dualities. The research demonstrates that many modern AI tasks, particularly GPT-style language models and reinforcement learning systems, can be modeled as predicting particle trajectories on graphs. For Cayley graphs of the symmetric group S_n, the authors establish a dual description in terms of discrete strings and propose string holographic images as natural candidates for data embeddings, extending the complexity equals volume principle from AdS/CFT to AI applications. This work hypothesizes potential extensions of such dualities across a range of AI systems, suggesting more efficient computational approaches for language modeling and general AI tasks.

CayleyPy Project Team
27 days ago
arXiv 2603.22195v1
cs.CV大语言模型

Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement

Recent advances in Multimodal Large Language Models (MLLMs) have enabled automated generation of structured layouts from natural language descriptions. Existing methods typically follow a code-only paradigm that generates code to represent layouts, which are then rendered by graphic engines to produce final images. However, they are blind to the rendered visual outcome, making it difficult to guarantee readability and aesthetics. In this paper, we identify visual feedback as a critical factor in layout generation and propose Visual Feedback Layout Model (VFLM), a self-improving framework that leverages visual feedback iterative refinement. VFLM is capable of performing adaptive reflective generation, which leverages visual information to reflect on previous issues and iteratively generates outputs until satisfactory quality is achieved. It is achieved through reinforcement learning with a visually grounded reward model that incorporates OCR accuracy. By rewarding only the final generated outcome, we can effectively stimulate the model's iterative and reflective generative capabilities. Experiments across multiple benchmarks show that VFLM consistently outperforms advanced MLLMs, existing layout models, and code-only baselines, establishing visual feedback as critical for design-oriented MLLMs. Our code and data are available at https://github.com/FolSpark/VFLM.

Junrong Guo +3
27 days ago
arXiv 2603.22187v1
cs.CL大语言模型端到端

Enhancing Document-Level Machine Translation via Filtered Synthetic Corpora and Two-Stage LLM Adaptation

This paper addresses the challenge of improving document-level machine translation using Large Language Models. The authors propose a two-stage fine-tuning strategy that augments training data by converting summarization data into document-level parallel data using LLMs. To ensure data quality, they filter the synthetic corpus using multiple metrics including sacreBLEU, COMET, and LaBSE-based cosine similarity. The approach tackles two key challenges: the scarcity of large-scale document-level parallel data and the tendency of LLMs to generate hallucinations and omissions. By leveraging LLMs' strength in modeling contextual information, this method aims to improve coherence across sentences in translation tasks.

Ireh Kim +2
27 days ago
arXiv 2603.22186v1
第 1 / 3 页下一页