Chaperone-Thinking-LQ-1.0, a 4-bit GPTQ + QLoRA fine-tuned DeepSeek-R1-32B model, has been open-sourced. It achieves 84% accuracy on MedQA, close to GPT-4o, while being only ~20GB in size and 1.6x faster than the base model.
This study investigates hallucinations in domain-adapted Large Language Models, specifically Llama-2 fine-tuned with the Lamini dataset. It found that while the model excels in training-similar tasks, its ability to reason about and recall new domain-specific information is limited, leading to hallucinations and a tendency for over-generation.
This paper argues that the prevailing post-training paradigm for LLMs, involving SFT and RL, effectively reverts to the "pre-train then fine-tune" approach, explicitly tailoring models to specific benchmarks. Empirical evidence shows that models post-trained from scratch can yield significant performance on reasoning datasets.
An ML team documented the technical challenges faced while fine-tuning and deploying Gemma-4. Key issues included PEFT's incompatibility with Gemma 4's custom layers, SFTTrainer silently breaking KV-sharing attention, and DeepSpeed ZeRO-3 saving half-empty LoRA adapters.
This research proposes an unsupervised method to identify community-specific slang and unique entities by analyzing the magnitude of semantic shift. Semantic shift is defined as the evolution of a word's encoded representation after fine-tuning a pre-trained Large Language Model (LLM) on a community-specific text corpus.
Um desenvolvedor treinou um modelo Qwen3.5-9B com LoRA para atuar como analista de dados agente, focando em autonomia através de pesos. O modelo alcançou 89% de conclusão de fluxos de trabalho de ponta a ponta sem intervenção humana, superando a falha total do modelo base.
The author is transitioning from fine-tuning dense 3B/7B transformers to NVIDIA's Nemotron 3 Nano (a hybrid Mamba-Attention-MoE architecture) for multi-task reasoning. They are seeking guidance on how the hybrid architecture impacts standard LoRA fine-tuning, as their prior experience is limited to dense models.
A self-taught user new to fine-tuning seeks advice on choosing between 3B and 7B LLM models for a multi-task reasoning project. The project involves understanding underlying questions, maintaining multiple perspectives, and handling messy inputs.
This content reviews the Qwen3.6 35B A3B Heretic model, praising it as the best uncensored 35B model the user has found. It highlights its ability to fit in 24GB VRAM, handle multi-turn tool calls, and its potential to benchmark higher than the original Qwen 3.6 model.
Aletheia introduces a gradient-guided layer selection method for LoRA fine-tuning, identifying the most task-relevant layers and applying adapters selectively with asymmetric rank. This approach achieves a significant 15-28% training speedup across diverse large language models and architectures while broadly matching downstream behavior.
This research introduces a rubric-based Generative Reward Model (GRM) to enhance Reinforced Fine-Tuning (RFT) for LLM Agents in Software Engineering (SWE) tasks. By providing richer learning signals beyond binary terminal rewards, this approach shapes intermediate behaviors and significantly improves the quality of the resolution process.
Discrete Tilt Matching (DTM) is a novel likelihood-free method for fine-tuning masked diffusion large language models (dLLMs), addressing the intractability of sequence-level marginal likelihoods in RL. It recasts fine-tuning as state-level matching, using a weighted cross-entropy objective with control variates for stability, and achieves strong results on various tasks like Sudoku and Countdown.
The title suggests that fine-tuning local AI models using
StepPRM-RTL is a novel framework that enhances LLM-based RTL code generation by combining stepwise trajectory modeling, process-reward modeling (PRM), and retrieval-augmented fine-tuning (RAFT). It uses dense feedback from a PRM to guide reinforcement-style updates and Monte Carlo Tree Search (MCTS) to enrich the training dataset.
This research introduces a data-efficient fine-tuning framework to teach large language models to effectively code-switch for reasoning tasks. It identifies beneficial code-switched behaviors, moving beyond treating code-switching as an error, through systematic analysis of diverse reasoning traces.
O1-Pruner introduces a length-harmonizing fine-tuning method aimed at improving reasoning capabilities through model pruning. This technique focuses on optimizing models for specific O1-like reasoning tasks.
This post explains how to use Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to enhance the tool-calling accuracy of small language models. It details how to leverage Amazon SageMaker AI training jobs to focus on training code and evaluate model quality.
This article explains how the LoRA (Low-Rank Adaptation) technique enables fine-tuning billion-parameter language models on consumer hardware like laptops. Instead of updating all parameters, LoRA adds tiny trainable modules, drastically reducing GPU memory requirements.
Polynomial Expansion Rank Adaptation (PERA) is a novel method to enhance low-rank adaptation (LoRA) for fine-tuning large language models. It introduces structured polynomial expansion into the low-rank factor space to model richer nonlinear high-order interactions, overcoming LoRA's linear limitations without increasing rank or inference cost.
This paper analyzes the interpretive behaviors of LLMs for automated code compliance using perturbation-based attribution analysis, comparing different fine-tuning strategies and model scales. Results show full fine-tuning yields more focused attribution patterns, and larger models prioritize specific textual elements like numerical constraints.