Risk-Aware Large Language Model Fine-Tuning with Tail-Robust Objectives and Feasible Gradient Updates

Authors
  • Yihang Wang

    University of Southern California, Los Angeles, USA

Abstract
This paper addresses the problem of unstable updates and difficulty in constraining behavioral boundaries in large language models during real-world applications due to data noise and distribution perturbations. A robust fine-tuning algorithm framework based on gradient projection and constraint optimization is proposed. The method uniformly models the fine-tuning process as a constrained optimization problem. On the objective side, worst-case robust risk and tail risk are introduced to enhance the resistance to abnormal samples and perturbation inputs. On the constraint side, a feasible region composed of output distribution offset constraints and parameter trust region constraints is constructed to limit distribution drift and parameter drift caused by fine-tuning. To ensure the feasibility of iterative updates, a gradient projection mechanism is designed to decompose and correct the original update direction into an effective descent direction that satisfies the approximation of the local feasible region. Simultaneously, primal-dual updates are used to dynamically allocate constraint pressure, forming a closed-loop synergy between constraint satisfaction and objective optimization. Furthermore, the framework is compatible with efficient parameter fine-tuning. Low-rank incremental parameterization restricts trainable updates to a low-dimensional subspace, reducing training overhead while preserving constraint-driven geometric control capabilities. This research provides a structured, interpretable, and reproducible optimization path for robust fine-tuning and offers a unified technical solution for adapting highly reliable large models to task requirements and constraint boundaries.
References
Cover Image
Downloads
Published
2025-03-30
Section
Articles
License
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Academic Scholar Hub Press also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.

How to Cite

Wang, Y. (2025). Risk-Aware Large Language Model Fine-Tuning with Tail-Robust Objectives and Feasible Gradient Updates. Journal of Computer Technology and Software, 4(3). https://doi.org/10.5281/zenodo.22791595