Mitigating API Hallucinations in Evolving Software Ecosystems with Controlled LoRA Fine-Tuning

Abstract

Large Language Models (LLMs) generate fluent code that can still be invalid for the environment it must run in. Under fixed dependency versions, a recurring failure is API hallucination: the model invokes a fabricated API, an outdated interface, or a plausible substitute for what the pinned library version requires. APIs released after the model's training data were collected are the hardest case, because the knowledge is missing outright and retraining the model for every library release is impractical.

This thesis asks whether small Low-Rank Adaptation (LoRA) adapters can inject post-cutoff API knowledge into a 7B instruction-tuned model (Qwen2.5-7B-Instruct) without corrupting its remaining behavior. An API counts as post-cutoff operationally when it entered the library in a recent release and the base model produces it on none of the tested surfaces. A teacher LLM generates training data grounded in library sources, checked mechanically and by manual review, and the adapters are trained with supervised fine-tuning (SFT) on target positives balanced with anchor and hard-negative examples that teach where the new API does not belong. The experiments cover one PyTorch API, then five, then twenty-five, all on the LibEvoBench completion benchmark.

A benchmark average cannot certify injection, so every claim is decomposed into per-API acquisition, target leakage onto unrelated code, a failure taxonomy separating fabricated, outdated, and wrong-but-valid answers, and retention on paired control benchmarks. Knowledge-free control adapters reproduce most of the aggregate benchmark gain with zero target output, and subtracting such a control from a trained adapter in weight space (task arithmetic) keeps target acquisition while moving unrelated rows at the measurement floor, so the acquisition metric, unlike the aggregate, tracks injected knowledge.

Twelve positive examples suffice to make the missing API appear, and the resulting adapter mentions it on 73.2% of unrelated rows, so the hard half of the problem is control rather than production. Anchored training restores that control, converting fabricated answers into correct ones without the anachronistic buildup of positive-only training. Both recipes then scale under replication across seeds. At five APIs, the matched-control composition acquires 75.2% of the target rows while direct SFT converts only at seed-dependent peaks, and at twenty-five APIs, on the target rows the benchmark asks, the composition reaches 62.8% and direct SFT 50.1%, the direct recipe cutting the phantom share on those rows from 44.2% to 7.6%.

The programme's one significant retention cost, a 2.88-percentage-point decline on GSM8K mathematical reasoning, belongs to the direct-SFT package, while both composition points change no paired control benchmark significantly, a contrast the cross-method design cannot attribute to a single component. The thesis therefore delivers a validated adaptation recipe together with the measurement discipline required to trust it, while release-scale coverage and a deployed library-version specialist remain open.