AO
A.N. Ojică
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
Large Language Models (LLMs) generate fluent code that can still be invalid for the environment it must run in. Under fixed dependency versions, a recurring failure is API hallucination: the model invokes a fabricated API, an outdated interface, or a plausible substitute for what the pinned library version requires. APIs released after the model's training data were collected are the hardest case, because the knowledge is missing outright and retraining the model for every library release is impractical.
This thesis asks whether small Low-Rank Adaptation (LoRA) adapters can inject post-cutoff API knowledge into a 7B instruction-tuned model (Qwen2.5-7B-Instruct) without corrupting its remaining behavior. An API counts as post-cutoff operationally when it entered the library in a recent release and the base model produces it on none of the tested surfaces. A teacher LLM generates training data grounded in library sources, checked mechanically and by manual review, and the adapters are trained with supervised fine-tuning (SFT) on target positives balanced with anchor and hard-negative examples that teach where the new API does not belong. The experiments cover one PyTorch API, then five, then twenty-five, all on the LibEvoBench completion benchmark.
A benchmark average cannot certify injection, so every claim is decomposed into per-API acquisition, target leakage onto unrelated code, a failure taxonomy separating fabricated, outdated, and wrong-but-valid answers, and retention on paired control benchmarks. Knowledge-free control adapters reproduce most of the aggregate benchmark gain with zero target output, and subtracting such a control from a trained adapter in weight space (task arithmetic) keeps target acquisition while moving unrelated rows at the measurement floor, so the acquisition metric, unlike the aggregate, tracks injected knowledge.
Twelve positive examples suffice to make the missing API appear, and the resulting adapter mentions it on 73.2% of unrelated rows, so the hard half of the problem is control rather than production. Anchored training restores that control, converting fabricated answers into correct ones without the anachronistic buildup of positive-only training. Both recipes then scale under replication across seeds. At five APIs, the matched-control composition acquires 75.2% of the target rows while direct SFT converts only at seed-dependent peaks, and at twenty-five APIs, on the target rows the benchmark asks, the composition reaches 62.8% and direct SFT 50.1%, the direct recipe cutting the phantom share on those rows from 44.2% to 7.6%.
The programme's one significant retention cost, a 2.88-percentage-point decline on GSM8K mathematical reasoning, belongs to the direct-SFT package, while both composition points change no paired control benchmark significantly, a contrast the cross-method design cannot attribute to a single component. The thesis therefore delivers a validated adaptation recipe together with the measurement discipline required to trust it, while release-scale coverage and a deployed library-version specialist remain open.
...
This thesis asks whether small Low-Rank Adaptation (LoRA) adapters can inject post-cutoff API knowledge into a 7B instruction-tuned model (Qwen2.5-7B-Instruct) without corrupting its remaining behavior. An API counts as post-cutoff operationally when it entered the library in a recent release and the base model produces it on none of the tested surfaces. A teacher LLM generates training data grounded in library sources, checked mechanically and by manual review, and the adapters are trained with supervised fine-tuning (SFT) on target positives balanced with anchor and hard-negative examples that teach where the new API does not belong. The experiments cover one PyTorch API, then five, then twenty-five, all on the LibEvoBench completion benchmark.
A benchmark average cannot certify injection, so every claim is decomposed into per-API acquisition, target leakage onto unrelated code, a failure taxonomy separating fabricated, outdated, and wrong-but-valid answers, and retention on paired control benchmarks. Knowledge-free control adapters reproduce most of the aggregate benchmark gain with zero target output, and subtracting such a control from a trained adapter in weight space (task arithmetic) keeps target acquisition while moving unrelated rows at the measurement floor, so the acquisition metric, unlike the aggregate, tracks injected knowledge.
Twelve positive examples suffice to make the missing API appear, and the resulting adapter mentions it on 73.2% of unrelated rows, so the hard half of the problem is control rather than production. Anchored training restores that control, converting fabricated answers into correct ones without the anachronistic buildup of positive-only training. Both recipes then scale under replication across seeds. At five APIs, the matched-control composition acquires 75.2% of the target rows while direct SFT converts only at seed-dependent peaks, and at twenty-five APIs, on the target rows the benchmark asks, the composition reaches 62.8% and direct SFT 50.1%, the direct recipe cutting the phantom share on those rows from 44.2% to 7.6%.
The programme's one significant retention cost, a 2.88-percentage-point decline on GSM8K mathematical reasoning, belongs to the direct-SFT package, while both composition points change no paired control benchmark significantly, a contrast the cross-method design cannot attribute to a single component. The thesis therefore delivers a validated adaptation recipe together with the measurement discipline required to trust it, while release-scale coverage and a deployed library-version specialist remain open.
...
Large Language Models (LLMs) generate fluent code that can still be invalid for the environment it must run in. Under fixed dependency versions, a recurring failure is API hallucination: the model invokes a fabricated API, an outdated interface, or a plausible substitute for what the pinned library version requires. APIs released after the model's training data were collected are the hardest case, because the knowledge is missing outright and retraining the model for every library release is impractical.
This thesis asks whether small Low-Rank Adaptation (LoRA) adapters can inject post-cutoff API knowledge into a 7B instruction-tuned model (Qwen2.5-7B-Instruct) without corrupting its remaining behavior. An API counts as post-cutoff operationally when it entered the library in a recent release and the base model produces it on none of the tested surfaces. A teacher LLM generates training data grounded in library sources, checked mechanically and by manual review, and the adapters are trained with supervised fine-tuning (SFT) on target positives balanced with anchor and hard-negative examples that teach where the new API does not belong. The experiments cover one PyTorch API, then five, then twenty-five, all on the LibEvoBench completion benchmark.
A benchmark average cannot certify injection, so every claim is decomposed into per-API acquisition, target leakage onto unrelated code, a failure taxonomy separating fabricated, outdated, and wrong-but-valid answers, and retention on paired control benchmarks. Knowledge-free control adapters reproduce most of the aggregate benchmark gain with zero target output, and subtracting such a control from a trained adapter in weight space (task arithmetic) keeps target acquisition while moving unrelated rows at the measurement floor, so the acquisition metric, unlike the aggregate, tracks injected knowledge.
Twelve positive examples suffice to make the missing API appear, and the resulting adapter mentions it on 73.2% of unrelated rows, so the hard half of the problem is control rather than production. Anchored training restores that control, converting fabricated answers into correct ones without the anachronistic buildup of positive-only training. Both recipes then scale under replication across seeds. At five APIs, the matched-control composition acquires 75.2% of the target rows while direct SFT converts only at seed-dependent peaks, and at twenty-five APIs, on the target rows the benchmark asks, the composition reaches 62.8% and direct SFT 50.1%, the direct recipe cutting the phantom share on those rows from 44.2% to 7.6%.
The programme's one significant retention cost, a 2.88-percentage-point decline on GSM8K mathematical reasoning, belongs to the direct-SFT package, while both composition points change no paired control benchmark significantly, a contrast the cross-method design cannot attribute to a single component. The thesis therefore delivers a validated adaptation recipe together with the measurement discipline required to trust it, while release-scale coverage and a deployed library-version specialist remain open.
This thesis asks whether small Low-Rank Adaptation (LoRA) adapters can inject post-cutoff API knowledge into a 7B instruction-tuned model (Qwen2.5-7B-Instruct) without corrupting its remaining behavior. An API counts as post-cutoff operationally when it entered the library in a recent release and the base model produces it on none of the tested surfaces. A teacher LLM generates training data grounded in library sources, checked mechanically and by manual review, and the adapters are trained with supervised fine-tuning (SFT) on target positives balanced with anchor and hard-negative examples that teach where the new API does not belong. The experiments cover one PyTorch API, then five, then twenty-five, all on the LibEvoBench completion benchmark.
A benchmark average cannot certify injection, so every claim is decomposed into per-API acquisition, target leakage onto unrelated code, a failure taxonomy separating fabricated, outdated, and wrong-but-valid answers, and retention on paired control benchmarks. Knowledge-free control adapters reproduce most of the aggregate benchmark gain with zero target output, and subtracting such a control from a trained adapter in weight space (task arithmetic) keeps target acquisition while moving unrelated rows at the measurement floor, so the acquisition metric, unlike the aggregate, tracks injected knowledge.
Twelve positive examples suffice to make the missing API appear, and the resulting adapter mentions it on 73.2% of unrelated rows, so the hard half of the problem is control rather than production. Anchored training restores that control, converting fabricated answers into correct ones without the anachronistic buildup of positive-only training. Both recipes then scale under replication across seeds. At five APIs, the matched-control composition acquires 75.2% of the target rows while direct SFT converts only at seed-dependent peaks, and at twenty-five APIs, on the target rows the benchmark asks, the composition reaches 62.8% and direct SFT 50.1%, the direct recipe cutting the phantom share on those rows from 44.2% to 7.6%.
The programme's one significant retention cost, a 2.88-percentage-point decline on GSM8K mathematical reasoning, belongs to the direct-SFT package, while both composition points change no paired control benchmark significantly, a contrast the cross-method design cannot attribute to a single component. The thesis therefore delivers a validated adaptation recipe together with the measurement discipline required to trust it, while release-scale coverage and a deployed library-version specialist remain open.
Analysing the Performance of Generative Models Trained in a Federated Manner
Exploring the Impact of GANs and Variational Auto-Encoders on Decentralized Data
Federated learning (FL) is an innovative approach in machine learning that enables model training across multiple decentralized devices or servers without sharing local data, thus preserving privacy and utilizing decentralized data. However, a significant challenge in FL is handling non-IID (Non-Identical and Independently Distributed) data, which can adversely affect performance. This paper investigates the impact of federated learning on the performance of various generative models, including Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), specifically in the context of image and tabular data generation tasks. Our study aims to determine how these generative models perform when trained in a federated manner compared to centralized training. We evaluate the models using several metrics, including classifier accuracy on generated images, Earth Mover’s Distance (EMD) for distribution comparison, resemblance, discriminability, downstream utility, and privacy metrics for tabular data. Experiments conducted on the MNIST and CIFAR-10 datasets for image generation, and the Adult and Abalone datasets for tabular data generation, reveal that VAEs exhibit robust and consistent performance across federated and centralized setups. In contrast, GANs show significant performance degradation under federated non-IID conditions. The results indicate that VAEs can effectively address the non-IID data challenge in FL by generating high-quality synthetic data, thereby enhancing model generalizability and stability. The framework used for executing the experiments in this study can be found at https://github.com/alexojica/research-project-experiments.
...
Federated learning (FL) is an innovative approach in machine learning that enables model training across multiple decentralized devices or servers without sharing local data, thus preserving privacy and utilizing decentralized data. However, a significant challenge in FL is handling non-IID (Non-Identical and Independently Distributed) data, which can adversely affect performance. This paper investigates the impact of federated learning on the performance of various generative models, including Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), specifically in the context of image and tabular data generation tasks. Our study aims to determine how these generative models perform when trained in a federated manner compared to centralized training. We evaluate the models using several metrics, including classifier accuracy on generated images, Earth Mover’s Distance (EMD) for distribution comparison, resemblance, discriminability, downstream utility, and privacy metrics for tabular data. Experiments conducted on the MNIST and CIFAR-10 datasets for image generation, and the Adult and Abalone datasets for tabular data generation, reveal that VAEs exhibit robust and consistent performance across federated and centralized setups. In contrast, GANs show significant performance degradation under federated non-IID conditions. The results indicate that VAEs can effectively address the non-IID data challenge in FL by generating high-quality synthetic data, thereby enhancing model generalizability and stability. The framework used for executing the experiments in this study can be found at https://github.com/alexojica/research-project-experiments.