ia3

chatglm 6b finetuning and alpaca finetuning

qwen models finetuning

Experiments on optimizing of Large Language Models (LLMs) using Parameter Efficient Fine-Tuning (PEFT) with techniques like LORA and IA3 for faster inference and reduced memory usage.

Fine-tuned chemical language model for predicting molecular lipophilicity in drug design. Explores parameter-efficient fine-tuning strategies (LoRA, BitFit, IA3), layer freezing techniques, and influence-based data selection. Balances accuracy and computational efficiency for molecular property prediction tasks.

A comprehensive benchmark suite comparing Parameter-Efficient Fine-Tuning (PEFT) techniques including LoRA, QLoRA, and IA3, against full fine-tuning for transformer-based text classification across DistilBERT, RoBERTa, and DeBERTa for sentiment classification, focusing on accuracy, efficiency, and hardware-aware optimization.

ESM-2 and IA3 parameter-efficient functional peptide classification with a Streamlit inference demo

Benchmarking QLoRA, DoRA and IA3 vs RAG on Llama-3.2-3B for legal contract risk analysis — 4-layer evaluation framework with statistical significance testing

Elixir port of HuggingFace's PEFT (Parameter-Efficient Fine-Tuning) library. Implements LoRA, AdaLoRA, IA3, prefix tuning, prompt tuning, and 30+ state-of-the-art PEFT methods for efficient neural network adaptation. Built for the BEAM ecosystem with native Nx/Axon integration.