LoRA fine-tuning requires choosing a rank parameter that nobody understands — too low and the model learns nothing, too high and it overfits

You are fine-tuning Llama 3 8B using LoRA (Low-Rank Adaptation). You must choose the rank (r): r=4, r=8, r=16, r=32, r=64, or r=128. What does this number mean? It determines the dimensionality of the low-rank update matrices. Higher rank = more learnable parameters = more capacity to learn your data = but also more capacity to overfit. r=4 might be too constrained for a complex task. r=128 might memorize your training data. The optimal rank depends on your dataset size, task complexity, model architecture, and which layers you apply LoRA to. Nobody knows how to choose without experimentation. Most people use r=16 because that is what the original LoRA paper used. So what? LoRA's entire value proposition is 'efficient fine-tuning without full parameter updates.' But the efficiency gain is partially offset by the hyperparameter search needed to find the right rank. A bad rank choice means wasted compute or a bad model. The LoRA paper tested ranks on specific benchmarks, but those results do not transfer to your dataset. A rank of 16 that works for coding tasks might be wrong for medical Q&A or legal document analysis. Every new fine-tuning project re-discovers the right rank through trial and error. Why does this persist? The optimal rank is a function of the intrinsic dimensionality of the task — a theoretical quantity that cannot be computed without running experiments. There is no closed-form relationship between dataset size, task complexity, and optimal LoRA rank. Research papers that propose adaptive rank methods (AdaLoRA, QLoRA with different ranks per layer) add more hyperparameters, not fewer.