LLM Fine-Tuning Notes: Training Preparation and Hyperparameter Tuning on a Single 40GB GPU
This article walks through the preparation and tuning involved in fine-tuning a roughly 9-billion-parameter large language model on a single 40GB GPU. It covers why base-model scale should match task complexity, how to manage memory constraints with low-rank adaptation (rank 32, alpha 64), gradient checkpointing, and mixed precision, and how to choose key hyperparameters such as a 5120 max length, a 1e-4 learning rate, and 2 epochs. It also explains how to structure a ~6k-sample dataset split between chain-of-thought multi-turn dialogue and tool-calling data, what to monitor beyond loss, common pitfalls like out-of-memory errors and overfitting, and a reusable preparation checklist for stable, reproducible fine-tuning.
LLM Fine-Tuning Notes: Training Preparation and Hyperparameter Tuning on a Single 40GB GPU
1. The Goal of Training Preparation: Not “Getting the Model to Run,” but “Getting It to Run Stably”
Fine-tuning a large model may look like launching a single training job, but in practice it’s closer to preparing a systems-engineering effort. Before training starts, a developer has to answer several questions at once: Does the base model’s size match the complexity of the task? Can the GPU memory handle the target sequence length? Are the learning rate, number of epochs, and low-rank adaptation settings reasonable? Can the data volume and data mix actually support the target capability?
After an earlier stage of basic fine-tuning with a lighter-weight base model, this round of practice swaps the base model for a base model at roughly the 9-billion-parameter level. The goal is to give the model stronger capabilities in complex instruction following, multi-turn dialogue, chain-of-thought reasoning, and tool-call planning. A larger parameter count usually means stronger expressive power, but it also brings higher memory pressure, longer training times, and a more noticeable risk of overfitting. So the core of the preparation phase isn’t simply stacking parameters — it’s finding a reusable balance point among model capability, training cost, memory limits, and data quality.
The training environment here is a single datacenter-grade training GPU with 40GB of memory. For a mid-sized model, this setup offers some training capability but also imposes clear constraints. Full-parameter fine-tuning would put enormous pressure on memory, so a parameter-efficient fine-tuning approach is a better fit: through low-rank adaptation and similar techniques, you train only a small number of added parameters, controlling memory use while preserving as much of the base model’s existing capability as possible.
2. Choosing the Base Model: Parameter Scale Should Match Task Complexity
Moving from a lightweight base to a mid-sized base
With a smaller-parameter base early on, training costs are low and memory pressure is small, which makes it well suited to quickly validating the data format, the training pipeline, and the feasibility of the task. But as the goal shifts from simple Q&A and basic text generation toward multi-turn reasoning, complex tool calling, and longer-context handling, a small model’s weaknesses in instruction following, context understanding, and format stability gradually become apparent.
That’s why this round upgrades the base model to roughly the 9-billion-parameter level. Models at this scale tend to be more stable at understanding complex tasks, planning tool calls, and holding coherence across multi-turn dialogue. At the same time, it’s no longer suitable for a crude training approach. With more parameters, backpropagation has to store more intermediate states, activations grow larger, and optimizer states take up more space. Even with low-rank adaptation, memory can’t be ignored entirely.
Base models vs. chat models
In terms of task type, a base model usually emphasizes language modeling and general text capability, making it suitable for domain continued pretraining, instruction fine-tuning, or task customization. If the goal is dialogue, tool calling, or complex instruction following, you generally need a matching high-quality data format and clear role constraints.
If the goal is to generate dialogue content that faces end users directly, pay special attention to three points:
- Role boundaries must be clear: roles such as system prompt, user input, assistant reply, and tool return must be clearly distinguished.
- Output format must be stable: especially in tool-calling scenarios, the model needs to learn when to call a tool, how to extract arguments, and how to interpret the returned result.
- The training objective must stay focused: you can’t train on everything indiscriminately, or the model may learn redundant content and even produce malformed output.
3. Memory Constraints: A Training Strategy on a Single 40GB GPU
Where memory goes
Memory usage during training doesn’t depend only on the number of model parameters. Even with parameter-efficient fine-tuning, memory is still affected by:
- Model weights: the memory occupied by the base model itself.
- Trainable parameters: added parameters such as low-rank adaptation layers, prompt layers, or task heads.
- Gradients: parameter gradients produced during backpropagation.
- Optimizer states: even when training only the added parameters, the optimizer still maintains their state.
- Activations: intermediate results produced during the forward pass — the biggest memory consumer in long-sequence training.
- Batch size and sequence length: larger batches and longer sequences make memory pressure more pronounced.
For 40GB of memory, handling a mid-sized model is feasible but requires a careful strategy. This is especially true for long-context tasks, where increasing sequence length sharply raises memory consumption. Attention computation, activation storage, and backpropagation are all affected by sequence length. When sequence length goes from 4096 to 8192, memory pressure may not rise linearly — it can climb rapidly.
A recommended baseline configuration
On a single 40GB GPU, it’s best to start with a conservative configuration to get the pipeline working, then gradually scale up. A relatively safe starting point:
max_length: 5120micro_batch_size: 1gradient_accumulation_steps: 16gradient_checkpointing: truemixed_precision: truelearning_rate: 1e-4epochs: 2low_rank_adaptation:rank: 32alpha: 64
The core ideas behind this configuration:
- Small-batch forward pass with multi-step gradient accumulation: when single-GPU memory is limited, set the micro batch to 1 and use gradient accumulation to simulate a larger effective batch.
- Enable gradient checkpointing: trade extra compute time for memory savings, reducing the pressure of storing activations.
- Enable mixed precision: on most training GPUs and frameworks, mixed precision helps improve throughput and reduce memory usage.
- Control the maximum sequence length: 5120 is a compromise between memory and context coverage.
What to do when 8192 length runs out of memory
In practice, there was a desire to set the maximum sequence length to 8192 to cover longer dialogues or tool-calling contexts. But on a single 40GB GPU, this setting easily led to out-of-memory errors. There are three main reasons:
- Larger attention intermediate states: increasing sequence length significantly increases attention-related intermediate computation.
- More activations stored: with long sequences, the intermediate results generated at each layer and each step take up more memory.
- Poor padding and truncation strategy for long samples: if sample lengths are unevenly distributed, some batches can see peak memory spike due to overly long samples.
Possible countermeasures include:
- Lower the maximum length to 5120 or 4096 to keep training stable first.
- Bucket data by length to avoid having too many overly long samples in the same batch.
- Filter out extremely long samples, or split them for separate processing.
- Use gradient checkpointing to reduce activation storage.
- Lower the micro batch and maintain an equivalent batch via gradient accumulation.
- Check whether unnecessary caching, logging, or debug output is enabled.
4. Analyzing the Key Training Parameters: Low Rank, Learning Rate, Epochs, and Length
Low-rank adaptation parameters: rank and alpha
In this training run, the core low-rank adaptation settings were:
rank: 32alpha: 64
The basic idea of low-rank adaptation is to introduce low-rank matrices alongside the original weights and train only the small set of added parameters, which significantly reduces the number of trainable parameters and optimizer overhead. For a mid-sized model, this approach is well suited to single-GPU or resource-constrained environments.
rank can be understood as the adapter’s capacity. A larger rank means stronger learnable expressiveness, but also more memory and higher overfitting risk; a smaller rank means lighter training, but may struggle to learn complex tasks.
alpha is typically used to control the scaling factor of the low-rank adaptation output. A common practice is to keep alpha in a certain ratio to rank — for example, alpha around twice rank. rank=32 and alpha=64 is a common starting point, suitable as a baseline configuration for general instruction fine-tuning and tool-calling fine-tuning.
Adjustment suggestions:
- If the task is simple classification or short-text generation, try a smaller
rank. - If tool calling is complex or has many instruction levels, moderately increase
rank. - If the training set is small, avoid blindly increasing
rank, or overfitting becomes likely. - If memory is tight, prioritize lowering
rankrather than increasing sequence length further.
Learning rate: 1e-4 as a common starting point
The learning rate here was set to:
learning_rate: 1e-4
For parameter-efficient fine-tuning methods like low-rank adaptation, the learning rate can usually be a bit higher than in full-parameter fine-tuning, because the number of added parameters is small and updates are more flexible. 1e-4 is a common starting point in many instruction fine-tuning scenarios.
But if the following occurs during training, consider adjusting the learning rate:
- Loss drops fast then oscillates violently: possibly the learning rate is too high.
- Malformed output and severe repetition: possibly unstable training.
- Loss drops very slowly: possibly the learning rate is too low or there’s an issue with the data template.
- Validation performance clearly degrades: possibly overfitting, so lower the learning rate or reduce epochs.
A range to try:
Conservative: 5e-5Common: 1e-4Aggressive: 2e-4
Whether to use learning rate warmup also depends on the data scale. When the data volume is small and samples differ widely, appropriate warmup helps stabilize training.
Epochs: 2 is usually more stable than 3
This run set:
epochs: 2
This choice comes from practical observation: for pure text instruction data, 2 epochs are usually enough for the model to absorb the data patterns well; training to 3 epochs makes overfitting likely. Typical signs of overfitting include:
- Training loss keeps dropping, but validation performance stops improving.
- Model output becomes increasingly template-like.
- The model clearly memorizes training samples, but generalization on new samples declines.
- Output becomes verbose and repetitive, or frequently uses fixed expressions seen in the training set.
- During tool calling, the model tends to mechanically copy call patterns from the training samples.
For scenarios with a few thousand samples, epoch control matters especially. When the data volume is small, too many epochs make the model quickly memorize surface features of the samples instead of learning genuinely generalizable capability.
So in pure-text or instruction fine-tuning, 2 epochs is often a safe starting point. If the data volume is large, the task is complex, and validation keeps improving, you can cautiously try a third epoch, but it must be paired with validation-based evaluation.
Maximum sequence length: 5120 as a compromise
The maximum length here was set to:
max_length: 5120
This length covers most multi-turn dialogues, reasoning processes, and tool-calling scenarios while staying relatively stable under 40GB of memory. By comparison, 8192 can hold more context but significantly increases memory pressure and makes training prone to instability.
Setting sequence length requires balancing these factors:
- Data length distribution: if the vast majority of samples are under 4096, setting 8192 instead adds meaningless computation.
- Memory limit: long sequences significantly amplify activation and attention computation pressure.
- Truncation strategy: if long samples are truncated, you must ensure the key output portion isn’t cut off.
- Training stability: overly long sequences can cause memory peak fluctuations and increase the risk of interrupted training.
A more reasonable approach is to first measure the length distribution of the training data, then choose a length that covers most samples. For example, if 90% of samples are within 5000 tokens, then 5120 is a suitable upper limit; if only a few samples are overly long, handle them separately rather than raising the global length for the sake of a tiny number of samples.
5. Dataset Design: The Capability Structure Behind 6k Samples
The total training data is about 6k, of which roughly 5k is multi-turn dialogue data with chain-of-thought, and about 1k is tool-calling data. This mix reflects a clear capability goal: first strengthen the model’s stability in multi-turn reasoning and dialogue, then add tool-calling capability.
Chain-of-thought multi-turn dialogue data
The roughly 5k chain-of-thought multi-turn dialogue samples are the main body of training. This kind of data is mainly used to improve the model’s:
- Multi-turn context understanding.
- Step-by-step reasoning ability.
- Ability to break down complex problems.
- Ability to explain things clearly to users.
- Ability to maintain role consistency across continuous dialogue.
Chain-of-thought data doesn’t mean more complex is better. High-quality chain-of-thought data should have these characteristics:
- Grounded reasoning: each reasoning step should be based on the user’s question, known facts, or prior context.
- Avoid unfounded expansion: don’t fabricate nonexistent facts just to appear “deep.”
- Concise and clear expression: the reasoning process should serve the final answer, not be verbose padding.
- Consistent format: roles, order, and output structure should stay consistent across multi-turn dialogue.
- Verifiable answers: the final answer should address the user’s question rather than drift off-topic.
If the chain-of-thought data quality is poor, the model may learn bad habits — for example, over-expansion, logic jumps, verbose repetition, or forgetting the core goal across multi-turn dialogue.
Tool-calling data
The roughly 1k tool-calling samples make up a smaller share than the chain-of-thought data, but they’re critical for real business scenarios. The core of tool-calling training isn’t simply getting the model to “call a tool” — it’s getting the model to understand call boundaries and the call flow.
Tool-calling data usually needs to cover these scenarios:
- Cases that need a tool call: for example, querying information, performing calculations, or accessing external data.
- Cases that don’t need a tool call: for example, common-sense Q&A, text polishing, or simple explanations.
- Argument extraction: identifying the arguments a tool needs from the user’s natural language.
- Missing arguments: when the user’s information is insufficient, the model should ask a follow-up question rather than forcing a call.
- Handling tool returns: the model needs to understand the tool’s returned result and produce a final answer.
- Handling call failures: the model should be able to recognize error messages and give an explanation or retry strategy.
Tool-calling data formats should stay consistent. It’s recommended to clearly distinguish system instructions, user input, tool calls, tool returns, and the final assistant reply. An example:
{"messages": [{"role": "system","content": "You are an intelligent assistant that can call tools when needed."},{"role": "user","content": "Please help me check the current task status."},{"role": "assistant","content": "I need to call the task-query tool."},{"role": "tool","content": "Task status: in progress."},{"role": "assistant","content": "The current task is still being processed."}]}
Tool-calling data especially needs to avoid two extremes:
- Over-calling: calling a tool for every question, leaving the model without autonomous judgment.
- Never calling: the model still fabricates an answer directly when it should call a tool.
So although 1k tool-calling samples isn’t a large number, they must be high-quality, cover the full range, and include positive examples, negative examples, and edge cases.
6. Training Monitoring: Don’t Just Watch the Loss
During training, loss is the most intuitive metric, but it isn’t the only one. For generative models — especially tasks involving dialogue, reasoning, and tool calling — you also need to watch output quality and format stability.
Baseline metrics to monitor
- Training loss: reflects how well the model fits the training set.
- Validation loss: used to judge generalization, and the key metric for spotting overfitting.
- Gradient norm: indicates whether training is stable; both exploding and vanishing gradients can cause anomalies.
- Learning rate changes: confirms the schedule behaves as expected.
- Memory usage: confirms whether the configuration is near its limit and whether there’s a risk of peak overflow.
- Training throughput: reveals whether per-step time is abnormally increasing.
Additional quality metrics for generation tasks
For chain-of-thought and tool-calling tasks, looking only at loss isn’t enough. You also need to periodically sample and check:
- Whether output follows the role setting.
- Whether tools are called correctly.
- Whether tool arguments are complete and accurate.
- Whether tools are misused when no call is needed.
- Whether output is too long or repetitive.
- Whether tool return results are handled correctly.
- Whether context stays consistent across multi-turn dialogue.
It’s recommended to carve out an independent validation set from the training data and run evaluation on it periodically. The validation set must not participate in training, or the evaluation results will be distorted.
7. Common Problems and Optimization Suggestions
Out of memory
Common problems:
- Maximum length set too high.
- Micro batch set too high.
- Gradient checkpointing not enabled.
- Long samples not filtered or bucketed.
- Using a parallelism or precision strategy unsuitable for the available memory.
Optimization suggestions:
- Prioritize lowering the maximum length.
- Drop the micro batch to 1.
- Increase gradient accumulation steps.
- Enable gradient checkpointing.
- Filter, split, or separately train overly long samples.
Overfitting
Common problems:
- Too many epochs.
- Small data scale but rank or learning rate too high.
- Insufficient validation coverage, so degradation goes undetected.
Optimization suggestions:
- Keep epochs around 2.
- Lower the learning rate or rank.
- Increase data diversity.
- Introduce a validation set and early stopping.
- Deduplicate training data to avoid repeated samples reinforcing memorization.
Unstable tool calling
Common problems:
- Too little tool-calling data.
- Missing negative examples and missing-argument samples.
- Inconsistent tool definitions and return formats.
- The assistant portion wasn’t correctly included in the loss during training.
Optimization suggestions:
- Define the tool-calling format clearly.
- Add samples that judge whether a call is needed.
- Add samples for missing arguments, error returns, and multi-turn confirmation.
- Check that the training template correctly masks non-target content.
- Periodically run regression tests with real business samples.
Answer loss caused by long-sample truncation
If a training sample exceeds the maximum length, simple truncation may cut off the final reply, which teaches the model an incomplete target sequence.
Optimization suggestions:
- When truncating, prioritize keeping the user’s question and the assistant’s final answer.
- Summarize, split, or apply sliding-window processing to overly long inputs.
- Set a higher length separately for long samples that must be preserved.
- Check the data preprocessing logic to ensure the loss mask covers the correct region.
8. A Reusable Training Preparation Checklist
Based on this practice, the preparation work before large-model fine-tuning can be summarized as this checklist:
Clarify the task goal
- Is it general dialogue, domain Q&A, complex reasoning, or tool calling?
- Different goals determine the data mix and parameter strategy.
Choose a suitable base model
- Parameter scale should match task complexity.
- Also consider context length, memory cost, and inference deployment cost.
Assess training resources
- How much memory does the single GPU have, and does it support mixed precision and gradient checkpointing?
- Is parameter-efficient fine-tuning needed?
- Is there a risk of memory peaks?
Design the data format
- Are role boundaries clear?
- Is the tool-calling format consistent?
- Does it include positive, negative, and edge-case samples?
Set baseline training parameters
- Low-rank rank and alpha.
- Learning rate.
- Epochs.
- Maximum sequence length.
- Batch size and gradient accumulation.
Establish a validation mechanism
- Is there an independent validation set?
- Does it evaluate format, tool calling, and reasoning quality?
- Can it detect overfitting or training anomalies in time?
Keep a rollback configuration
- Change only one key variable at a time.
- Save the checkpoint corresponding to the best validation metric.
- Avoid modifying too many parameters at once, which makes problems hard to isolate.
9. Summary
The core lesson from this training preparation can be summed up in one sentence: Large-model fine-tuning isn’t about having more parameters — it’s about forming a closed loop between model scale, data quality, memory budget, and training strategy.
After upgrading the base model from a smaller scale to roughly the 9-billion-parameter level on a single 40GB training GPU, training can no longer rely on a simple configuration. At this point you need to handle sequence length, low-rank adaptation parameters, learning rate, and epochs more carefully. A maximum length of 5120, 2 epochs, a learning rate of 1e-4, and a low-rank adaptation configuration of rank 32 and alpha 64 together form a relatively safe starting point.
On the data side, about 6k samples isn’t about simply piling up quantity — it uses about 5k chain-of-thought multi-turn dialogue samples to strengthen reasoning and multi-turn ability, then about 1k tool-calling samples to fill in external-task capability. This data mix shows that the key to fine-tuning isn’t just getting the model to “memorize answers,” but getting it to learn to stably produce expected behavior in specific scenarios.
For later optimization, three directions are worth focusing on:
- Data quality over data quantity: especially for chain-of-thought and tool-calling data, logic must be correct, formats consistent, and boundaries clear.
- Length strategy should be based on data distribution: don’t blindly chase a larger max_length; choose an appropriate upper limit based on sample length distribution.
- Epochs must be paired with a validation set: in small-data scenarios, 2 epochs are often enough; be cautious about a third epoch to avoid overfitting.
Overall, the more solid the preparation phase, the more stable the subsequent training, and the easier results are to reproduce. For developers just starting out with large-model fine-tuning, getting the pipeline working first and then gradually optimizing parameters is a safer way to get started.
