FireFlash: Training a 300B Terminal and Coding Agent with Full-Parameter RL to Match Flagship Models at ~30x Lower Cost in Under 5 Days

    This article breaks down FireFlash, a terminal and coding agent built by applying full-parameter reinforcement learning (RL) to a 300B-class, low-latency open-source base model. The resulting agent reaches a 46.2% Pass@1 on a mainstream terminal benchmark—on par with two leading closed-source flagship models—while cutting average per-task cost by more than 30x, all from a single training run that used 64 flagship GPUs in under 5 days. We explain why the recipe commits to full-parameter RL, how verifiable rewards handle credit assignment across long-horizon tasks, and which engineering choices make a five-day run feasible. You will also find a reproduction checklist, cost-optimization strategies, and a troubleshooting guide for common failure modes, so teams with limited compute can train a cost-efficient coding agent.

    1. The takeaway: a new working point for cost-effective agents

    Recently, a research team published FireFlash in a public technical blog: a terminal and coding agent obtained by running full-parameter reinforcement learning (RL) on a 300B-class, low-latency open-source model (hereafter the “base model”).

    It reports three headline numbers:

    • Performance parity: a 46.2% Pass@1 (single-attempt success rate) on a mainstream terminal benchmark, on par with two leading closed-source flagship models;
    • Sharp cost drop: average cost per task more than 30x lower than one of those flagships running at its highest inference effort;
    • Affordable training: the entire run used only 64 flagship GPUs and took under 5 days, with the full recipe published.

    Measuring an agent by score alone is misleading. Figure 1 plots terminal-benchmark Pass@1 against average per-task cost on the same chart, with the cost axis on a log scale, a dashed line marking the “Pareto frontier,” and an arrow showing the direction and magnitude of this method’s improvement over the base model. The Pareto frontier is the boundary in the capability–cost plane beyond which no option is “both cheaper and stronger.” Any model that matches or exceeds performance at lower cost pushes that frontier outward.

    The value of this recipe therefore goes beyond leaderboard scores. It turns “high-performance agents” from a scarce resource available only through a few closed-source services into a capability that smaller teams can approach with limited compute.

    A note on evaluation methodology: In Table 1 of the original, the base model’s scores come from its officially published results, while FireFlash’s scores come from the team’s own evaluation framework (harness). The testing conditions are not identical, so keep this in mind when interpreting the size of the improvement.


    2. Why terminal and coding agents are hard, and worth building

    Terminal and coding tasks are a litmus test for agents because they share three defining traits:

    • Long-horizon: fixing a single bug may require dozens of interactions—locating files, reading context, editing code, running tests, and continuing to debug based on errors—with trajectories often spanning dozens to hundreds of turns;
    • Sparse rewards: a trajectory typically yields only a “success/failure” signal at the end, and intermediate steps cannot be judged directly;
    • Verifiable: the final result can be assessed by objective signals such as automated tests, compilation, and command exit status, with no human scoring required.

    “Verifiable” is the critical trait. It gives RL cheap and reliable environmental feedback, and verifiable tasks are exactly where RL tends to work best. But long horizon and sparse reward together create a series of engineering problems:

    1. Credit assignment: when a run fails, which step went wrong? Was it choosing the wrong file, or misusing a command?
    2. Runaway inference cost: context accumulates across interactions, so if the model is left to “think for a long time before acting,” token cost balloons.
    3. Heavy training overhead: full-parameter training must hold multiple parameter copies at once—inference, policy, optimizer states—raising the engineering bar.
    4. Slow and unstable sampling: long-tail issues such as environment startup failures, container crashes, and non-converging overlong trajectories can significantly slow overall progress.

    FireFlash’s core claim is that an end-to-end solution can address these problems simultaneously. The original also notes that the improvement transferred to two software-engineering benchmarks—one a deeper software-engineering benchmark emphasizing long-horizon synthesis, and one an enhanced-verification version of a software-engineering repair benchmark. This shows that “lower cost” was not achieved by avoiding hard problems.


    3. Cost breakdown: where the 30x comes from

    To understand the 30x gap, start with what drives per-task cost. In terminal tasks, cost comes mainly from context accumulated across multi-turn interactions. If an agent consumes t tokens per step over N steps, a run’s output is roughly N×t; as conversation history and tool returns keep piling up, input volume grows with the number of turns as well. This can be approximated as:

    1. Cost_task ≈ Σ(i=1..N) [ InputTokens_i × Price_in + OutputTokens_i × Price_out ]

    That points to three levers for cost optimization: leaner per-turn output, fewer interaction turns, and more compact input context.

    1. Diminishing returns from “maximum inference effort”: many frontier models expose a “reasoning effort” or “thinking budget” parameter that, when raised, generates longer internal reasoning. There is a real gain on hard problems, but token cost grows almost linearly or even super-linearly, and on simple tasks the trade-off is often not worth it. By contrast, FireFlash bakes “how to interact with the environment efficiently” directly into the policy itself: the model learns to take fewer detours, fix things correctly on the first try, and converge sooner. The original uses “maximum inference effort” as the cost baseline precisely to highlight this contrast.
    2. A cheaper base: the base model is positioned for low latency and is billed well below flagship models, so its per-token price advantage at the same token volume directly amplifies the overall cost advantage.
    3. Transparent billing basis: because the team deploys the model itself, costs are counted at the base model’s published price. This avoids gray areas in self-hosting such as utilization and idle cost, keeping the comparison close to a real calling scenario.

    In one line: rather than repeatedly “thinking” at inference time, learn to “do” during training.


    4. Method breakdown: how one recipe achieves both “strong” and “cheap”

    Note: This section summarizes the design logic of comparable recipes based on public information and common industry practice. It helps readers understand the reasoning, and is not a line-by-line reconstruction of every implementation detail of the original study.

    4.1 Base model choice: 300B-class, low-latency, and smart enough

    Choosing a 300B-class base is a careful balance point:

    • Stronger reasoning and instruction-following than models in the tens of billions: a prerequisite for long-horizon tasks;
    • Far smaller than super-large models in the hundreds of billions or more: single-pass compute and memory overhead stay manageable, and per-token cost stays low;
    • Low-latency positioning: fast responses and high throughput improve the agent’s interaction experience and push down calling costs.

    This lays the first cornerstone for the later “low cost.”

    4.2 Training approach: why commit to full-parameter RL

    Common agent-training methods and their trade-offs:

    Approach Core idea Strengths Limitations
    Rejection sampling + supervised fine-tuning Sample correct trajectories, then imitate Stable, low cost Ceiling limited by sample quality; slow iteration
    Preference optimization (e.g., DPO-style) Uses paired preference data Relatively relaxed data requirements Preference data hard to build; weak long-horizon credit assignment
    Parameter-efficient RL (e.g., LoRA) Trains only a few added parameters Saves memory, trains fast Limited ability to change deep model behavior
    Full-parameter RL Updates all model parameters Higher performance ceiling; genuinely reshapes the policy distribution Engineering-heavy, resource-intensive

    The capability gaps of long-horizon agents usually lie not in “surface style” but in deep behaviors like search, planning, and error correction. FireFlash chooses full-parameter RL precisely to change the model’s interaction policy at its root.

    4.3 Environment engineering: making long-horizon sampling stable and fast

    The throughput ceiling of RL is largely determined by the sampling environment. Engineering typically includes:

    • Container isolation: each trajectory runs in its own container preloaded with common toolchains; layered images and dependency caching reduce repeated installs;
    • Configurable execution timeouts: prevents a hung command from blocking the whole sampling batch;
    • Resource quotas: cap CPU, memory, and disk so a single trajectory cannot affect the whole machine;
    • Container pool reuse: pre-warm a pool of containers and cycle through them to cut cold-start overhead;
    • Scripted reward verification: return objective results by running tests, checking exit codes, and so on.

    For this kind of workload, general cloud capabilities all come in handy:

    • Container platforms host large numbers of short-lived sampling environments;
    • Object storage holds trajectories, checkpoints, and evaluation artifacts;
    • Log services accumulate training and sampling logs;
    • Monitoring and alerting watch key metrics such as GPU utilization, sampling success rate, and queue backlog.

    4.4 Algorithm design: four key choices under long-horizon, sparse rewards

    • Outcome-based reward: give a 0/1 reward based only on whether the task was completed, without relying on human-labeled intermediate steps. This naturally fits verifiable tasks.
    • Within-group relative baseline: sample multiple trajectories for the same task and use the group’s average as a baseline to compute advantage, e.g., (r - mean)/std. Because difficulty is the same within a group, the baseline cancels difficulty-induced variance; it also removes the overhead of training a separate value network (critic). GRPO-style related methods take exactly this approach.
    • Controlling distribution shift and entropy collapse:
      • Use a PPO-style clipped objective to constrain deviation between the new and old policies; asymmetric clipping (“clip-higher”) can ease rapid entropy collapse;
      • Add moderate KL regularization to keep the model from drifting from its base capabilities just to game the reward;
      • Monitor entropy and, when anomalies appear, adjust temperature, clipping range, or reward design in time.
    • Partial trajectories and length normalization: save and resume overlong trajectories instead of discarding them; when computing loss, mask out non-model-generated content such as prompts and tool returns, and normalize across trajectories of different lengths so long trajectories do not dominate the gradient.

    4.5 Performance engineering: the key to finishing in 5 days

    To finish training within 5 days on 64 GPUs, controlling time and throughput is just as important:

    • Heterogeneous deployment of training and inference: deploy the training cluster and the inference-engine cluster separately and scale each independently, avoiding memory contention;
    • Asynchronous pipeline: overlap sampling and training so that while one side computes, the other samples, reducing mutual waiting;
    • In-batch concurrent sampling: run multiple trajectories of the same task in parallel, hiding the time of long-tail trajectories inside overall throughput;
    • Kernel and memory optimization: operator fusion, gradient checkpointing, paged KV cache, and sequence parallelism ease the memory pressure of long-sequence training and sampling.

    5. Engineering view: a simplified training loop

    The pseudocode below abstracts the core flow; a real implementation must also fill in distributed communication, checkpoint/resume, monitoring instrumentation, and more.

    1. # Illustrative pseudocode; flow only
    2. MAX_TURNS = 60 # max interaction turns per trajectory
    3. GROUP = 8 # trajectories sampled per task, for within-group comparison
    4. BETA = 0.02 # KL penalty coefficient, discourages drift from the reference model
    5. for step in range(total_steps):
    6. prompts = pool.sample(batch_size)
    7. # 1) Parallel sampling: each trajectory runs in its own container
    8. trajs = []
    9. for p in prompts:
    10. for _ in range(GROUP):
    11. env = env_pool.acquire(p) # attach repo snapshot and dependency image
    12. traj = env.rollout(actor, p, max_turns=MAX_TURNS, temperature=T)
    13. traj.reward = verifier.verify(p, env) # returns only 0/1
    14. trajs.append(traj)
    15. env_pool.release(env) # return and clean up the container
    16. # 2) Within-group relative advantage: multiple trajectories of one task act as each other's baseline
    17. adv = group_relative_baseline(trajs, group_size=GROUP)
    18. # 3) Update: compute loss only over model-generated tokens
    19. loss = pg_loss_with_kl(trajs, adv, beta=BETA, clip=eps)
    20. actor.update(loss)
    21. reference.update_ema(actor) # periodically sync the reference model

    Three details are easy to get wrong:

    • Keep the reward “pure”: the verifier returns only success or failure, and never treats failure stack traces or test output as a reward signal. Otherwise the model learns to “please the verifier” instead of actually completing the task.
    • Align advantage computation with grouping: after concurrent sampling, trajectories easily get reordered, so you must regroup by task before computing the within-group baseline, or the advantage will be wrong.
    • Be precise with loss masking: tool returns appear in training samples, but they are environment output, not the model’s decision. They must be masked when computing loss, or the model will imitate tool output rather than learn how to decide.

    6. Reproduction checklist: turning the recipe into an actionable plan

    6.1 Resource estimate

    A rough conversion: 64 GPUs for under 5 days comes to less than 320 GPU·days in total. This roughly means an 8-node cluster with 8 GPUs per node running for a week can cover the main training, including the compute consumed by parallel sampling.

    By contrast, a single training run of a top closed-source model often costs thousands or even tens of thousands of GPU·days. The gap of one to two orders of magnitude underscores the leverage of efficient algorithms and careful engineering. For small and mid-sized teams, this is a budget that is “just within reach.”

    The above is a rough estimate based on public information; actual cost depends on hardware type, parallelism strategy, and task scale.

    6.2 Phased roadmap

    Phase Suggested duration Core work Deliverables
    Preparation 1–2 days Build the environment pool, write verifiers, curate the task pool Reproducible task and environment sets
    Cold start 0.5 day Small-scale sampling; remove all-correct / all-wrong tasks A training set of moderate difficulty
    Main training 2–3 days Asynchronous sampling + full-parameter RL, with periodic evaluation and checkpointing Multiple checkpoints and curves
    Evaluation 0.5 day Multi-benchmark, repeated evaluation, consistent comparison methodology Capability–cost curves

    6.3 Key configuration suggestions

    1. rollout:
    2. concurrency: 512 # containers running at once; scale to cluster size
    3. max_turns: 60 # max interaction turns per trajectory
    4. temperature: 1.0 # sampling temperature; balances exploration and stability
    5. partial_rollout: true # save and resume overlong trajectories to reduce wasted samples
    6. task_filter:
    7. keep_pass_rate: [0.1, 0.9] # drop all-correct / all-wrong tasks; focus on the "learnable" band
    8. dedup_by_repo: true # deduplicate by repo to avoid re-sampling tasks from the same source
    9. training:
    10. group_size: 8 # trajectories sampled per group
    11. clip_eps_low: 0.2 # lower clipping bound
    12. clip_eps_high: 0.28 # asymmetric upper bound to ease entropy collapse
    13. kl_coef: 0.02 # KL penalty coefficient
    14. loss_mask: "generated_tokens_only" # compute loss only over model-generated tokens

    7. Cost and efficiency: training agents requires frugality too

    Cost control during training also determines whether a recipe can be reproduced:

    • Stratify by task difficulty: all-correct tasks teach nothing, and all-wrong tasks yield all-zero rewards with no within-group variance, so they provide no gradient signal. Filtering out these two categories lets sampling compute focus on the “learnable band.”
    • Isolate training and evaluation sets: split by repo or source to prevent test answers from “leaking” into training early.
    • Cut wasted interactions: set a turn limit on trajectories that repeat failures or spin in place, and terminate them early.
    • Mixed precision and parallelism optimization: when memory is tight, enable gradient checkpointing and sequence parallelism to balance throughput and memory.
    • Pick a point along the cost curve: at deployment, you do not necessarily need the “optimal” post-training checkpoint. If a slightly earlier checkpoint loses only a small amount of performance but is significantly cheaper, it may be the better business choice.

    8. Common failure modes and troubleshooting

    Symptom Possible cause Where to look
    Reward rises fast early, then plateaus or falls Entropy collapse, premature policy convergence Raise sampling temperature, increase the asymmetric clipping upper bound, lower the learning rate slightly
    Reward rises, but real pass rate does not Reward being gamed, verifier loopholes Check whether the verifier covers edge cases; add adversarial tests
    Turn count keeps growing, cost rises instead of falling Model stuck in unproductive exploration loops Set a max-turn limit; add a turn penalty or efficiency reward
    Training throughput fluctuates heavily Long-tail trajectories, environment failures Partial-trajectory resumption, container health checks and auto-rebuild
    One benchmark rises while another falls Training task distribution too narrow, overfitting Deduplicate tasks by source and repo; mix tasks from multiple sources
    Old and new policies differ too much, training unstable Sampling and training policies out of sync Shorten the sync interval, enable importance-sampling correction

    9. Significance and limitations: a level-headed view

    Three points of significance:

    1. Capability leveling: a 300B-class open-source base plus a public recipe means more teams have a chance to approach flagship-level performance;
    2. Democratized training: roughly three hundred GPU·days turns “agent-specific reinforcement learning” from a big-company exclusive into general practice;
    3. Governable cost: treating “capability–cost” as a first-class goal gives large-scale agent deployment a measurable handle.

    Three limitations:

    • The conclusions rest mainly on verifiable terminal and coding tasks; whether they transfer to open-ended settings that are hard to score automatically remains to be seen;
    • A single public result is affected by conditions such as the evaluation framework and random seed, and needs broader, repeatable independent verification;
    • Safety risks in long-horizon agents cannot be ignored. When the model has permission to execute commands and modify files, both training and deployment should happen in isolated environments, with permission boundaries and audit logs on tool calls.

    For enterprise adoption, a three-step approach:

    1. Start with internal, high-frequency, verifiable tasks (e.g., automated test fixing, log triage, script generation) and build a dedicated task pool and verifiers;
    2. Do a small-scale reproduction based on the public recipe to validate gains in your own scenario;
    3. Bring “per-task cost” into model selection and launch evaluation, so agents are continuously measured and optimized like services.

    10. Conclusion

    FireFlash’s lesson is simple: in the age of agents, competitiveness is decided not only by how big a model is, but by how efficient the method for training agents is. When 64 GPUs and under 5 days of training buy capability on par with flagship models plus an order-of-magnitude lower per-task cost, what developers and businesses should really do is re-examine their own task pools, verifiers, and environment-engineering capabilities. These are what turn a general recipe into business productivity.