AI Engineering in Practice: Four Implementation Paths Seen in Open Source Projects

Start With a Hard Requirement: It Has to Run

When you go through open-source leaderboards, the most informative filter is not the star count. It is whether you can clone the repo and get a visible result within half an hour. Projects that run usually share three traits: dependencies converge into a single package manager, configuration lives in environment variables rather than in edits to source files, and there is a minimal end-to-end entry point — a script, a demo, a single command. A project that does not run may still have value, but it behaves more like the code shipped with a paper or a tool the author built for their own use, and the evaluation cost far outweighs the return.

The projects in this roundup fall into four categories: multi-agent orchestration, automatic generation of slide decks from documents, visual node-based game prototypes, and exposing a local media processing toolchain to coding agents. Each is described below in the same order: the problem it solves, its key design choices, where its limits are, and how to evaluate it.

Multi-Agent Orchestration: the Hard Part Was Never “More Than One Agent”

The actual problem

When a single agent handles planning, retrieval, coding, and verification all at once, its context gets polluted by process content very quickly: trial-and-error logs, repeated file reads, contradictory intermediate conclusions piling up in the window. The model’s effective attention is diluted in the process. Splitting the work across roles is really a way of cutting one long context into several short ones, and passing information between them as structured results instead of free-form conversation.

The right shape for shared context

A common mistake is to let every agent read the same complete chat transcript. That simply replicates the pollution problem across copies. The sturdier approach is a blackboard: it holds only structured facts, artifact handles, and decision records — never raw conversation.

# orchestrator.py -- minimal skeleton for multi-agent orchestration
# (illustrative code, not the API of a specific framework)
from dataclasses import dataclass, field
@dataclass
class SharedContext:
    task: str                                       # task definition, immutable throughout
    facts: dict = field(default_factory=dict)       # confirmed conclusions: contracts, paths, parameters
    artifacts: dict = field(default_factory=dict)   # artifact handles: files, branches, chart IDs
    decisions: list = field(default_factory=list)   # key trade-offs and rationale, for traceability
class Agent:
    def __init__(self, name: str, tools: list[str], max_steps: int = 8):
        self.name = name
        self.tools = tools          # least privilege: each role gets only the tools it needs
        self.max_steps = max_steps
    def run(self, ctx: SharedContext, goal: str) -> dict:
        # 1) Inject only the fact slices relevant to the goal into the prompt
        # 2) Loop over tool calls, writing results into ctx.facts / ctx.artifacts
        # 3) Return a structured result; never send long process logs back to the orchestrator
        ...

Seven design points that have to be settled

• Split roles by context type, not by seniority. Research roles read a lot and write little; implementation roles need filesystem access; verification roles need execution and test permissions. Mixing those permissions together is where most orchestration projects start to go out of control.
• Make the context compression policy explicit. Define what counts as a “confirmed fact,” what can be dropped, and how often summaries are refreshed. A shared context with no eviction rule only grows monotonically.
• Version the message contracts. Agents exchange JSON Schema, not natural language. Once free text is allowed between agents, errors get amplified along the chain into noise you cannot debug.
• Handle concurrency and write conflicts. When several agents modify the same files at once, separate work areas or use file locks; otherwise outputs overwrite each other and the cause is hard to locate.
• Put human-in-the-loop (HITL) checkpoints in place. Deleting data, publishing, sending external requests, and any irreversible file operation must require explicit human confirmation. This one is not optional.
• Instrument observability. Record the input summary, output, token count, and latency for every call each agent makes. An orchestration system without traces cannot answer “which step went wrong.”
• Cap cost and rate. Set budget ceilings and maximum step counts for the whole chain, so a self-reinforcing loop does not burn through both the bill and wall-clock time.

When evaluating projects in this category, go straight to the context management code. If the entire logic is “concatenate history into the prompt,” it is only a loop wrapper and there is little to gain from reading the source. If it has clear fact extraction, compression, and permission boundaries, that is the part worth copying.

Document to Slide Deck: the Structured Middle Layer Decides Whether It Works

Turning a document into an editable presentation is easy to misread as a “typesetting task.” It is really an information extraction and restructuring task. A reliable pipeline usually has six stages: parsing, structuring, content selection, layout mapping, rendering, and validation.

The key requirement is that a renderer-independent structured representation exists in the middle. Feeding document parse output directly into render templates tangles layout constraints with content semantics, so any single adjustment forces a full pipeline rerun.

{
  "meta": { "ratio": "16:9", "theme": "plain", "lang": "en" },
  "slides": [
    {
      "layout": "title_content",
      "title": "Current State and Goals",
      "bullets": ["Manual effort: about 8 hours per deck", "Goal: consistent structure, editable output"],
      "chart": { "type": "bar", "series": [{ "name": "Hours", "data": [8, 3] }] }
    },
    {
      "layout": "image_caption",
      "asset": "diagram/architecture.png",
      "caption": "Overall architecture"
    }
  ]
}

With that layer in place, three things become easy. Switching themes or aspect ratios for the same content only requires changing meta. Problems such as text overflow or charts running off the slide can be validated statically before rendering. And native charts and shapes are emitted as real objects rather than whole-page screenshots, so users can keep editing.

The division of labor between machine and human also needs to be stated clearly. Machines are good at compressing a long document into a hierarchical outline, unifying terminology, and producing a first draft of charts. They are not good at judging how to rank information by importance, controlling visual rhythm, or handling brand and compliance requirements. So “the machine produces the draft, the human sets the layout” is not a compromise — it is the best available split today. A workable review checklist looks roughly like this: one claim per slide, no more than six bullets in the body, complete axis labels and units on every chart, consistent fonts and colors across slides, and action items on the conclusion slide.

Visual Node-Based Prototyping Tools: Be Clear About the Ceiling

Tools that produce 2D/3D gameplay by dragging nodes are, at their core, a way of writing event-condition-action rules as a directed graph. Their advantage is a short feedback loop: logic, asset references, and scene editing all happen in the same interface, so even non-programmers can build a playable prototype in a day or two. That makes them well suited to validating core mechanics and to teaching demos.

The ceiling is equally clear.

First, performance and tuning headroom. For areas that need fine-grained control — complex render pipelines, memory layout, multithreaded scheduling — the node graph abstraction often has no corresponding knob.

Second, engineering collaboration. Large projects need code-level refactoring, unit tests, continuous integration, and branch merges, and graphical projects have limited ability to diff and resolve conflicts as text.

Third, the asset pipeline. Importing, compressing, generating LODs, and packaging high-fidelity art and audio usually has to be handled outside the tool.

The pragmatic rule: use node-based tools for validating mechanics, rapid prototyping, and teaching scenarios. Once you enter a performance-sensitive phase or need sustained collaboration across many people, migrate to a code-based project early, or adopt a hybrid mode — keep logic in the visual layer and push performance hotspots down into scripts or plugins.

Exposing Local Media Processing as Agent Tools

Giving a coding agent audio and video capability is the highest-value pattern in this roundup: the capability itself comes from a mature command-line toolchain, and the agent only orchestrates parameters rather than implementing algorithms.

The hard part is not whether a command can be invoked, but how to invoke it safely. At minimum, five things need to be covered.

{
  "name": "normalize_loudness",
  "description": "Loudness-normalize a local audio file and write the result to a new file without overwriting the source",
  "parameters": {
    "input":  { "type": "string", "required": true },
    "output": { "type": "string", "required": true },
    "target_lufs": { "type": "number", "default": -16 }
  },
  "constraints": ["output must reside inside the working directory", "timeout = 120s", "source file is read-only"]
}

#!/usr/bin/env bash
set -euo pipefail
root="$(realpath "$WORKDIR")"
out="$(realpath -m "$2")"
case "$out" in
  "$root"/*) : ;;                                  # reject path escapes outright
  *) echo "refuse: output outside workdir" >&2; exit 2 ;;
esac
# pass arguments as an array; never concatenate shell strings, to avoid command injection
"${MEDIA_TOOL[@]}" -i "$1" -filter loudnorm -o "$out"

The remaining points: timeouts and output size limits, so a single job cannot hang the whole pipeline; structured error codes, so the agent can tell “bad parameter” from “file not found” and correct itself; artifact registration, writing generated files back into a manifest so later steps can reference them; and processing assets locally, avoiding uploads of unpublished raw material to external services — in enterprise content production that is often a hard requirement.

The Underlying Variables: Sparse Base Models and Cheaper Inference Rewrite the Math

Two parallel trends in the industry are worth looking at separately.

The first is the spread of the sparse mixture-of-experts (MoE) architecture. It decouples total parameter count from the parameters actually activated per token: knowledge capacity comes from total parameters, while per-inference compute is controlled by activated parameters. At the same time, larger base model weights have been released progressively, turning “self-hosted deployment plus domain fine-tuning” from an option available to a few organizations into something mid-sized and larger teams can evaluate — at the cost of running and operating their own inference cluster.

The second is that inference unit prices keep falling, with the input side now in the range of roughly one dollar per million tokens. This matters to the application layer more directly than benchmark scores do: a document understanding task that needs 500,000 input tokens costs about $0.50 in inference, while manually organizing comparable documents typically takes hours of work. Once the cost gap reaches one or two orders of magnitude, three previously uneconomical practices become viable: high-coverage multi-candidate sampling with reranking, one-pass analysis of an entire corpus in a long context, and a lenient retry-on-failure policy. The document-to-slides and media orchestration workflows mentioned above are exactly what this curve has moved from demo to daily use.

One caveat: a lower unit price does not mean the quality bar is met. Any agent workflow aimed at production should keep an offline evaluation set and regression cases, and treat “per-step success rate,” “rework rate,” and “cost per task” as release gates rather than deciding on the strength of a single demo.

A Selection Checklist: Judging in Half a Day Whether a Project Is Worth the Investment

1. Dependencies and configuration: Can dependencies be pinned to fixed versions? Is all configuration externalized?
2. Minimum runnable path: How many steps from zero to a visible result? Are network credentials required to get there?
3. Permission model: Does it distinguish read-only from write operations? Are there sandbox or working-directory constraints?
4. Context management: For multi-agent projects, is there compression and fact extraction logic, or just history concatenation?
5. Intermediate representation: For content generation projects, are semantics decoupled from rendering, and are outputs still editable?
6. Observability: Are steps, tokens, and latency recorded? Can a failure be reproduced?
7. Licensing and compliance: Does the open-source license permit commercial use? Can data processing be kept fully local?
8. Migration cost: If the project stops being maintained, can its outputs and configuration be consumed directly by an alternative?

Run projects through these eight points and most are eliminated in the first ten minutes; what remains are the candidates worth reading the source code for. The genuinely scarce capability is not one more tool that runs, but embedding tools into an engineering pipeline that has budgets, permissions, and traceability — which is the direction all the projects in this roundup point toward.