Building an E-commerce Copy System with Open-Source LLMs
This guide explains how to design a multi-step copy generation system on open-source large language models for white-label products. It covers model selection, a five-layer architecture, prompts, tool calls, error handling, advertising-compliance checks, conversion evaluation, deployment sizing, and troubleshooting.
Why white-label products need copy at scale
The main problem with white-label products is sameness. Titles, selling points, and detail pages for the same category often look alike, so buyers cannot tell products apart and sellers compete mainly on price. Branded copy gives a product its own voice, but writing it by hand is expensive, and promotions can involve a large number of items. That is why many teams try to generate copy in batches with large language models.
A simple question-and-answer call only returns a block of text. E-commerce copy is closer to a workflow: understand the product, look up market information, write a draft, then check for compliance issues. Any step can fail—for example, search returns nothing, or the selling points do not match the target audience. The system therefore needs the model to plan across steps, call tools, and revise when it finds a problem. This is often called long-horizon task ability, or agent ability.
This article describes the design and implementation of a copy generation system based on open-source models. The code is illustrative. It focuses on the flow and boundary conditions; readers should fill in details for their own models and service environments.
How to choose an open-source model
Compare candidates on five dimensions.
- Tool calling: the model must be able to emit structured function-call requests. Models that only generate free text and cannot reliably call a search API are a poor fit.
- Context length: product data, search results, and multi-turn revisions accumulate. If the context window is too short, the model drops earlier selling points or constraints.
- Chinese writing quality: the copy needs to fit Chinese usage and category conventions. Test with your own product samples instead of relying only on general leaderboards.
- License: before commercial use, confirm that the model license allows it and note any extra restrictions.
- Deployment cost: GPU specifications, the inference stack, memory use at the target concurrency, and latency per generation.
Start with two or three candidates, run the same product batch through each, and record tool-call success rate, compliance issues, and generation time before choosing one.
System architecture
The system has five layers. Each layer should have clear inputs and outputs so failures are easier to trace.
- User interaction: accept product information and return copy plus task status. Example input:
{"name": "stainless steel thermos", "attributes": {"capacity": "500ml", "material": "304 stainless steel"}, "target_audience": "commuters"}
- Task orchestration: split the goal into research, selling-point extraction, draft writing, and compliance review. Each task has an explicit state: pending, running, succeeded, failed, or needs human handling.
- Model execution: call the open-source model in order to complete each subtask.
- External tools: search, a product knowledge base, image analysis, and similar interfaces. Every tool needs a timeout and a uniform error response format.
- Platform support: task queue, logs, retries, and monitoring.
Suggested retries: retry a timed-out tool call twice, with increasing intervals. If the model output has a bad format, ask the model to emit it once more. After repeated failures, mark the task as needing human handling. Do not retry forever.
Core implementation
Prompt design
The system prompt sets the model’s role, steps, and output format. A usable prompt should include the following:
You are an e-commerce brand copywriter. Using the product information provided, complete these steps:1. Extract three core selling points. Each point must be supported by the input.2. Call the search tool for recent consumer concerns in this category. If results are empty or unrelated, state "no reliable trend information" and write from the product information only.3. Write a detail-page draft of no more than 200 characters.4. Output JSON with fields selling_points (array), copy (string), and needs_human_review (boolean).Do not use absolute claims such as "best", "number one", "only", or "national-level", and do not invent product specifications. If a specification cannot be confirmed, set needs_human_review to true.
If the model follows the field constraints, the program can parse the result directly. Prohibitions in the prompt only reduce the chance of violations. A rules engine still has to check the output.
Tool registration and execution
When calling an open-source model through an OpenAI-compatible API, the server must enable tool-call parsing. Otherwise, function calls in the model output will not be recognized. The code below defines a search tool and a single execution entry point.
import jsonTOOLS = [{"type": "function","function": {"name": "search_market_trends","description": "Search recent consumer trends and concerns for a product category","parameters": {"type": "object","properties": {"keyword": {"type": "string", "description": "Category keyword"}},"required": ["keyword"],},},}]def search_market_trends(keyword: str) -> dict:# Connect a real search API here. This example returns fixed data.return {"keyword": keyword, "trends": ["eco-friendly materials", "minimal design", "value for money"]}TOOL_FUNCS = {"search_market_trends": search_market_trends}def run_tool_call(call) -> dict:"""Run one tool call and return a uniform result for the tool message."""try:args = json.loads(call.function.arguments)func = TOOL_FUNCS[call.function.name]return {"ok": True, "data": func(**args)}except Exception as e:return {"ok": False, "error": str(e)}
Correction loop
The hard part of a long-horizon task is fault tolerance. The pseudocode below shows termination conditions and error branches. call_llm should set a timeout on each request. check_compliance should be implemented against the actual banned-term list.
import jsonimport timeMAX_TURNS = 8 # maximum reasoning turnsTOTAL_TIMEOUT = 60 # total time limit for one task, in secondsdef run_copy_task(product_info: dict) -> dict:"""Pseudocode: termination conditions and error branches."""deadline = time.monotonic() + TOTAL_TIMEOUTmessages = [{"role": "system", "content": SYSTEM_PROMPT},{"role": "user", "content": json.dumps(product_info, ensure_ascii=False)},]for turn in range(MAX_TURNS):if time.monotonic() > deadline:return {"status": "timeout", "turns": turn}try:msg = call_llm(messages, tools=TOOLS)except Exception as e:return {"status": "llm_error", "detail": str(e)}if msg.tool_calls:# Keep the assistant message that contains tool_calls.# Tool results must match those ids.messages.append({"role": "assistant","content": msg.content or "","tool_calls": [{"id": c.id, "type": "function","function": {"name": c.function.name, "arguments": c.function.arguments}}for c in msg.tool_calls],})for call in msg.tool_calls:result = run_tool_call(call)messages.append({"role": "tool","tool_call_id": call.id,"content": json.dumps(result, ensure_ascii=False),})continuetry:result = json.loads(msg.content)copy_text = result["copy"]except (json.JSONDecodeError, KeyError, TypeError):messages.append({"role": "user", "content": "The output is not valid JSON. Return the required fields again."})continueif result.get("needs_human_review"):return {"status": "needs_human_review", "result": result}hits = check_compliance(copy_text) # list of matched banned termsif not hits:return {"status": "done", "result": result, "turns": turn + 1}messages.append({"role": "assistant", "content": msg.content})messages.append({"role": "user","content": f"The copy matched banned terms: {hits}. Remove those terms, keep product facts unchanged, and output JSON again."})return {"status": "max_turns_exceeded"}
The loop ends in one of four ways: normal completion, handoff for human review, exceeding the turn limit, or a timeout or model-call error. Each ending must return a clear status for later reporting.
How to avoid prohibited advertising claims
Avoiding banned wording needs three layers. The first is the prompt, which tells the model not to use absolute claims. The second is a rules engine that checks output with a term list and pattern matching. The third is human review. For tightly regulated categories such as food, health products, and cosmetics, compliance staff should review copy before it is published.
Legal or compliance staff should maintain the term list. Simple substring matching creates false positives—for example, a match on “best” can hit “bestseller” or a similar ordinary word. Prefer word-level matching, and sample hits for manual review. This article only describes the technical flow. Specific legal requirements should follow the applicable regulations and professional legal advice.
How to evaluate conversion
Evaluate with a controlled comparison, not only with before-and-after launch numbers. Use these steps.
- Build an evaluation set: sample products by category and keep human-written copy as the control. Sample size should come from a power calculation based on the baseline conversion rate and the difference you want to detect.
- Offline metrics: compliance pass rate, human scores (ideally independent scores from several reviewers, then averaged), and time per generation.
- Online metrics: randomly split traffic into two groups, show model copy to one and human copy to the other, and watch click-through rate and order rate for at least one full sales cycle.
Conversion is also affected by price, hero images, reviews, and promotions. Copy is only one factor, so conclusions should come from the randomized comparison.
Deployment and cost
The system can run in the cloud or on your own GPU servers. The example below uses cloud containers; compare options against your own constraints.
A common stack is:
- Container service: managed Kubernetes such as Baidu AI Cloud CCE, Alibaba Cloud ACK, or Tencent Cloud TKE can host both the model service and the application service.
- Message queue: Kafka, RocketMQ, or a vendor-managed queue, for peak smoothing and asynchronous task dispatch.
- Logging and monitoring: record each model output, tool-call arguments, and latency so infinite loops and timeouts can be found.
- Alternative: a self-managed Kubernetes cluster plus your own GPU servers. Unit cost may be lower, but operations effort is higher.
Capacity planning starts from a few assumptions: daily task volume, peak concurrency during a promotion, time per task, and stable concurrency per GPU. The estimates are:
Concurrent tasks needed ≈ peak tasks per second × seconds per task
GPUs needed ≈ concurrent tasks needed ÷ stable concurrency per GPU × redundancy factor
For example, 600 tasks per minute at peak is 10 per second. At 20 seconds per task, about 200 tasks are in flight. If one GPU stably handles 8 concurrent tasks and the redundancy factor is 1.5, about 38 GPUs are required. These numbers are only an example. Use load-test results.
For availability, run at least two model-service replicas. The task queue should redeliver failed work, task state should be stored in a database, and a restarted service should resume from the last checkpoint.
Troubleshooting checklist
- The model never calls tools: check that the server has tool-call parsing enabled, that the tools JSON Schema is valid, and that the prompt explicitly asks for search first.
- Tool messages fail: check that
tool_call_idmatches the id in the assistant message, and that the assistant message still containstool_calls. - Output is not valid JSON: lower the sampling temperature, include a field example in the prompt, and cap format retries.
- Tasks time out: inspect per-turn latency in the logs and separate slow model inference from a slow tool API.
- Copy is blocked repeatedly: check whether the banned-term list is matching normal words, such as a substring hit inside an ordinary phrase.
- Tasks loop: check that
MAX_TURNSis enforced, and whether the same tool is called again and again with the same arguments.
Summary
Long-horizon ability in large models can support a fuller copy production flow, but it does not replace rules or human judgment. Reliability mainly comes from clear steps and output formats, callable tool interfaces, explicit stop conditions, an independent compliance check, and evaluation on real data. Build a prototype on a small product batch, validate it, and then scale. That keeps both risk and cost easier to control.
