Large Language Models (LLMs): How They Work, Architecture Types, and a Practical Selection Guide
This article explains what a large language model is at a technical level, why generality is its defining trait, and how to make the engineering decisions that surround it. It walks through the Transformer-based training pipeline (unsupervised pre-training, followed by supervised fine-tuning and alignment), the three main model architectures — encoder-only, encoder-decoder, and decoder-only — and the tasks each one fits. It then compares LLMs with traditional fine-tuned models across data requirements, task generality, cold-start cost, inference cost, and quality ceiling, lists the key dimensions for evaluating base models, introduces prompting and reasoning techniques such as Chain-of-Thought, ReAct, and Self-Ask, and maps out application patterns from conversational assistants to tool calling and agents. Readers will leave with a framework for architecture selection, base-model choice, capability elicitation, and application design.
Rethinking LLMs: From “It Works” to “General Purpose”
Over the past few years, a wave of applications built around large language models has moved into the mainstream, and the term “LLM” now shows up constantly in technical discussions. So what exactly is a large language model (LLM)?
Technically, an LLM is a class of deep neural network models that are pre-trained on massive text corpora with a single core objective: predicting the next token. They typically use the Transformer architecture and rely on self-attention to capture long-range dependencies between words. Training happens in two broad stages. First comes unsupervised pre-training, where the model learns general patterns of language and world knowledge from a corpus that spans nearly every topic. Then comes supervised fine-tuning and human feedback alignment techniques such as RLHF, which shape the model’s behavior to better match human intent and preferences.
The biggest difference from traditional models is generality. Conventional machine learning often follows a “one task, one model” approach, whereas an LLM has zero-shot and few-shot generalization ability. A single base model can handle translation, summarization, question answering, code generation, and more simply through different prompts. That is the fundamental reason LLMs attracted so much attention.
A Technical Map: The Three Main Architectures
Viewed as a whole, Transformer-based models fall into three architectural paradigms, and understanding the differences is the starting point for any selection decision.
- Encoder-only: Uses bidirectional attention and excels at deep understanding and representation of input text. It fits “understanding” tasks such as text classification, information extraction, and semantic matching. It is not built for long-form free generation.
- Encoder-Decoder: The encoder understands the input and the decoder generates the output, which naturally suits sequence-to-sequence scenarios such as machine translation and text rewriting.
- Decoder-only: Generates text autoregressively, one token at a time, and in theory can frame every task as “text continuation.” Thanks to strong scalability and generation ability, it has become the mainstream paradigm for large models in industry, and the vast majority of general-purpose conversational models follow this path.
Which architecture to choose comes down to whether the business is primarily about understanding or generation. Understanding-heavy work can lean toward an encoder-based approach, while generation and dialogue favor a decoder-based one.
LLMs vs. Traditional Fine-Tuned Models: How to Choose
In practice, “should we use a large model or keep fine-tuning for this task” is a frequent question. The two are not substitutes; they should be weighed against the scenario.
| Dimension | Traditional fine-tuned model | Large language model (LLM) |
|---|---|---|
| Data requirements | Needs a fair amount of labeled data | Can start with zero-shot or few-shot |
| Task generality | Purpose-built for a single task | General across multiple tasks |
| Cold-start cost | Higher | Lower |
| Inference cost | Low, easy to deploy | Higher, requires compute resources |
| Quality ceiling | Limited by data and scale | Stronger generalization |
The practical guidance: when task boundaries are clear, data is plentiful, latency and cost are sensitive, and results must be stable and controllable, a traditional fine-tuning approach still offers good value. When facing open-ended generation, question answering, or creative work — or when you want one base model to cover many tasks — an LLM is the better first choice. Many mature solutions adopt a hybrid “LLM + fine-tuning” strategy: use a general-purpose large model for a cold start, then apply lightweight fine-tuning or alignment with a small amount of domain data, balancing generality with specialization.
Key Dimensions for Selecting a Base Model
Once the architecture is settled, how do you pick the right base model? Evaluate it across the following dimensions:
- Capability coverage: Whether the model meets expectations on your target scenarios — reasoning, code, multi-turn dialogue, long-context understanding.
- Parameter scale and inference cost: Larger models are usually more capable, but they also raise inference overhead and the bar for deployment, so balance quality against cost.
- Context length: Context window size determines how much information the model can process in one pass and directly affects scenarios such as document Q&A and long-document summarization.
- Deployment model: Whether to use a managed service in the cloud or a private deployment, depending on data compliance, latency, and operations requirements.
- Ecosystem and tooling: Whether it comes with complete development APIs, fine-tuning support, monitoring and alerting, and observability.
For teams that want to move quickly, a mature AI cloud platform — one that bundles large model services with a development toolchain — can lower the barrier from selection to production, letting you focus effort on the business rather than the underlying infrastructure.
Capability Elicitation: Getting More Out of the Same Model
With the same base model, different prompting and reasoning strategies can produce noticeably different results. These are the approaches commonly used in industry:
- Chain-of-Thought (CoT): Guides the model to “think step by step,” breaking complex reasoning into intermediate steps, which significantly improves accuracy on math and logic tasks.
- ReAct (reasoning plus acting): Has the model alternate between reasoning and calling external tools or retrieval, performing better on tasks that need outside knowledge or real-time information.
- Self-Ask: The model actively decomposes a large question into sub-questions, solves them one by one, and then aggregates the answers — well suited to complex problems that require multi-hop reasoning.
The common thread is that carefully designed prompts and workflows “draw out” the model’s latent capability without modifying model weights. In engineering terms, these patterns can be packaged as reusable prompt templates and orchestration flows, forming stable capability components.
Applications: From Chat to Agents
The value of an LLM ultimately has to show up in shipped products. Application forms fall into three broad categories:
- Conversational applications: Intelligent Q&A, content creation, and office assistants — the most direct entry point for LLMs.
- Plugins and tool calling: The model calls search, computation, databases, and external APIs, going beyond its own knowledge and compute limits to complete more realistic end-to-end tasks.
- Agents: With an LLM as the “brain,” an agent combines planning, memory, tool use, and autonomous execution, and can independently complete multi-step complex tasks. This is an important direction in how applications are evolving.
Summary
A large language model is not a single black box but a layered engineering discipline: architecture selection, base-model choice, capability elicitation, and application delivery. Understand the split between the encoder-only, encoder-decoder, and decoder-only paradigms, grasp the trade-off logic that complements traditional fine-tuned models, and combine elicitation techniques such as Chain-of-Thought and ReAct with application forms such as agents — and you can turn a general-purpose large model into business productivity that actually ships and can be measured. For teams ready to go deeper, the recommendation is to start from one clearly defined, high-value scenario, iterate quickly, and gradually build your own large model application capability.
