Migrating Local LLM Models: Hash-Named Weights, Manifests, and Model Directory Conventions Explained
Local inference tools and desktop model managers have become the default choice for developers experimenting with large language models on their own machines, yet model files are rarely interchangeable between them. This article uses migrating hash-named weight files as the entry point and walks through the full practice: identifying a model, comparing manifests, reorganizing directories, and verifying the result after migration. It also covers quantization levels, rough resource estimation, copy versus hard link versus symlink trade-offs, and a troubleshooting table. On Baidu AI Cloud, teams can build a unified model repository on top of object storage and container services so that local and cloud environments share one standardized workflow.
1. Where the Problem Comes From: What Hash Naming Really Means
1.1 Content-addressable storage: the name is computed
Early model management was straightforward: a model was a weight file with a file name. As model sources multiplied and updates got faster, this approach exposed three problems:
- The same weights were downloaded repeatedly: different tags of the same model often point to identical weights, so name-based storage creates duplicate copies.
- Tampered or truncated files were hard to notice: incomplete files left behind by an interrupted download are easy to miss when there is no verification mechanism.
- No consistent metadata: architecture type, tokenizer vocabulary, chat template, quantization scheme, and similar information was scattered, and loading depended on convention.
Content-Addressable Storage (CAS) was designed to address exactly these issues. Take a large-model weight format such as GGUF: a single file can be anywhere from a few GB to tens of GB, while the accompanying chat template, stop sequences, sampling parameters, and other metadata change with every model revision. Managing that by file name quickly leads to duplicated storage and version confusion. The CAS approach is to compute a digest of the file contents (SHA256, for instance) and use that digest as the file’s unique identifier in the storage layer. This yields several immediate benefits:
- Deduplication by content: identical content is stored once, and different tags can share the same weights.
- Verifiable integrity: recomputing the digest and comparing it against the manifest at any time tells you whether a file is damaged.
- Separation of reference and entity: logical names (model name, tag, version) are recorded in a manifest, which indirectly points at the actual data.
Local inference tools commonly combine blob (binary large object) storage + a manifest, a design that follows the same lineage as layered container image storage. The cost is that a human-readable model name becomes an unreadable hash string.
1.2 Every tool has a different “model library convention”
Different inference tools expect different things from a model library. Broadly, they fall into three categories:
- Content-addressed: weights are stored under hash file names, and a manifest describes how the model is composed. This suits scenarios with frequent model updates and a focus on deduplication, but the native files are inconvenient to move around directly.
- Convention-based directories: you point the tool at a model library root, and it scans and identifies models by walking a hierarchy such as
publisher/model name/quantization level/weight file. This structure needs no database, is easy to organize by hand, and is easy to migrate between machines. - Index/registry based: a separate registry or index file records model locations and configuration. New models are added either through an import function or by editing the configuration manually.
When the weights come from a category 1 tool and need to be used by a category 2 tool, the problem reduces to three steps: find the real weight file, confirm which model it is, and place it according to the target tool’s convention. On top of that, some desktop inference applications in earlier versions offered no direct GGUF import entry point, or recognized imported models unreliably — in those cases, placing files by hand is the most dependable route.
1.3 Three things to settle before you migrate
Before starting, clarify the following. The more thorough the up-front planning, the less rework later.
- Model identity: source repository, model family, parameter count, precision type, and quantization level? Together they determine the model’s RAM or VRAM requirements.
- Hardware budget: how much RAM/VRAM is available? How long a context do you need to support? Are other programs also occupying VRAM?
- Migration method: copy the file, create a hard link, or use a symbolic link? The choice depends on free disk space, the partition the file lives on, and whether you need to switch back and forth between the old and the new tool.
2. Step 1: Locate the Model’s Real Storage Location
2.1 Typical directory structure
Local inference tools that manage models in a content-addressed way usually create a hidden data directory under the user’s home directory, with two layers inside:
<data root>/├── blobs/ # Actual data: weights, chat templates, parameter configs, license text, etc.└── manifests/ # Metadata: each model's name, tags, and the parts it is composed of
File names in the blobs directory are the content digests themselves, for example sha256-500a8806e85ee... — no extension, and no per-model folders. The manifests directory, by contrast, is organized by the hierarchy source site / publisher / model name / tag, and each level can be treated as a “model key”. For a model downloaded from a community model hub, the manifest path typically retains a fragment of the source domain and the publisher namespace. In other words, reading through the manifests directory gives you a complete model inventory: which models exist, where they came from, and which tags each one has.
2.2 Finding the data directory
Data root locations vary by operating system and by tool. A common order of investigation:
- Hidden directory in the user’s home directory (such as
~/.<toolname>/), the first choice for most tools; - The operating system’s application data directory:
- Linux: usually
~/.local/share/<app>/or~/.config/<app>/; - Windows: usually
%LOCALAPPDATA%or%APPDATA%; - macOS: usually
~/Library/Application Support/<app>/;
- Linux: usually
Directories customized through environment variables: some tools let you specify the model location via an environment variable, for example:
# Linux / macOSecho $HOMEenv | grep -i -E 'model|cache|data|home'# Windows (PowerShell)$env:USERPROFILEGet-ChildItem Env: | Where-Object { $_.Name -match 'MODEL|CACHE|HOME|DATA' }
Once you have a candidate directory, file size confirms it quickly: real weight files are single large files of several GB or more, while chat templates and parameter configs are only a few hundred bytes to a few KB. For example:
# Windows: list files in the blobs directory by size to spot the real weight filesGet-ChildItem "$env:USERPROFILE\.<dataDir>\blobs" -File |Sort-Object Length -Descending |Select-Object -First 10 Name, @{Name='SizeGB';Expression={[math]::Round($_.Length/1GB,2)}}
2.3 Reading the model inventory correctly
On Linux or macOS, recursively listing the manifests directory shows the full model inventory. For deep hierarchies, you can also use a wildcard (with the globstar option enabled) or find:
# Option 1: recursively list all manifest filesls -laR ~/.ollama/models/manifests/# Option 2: list only the full paths of manifest filesshopt -s globstarls -l ~/.ollama/models/manifests/**/*.json# Option 3: use find to locate every manifest filefind ~/.ollama/models/manifests -type f
On Windows, use PowerShell’s recursive enumeration:
Get-ChildItem "$env:USERPROFILE\.ollama\models\manifests" -Recurse -File |Select-Object FullName, Length
Note: on Windows, directories whose names start with a dot are hidden by default in File Explorer. Enable “Hidden items” in the View options, or type the path directly into the address bar.
2.4 Reading the manifest: the structure determines what you can get out of it
A manifest is JSON metadata that typically contains one or more “layers”, describing the model weights, chat template, parameter config, license information, and so on. A simplified example:
{"schemaVersion": 2,"config": {"model_format": "gguf","parameter_size": "14B","quantization": "Q4_K_M","family": "llama","families": ["llama"],"template": "{{ .System }}\n{{ .Prompt }}","stop": ["<|im_end|>"]},"layers": [{ "mediaType": "application/vnd.local.image.model", "digest": "sha256:500a8806e85ee...", "size": 9060000000 },{ "mediaType": "application/vnd.local.image.template", "digest": "sha256:...", "size": 1289 },{ "mediaType": "application/vnd.local.image.params", "digest": "sha256:...", "size": 64 }]}
Two points matter here:
- Locate the weights by mediaType, not by file name: the layer whose media type is the model weight has a
digestfield holding the hash of the corresponding file in the blobs directory; the template and parameter layers that follow are only a few KB. - The manifest path is the only human-readable clue: the file name cannot be interpreted, but the path under
manifestsstates the source, the publisher, and the model name plainly.
If you already know a model’s name, you can go straight to it:
ls -l ~/.ollama/models/manifests/*/*/Qwen3-14B/latest# Or read the content directlycat ~/.ollama/models/manifests/*/*/Qwen3-14B/latest
3. Step 2: Verify Digests and Quantization Info to Avoid Mix-Ups
3.1 Building the “manifest path → digest → weight file” mapping
Use a script to link the human-readable manifest path with the hash and size of the weight file:
import jsonfrom pathlib import Pathmanifests = Path.home() / ".ollama" / "models" / "manifests"for p in sorted(manifests.rglob("*")):if not p.is_file():continuetry:meta = json.loads(p.read_text(encoding="utf-8"))except json.JSONDecodeError:continuemodel_key = p.parent.relative_to(manifests).as_posix()for layer in meta.get("layers", []):if "model" in layer.get("mediaType", ""):digest = layer["digest"].replace("sha256:", "")size_gb = layer.get("size", 0) / 1e9print(f"{model_key}\t{layer['mediaType']}\tsha256:{digest[:12]}…\t{size_gb:.2f} GB")
The output is usually a comparison table like the one below. Different sources and different quantization levels can correspond to the same model family, so always identify a file by the combination of manifest path + quantization level + the first few digest characters:
| Model (model key in the path) | Quantization | First 16 chars of digest | Weight size |
|---|---|---|---|
| Example model A (~14B) | Q4_K_M | 500a8806e85ee... |
~9.0 GB |
| Example model B (~9B) | Q4_K_M | 9437f5bf0dd0c... |
~5.5 GB |
| Example model B (~9B) | Q8_0 | 809626574d0cb... |
~9.7 GB |
| Example multimodal model C | Q4_K_M | 3f72a20a06f6... |
~8.5 GB |
| Example multimodal model C | Q4_K_M | dff0ffba4c90b... |
~8.5 GB |
The table above is illustrative only: digest-to-size relationships vary by model, so trust the actual output of the script on your own machine.
3.2 Verify weight integrity with the file header
When you hit a “model failed to load” error or loading aborts halfway, the first step is to confirm whether the file itself is intact. A GGUF file starts with the fixed magic number GGUF (hex 47 47 55 46), and a few lines of code check it quickly:
from pathlib import Pathdef check_gguf(path: Path) -> None:with path.open("rb") as f:magic = f.read(4)version = int.from_bytes(f.read(4), "little")print(f"{path.name}: magic={magic!r}, version={version}")
- If the magic number is not
GGUF, the file is not a weight file (it may be a template layer, a parameter layer, or a corrupted file). - If the version number is abnormal (0 or an extremely large value, for instance), the file was most likely truncated.
- A stricter approach is to recompute the full SHA256 of the file and compare it against the
digestin the manifest one by one. This is the most reliable method, but it requires reading several GB or even tens of GB in full and takes time, so it is best run once before a critical model goes live.
4. Step 3: Reorganize Models into the Convention Directory
4.1 The target tool’s directory convention
Convention-based tools typically scan a root directory and resolve model identity from the following hierarchy:
<model library root>/└── <publisher>/<model name>/<quantization level>/<weight file>.gguf
For example:
~/models/Qwen/Qwen3-14B/Q4_K_M/model.gguf~/models/community/Qwen3.5-9B/Q4_K_M/model.gguf
In this structure:
- Directory names carry the “model key” role, and any quantization-level string appearing in a directory name is picked up by the tool as the model’s quantization tag;
- Whether a model shows up in the list depends on the directory hierarchy being correct; the weight file name generally does not affect recognition;
- The model library root can be set in the software’s settings. Put it on a disk with plenty of space to avoid occupying the system drive.
4.2 Plan the model library layout
Before you start, settle the root directory and the naming rules. For example:
- Layer by
publisher/model name, matching the naming used in the manifest path so the model list maps easily onto the original tool’s list; - Make the quantization level its own layer, so different quantized versions of the same model can coexist and be compared for quality and speed;
- Keep file names simple, avoiding spaces, non-ASCII characters, and special characters — some tools handle non-ASCII characters or spaces in paths poorly;
- If several tools share one library, keep the root in a single location to reduce duplicate downloads.
4.3 Copy, hard link, or symbolic link?
Moving weights out of the blobs directory into the model library comes down to three options:
| Method | Disk usage | Does the original tool still work? | When to use it |
|---|---|---|---|
| Copy | Uses a full extra copy of the space | Yes | Plenty of free space; you plan to uninstall the old tool after migration |
| Hard link | Almost no extra space | Yes | Old and new tools need to coexist long term (same file system) |
| Symbolic link | Only the link itself | Yes | Weights live on another disk or partition; the original files are read-only |
Three points deserve emphasis:
- Hard links do not consume additional data space: two paths point to the same data. Even if you delete the file in the original directory, the data remains valid as long as the link exists.
- Hard links have boundaries: they generally cannot be created across file systems or partitions, and support for scanning non-regular files (symbolic links, hard links) varies from tool to tool. If the target tool cannot recognize a link, fall back to copying.
- Symbolic links usually require extra privileges on Windows: creating one typically needs administrator rights or Developer Mode, and cross-device links need additional authorization. Hard links within the same partition generally need no special privileges, so on Windows prefer hard links first.
A script that handles all of the above — it tries a hard link first, falls back to copying on failure, and takes the model library root explicitly:
import json, os, shutilfrom pathlib import PathMANIFESTS = Path.home() / ".ollama" / "models" / "manifests"BLOBS = MANIFESTS.parent / "blobs"MODEL_ROOT = Path.home() / "models" # change as neededdef export_models():for p in sorted(MANIFESTS.rglob("*")):if not p.is_file():continuetry:meta = json.loads(p.read_text(encoding="utf-8"))except json.JSONDecodeError:print(f"[skip] cannot parse manifest: {p}")continue# 1. Derive the target model key from the relative manifest path,# keeping the "source/publisher/model name/tag" structuremodel_key = p.parent.relative_to(MANIFESTS).as_posix()# e.g. modelscope.cn/Qwen/Qwen3-14B/latest -> Qwen/Qwen3-14Bparts = model_key.split("/")if len(parts) >= 3:model_key = "/".join(parts[1:3])quant = meta.get("quantization") or meta.get("config", {}).get("quantization", "unknown")target_dir = MODEL_ROOT / model_key / quanttarget_dir.mkdir(parents=True, exist_ok=True)# 2. Separate the weight layer from the supporting files by media typefor layer in meta.get("layers", []):digest = layer["digest"].replace("sha256:", "")src = BLOBS / f"sha256-{digest}"if not src.exists():print(f"[missing] {src}")continueif "model" in layer.get("mediaType", ""):dst = target_dir / "model.gguf"else:dst = target_dir / f"{digest[:12]}.layer"if dst.exists():continuetry:os.link(src, dst) # hard linkprint(f"[link] {src.name} -> {dst}")except OSError:shutil.copy2(src, dst) # fall back to copyprint(f"[copy] {src.name} -> {dst}")if __name__ == "__main__":export_models()
5. Step 4: Choosing a Quantization Level and Estimating Resources
5.1 Common quantization levels
Quantization levels in the GGUF ecosystem broadly follow this pattern (a smaller number means more aggressive compression):
- Q8_0: 8-bit quantization close to the original precision, with the smallest quality loss, but a size that is nearly twice that of the Q4 versions;
- Q6_K, Q5_K_M: modest precision loss, suitable when VRAM or RAM is plentiful;
- Q4_K_M: the community-recognized sweet spot; most everyday chat and coding-assistance tasks get usable quality;
- Q3, Q2 series: smaller in size, but quality can drop noticeably on complex reasoning and long-context tasks;
- IQ series (such as IQ4_XS): typically retains more precision than traditional quantization at the same size, at the cost of a small extra dequantization overhead.
5.2 Rough resource estimation
A common rule of thumb: every billion parameters takes about 0.6–0.7 GB at Q4_K_M and about 1.0 GB at Q8_0. On top of that, add:
- KV cache: grows with context length; the exact size depends on the number of layers, the number of attention heads, and whether grouped-query attention is used, ranging from a few GB to more than ten GB;
- Inference framework overhead: CUDA context, compute buffers, and similar;
- Multimodal projection weights: the projection layers needed for vision, audio, and other inputs are usually stored separately and consume additional space.
As a result, a consumer GPU with 24 GB of VRAM generally runs a 14B-class Q4_K_M model comfortably; switch to Q8_0 or a long context and you may exceed VRAM capacity, forcing some layers to be offloaded to RAM and slowing inference down.
5.3 The extra risk that comes with duplication
The relationship above shows that storing large models redundantly costs more than disk space: when several copies of the same model exist, it is easy to overwrite a newer version with an older one, or to load weights under inconsistent configuration, making results irreproducible. Preferring hard links or a single unified model library during migration is precisely how you avoid this kind of “version drift”.
6. Step 5: The Post-Migration Verification Checklist
Finishing the migration is not the same as succeeding at it. Confirm each item below:
- The model is visible: it appears in the target tool’s model list, and its parameter count and quantization level match the manifest;
- Context length: is the default value what you expected? Adjust manually if needed;
- Chat template: are the role markers correct across multiple turns, with no confusion about who is speaking?
- Stop conditions: does the response terminate normally instead of repeating the same sentence over and over?
- Hardware offload: does the number of GPU-offloaded layers match expectations, and is the VRAM usage reported by
nvidia-smistable? - Inference speed baseline: record time-to-first-token and generation speed (tokens/s) as a reference for later comparisons;
- Multimodal capability: if the model ships multimodal projection files, confirm the target tool loads them correctly and recognizes image/audio input;
- Special capabilities: if the model depends on a particular attention implementation (some sliding-window-style models rely on it for long-context support), confirm the target tool supports it — otherwise you may need to reduce the context length.
If any item does not behave as expected, suspect two causes first: the model file is incomplete (go back to 3.2 to verify), or the target tool’s directory convention or model configuration is not being recognized correctly (go back to 4.1 to check the hierarchy).
7. Consistent Model Governance Across Cloud and Local: From Manual Copying to a Unified Repository
The workflow above solves migration for a single machine with a handful of models. Once the count grows to dozens, the team has multiple machines, or models must be synchronized across environments, manual copying quickly becomes a burden. The more sensible approach is to treat models as assets that need version management: build a unified model repository and have both local tools and cloud inference services pull from the same source.
7.1 Using object storage as a model repository
- Unified naming and versioning: the “bucket + directory prefix” structure of object storage naturally expresses the hierarchy
publisher/model name/quantization level/version, and can mirror the local model library convention; - Integrity checks: record each file’s ETag (content digest) at upload time and compare after download, replacing manual verification;
- Hot/cold tiering: keep frequently used models in standard storage and move historical versions to infrequent-access or archive storage to lower long-term cost;
- Permissions and auditing: access control policies limit the readable scope, and operation logs support compliance audits.
On Baidu AI Cloud, enterprises can host a model repository on object storage (BOS) and pair it with a content delivery network (CDN) to accelerate downloads from nearby edge locations, so developers get models quickly in their local environments.
7.2 Standardizing the inference environment with containers
Model loading behavior depends not only on the weights but also on the inference engine version, drivers, dependent libraries, and launch parameters. It is advisable to package the inference service as an image, deploy it on Baidu AI Cloud Container Engine (CCE), and pin the following settings in the image:
- model version and quantization level;
- launch parameters such as context length and batch size;
- chat template and stop-sequence configuration.
Only then does model behavior in local experiments match the online service, avoiding the “works locally, behaves differently in production” situation.
7.3 Observability: from “does it work” to “how well does it work”
After launch, keep watching these metrics:
- Time-to-first-token and generation throughput;
- VRAM usage and batch queueing;
- Output quality: run periodic regression on an evaluation set to catch quality degradation caused by quantization or version changes.
Baidu AI Cloud Log Service (BLS) can collect application logs and runtime metrics centrally to support troubleshooting. Note that responsibility for producing, training, and using model data still rests with the enterprise itself, and applicable laws, regulations, and the license terms set by the model provider must be respected.
8. Troubleshooting Common Issues
| Symptom | Likely cause | How to fix it |
|---|---|---|
| The model does not appear in the target tool | The directory hierarchy does not match the convention; the quantization-level directory is missing | Check the path against 4.1; confirm the root directory setting is correct |
| Loading fails with a “file format error” | You copied the template or parameter layer instead of the weight layer | Go back to 3.1 and re-locate the weights by mediaType |
| Loading aborts midway | The file was truncated; the disk is out of space | Verify the file header as in 3.2; free up space and copy again |
| Output repeats and never stops | The chat template or stop sequences are not configured correctly | Specify the template and stop sequences explicitly in the model configuration |
| Slow speed, low VRAM usage | Quantization is too high or the context is too long, so some layers are offloaded to RAM | Switch to a lower quantization level, or shorten the context |
| Multimodal images are not recognized | The projection weights were not migrated together | Copy the multimodal projection files into the model directory too, or point to them separately in the tool |
9. Summary
Hash naming, at its core, decouples “name” from “data”. Once you understand that, migration follows a clear main line:
- Locate the data directory and find
blobsandmanifests; - Read the manifests to build the mapping “model key → digest → weight file”;
- Verify integrity by checking the file header and version number;
- Reorganize by convention so the directory structure expresses model identity;
- Choose the migration method, using hard links to balance space and compatibility;
- Pick a sensible quantization level, trading quality against resources;
- Verify item by item that chat behavior, stop sequences, hardware offload, and speed all match expectations;
- Once you scale up, build a unified model repository on object storage and container services so local and cloud share one workflow.
On Baidu AI Cloud, this approach extends smoothly from single-machine experimentation to enterprise adoption: object storage (BOS) holds model assets, Container Engine (CCE) standardizes inference environments, and Log Service (BLS) with cloud monitoring provide observability. For developers, the goal is not “moving files into another directory” but managing models in a way that is traceable, reusable, and verifiable.
