Evaluating and Applying Large Language Models in Office Work and Software Development

1. Evaluate by Task, Not by a Single Overall Ranking

A common mistake when discussing large language model capabilities is trying to answer “which model is strongest” with a single overall ranking. In practice, models perform differently in instruction understanding, long-document processing, code generation, structured output, tool calling, and multi-turn collaboration. Even with the same model, the final result can be significantly affected by whether the task description is clear, whether the input materials are well structured, and whether the output is checked.

A more practical approach is therefore to define the task first and then compare results through small-scale tests. Evaluation can use the following dimensions:

• Accuracy: whether facts, formulas, code logic, and citations are correct.
• Completeness: whether required fields, steps, or boundary conditions are missing.
• Format stability: whether the model can output the specified structure and handle tables, checklists, or fixed templates.
• Verifiability: whether results can be traced, reviewed, and revised.
• Collaboration cost: whether obtaining a usable result requires extensive additional clarification.

Model output should be treated as a draft for review, not as inherently trustworthy final output. This is especially important for financial data, contract terms, production code, and external-facing materials, where human review and necessary automated validation cannot be omitted.

2. Document Tasks: Turn Repetitive Operations into Reviewable Workflows

Document processing often includes information extraction, format organization, content revision, and version review. Large language models can help with initial organization, but the result depends on input quality and clearly defined task boundaries.

2.1 Extracting Information from PDFs

When working with multiple quotations, reports, or scanned materials, you can first ask the model to extract content using fixed fields such as name, date, amount, delivery scope, and notes, and then organize the results into a table. Compared with a vague request to “summarize the file,” explicit fields make omissions easier to find and support item-by-item verification.

For scanned documents, consider that text recognition may introduce misspellings, missing lines, or table misalignment. Mark uncertain content and verify it against the original. For key fields such as amounts, identifiers, and dates, perform item-by-item comparison rather than judging accuracy from a summary alone.

If files need to be merged, annotated, or filled into forms, check the file structure and final rendering first. Automation can reduce repetitive work, but it cannot replace confirmation of page order, field positions, and signature areas.

2.2 Editing Word Processing Documents

When drafting or revising a document, provide the purpose, audience, tone, length, and content that must be retained. For example, a request to “rewrite the technical description for managers, retain risks and dependencies, and organize it by background, approach, constraints, and conclusion” is usually more likely to produce a usable result than “polish this.”

When modifying an existing document, ask the model to list the changes and the reasons for them instead of returning only a rewritten full text. This reduces the risk of important information being accidentally deleted or altered and helps reviewers quickly judge whether the changes are reasonable. For documents involving multiple collaborators, keep version records and distinguish factual corrections, language improvements, and opinion changes.

2.3 Analyzing Spreadsheets

Large language models can help explain data fields, construct analysis approaches, check formulas, summarize trends, and suggest chart designs. Before actual analysis, confirm how null values, duplicate records, units, date formats, and outliers should be handled. If data definitions are inconsistent, conclusions may still be distorted even when the calculation process appears correct.

It is recommended to split analysis tasks into several stages:

1. Explain field meanings and statistical definitions.
2. Check data quality and anomalous records.
3. Define the required metrics and calculation methods.
4. Perform calculations and spot-check key results.
5. Explain conclusions and limitations with charts or text.

For example, when summarizing business data by month, confirm the rules for dividing months, how returned records are handled, and whether the statistical scope includes unfinished projects. For important metrics, recalculate with spreadsheet formulas or scripts rather than relying only on natural-language reasoning.

2.4 Creating Presentation Materials

Models can help design presentation structure, compress long text, generate page titles, and suggest chart approaches. However, presentation quality depends not only on wording but also on information hierarchy, page density, chart accuracy, and narrative order.

A safer process is to generate an outline first, confirm the audience and conclusions, and then expand the page content. After that, check each page to see whether claims and evidence correspond, and verify numbers, legends, and sources. Do not stuff long paragraphs directly into slides, and do not treat model-generated chart descriptions as the data itself. For external presentations, also unify fonts, colors, terminology, and layout standards.

3. Development Tasks: Involve Models in the Workflow, Not as a Substitute for Engineering Judgment

In software development, large language models are suitable for helping understand code, generate drafts, suggest test ideas, and summarize changes. They do not inherently know all project constraints and may overlook compatibility, security, and historical behavior. Output therefore needs to be constrained by engineering workflows.

3.1 Generating Clear Commit Messages

A commit message should accurately summarize the purpose and impact scope of the change. Writing only “fix issue” or “update code” makes it difficult for team members to understand the change and is also unhelpful for locating historical causes later.

First inspect the diff for the current change, then organize the text around type, scope, action, and necessary explanation. For example:

type(scope): brief description of the change

The type can use general categories such as new feature, bug fix, documentation update, or refactor. After generating the message, developers should still confirm that it matches the actual changes, avoiding situations where the model writes unimplemented goals into the record or omits compatibility impact.

3.2 Move Testing Ahead of Implementation

The core idea of test-driven development is to describe expected behavior with tests first, then write the implementation, and finally improve the code structure. A common cycle is:

1. Red: Add a test that expresses the requirement and currently fails.
2. Green: Write the minimum implementation needed to pass the test.
3. Refactor: Improve the structure under test protection while keeping behavior unchanged.

Models can help propose boundary conditions, add test examples, or explain failure messages, but developers still need to judge whether the tests cover real requirements. Testing only the normal path is insufficient; consider empty input, invalid input, insufficient permissions, duplicate requests, and failing dependencies.

3.3 Code Review and Risk Control

When asking a model to review code, provide relevant context, expected behavior, and the change scope, and require issues to be listed by severity together with impact and reproduction conditions. Do not ask only “are there any issues,” because broad instructions tend to produce generic advice.

Review results should be compared line by line with the actual code. Focus on boundary conditions, error handling, resource release, input validation, concurrency behavior, and permission control. For proposed changes, confirm them through tests, static checks, or human review rather than copying them directly into production.

4. Build Reusable Validation Benchmarks

When selecting models, prepare a set of test tasks close to actual work rather than making decisions based on a single demonstration. Test materials should cover typical tasks, complex tasks, and error-prone boundary cases, and use a consistent scoring standard.

For example, record the accuracy rate, number of omissions, format compliance, amount of manual editing, and verification time for each task. When sensitive data is involved, first assess data classification, access permissions, and organizational policy to avoid entering content that should not be shared externally into unapproved services. For important conclusions, retain source materials, generated results, and human revision records for traceability.

5. Implementation Advice: Start with Low-Risk, High-Repetition Work

Adopting large language models does not require changing the entire workflow at once. A safer path is to start with tasks that are frequent, easy to verify, and low-impact when errors occur, such as categorizing materials, organizing drafts, drafting test cases, and generating change descriptions. After confirming benefits and risks, gradually expand to more complex scenarios.

Ultimately, model capability is only one part of efficiency improvement. Clear inputs, stable templates, clear responsibility boundaries, automated checks, and human review together determine whether results are reliable. Breaking tasks into smaller pieces, writing acceptance criteria clearly, and verifying key data thoroughly are what allow large language models to truly integrate into daily office and development workflows.