Equity Penetration Analysis with LLMs and Knowledge Graphs: Architecture, Reasoning, and Guardrails

    This article presents a practical design for equity penetration analysis that combines large language models with knowledge graphs. It covers the main challenges, a four-layer system architecture, graph schema and extraction quality controls, control-determination reasoning, a code example for model-assisted analysis, RAG-based hallucination control, data security, common errors, and infrastructure choices. The emphasis is on a clear division of labor: models read and interpret unstructured text, while deterministic programs calculate ratios, apply time filters, and retain evidence.

    1. Challenges of Equity Penetration and the Role of LLMs

    Equity penetration analysis is a foundational task in enterprise risk management and investment due diligence. It must answer two questions: who is the ultimate controller of a company, and whether two companies have a concealed affiliation.

    Traditional approaches rely on rule engines and manual review. When ownership structures involve multiple layers of nesting, cross-shareholding, nominee arrangements, and concert party agreements, these approaches face three typical problems. First, data is often stale; equity changes may only be discovered through annual reports or public business registration disclosures. Second, hidden relationships are difficult to find. For example, two companies may have no direct shareholding relationship but share the same senior executive or a relative of that executive in a key position. Third, rule maintenance is costly, because every new ownership arrangement may require manually added rules.

    The value of LLMs here lies mainly in processing unstructured text. Key information in annual reports, announcements, litigation records, and agreement clauses is difficult to extract reliably with regular expressions and fixed rules. LLMs can read long text and identify entities and relationships. However, LLMs should not make all judgments directly. Ratio calculations, threshold decisions, and evidence retention require deterministic programs. The design below follows this division of labor: models read and interpret, while rules and programs calculate and store.

    2. System Architecture and Boundaries

    A usable equity penetration system usually has four layers.

    1. Data collection and preprocessing layer: Acquire data from business registration records, annual reports, equity change records, judicial announcements, and other sources. Original files must retain version information, including source, acquisition time, and file hash. Scanned documents are converted to text through OCR before entering the extraction process.
    2. Graph construction and storage layer: Extract entities and relationships and write them into a graph database. Each relationship must record the source document ID and effective time for later traceability.
    3. Reasoning and analysis layer: Based on a user request, retrieve the subgraph around the target company from the graph database, combine it with calculated ratio results, and pass the result to the LLM for summarization and explanation.
    4. Review and output layer: Perform structural validation on model output, route low-confidence conclusions to manual review, and generate the final report.

    Each layer needs clear boundaries. The data collection layer must not send conclusions directly to the reasoning layer, and conclusions from the reasoning layer must not be written back into the graph unless they have been manually confirmed.

    3. Graph Schema, Extraction Quality, and Evaluation

    The accuracy of equity penetration depends on whether graph data is accurate and complete. It is advisable to fix the schema before starting extraction.

    Type Field examples Notes
    Node: Company Unified social credit code, name, registered location Use the credit code as the primary key; names may collide.
    Node: Natural person Pseudonymized identifier, name (encrypted), identity category Raw identity document numbers are not stored in the graph.
    Edge: Capital contribution Shareholding ratio, subscribed amount, contribution date Ratios follow source documents and are not inferred by the model.
    Edge: Employment Position, start and end dates Used to identify overlapping appointments.
    Edge: Agreement Agreement type (concert party, nominee holding, voting rights entrustment), signing date, validity period Agreement relationships must include source-text evidence.

    Every edge should include four fields: valid_from, valid_to, source_doc_id, and ingest_time. After an equity change, old relationships should be given an expiration time rather than deleted; otherwise, the system may answer new questions with old data.

    Before extraction results enter the graph, it is advisable to define three confidence bands. For example, relationships with confidence of at least 0.9 are automatically ingested, those between 0.7 and 0.9 enter a review queue, and those below 0.7 are discarded. These thresholds are only a starting point and should be calibrated against sampling results.

    For quality evaluation, the following metrics are recommended:

    • Precision (P), recall (R), and F1, calculated separately by relationship type.
    • Weekly sampling of newly ingested relationships at a rate of no less than 5%, with at least 100 samples per relationship type.
    • Error categorization. Common types include entity false merges (same-name companies or natural persons), ratio misreading, use of the wrong time version, and agreement-type misclassification.

    4. Reasoning Stage: Control Determination Cannot Rely on the Model Alone

    Determining the ultimate controller requires distinguishing direct shareholding, indirect shareholding, and voting rights. Shareholding ratio is only one aspect.

    A simple example illustrates how to calculate indirect shareholding. A natural person, Zhang San, holds 40% of Company A, and Company A holds 60% of Company B. Zhang San therefore indirectly holds 24% of Company B through Company A (40% × 60%). This number is calculated by a program and is not generated by the model. Indirect shareholding through multiple paths must be calculated separately and then summed.

    Voting rights and shareholding ratios are not always aligned. Voting rights arrangements in the articles of association, voting rights entrustment agreements, concert party agreements, and nominee holding agreements may all change the control relationship. This content is often buried in contract clauses or announcement text, so the model needs to read and extract it, after which legal staff confirm its validity.

    The recommended division of labor is as follows:

    • Programs handle: path enumeration, ratio calculation, time filtering, and threshold decisions.
    • LLMs handle: reading agreement text, summarizing control relationships, explaining the reasoning process, and generating risk warnings.
    • Humans handle: confirming agreement validity, processing low-confidence conclusions, and signing off on final opinions.

    Conclusions should be output with a confidence level. Low confidence or incomplete evidence should be routed to legal manual review.

    5. Engineering Code Example

    The following code assumes a generic model service compatible with the OpenAI Chat Completions format. The API endpoint, model name, and key are passed through parameters or environment variables, and the code contains no real credentials. The input graph query result is assumed to contain an evidences list, where each item has an evidence_id field.

    1. import json
    2. import os
    3. import requests
    4. REQUIRED_FIELDS = {"ultimate_controller", "control_basis", "evidence_ids", "confidence", "risk_warnings"}
    5. CONFIDENCE_LEVELS = {"high", "medium", "low"}
    6. PROMPT_TEMPLATE = """You are an equity penetration analysis assistant. Reason about the target company's ultimate controller only from the graph data below.
    7. Rules:
    8. 1. Use only information in the input data. Do not fabricate shareholders, shareholding ratios, or agreements.
    9. 2. If the information is insufficient for a determination, set ultimate_controller to "unknown".
    10. 3. Every conclusion must cite evidence_id values from the input.
    11. 4. Output only one JSON object with the following fields: ultimate_controller, control_basis, evidence_ids, confidence (high/medium/low), risk_warnings.
    12. Graph data (JSON):
    13. {graph_json}
    14. """
    15. def build_prompt(graph_query_result):
    16. graph_json = json.dumps(graph_query_result, ensure_ascii=False)
    17. return PROMPT_TEMPLATE.format(graph_json=graph_json)
    18. def validate_result(data, allowed_evidence_ids):
    19. if not isinstance(data, dict) or not REQUIRED_FIELDS.issubset(data):
    20. return False
    21. if data["confidence"] not in CONFIDENCE_LEVELS:
    22. return False
    23. if not isinstance(data["evidence_ids"], list):
    24. return False
    25. if any(eid not in allowed_evidence_ids for eid in data["evidence_ids"]):
    26. return False
    27. if data["ultimate_controller"] != "unknown" and not data["evidence_ids"]:
    28. return False
    29. return True
    30. def analyze_ownership_chain(api_endpoint, model_name, graph_query_result):
    31. api_key = os.environ["LLM_API_KEY"]
    32. allowed_ids = {e["evidence_id"] for e in graph_query_result.get("evidences", [])}
    33. headers = {
    34. "Content-Type": "application/json",
    35. "Authorization": f"Bearer {api_key}",
    36. }
    37. payload = {
    38. "model": model_name,
    39. "messages": [{"role": "user", "content": build_prompt(graph_query_result)}],
    40. "temperature": 0.1,
    41. "max_tokens": 2000,
    42. }
    43. try:
    44. resp = requests.post(api_endpoint, headers=headers, json=payload, timeout=30)
    45. resp.raise_for_status()
    46. content = resp.json()["choices"][0]["message"]["content"]
    47. result = json.loads(content)
    48. except (requests.exceptions.RequestException, KeyError, IndexError, json.JSONDecodeError) as e:
    49. print(f"Call or parsing failed; routing to manual review: {e}")
    50. return {"status": "manual_review"}
    51. if not validate_result(result, allowed_ids):
    52. return {"status": "manual_review"}
    53. return {"status": "ok", "data": result}

    There are several points to note in the code. First, the model sometimes wraps JSON in Markdown code blocks; in that case json.loads will fail and the code routes the result to manual review. In production, you can strip code block markers before parsing or use the JSON output mode provided by the model service. Second, the validation function checks only structure and evidence references; it cannot verify legal validity, so human confirmation is still required. Third, timeouts and failures have a fallback path, so a single failed call does not interrupt the entire batch.

    6. Retrieval Augmentation and Hallucination Control

    LLMs tend to generate plausible but unsupported content when evidence is insufficient. Retrieval-augmented generation (RAG) can reduce this risk. The approach is to first retrieve relevant text snippets and graph nodes from a vector database and graph database, then pass the retrieval results together with evidence IDs to the model.

    RAG effectiveness depends on retrieval quality. When the retrieved evidence is complete, model hallucinations decrease noticeably. When a key agreement is missed or a snippet is truncated, the model may still produce an incorrect conclusion. Therefore, RAG can reduce hallucinations but cannot eliminate them. The system should record the evidence cited by each conclusion and support tracing from a conclusion back to the original document.

    7. Data Security and Compliance

    Equity penetration involves natural person information and enterprise operating information. Processing must comply with applicable personal information protection laws and regulations, and the legality of data sources and usage authorization must be confirmed. The following measures are recommended:

    • Store natural person information in pseudonymized form, use internal identifiers in the graph, and keep sensitive fields such as identity document numbers out of the graph.
    • Control access by role and follow the principle of least privilege.
    • Keep complete audit logs, including who queried which company, at what time, and which data was accessed.
    • Clarify data retention periods and cross-border transfer restrictions with model service providers; sensitive data must not be sent to services that have not been approved.
    • In report output, display only authorized information and avoid pasting raw data wholesale into external documents.

    8. Common Errors and Troubleshooting

    Common errors in practice can be grouped into several categories.

    • Treating an external investment ratio as control. A 30% stake does not necessarily confer control; articles of association, voting rights arrangements, and board seats must also be checked.
    • Ignoring voting rights entrustment and concert party agreements. These relationships are often not reflected in business registration and require separate extraction from agreement text.
    • Using outdated equity data. Queries must filter edges by time point and confirm that the query date matches the report date.
    • Entity disambiguation errors. Same-name companies or natural persons can cause incorrect graph node merges. The unified social credit code and pseudonymized identifiers should be used as primary keys, with names only as auxiliary attributes.
    • Failing to degrade after parsing errors. When model output format is abnormal, the system should route to manual review rather than directly output an empty or default conclusion.

    During troubleshooting, first confirm whether the subgraph data is complete, then check ratio calculation results, and finally check the correspondence between model output and evidence IDs.

    9. Infrastructure Selection

    An equity penetration system typically requires the following types of infrastructure: object storage for original files, a graph database for relationship networks, a vector database for retrieving textual evidence, a message queue for scheduling collection and extraction tasks, and logging and monitoring systems for tracking extraction latency and model call failure rates. When selecting infrastructure, compare data volume, query latency, operational cost, and compliance requirements; there is no need to prioritize a particular vendor.

    10. Scope

    This article describes a technical method and does not constitute legal advice or investment advice. Final conclusions on ultimate controller determination, related-party transaction review, and risk ratings should be reviewed and owned by qualified legal and risk control personnel. Any conclusion produced by the system should include evidence sources and confidence levels, and manual modification records should be retained.