The Debate Between “Appearance” and “Essence” in AI Reasoning

When large language models (LLMs) solve mathematical problems, their outputs display clear logical chains: “Extract known conditions → Apply Pythagorean theorem → Reach conclusion.” This human-like reasoning raises fundamental questions: Do these textual logic steps correspond to distinct computational states within neural networks?

Traditional research focuses on textual analysis of Chain-of-Thought (CoT):

• Key sentence identification: Locating core reasoning steps through attention weights
• Behavior classification: Labeling cognitive patterns like verification and backtracking
• Relationship mapping: Constructing dependency networks between reasoning steps

These methods reveal only surface-level structures. They cannot confirm whether geometric representations corresponding to reasoning actions exist in hidden layers - analogous to observing facial expressions without detecting neuronal firing patterns.

Geometric Representation Discovery: Breakthrough Experimental Design

Researchers from KAIST and an AI lab employed innovative verification methods:

1. Dataset Construction: Collected mathematical problems containing five typical reasoning actions (information extraction, formula recall, problem decomposition, computation execution, result verification)
2. Dynamic Tracking: Continuously captured time-series data of hidden layer activation vectors during model reasoning
3. Geometric Analysis: Used t-SNE dimensionality reduction to map high-dimensional vectors to 2D space, observing point cloud distributions for different reasoning actions

The experiment revealed striking results: In hidden layer space, point clouds corresponding to different reasoning actions formed clearly separated clusters. When recalling formulas, activation vectors spontaneously migrated to specific geometric regions with distinct boundaries from problem decomposition areas.

Dynamic Geometry Mechanisms

These geometric representations exhibit three key characteristics:

1. Self-Organization

Models form reasoning functional partitions without explicit instructions. In Transformer architectures, all layers share parameters but activate different neuron subsets through self-attention mechanisms - similar to an orchestra producing different melodies through varied instrument combinations.

2. State Transition

Reasoning processes manifest as trajectory movements in geometric space. For geometric proofs:

Initial state: Problem comprehension (Point A)
→ State transition: Extract known conditions (A→B)
→ State jump: Recall relevant theorems (B→C)
→ Path optimization: Construct auxiliary lines (C→D)
→ Final convergence: Reach conclusion (D→E)

Each state transition corresponds to directional movement of activation vectors in geometric space.

3. Functional Generalization

Untrained reasoning combinations still map to correct geometric regions. When testing compound reasoning (“decompose problem then recall formula”), activation vectors sequentially passed through decomposition and recall zones, forming continuous geometric paths.

Key Technical Breakthroughs

The research team achieved visualization through:

1. Dynamic Attention Tracking

# Pseudocode: Capture attention patterns for specific reasoning steps
def track_attention(model, input_text, target_phase):
    outputs = model(input_text, output_attentions=True)
    attention_weights = outputs.attentions  # Get multi-head attention matrices
    phase_mask = create_phase_mask(target_phase)  # Create reasoning phase mask
    return apply_mask(attention_weights, phase_mask)

Dynamic masking isolates attention distribution patterns across reasoning stages.

2. Geometric Feature Extraction

Contrastive learning framework for geometric encoder training:

Input: Activation vectors from different instances of same reasoning action
Output: Minimize intra-class distance, maximize inter-class distance
Loss function: InfoNCE loss + Center loss

This encoder maps high-dimensional activation vectors to semantically interpretable geometric features.

3. State Visualization System

Interactive 3D visualization platform featuring:

• Real-time display of geometric trajectories during reasoning
• Rotation/zoom capabilities for hidden layer inspection
• Annotation of geometric regions corresponding to key reasoning actions
• Numerical query function for activation vectors

Practical Implications for AI Development

This research opens new optimization directions:

1. Reasoning Efficiency

Geometric constraint optimization of attention mechanisms enables faster convergence of activation vectors to target regions. Experiments showed 27% faster reasoning in geometric proof tasks.

2. Enhanced Interpretability

After establishing mappings between geometric features and reasoning actions, developers can create explanation tools:

When activation vectors enter "recall zone": Auto-annotate "Model retrieving relevant knowledge"
When abnormal trajectory jumps occur: Trigger warning "Potential logical discontinuity in reasoning"

3. Improved Training Strategies

Curriculum learning guided by geometric space:

1. Initial stage: Train basic reasoning in simple geometric regions
2. Intermediate stage: Expand geometric space for complex reasoning exploration
3. Final stage: Optimize reasoning path continuity across full space

This approach improved mathematical reasoning accuracy by 15-20%.

Future Research Directions

Current limitations include:

1. Geometric stability: Whether different architectures (RNN, MLP) exhibit similar geometric properties
2. Multimodal extension: Whether visual reasoning tasks share isomorphic geometric representations with language tasks
3. Dynamic adaptation: Whether geometric partitions morph during continual learning

Researchers are exploring applications in:

• Logical structure analysis for code generation tasks
• Reasoning path validation in medical diagnosis
• Decision process auditing for autonomous vehicles

This breakthrough reveals the geometric nature of AI “thinking” while opening new pathways for developing more efficient, trustworthy reasoning models. When we can visualize AI’s thought trajectories, the boundaries of human-machine collaboration will fundamentally transform.