06-consolidation-update-vs-link-distinction

Dec 7, 2025

UPDATE vs LINK Distinction

Improvement: #3 of 4 Parent Doc: Experiment Improvements Status: Design Phase

Problem Statement

Current threshold-based decision logic cannot reliably distinguish between two fundamentally different scenarios that may have similar similarity scores:

UPDATE: Same entity, property changed (old info replaced by new)
LINK: Related entities, both valid (need connection, not replacement)

Example Confusion

Case 1: UPDATE (Score: 0.87)

Memory A: "Q1 마케팅 캠페인 예산은 5000만원입니다."
Memory B: "Q1 마케팅 캠페인 예산이 6000만원으로 증액됨."

Decision: UPDATE (correct)
Reason: Same entity (Q1 마케팅 캠페인), property (예산) changed (5000 → 6000)

Case 2: LINK (Score: 0.85)

Memory A: "Q1 OKR: 사용자 증가 20%, 매출 30억 목표"
Memory B: "Q2 OKR: 사용자 증가 15%, 매출 35억 목표. Q1 성과 기반."

Decision: CREATE + LINK (correct)
Reason: Different entities (Q1 vs Q2), both valid, sequential relationship

Current Problem

Both cases fall in 0.80-0.95 range (UPDATE threshold), but:

Case 1 should replace old info
Case 2 should create new and link to old

Current logic cannot distinguish based on score alone.

Design Goals

Primary Goals

Semantic Distinction: Distinguish "property change" from "related entity"
Relationship Detection: Identify sequential, causal, reference relationships
Accurate Decision: Route to UPDATE vs CREATE+LINK correctly
Explainability: Provide clear reasoning for decision

Non-Goals

Perfect Accuracy: Human judgment varies, aiming for > 85% accuracy
All Relationship Types: Focusing on core types (sequential, causal, update)
Graph Inference: Not inferring implicit relationships (explicit only)

Core Concepts

UPDATE Conditions

An UPDATE occurs when:

Same Subject: Both memories refer to the same entity/topic
Property Changed: A specific attribute/value has changed
Replacement Intent: New information supersedes old (old becomes outdated)
Temporal Progression: Later information updates earlier

Examples:

Budget amount changed: 5000만원 → 6000만원
Meeting time changed: 2시 → 3시
Status changed: 검토중 → 승인됨
Scope expanded: 3개 기능 → 5개 기능 추가

LINK Conditions

A LINK occurs when:

Related Subjects: Different but related entities/topics
Both Valid: Each has independent, valid information
Relationship Present: Sequential, causal, reference, or thematic connection
No Replacement: Neither supersedes the other

Examples:

Sequential: Q1 OKR → Q2 OKR
Causal: GDPR 요구사항 → 개인정보처리방침 변경
Reference: 예산 승인 → 캠페인 실행
Thematic: 마케팅 전략 회의 → 실행 계획

Approach: Property-Level Change Detection

Core Idea

Analyze whether new memory changes properties of existing entity vs introduces new entity.

Input: Memory A + Memory B
  ↓
Semantic Analysis (LLM)
  ↓
Property Change Detection
  ↓
Decision: UPDATE or CREATE+LINK

Change Analysis Schema

interface PropertyChangeAnalysis {
  // Core judgment
  decision: 'UPDATE' | 'CREATE_AND_LINK' | 'CREATE_UNRELATED';

  // Detailed analysis
  subject_analysis: {
    same_subject: boolean;           // Same entity?
    subject_A: string;               // Entity in Memory A
    subject_B: string;               // Entity in Memory B
    relationship: SubjectRelationship;
  };

  // Property changes (for UPDATE)
  property_changes?: PropertyChange[];

  // Relationship (for LINK)
  relationship_type?: RelationshipType;

  // Confidence and reasoning
  confidence: number;                // 0-1
  reasoning: string;
  key_factors: string[];
}

enum SubjectRelationship {
  IDENTICAL = 'identical',           // Same entity
  SEQUENTIAL = 'sequential',         // Q1 → Q2
  CAUSAL = 'causal',                 // Cause → Effect
  HIERARCHICAL = 'hierarchical',     // Parent → Child
  REFERENCE = 'reference',           // Mentions/refers to
  THEMATIC = 'thematic',             // Same theme, different focus
  UNRELATED = 'unrelated',
}

interface PropertyChange {
  property: string;                  // Which property changed
  old_value: string;                 // Previous value
  new_value: string;                 // New value
  change_type: 'value_change' | 'addition' | 'removal' | 'expansion';
}

enum RelationshipType {
  SEQUENTIAL = 'sequential',         // Time-based progression
  CAUSAL = 'causal',                 // Cause and effect
  PREREQUISITE = 'prerequisite',     // Dependency
  ELABORATION = 'elaboration',       // Adds detail
  REFERENCE = 'reference',           // Mentions
  ALTERNATIVE = 'alternative',       // Different option
}

LLM-Based Property Analysis

Analysis Prompt

const PROPERTY_ANALYSIS_PROMPT = `
당신은 두 메모리를 분석하여 UPDATE인지 LINK인지 판단하는 전문가입니다.

## 판단 기준

### UPDATE 조건
- 같은 대상(entity)에 대한 정보
- 특정 속성(property)이 변경됨
- 새 정보가 이전 정보를 대체함

예시:
Memory A: "Q1 예산 5000만원"
Memory B: "Q1 예산 6000만원으로 증액"
→ UPDATE (같은 Q1 예산, 금액 변경: 5000→6000)

### CREATE + LINK 조건
- 관련은 있지만 다른 대상들
- 각각 독립적으로 유효한 정보
- 시간순, 인과관계, 참조 등의 관계

예시:
Memory A: "Q1 OKR: 사용자 20% 증가"
Memory B: "Q2 OKR: 사용자 15% 증가, Q1 기반"
→ CREATE + LINK (다른 분기, 둘 다 유효, 시간순 관계)

---

이제 다음을 분석해주세요:

Memory A:
"""
{memoryA}
"""

Memory B:
"""
{memoryB}
"""

JSON 형식으로 답변:
{
  "decision": "UPDATE|CREATE_AND_LINK|CREATE_UNRELATED",
  "subject_analysis": {
    "same_subject": true/false,
    "subject_A": "Memory A의 핵심 대상",
    "subject_B": "Memory B의 핵심 대상",
    "relationship": "identical|sequential|causal|hierarchical|reference|thematic|unrelated"
  },
  "property_changes": [
    {
      "property": "변경된 속성명",
      "old_value": "이전 값",
      "new_value": "새 값",
      "change_type": "value_change|addition|removal|expansion"
    }
  ],
  "relationship_type": "sequential|causal|prerequisite|elaboration|reference|alternative",
  "confidence": 0.0~1.0,
  "reasoning": "판단 근거를 명확히 설명",
  "key_factors": ["판단에 중요했던 요소들"]
}

주의사항:
1. 같은 키워드가 있어도 대상이 다르면 CREATE + LINK (예: Q1 vs Q2)
2. 명확한 속성 변경이 있으면 UPDATE
3. 시간순(Q1→Q2), 인과(A→B), 참조 관계가 있으면 LINK
`;

async function analyzePropertyChange(
  memoryA: MemoryNode,
  memoryB: MemoryNode,
  llm: LLMClient
): Promise<PropertyChangeAnalysis> {
  const prompt = PROPERTY_ANALYSIS_PROMPT
    .replace('{memoryA}', memoryA.content)
    .replace('{memoryB}', memoryB.content);

  const response = await llm.generate({
    prompt,
    model: 'gpt-4o',  // Need good reasoning
    temperature: 0.0,
    response_format: { type

Dynamic Factor Weights

Core Idea

Adjust similarity factor weights based on potential decision type.

interface DynamicWeights {
  content: number;
  people: number;
  threadId: number;
  subject: number;
  entities: number;
}

// Default weights (baseline)
const DEFAULT_WEIGHTS: DynamicWeights = {
  content: 0.50,
  people: 0.15,
  threadId: 0.15,
  subject: 0.10,
  entities: 0.10,
};

// Adjusted for UPDATE detection
const UPDATE_WEIGHTS: DynamicWeights = {
  content: 0.35,    // Lower - content differs when property changes
  people: 0.15,
  threadId: 0.25,   // Higher - same thread indicates UPDATE
  subject: 0.15,    // Higher - same subject critical
  entities: 0.10,
};

// Adjusted for LINK detection
const LINK_WEIGHTS: DynamicWeights = {
  content: 0.30,    // Lower - content can differ
  people: 0.20,     // Higher - same people, related topics
  threadId: 0.10,   // Lower - can be different threads
  subject: 0.15,
  entities: 0.25,   // Higher - same entities indicate relationship
};

function adjustWeights(
  baseWeights: DynamicWeights,
  context: {
    same_thread: boolean;
    same_people: boolean;
    same_entities: boolean;
    similarity_score: number;
  }
): DynamicWeights {
  // If same thread + high similarity → likely UPDATE
  if (context.same_thread && context.similarity_score > 0.75) {
    return UPDATE_WEIGHTS;
  }

  // If same entities but different thread → likely LINK
  if (context.same_entities && !context.same_thread) {
    return LINK_WEIGHTS;
  }

  // Default
  return baseWeights;
}

Enhanced Decision Flow

Current Flow (Threshold-Based)

Similarity Score
      ↓
  >= 0.95 → SKIP
  0.80-0.95 → UPDATE   ← Problem: Cannot distinguish UPDATE vs LINK
  0.50-0.80 → CREATE + LINK
  < 0.50 → CREATE

New Flow (Property-Based)

┌─────────────────────────────────────────────────┐
│          Step 1: Similarity Search              │
│          (Get top-k candidates)                 │
└─────────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────┐
│     Step 2: Initial Threshold Classification    │
├─────────────────────────────────────────────────┤
│  >= 0.95 → SKIP (duplicate)                     │
│  < 0.50  → CREATE (unrelated)                   │
│  0.50-0.95 → AMBIGUOUS (needs analysis) ────┐   │
└─────────────────────────────────────────────┘   │
                                                  │
                    ┌─────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────────┐
│  Step 3: Property Change Analysis (LLM)         │
│  - Same subject?                                │
│  - Property changed?                            │
│  - Relationship type?                           │
└─────────────────────────────────────────────────┘
                    ↓
        ┌───────────┼───────────┐
        ↓           ↓           ↓
   Same Subject  Related    Unrelated
   Property Chg  Subjects
        ↓           ↓           ↓
      UPDATE    CREATE+LINK   CREATE

Implementation

async function decideWithPropertyAnalysis(
  newMemory: NewMemoryInput,
  similarityResults: SimilarityResult[],
  llm: LLMClient,
  getMemoryNode: (id: string) => Promise<MemoryNode>
): Promise<DecisionResult> {
  // Step 1: Get best match
  const bestMatch = similarityResults[0];

  // Step 2: Clear cases (no LLM needed)
  if (bestMatch.score >= 0.95) {
    return { decision: DecisionType.SKIP, reason: 'Duplicate', ... };
  }
  if (bestMatch.score < 0.50) {
    return { decision: DecisionType.CREATE, reason: 'Unrelated', ... };
  }

  // Step 3: Ambiguous range (0.50-0.95) - analyze with LLM
  const existingMemory = await getMemoryNode(bestMatch.memoryId);

  const analysis = await analyzePropertyChange(
    existingMemory,
    newMemory,
    llm
  );

  // Step 4: Decision based on analysis
  if (analysis.decision === 'UPDATE') {
    // Verify score is high enough for UPDATE (> 0.70)
    if (bestMatch.score >= 0.70) {
      return {
        decision: DecisionType.UPDATE,
        targetMemoryId: existingMemory.id,
        reason: analysis.reasoning,
        action: {
          type: 'update',
          targetId: existingMemory.id,
          propertyChanges: analysis.property_changes,
          createVersion: true,
        },
        metadata: {
          confidence: analysis.confidence,
          subject_relationship: analysis.subject_analysis.relationship,
        },
      };
    } else {
      // Score too low for UPDATE, fallback to CREATE
      return {
        decision: DecisionType.CREATE,
        reason: `Analysis suggests UPDATE but score too low (${bestMatch.score})`,
        ...
      };
    }
  }

  if (analysis.decision === 'CREATE_AND_LINK') {
    return {
      decision: DecisionType.CREATE,
      reason: analysis.reasoning,
      action: {
        type: 'create',
        memoryData: newMemory,
        relatedMemoryIds: [existingMemory.id],
        relationshipType: analysis.relationship_type || 'related_to',
      },
      metadata: {
        confidence: analysis.confidence,
        relationship_type: analysis.relationship_type,
      },
    };
  }

  // CREATE_UNRELATED
  return {
    decision: DecisionType.CREATE,
    reason: analysis.reasoning,
    action: { type

Test Cases: UPDATE vs LINK Boundary

Test Case Set (20 cases)

UPDATE Cases (10)

UPDATE_BOUNDARY_001: Budget amount change

{
  "existing": "Q1 마케팅 예산 5000만원",
  "new": "Q1 마케팅 예산 6000만원으로 증액",
  "expected": "UPDATE",
  "score_range": [0.82, 0.90],
  "reasoning": "Same subject (Q1 예산), property changed (금액: 5000→6000)"
}

UPDATE_BOUNDARY_002: Meeting time change

{
  "existing": "프로젝트 킥오프 미팅 2시",
  "new": "프로젝트 킥오프 미팅 3시로 변경",
  "expected": "UPDATE",
  "score_range": [0.85, 0.92],
  "reasoning": "Same meeting, time property changed (2시→3시)"
}

UPDATE_BOUNDARY_003: Status change

{
  "existing": "계약서 검토 중. 법무팀 확인 대기.",
  "new": "계약서 검토 완료. 최종 승인됨.",
  "expected": "UPDATE",
  "score_range": [0.78, 0.88],
  "reasoning": "Same contract, status changed (검토중→완료)"
}

LINK Cases (10)

LINK_BOUNDARY_001: Quarterly OKR sequence

{
  "existing": "Q1 OKR: 사용자 20% 증가 목표",
  "new": "Q2 OKR: 사용자 15% 추가 증가, Q1 성과 기반",
  "expected": "CREATE_AND_LINK",
  "score_range": [0.72, 0.85],
  "relationship": "sequential",
  "reasoning": "Different subjects (Q1 vs Q2), both valid, sequential"
}

LINK_BOUNDARY_002: Cause-effect relationship

{
  "existing": "GDPR 규정 준수 요구사항 발표",
  "new": "개인정보처리방침 업데이트. GDPR 대응.",
  "expected": "CREATE_AND_LINK",
  "score_range": [0.68, 0.80],
  "relationship": "causal",
  "reasoning": "Different subjects (규정 vs 방침), causal relationship"
}

LINK_BOUNDARY_003: Prerequisite relationship

{
  "existing": "Q1 예산 6000만원 승인 완료",
  "new": "Q1 마케팅 캠페인 실행 시작. 예산 집행.",
  "expected": "CREATE_AND_LINK",
  "score_range": [0.75, 0.88],
  "relationship": "prerequisite",
  "reasoning": "Different subjects (예산 승인 vs 캠페인 실행), dependency"
}

Few-Shot Examples for LLM

UPDATE Examples

const UPDATE_EXAMPLES = [
  {
    memoryA: "Q1 마케팅 캠페인 예산은 5000만원입니다.",
    memoryB: "Q1 마케팅 캠페인 예산이 6000만원으로 증액되었습니다.",
    decision: "UPDATE",
    reasoning: "같은 'Q1 마케팅 캠페인 예산'에 대한 정보. 예산 금액이 5000만원에서 6000만원으로 변경됨.",
    property_changes: [
      { property: "예산", old_value: "5000만원", new_value: "6000만원", change_type: "value_change" }
    ],
  },
  {
    memoryA: "프로젝트 킥오프 미팅 1월 15일 오후 2시",
    memoryB: "프로젝트 킥오프 미팅 시간 변경: 1월 15일 오후 3시",
    decision: "UPDATE",
    reasoning: "같은 '프로젝트 킥오프 미팅'의 시간이 2시에서 3시로 변경됨.",
    property_changes: [
      { property: "시간", old_value: "오후 2시", new_value: "오후 3시", change_type: "value_change" }
    ],
  },
];

LINK Examples

const LINK_EXAMPLES = [
  {
    memoryA: "Q1 OKR: 사용자 증가 20%, 매출 30억 목표. 달성률 25%/32억.",
    memoryB: "Q2 OKR: 사용자 증가 15% (누적 40%), 매출 35억. Q1 성과 기반.",
    decision: "CREATE_AND_LINK",
    reasoning: "다른 분기의 OKR (Q1 vs Q2). 각각 독립적으로 유효하며, 시간순 연결 관계.",
    relationship_type: "sequential",
  },
  {
    memoryA: "GDPR 규정 준수 요구사항 발표. 개인정보 처리 기준 강화.",
    memoryB: "개인정보처리방침 업데이트 완료. GDPR 요구사항 반영.",
    decision: "CREATE_AND_LINK",
    reasoning: "다른 대상 (규정 요구사항 vs 방침 업데이트). 인과관계 (규정→방침 변경).",
    relationship_type: "causal",
  },
];

Validation Experiment

Experiment 3: UPDATE vs LINK Accuracy

Goal: Validate property-based distinction outperforms threshold-only approach

Test Cases: 40 cases

20 UPDATE boundary cases (score 0.70-0.95)
20 LINK boundary cases (score 0.50-0.90)

Baseline: Current threshold-based logic (score >= 0.80 → UPDATE)

Test: Property-based analysis + LLM judgment

Metrics:

Overall accuracy (correct UPDATE vs LINK decision)
UPDATE precision/recall
LINK precision/recall
Confusion rate (UPDATE misclassified as LINK, vice versa)

Success Criteria:

Overall accuracy > 85%
UPDATE precision > 85%
LINK precision > 85%
Confusion rate < 15%

Performance Considerations

Latency

LLM Call: ~1-2 seconds per analysis
Optimization: Only call LLM for ambiguous cases (0.50-0.95 range)
Caching: Cache analysis results for repeated comparisons

Cost

Per Analysis: ~$0.001-0.002 (GPT-4o)
Optimization: Skip LLM for clear cases (> 0.95, < 0.50)
Expected: ~50% of cases need LLM (ambiguous range)

Accuracy vs Speed Trade-off

interface DecisionConfig {
  use_property_analysis: boolean;
  llm_threshold_min: number;   // Only use LLM if score > this
  llm_threshold_max: number;   // Only use LLM if score < this
}

// Fast mode (lower accuracy)
const FAST_CONFIG: DecisionConfig = {
  use_property_analysis: false,  // Threshold-only
  llm_threshold_min: 0,
  llm_threshold_max: 0,
};

// Balanced mode (recommended)
const BALANCED_CONFIG: DecisionConfig = {
  use_property_analysis: true,
  llm_threshold_min: 0.50,       // Skip LLM if too low
  llm_threshold_max: 0.95,       // Skip LLM if too high
};

// Accurate mode (slower)
const ACCURATE_CONFIG: DecisionConfig = {
  use_property_analysis: true,
  llm_threshold_min: 0.30,       // Use LLM more aggressively
  llm_threshold_max: 0.98,
};

Implementation Plan

Step 1: Types and Schemas (Day 5, Morning)

Tasks:

Define PropertyChangeAnalysis type
Define SubjectRelationship enum
Define RelationshipType enum

Deliverable: lib/types/property-change.ts

Step 2: LLM Analysis (Day 5, Afternoon)

Tasks:

Implement property analysis prompt
Implement analyzePropertyChange()
Add few-shot examples

Deliverable: lib/consolidation/property-change-detector.ts

Step 3: Dynamic Weights (Day 6, Morning)

Tasks:

Implement weight adjustment logic
Integrate with similarity calculation
Test weight variations

Deliverable: lib/consolidation/dynamic-weights.ts

Step 4: Enhanced Decision Flow (Day 6, Morning)

Tasks:

Update DecisionTreeEngine to use property analysis
Implement new decision flow
Add confidence thresholds

Deliverable: Updated lib/consolidation/engine.ts

Step 5: Testing and Validation (Day 6, Afternoon)

Tasks:

Create 40 UPDATE/LINK boundary test cases
Run Experiment 3
Analyze confusion matrix
Tune thresholds and weights

Deliverable: Experiment run + analysis report

Success Criteria

Phase 3 Success Metrics

UPDATE/LINK distinction accuracy > 85%
UPDATE precision > 85%
LINK precision > 85%
Confusion rate < 15%
Latency < 2.5 seconds per decision (with LLM)
Cost < $0.002 per decision

Go/No-Go for Phase 4

Proceed if:

Accuracy > 80%
Confusion rate < 20%
Latency < 3 seconds

Block/Revise if:

Accuracy < 75%
Confusion rate > 25%
Latency > 5 seconds

Change Log

Date	Author	Change
2025-12-06	Claude	Initial UPDATE vs LINK distinction design

‹

›

06-consolidation-update-vs-link-distinction

UPDATE vs LINK Distinction

Problem Statement

Example Confusion

Current Problem

Design Goals

Primary Goals

Non-Goals

Core Concepts

UPDATE Conditions

LINK Conditions

Approach: Property-Level Change Detection

Core Idea

Change Analysis Schema

LLM-Based Property Analysis

Analysis Prompt

Dynamic Factor Weights

Core Idea

Enhanced Decision Flow

Current Flow (Threshold-Based)

New Flow (Property-Based)

Implementation

Test Cases: UPDATE vs LINK Boundary

Test Case Set (20 cases)

UPDATE Cases (10)

LINK Cases (10)

Few-Shot Examples for LLM

UPDATE Examples

LINK Examples

Validation Experiment

Experiment 3: UPDATE vs LINK Accuracy

Performance Considerations

Latency

Cost

Accuracy vs Speed Trade-off

Implementation Plan

Step 1: Types and Schemas (Day 5, Morning)

Step 2: LLM Analysis (Day 5, Afternoon)

Step 3: Dynamic Weights (Day 6, Morning)

Step 4: Enhanced Decision Flow (Day 6, Morning)

Step 5: Testing and Validation (Day 6, Afternoon)

Success Criteria

Phase 3 Success Metrics

Go/No-Go for Phase 4

Related Documentation

Change Log