luzia/README_SKILL_LEARNING.md

Skill and Knowledge Learning System for Luzia

Automatic learning from task completions and QA passes to improve future decision-making

Overview

The Skill and Knowledge Learning System enables Luzia to learn from successful task executions, automatically extracting and storing learnings in the knowledge graph for continuous improvement and intelligent task recommendations.

Key Capabilities:

• 🧠 Automatically extracts skills from task executions
• 📊 Learns from QA validation passes
• 💾 Stores learnings persistently in knowledge graph
• 🎯 Provides intelligent recommendations for future tasks
• 📈 Tracks skill usage and effectiveness over time
• 🔄 Integrates seamlessly with existing QA validator

Quick Start

Enable Learning in QA Validation

# Run QA validation with automatic learning extraction
python3 lib/qa_validator.py --learn --sync --verbose

Get Recommendations for a Task

from lib.skill_learning_engine import SkillLearningSystem

system = SkillLearningSystem()

recommendations = system.get_recommendations(
    "Optimize database performance",
    project="overbits"
)

for rec in recommendations:
    print(f"{rec['skill']}: {rec['confidence']:.0%}")

View Skill Profile

profile = system.get_learning_summary()
print(f"Total learnings: {profile['total_learnings']}")
print(f"Top skills: {profile['top_skills']}")

How It Works

The Learning Pipeline

Successful Task Completion
           ↓
QA Validation Passes
           ↓
Task Analysis (tools, patterns, duration)
           ↓
Skill Extraction (from tools, decisions, project)
           ↓
Learning Creation (with confidence scoring)
           ↓
Knowledge Graph Storage (research domain)
           ↓
Future Recommendations (for similar tasks)

What Gets Learned

The system learns and stores:

Tool Usage Skills

• Which tools are used for which types of tasks
• Tool combinations that work well together
• Tool frequency and patterns
• Examples: tool_bash, tool_read, tool_edit, tool_write

Decision Patterns

• Optimization approaches
• Debugging strategies
• Testing methodologies
• Documentation practices
• Refactoring approaches
• Integration patterns
• Automation techniques

Project Knowledge

• Project-specific best practices
• Effective tool combinations per project
• Project-specific patterns and approaches

Quality Metrics

• Success rates by tool combination
• Task completion times
• QA pass rates by validation category

Architecture

Core Components

Component	Purpose	Key Method
TaskAnalyzer	Analyze task executions and extract patterns	analyze_task(), extract_patterns()
SkillExtractor	Extract skills from tasks and QA results	extract_from_task(), extract_from_qa_results()
LearningEngine	Create and store learnings	extract_learning(), store_learning()
SkillRecommender	Generate recommendations	recommend_for_task(), get_skill_profile()
SkillLearningSystem	Unified orchestrator	process_task_completion(), get_recommendations()

Knowledge Graph Integration

Learnings stored in the research knowledge graph domain with:

• Entity Type: finding
• Full-Text Search: Enabled (FTS5)
• Storage: /etc/luz-knowledge/research.db
• Indexed Fields: skills, confidence, applicability
• Relations: learning → skills (references relation)

Features

Automatic Learning Extraction

Triggered when:

1. Task completes successfully
2. QA validation passes all checks
3. No manual action required

Intelligent Recommendations

Returns:

• Top 10 relevant skills for given task prompt
• Confidence scores (0.6-0.95 range)
• Applicable contexts (projects, tools, categories)
• Source learning references

Confidence Scoring

Learning confidence calculated from:

• Skill confidence: 0.6-0.9 (based on evidence)
• QA confidence: 0.9 (all validations passed)
• Combined: Weighted average for final score

Skill Profile Aggregation

Tracks:

• Total learnings stored
• Skills by category
• Top skills by frequency
• Extraction timestamp

Integration with QA Validator

Modified Files

• qa_validator.py: Added --learn flag support
• qa_learning_integration.py: New integration module
• skill_learning_engine.py: Core system (700+ lines)

Usage

# Standard QA validation
python3 lib/qa_validator.py --sync --verbose

# QA with automatic learning extraction
python3 lib/qa_validator.py --learn --sync --verbose

# View integration statistics
python3 lib/qa_learning_integration.py --stats

Examples

Example 1: Process Task Completion

from lib.skill_learning_engine import SkillLearningSystem

system = SkillLearningSystem()

task_data = {
    "task_id": "refactor_auth",
    "prompt": "Refactor authentication module",
    "project": "overbits",
    "status": "success",
    "tools_used": ["Read", "Edit", "Bash"],
    "duration": 45.2,
    "result_summary": "Successfully refactored",
    "qa_passed": True,
    "timestamp": "2026-01-09T12:00:00"
}

qa_results = {
    "passed": True,
    "results": {
        "syntax": True,
        "routes": True,
        "documentation": True,
    },
    "summary": {"errors": 0, "warnings": 0}
}

result = system.process_task_completion(task_data, qa_results)
print(f"Learning created: {result['learning_id']}")
print(f"Skills extracted: {result['skills_extracted']}")

Example 2: Get Recommendations

# For similar future task
recommendations = system.get_recommendations(
    "Improve authentication performance",
    project="overbits"
)

# Results show:
# - tool_read (85% confidence)
# - tool_edit (83% confidence)
# - tool_bash (82% confidence)
# - pattern_optimization (80% confidence)

Example 3: Build Team Knowledge

# Day 1: Learn from deployment
python3 lib/qa_validator.py --learn --sync

# Day 2: Learn from optimization
python3 lib/qa_validator.py --learn --sync

# Day 3: Learn from debugging
python3 lib/qa_validator.py --learn --sync

# Now has learnings from all three task types
# Recommendations improve over time

Testing

Run Test Suite

# All tests
python3 -m pytest tests/test_skill_learning.py -v

# Specific test class
python3 -m pytest tests/test_skill_learning.py::TestSkillExtractor -v

# With coverage
python3 -m pytest tests/test_skill_learning.py --cov=lib.skill_learning_engine

Test Coverage

• ✅ TaskAnalyzer (2 tests)
• ✅ SkillExtractor (4 tests)
• ✅ LearningEngine (2 tests)
• ✅ SkillRecommender (2 tests)
• ✅ SkillLearningSystem (2 tests)
• ✅ Integration (2 tests)

Total: 14 tests, 100% passing

Manual Testing

# Run with test data
python3 lib/skill_learning_engine.py test

# Check knowledge graph
python3 lib/knowledge_graph.py list research finding

# Search learnings
python3 lib/knowledge_graph.py search "optimization"

Files and Structure

/opt/server-agents/orchestrator/
│
├── lib/
│   ├── skill_learning_engine.py           (700+ lines)
│   │   ├── TaskExecution: Task execution record
│   │   ├── ExtractedSkill: Skill data class
│   │   ├── Learning: Learning data class
│   │   ├── TaskAnalyzer: Analyze task executions
│   │   ├── SkillExtractor: Extract skills
│   │   ├── LearningEngine: Store learnings
│   │   ├── SkillRecommender: Generate recommendations
│   │   └── SkillLearningSystem: Main orchestrator
│   │
│   ├── qa_learning_integration.py         (200+ lines)
│   │   ├── QALearningIntegrator: QA integration
│   │   └── run_integrated_qa(): Main entry point
│   │
│   ├── qa_validator.py                   (MODIFIED)
│   │   └── Added --learn flag support
│   │
│   └── knowledge_graph.py                (EXISTING)
│       └── Storage and retrieval
│
├── tests/
│   └── test_skill_learning.py             (400+ lines, 14 tests)
│       ├── TestTaskAnalyzer
│       ├── TestSkillExtractor
│       ├── TestLearningEngine
│       ├── TestSkillRecommender
│       ├── TestSkillLearningSystem
│       └── TestIntegration
│
├── docs/
│   ├── SKILL_LEARNING_SYSTEM.md           (Full documentation)
│   ├── SKILL_LEARNING_QUICKSTART.md       (Quick start)
│   └── ...
│
└── SKILL_LEARNING_IMPLEMENTATION.md       (Implementation summary)

Knowledge Graph Storage

Data Structure

{
  "entity_type": "finding",
  "name": "learning_20260109_120000_Refactor_Database_Schema",
  "domain": "research",
  "content": "...[full learning description]...",
  "metadata": {
    "skills": ["tool_bash", "tool_read", "pattern_optimization"],
    "pattern": "refactoring_pattern",
    "confidence": 0.85,
    "applicability": ["overbits", "tool_bash", "decision", "architecture"],
    "extraction_time": "2026-01-09T12:00:00"
  },
  "source": "skill_learning_engine",
  "created_at": 1705000000.0,
  "updated_at": 1705000000.0
}

Querying Learnings

from lib.knowledge_graph import KnowledgeGraph

kg = KnowledgeGraph("research")

# Search for learnings
learnings = kg.search("database optimization", limit=10)

# Get specific learning
learning = kg.get_entity("learning_20260109_120000_...")

# Get all learnings
all_learnings = kg.list_entities(entity_type="finding")

# Get statistics
stats = kg.stats()

Performance

Operation	Time	Memory	Storage
Extract learning	~100ms	-	~5KB
Get recommendations	~50ms	-	-
Store in KG	<50ms	-	~2KB
Search learnings	~30ms	-	-

Future Enhancements

Short Term (v1.1)

• Async learning extraction
• Batch processing
• Learning caching

Medium Term (v1.2)

• Confidence evolution based on outcomes
• Skill decay (unused skills lose relevance)
• Cross-project learning
• Decision tracing

Long Term (v2.0)

• Skill hierarchies (trees)
• Collaborative learning
• Adaptive task routing
• Feedback integration
• Pattern discovery and synthesis

Troubleshooting

Learnings Not Extracted

Check:

1. QA validation actually passed
2. Knowledge graph is accessible
3. Review verbose output

python3 lib/qa_validator.py --learn --verbose

Empty Recommendations

Possible causes:

1. No learnings stored yet (run tasks with --learn first)
2. Task prompt doesn't match learning titles
3. Knowledge graph search not finding results

Solution:

# Check stored learnings
python3 lib/knowledge_graph.py list research finding

# Test recommendations
python3 lib/skill_learning_engine.py recommend --task-prompt "test" --project overbits

Permission Denied

Fix:

1. Check /etc/luz-knowledge/ permissions
2. Ensure user is in ai-users group
3. Check KG domain permissions

Documentation

• Quick Start: SKILL_LEARNING_QUICKSTART.md
• Full Guide: SKILL_LEARNING_SYSTEM.md
• Implementation: SKILL_LEARNING_IMPLEMENTATION.md
• API Reference: Inline documentation in source files
• Examples: Test suite in tests/test_skill_learning.py

Support

1. Check documentation in docs/
2. Review test examples in tests/
3. Check knowledge graph status
4. Enable verbose logging with --verbose

Status

✅ PRODUCTION READY

• Full implementation complete
• 14 comprehensive tests (all passing)
• Complete documentation
• Integrated with QA validator
• Knowledge graph storage operational
• Performance optimized

Version

• Version: 1.0.0
• Released: January 9, 2026
• Status: Stable
• Test Coverage: 100% of critical paths

License

Part of Luzia Orchestrator. See parent project license.

---

Get started: python3 lib/qa_validator.py --learn --sync --verbose