luzia/STRUCTURAL-ANALYSIS.md

# Structural Analysis Tool

Structural analysis capabilities for scanning project code structures, generating analysis reports, and saving structure data to the shared knowledge graph for cross-project learning.

## Overview

The Structural Analysis Tool provides:

- **AST-based Code Analysis**: Python Abstract Syntax Tree parsing for precise code structure analysis
- **Comprehensive Metrics**: Lines of code, complexity, functions, classes, imports, and more
- **Complexity Assessment**: Cyclomatic complexity calculation and function-level analysis
- **Code Quality Metrics**: Comment ratio, code distribution, blank line analysis
- **Hotspot Identification**: Automatically identifies complex modules requiring refactoring
- **Pattern Detection**: Detects design patterns like context managers, dataclasses, etc.
- **Knowledge Graph Integration**: Saves analysis results to shared knowledge graph for cross-project learning
- **JSON-based Reports**: Machine-readable analysis reports for integration with other tools
- **Actionable Recommendations**: Generated improvement recommendations based on analysis

## Installation

The tool is built into the Luzia orchestrator. No additional installation required.

### Dependencies

- Python 3.8+
- `ast` module (built-in)
- `pathlib` module (built-in)
- Optional: Knowledge graph support requires `/opt/server-agents/orchestrator/lib/knowledge_graph.py`

## Usage

### Via Luzia CLI

#### Analyze Current Orchestrator

```bash
luzia structure
```

#### Analyze a Specific Project

```bash
luzia structure <project_name>
```

Example:
```bash
luzia structure musica
luzia structure overbits
```

#### Analyze Specific Directory

```bash
luzia structure . path/to/src
```

#### Output Options

**JSON Output** (for programmatic use):
```bash
luzia structure --json
```

**Suppress Knowledge Graph Save** (quick analysis only):
```bash
luzia structure --no-kg
```

**Combine Options**:
```bash
luzia structure musica --json --no-kg
```

### Direct CLI

```bash
python3 lib/structural_analysis.py /path/to/project [--name project_name] [--json] [--no-kg]
```

Examples:
```bash
python3 lib/structural_analysis.py . --name orchestrator
python3 lib/structural_analysis.py /home/musica --name musica --json
python3 lib/structural_analysis.py /home/overbits/numerover --json --no-kg
```

## Output

### Console Output

The tool prints a human-readable summary:

```
============================================================
Structural Analysis Report: orchestrator
============================================================

Code Metrics:
  Total Lines: 4044
  Code Lines: 3115
  Comment Lines: 206
  Functions: 149
  Classes: 16

Complexity Assessment: low
  Average Cyclomatic Complexity: 0.0

Code Quality:
  Code Ratio: 77.03%
  Comment Ratio: 6.61%
  Assessment: Needs more documentation

Top Hotspots (Complex Modules):
  1. daemon.py
     Avg Complexity: 12.1
  2. orchestrator.py
     Avg Complexity: 8.5

Recommendations:
  • Increase code documentation - aim for 10%+ comment ratio
  • Focus refactoring on 2 high-complexity modules

============================================================
```

### JSON Report

Generated report saved to: `structure-analysis-YYYYMMDD-HHMMSS.json`

Contains:
```json
{
  "project": "orchestrator",
  "path": "/opt/server-agents/orchestrator",
  "timestamp": "2026-01-09T00:34:01.014546",
  "analysis": {
    "directory": ".",
    "file_count": 10,
    "files": { ... },
    "summary": {
      "total_lines": 4044,
      "code_lines": 3115,
      "comment_lines": 206,
      "blank_lines": 723,
      "functions": 149,
      "classes": 16,
      "imports": 87,
      "cyclomatic_complexity": 623
    }
  },
  "dependency_graph": { ... },
  "patterns": { ... },
  "insights": {
    "complexity_assessment": { ... },
    "code_quality_metrics": { ... },
    "hotspots": [ ... ],
    "recommendations": [ ... ]
  }
}
```

### Knowledge Graph Storage

Analysis results are saved to the shared knowledge graph (`/etc/luz-knowledge/projects.db`) as:

- **Entity**: `{project_name}-structure-analysis` (type: `architecture`)
  - Contains project metrics and timestamp
  - Includes all insights and recommendations as observations

- **Components**: One entity per function/class (type: `component`)
  - `{project_name}-ClassName`
  - `{project_name}-function_name`
  - Related to main analysis with `contains` relation

Example query:
```bash
luzia docs "orchestrator-structure-analysis"
luzia docs --show orchestrator-structure-analysis
```

## Metrics Explained

### Code Metrics

| Metric | Description |
|--------|-------------|
| Total Lines | All lines in files (code + comments + blank) |
| Code Lines | Lines containing actual code |
| Comment Lines | Lines starting with `#` |
| Blank Lines | Empty lines |
| Functions | Total function/method count |
| Classes | Total class count |
| Imports | Total import statements |
| Cyclomatic Complexity | Sum of all function complexities |

### Complexity Assessment

**Cyclomatic Complexity** measures decision paths in code:

- 1: Linear function, no branches
- 2-5: Simple, typical function
- 5-10: Moderate complexity
- 10+: High complexity, refactor recommended

**Assessment Levels**:
- **Low**: Avg complexity < 5 (healthy)
- **Moderate**: Avg complexity 5-10 (needs attention)
- **High**: Avg complexity > 10 (refactor urgently)

### Code Quality Metrics

| Metric | Ideal Value | Assessment |
|--------|-------------|------------|
| Comment Ratio | 10%+ | Good documentation |
| Code Ratio | 75%+ | Balanced structure |
| Blank Ratio | < 25% | Reasonable spacing |

## Hotspots

Identifies complex modules (functions with avg complexity > 8):

```json
"hotspots": [
  {
    "file": "/opt/server-agents/orchestrator/daemon.py",
    "complexity": 24,
    "functions": 2,
    "avg_complexity_per_function": 12.0
  }
]
```

Action: Review and refactor high-complexity functions in these files.

## Recommendations

Auto-generated based on analysis:

1. **Complexity**: If avg complexity > 10, recommend refactoring
2. **Documentation**: If comment ratio < 10%, recommend more comments
3. **Hotspots**: If complex modules found, focus refactoring effort there

## Design Patterns Detected

The tool identifies:

- **Context Managers**: Classes with `__enter__` and `__exit__` methods
- **Dataclasses**: Classes decorated with `@dataclass`

Future enhancements:
- Singleton pattern (class-level instance checks)
- Factory pattern (static creation methods)
- Observer pattern (listener registration)
- Decorator pattern (wrapper functions)

## Advanced Usage

### Analyze Multiple Projects

```bash
# Orchestrator
luzia structure

# All projects
for project in musica overbits dss; do
  luzia structure $project
done
```

### Compare Reports

```bash
# Generate baseline
luzia structure orchestrator --no-kg > baseline.json

# Later analysis
luzia structure orchestrator --no-kg > current.json

# Compare
diff baseline.json current.json
```

### Extract Specific Metrics

```bash
# Get complexity scores only
luzia structure --json | jq '.insights.complexity_assessment'

# Get hotspots
luzia structure --json | jq '.insights.hotspots'

# Get recommendations
luzia structure --json | jq '.insights.recommendations'
```

## Knowledge Graph Queries

### Search Knowledge Graph

```bash
# Find all structural analyses
luzia docs "structure-analysis"

# Find specific project
luzia docs "musica-structure-analysis"

# Show details
luzia docs --show orchestrator-structure-analysis
```

### List Components

Components from analyzed projects are stored as entities. They can be searched:

```bash
# Find functions
luzia docs "orchestrator-route_" | grep function

# Find classes
luzia docs "orchestrator-Router"
```

## Troubleshooting

### No Python Files Found

**Issue**: "No Python files found" error

**Solution**: Ensure project path contains `.py` files. Check the path is correct.

### Syntax Errors in Project

**Issue**: "Syntax error" in file analysis

**Solution**: The tool reports syntax errors but continues. Check the specific file for Python syntax issues.

### Knowledge Graph Not Available

**Issue**: Can't save to knowledge graph (warning message)

**Solution**: Use `--no-kg` flag to skip knowledge graph save. This is normal if running without admin access.

### Very High Complexity Scores

**Issue**: Cyclomatic complexity seems too high

**Solution**: Large functions with multiple branches (if/else, loops, exception handlers) are correctly identified as complex. This is intentional - it's a sign that the function should be refactored.

## Development Notes

### Adding Custom Metrics

Edit `CodeMetrics` dataclass to add new metrics. Add calculation in `ASTAnalyzer.visit_*` methods.

### Custom Pattern Detection

Extend `_detect_class_patterns` and `_detect_patterns` methods to detect more design patterns.

### Customizing Reports

Modify `_generate_insights()` and `print_summary()` for custom report formats.

## Best Practices

1. **Run regularly**: Quarterly or after major changes
2. **Track trends**: Save baseline, compare over time
3. **Act on hotspots**: Address high-complexity modules proactively
4. **Document code**: Improve comment ratio to 10%+
5. **Refactor iteratively**: Address complexity gradually

## See Also

- `/opt/server-agents/orchestrator/lib/structural_analysis.py` - Full source code
- `luzia docs` - Knowledge graph search
- `luzia qa` - QA validation (related tool)

## Version

- Tool Version: 1.0.0
- Last Updated: 2026-01-09
- Python: 3.8+
- Dependencies: ast (built-in)

---

**Generated by**: Luzia Structural Analysis Tool
**Purpose**: Code intelligence for self-improving orchestration