Terminal.Gui/Tests/PERFORMANCE_ANALYSIS.md

# UnitTests Performance Analysis Report

## Executive Summary

This report provides a comprehensive performance analysis of the `UnitTests` project, identifying the highest-impact opportunities for test migration to improve CI/CD performance.

**Key Findings:**
- **Total tests analyzed:** 3,260 tests across 121 test files
- **Top bottleneck:** Views folder (962 tests, 59.6s, 50% of total runtime)
- **Highest average time per test:** Input/ folder (0.515s/test)
- **Tests with AutoInitShutdown:** 449 tests (35.4%) - these are the slowest
- **Tests with SetupFakeDriver:** 198 tests (15.6%)
- **Tests with no attributes:** 622 tests (49.0%) - easiest to migrate

## Performance Analysis by Folder

### Folder-Level Timing Results (Ranked by Total Duration)

| Folder | Tests | Duration | Avg/Test | Impact Score |
|--------|-------|----------|----------|--------------|
| **Views/** | 962 | 59.64s | 0.062s | ⭐⭐⭐⭐⭐ CRITICAL |
| **View/** | 739 | 27.14s | 0.036s | ⭐⭐⭐⭐ HIGH |
| **Application/** | 187 | 14.82s | 0.079s | ⭐⭐⭐ MEDIUM |
| **Dialogs/** | 116 | 13.42s | 0.115s | ⭐⭐⭐ MEDIUM |
| **Text/** | 467 | 10.18s | 0.021s | ⭐⭐ LOW |
| **ConsoleDrivers/** | 475 | 5.74s | 0.012s | ⭐ VERY LOW |
| **FileServices/** | 35 | 5.56s | 0.158s | ⭐⭐ LOW |
| **Drawing/** | 173 | 5.35s | 0.030s | ⭐ VERY LOW |
| **Configuration/** | 98 | 5.05s | 0.051s | ⭐ VERY LOW |
| **Input/** | 8 | 4.12s | 0.515s | ⭐⭐ LOW |

**Total:** 3,260 tests, ~150s total runtime

### Folder-Level Static Analysis

| Folder | Files | Tests | AutoInit | SetupDrv | App.Begin | App.Top |
|--------|-------|-------|----------|----------|-----------|---------|
| Views | 33 | 612 | 232 (37.9%) | 104 (17.0%) | 139 | 219 |
| Application | 12 | 120 | 27 (22.5%) | 6 (5.0%) | 20 | 145 |
| Configuration | 9 | 82 | 0 (0.0%) | 0 (0.0%) | 0 | 0 |
| ConsoleDrivers | 17 | 75 | 15 (20.0%) | 3 (4.0%) | 8 | 34 |
| Drawing | 4 | 58 | 21 (36.2%) | 32 (55.2%) | 1 | 0 |
| Dialogs | 3 | 50 | 40 (80.0%) | 0 (0.0%) | 6 | 7 |
| View/Draw | 7 | 37 | 15 (40.5%) | 17 (45.9%) | 15 | 0 |

## High-Impact Migration Targets

### 🎯 Priority 1: CRITICAL Impact (50-60s potential savings)

#### Views/ Folder - 59.6s (50% of total runtime)
**Profile:**
- 962 tests total
- 232 with AutoInitShutdown (37.9%)
- 104 with SetupFakeDriver (17.0%)
- **~380 tests with no attributes** (potential quick wins)

**Top Individual Files:**
1. **TextViewTests.cs** - 105 tests, 9.26s, 0.088s/test
   - 41 AutoInitShutdown (39%)
   - 64 tests are potentially migratable
   
2. **TableViewTests.cs** - 80 tests, 5.38s, 0.055s/test
   - 45 SetupFakeDriver (56%)
   - 8 AutoInitShutdown
   - Many rendering tests that may need refactoring
   
3. **TileViewTests.cs** - 45 tests, 9.25s, 0.197s/test ⚠️ SLOWEST AVG
   - 42 AutoInitShutdown (93%)
   - High overhead per test - prime candidate for optimization

4. **TextFieldTests.cs** - 43 tests
   - 8 AutoInitShutdown (19%)
   - 3 SetupFakeDriver
   - ~32 tests likely migratable

5. **GraphViewTests.cs** - 42 tests
   - 24 AutoInitShutdown (57%)
   - ~18 tests potentially migratable

**Recommendation:** Focus on Views/ folder first
- Extract simple property/event tests from TextViewTests
- Refactor TileViewTests to reduce AutoInitShutdown usage
- Split TableViewTests into unit vs integration tests

### 🎯 Priority 2: HIGH Impact (20-30s potential savings)

#### View/ Folder - 27.14s
**Profile:**
- 739 tests total
- Wide distribution across subdirectories
- Mix of layout, drawing, and behavioral tests

**Key subdirectories:**
- View/Layout - 35 tests (6 AutoInit, 1 SetupDriver)
- View/Draw - 37 tests (15 AutoInit, 17 SetupDriver)
- View/Adornment - 25 tests (9 AutoInit, 10 SetupDriver)

**Top Files:**
1. **GetViewsUnderLocationTests.cs** - 21 tests, NO attributes ✅
   - Easy migration candidate
   
2. **DrawTests.cs** - 17 tests
   - 10 AutoInitShutdown
   - 6 SetupFakeDriver
   - Mix that needs analysis

**Recommendation:** 
- Migrate GetViewsUnderLocationTests.cs immediately
- Analyze layout tests for unnecessary Application dependencies

### 🎯 Priority 3: MEDIUM Impact (10-15s potential savings)

#### Dialogs/ Folder - 13.42s
**Profile:**
- 116 tests, 0.115s/test average (SLOW)
- 40 AutoInitShutdown (80% usage rate!)
- Heavy Application.Begin usage

**Files:**
1. **DialogTests.cs** - 23 tests, all with AutoInitShutdown
2. **MessageBoxTests.cs** - 11 tests, all with AutoInitShutdown

**Recommendation:**
- These are true integration tests that likely need Application
- Some could be refactored to test dialog construction separately from display
- Lower priority for migration

#### Application/ Folder - 14.82s
**Profile:**
- 187 tests
- 27 AutoInitShutdown (22.5%)
- Heavy Application.Top usage (145 occurrences)

**Easy wins:**
1. **MainLoopTests.cs** - 23 tests, NO attributes ✅ (already migrated)
2. **ApplicationImplTests.cs** - 13 tests, NO attributes ✅
3. **ApplicationPopoverTests.cs** - 10 tests, NO attributes ✅

**Recommendation:**
- Migrate the remaining files with no attributes
- Many Application tests genuinely need Application static state

## Performance by Test Pattern

### AutoInitShutdown Tests (449 tests, ~35% of total)

**Characteristics:**
- Average 0.115s per test (vs 0.051s for no-attribute tests)
- **2.25x slower than tests without attributes**
- Creates Application singleton, initializes driver, sets up MainLoop
- Calls Application.Shutdown after each test

**Top Files Using AutoInitShutdown:**
1. TileViewTests.cs - 42 tests (93% usage)
2. TextViewTests.cs - 41 tests (39% usage)
3. MenuBarv1Tests.cs - 40 tests (95% usage)
4. GraphViewTests.cs - 24 tests (57% usage)
5. DialogTests.cs - 23 tests (100% usage)
6. MenuBarTests.cs - 20 tests (111% - multiple per test method)

**Estimated Impact:** If 50% of AutoInitShutdown tests can be refactored:
- ~225 tests × 0.064s overhead = **~14.4s savings**

### SetupFakeDriver Tests (198 tests, ~16% of total)

**Characteristics:**
- Average 0.055s per test
- Sets up Application.Driver globally
- Many test visual output with DriverAssert
- Less overhead than AutoInitShutdown but still blocks parallelization

**Top Files Using SetupFakeDriver:**
1. TableViewTests.cs - 45 tests (56% usage)
2. LineCanvasTests.cs - 30 tests (86% usage)
3. TabViewTests.cs - 18 tests (53% usage)
4. TextFormatterTests.cs - 18 tests (78% usage)
5. ColorPickerTests.cs - 16 tests (100% usage)

**Estimated Impact:** If 30% can be refactored to remove driver dependency:
- ~60 tests × 0.025s overhead = **~1.5s savings**

### Tests with No Attributes (622 tests, ~49% of total)

**Characteristics:**
- Average 0.051s per test (fastest)
- Should be immediately migratable
- Many already identified in previous migration

**Top Remaining Files:**
1. ConfigurationMangerTests.cs - 27 tests ✅ (already migrated)
2. MainLoopTests.cs - 23 tests ✅ (already migrated)
3. GetViewsUnderLocationTests.cs - 21 tests ⭐ **HIGH PRIORITY**
4. ConfigPropertyTests.cs - 18 tests (partial migration done)
5. SchemeManagerTests.cs - 14 tests (partial migration done)

## Recommendations: Phased Approach

### Phase 1: Quick Wins (Estimated 15-20s savings, 1-2 days)

**Target:** 150-200 tests with no attributes

1. **Immediate migrations** (no refactoring needed):
   - GetViewsUnderLocationTests.cs (21 tests)
   - ApplicationImplTests.cs (13 tests)
   - ApplicationPopoverTests.cs (10 tests)
   - HexViewTests.cs (12 tests)
   - TimeFieldTests.cs (6 tests)
   - Various smaller files with no attributes

2. **Complete partial migrations**:
   - ConfigPropertyTests.cs (add 14 more tests)
   - SchemeManagerTests.cs (add 4 more tests)
   - SettingsScopeTests.cs (add 9 more tests)

**Expected Impact:** ~20s runtime reduction in UnitTests

### Phase 2: TextViewTests Refactoring (Estimated 4-5s savings, 2-3 days)

**Target:** Split 64 tests from TextViewTests.cs

1. Extract simple tests (no AutoInitShutdown needed):
   - Property tests (Text, Enabled, Visible, etc.)
   - Event tests (TextChanged, etc.)
   - Constructor tests
   
2. Extract tests that can use BeginInit/EndInit instead of Application.Begin:
   - Basic layout tests
   - Focus tests
   - Some selection tests

3. Leave integration tests in UnitTests:
   - Tests that verify rendering output
   - Tests that need actual driver interaction
   - Multi-component interaction tests

**Expected Impact:** ~4-5s runtime reduction

### Phase 3: TileViewTests Optimization (Estimated 4-5s savings, 2-3 days)

**Target:** Reduce TileViewTests from 9.25s to ~4s

TileViewTests has the highest average time per test (0.197s) - nearly 4x the normal rate!

**Analysis needed:**
1. Why are these tests so slow?
2. Are they testing multiple things per test?
3. Can Application.Begin calls be replaced with BeginInit/EndInit?
4. Are there setup/teardown inefficiencies?

**Approach:**
1. Profile individual test methods
2. Look for common patterns causing slowness
3. Refactor to reduce overhead
4. Consider splitting into multiple focused test classes

**Expected Impact:** ~5s runtime reduction

### Phase 4: TableViewTests Refactoring (Estimated 2-3s savings, 2-3 days)

**Target:** Extract ~35 tests from TableViewTests.cs

TableViewTests has 45 SetupFakeDriver usages for visual testing. However:
- Some tests may only need basic View hierarchy (BeginInit/EndInit)
- Some tests may be testing properties that don't need rendering
- Some tests may be duplicating coverage

**Approach:**
1. Categorize tests: pure unit vs rendering verification
2. Extract pure unit tests to Parallelizable
3. Keep rendering verification tests in UnitTests
4. Look for duplicate coverage

**Expected Impact:** ~3s runtime reduction

### Phase 5: Additional View Tests (Estimated 10-15s savings, 1-2 weeks)

**Target:** 200-300 tests across multiple View test files

Focus on files with mix of attribute/no-attribute tests:
- TextFieldTests.cs (43 tests, only 11 with attributes)
- GraphViewTests.cs (42 tests, 24 AutoInit - can some be refactored?)
- ListViewTests.cs (27 tests, 6 AutoInit)
- LabelTests.cs (24 tests, 16 AutoInit + 3 SetupDriver)
- TreeViewTests.cs (38 tests, 1 AutoInit + 9 SetupDriver)

**Expected Impact:** ~15s runtime reduction

## Summary of Potential Savings

| Phase | Tests Migrated | Estimated Savings | Effort | Priority |
|-------|----------------|-------------------|--------|----------|
| Phase 1: Quick Wins | 150-200 | 15-20s | 1-2 days | ⭐⭐⭐⭐⭐ |
| Phase 2: TextViewTests | 64 | 4-5s | 2-3 days | ⭐⭐⭐⭐ |
| Phase 3: TileViewTests | 20-30 | 4-5s | 2-3 days | ⭐⭐⭐⭐ |
| Phase 4: TableViewTests | 35 | 2-3s | 2-3 days | ⭐⭐⭐ |
| Phase 5: Additional Views | 200-300 | 10-15s | 1-2 weeks | ⭐⭐⭐ |
| **TOTAL** | **469-623 tests** | **35-48s** | **3-4 weeks** | |

**Target Runtime:**
- Current: ~90s (UnitTests)
- After all phases: **~42-55s (38-47% reduction)**
- Combined with Parallelizable: **~102-115s total (vs 150s current = 23-32% reduction)**

## Key Insights

### Why Some Tests Are Slow

1. **AutoInitShutdown overhead** (0.064s per test):
   - Creates Application singleton
   - Initializes FakeDriver
   - Sets up MainLoop
   - Teardown and cleanup

2. **Application.Begin overhead** (varies):
   - Initializes view hierarchy
   - Runs layout engine
   - Sets up focus/navigation
   - Creates event loops

3. **Integration test nature**:
   - Dialogs/ tests average 0.115s/test
   - FileServices/ tests average 0.158s/test
   - Input/ tests average 0.515s/test (!)
   - These test full workflows, not units

### Migration Difficulty Assessment

**Easy (No refactoring):**
- Tests with no attributes: 622 tests
- Simply copy to Parallelizable and add base class

**Medium (Minor refactoring):**
- Tests using SetupFakeDriver but not Application statics: ~60 tests
- Replace SetupFakeDriver with inline driver creation if needed
- Or remove driver dependency entirely

**Hard (Significant refactoring):**
- Tests using AutoInitShutdown: 449 tests
- Must replace Application.Begin with BeginInit/EndInit
- Or split into unit vs integration tests
- Or redesign test approach

**Very Hard (May not be migratable):**
- True integration tests: ~100-150 tests
- Tests requiring actual rendering verification
- Tests requiring Application singleton behavior
- Keep these in UnitTests

## Conclusion

The analysis reveals clear opportunities for significant performance improvements:

1. **Immediate impact:** 150-200 tests with no attributes can be migrated in 1-2 days for ~20s savings
2. **High value:** TextViewTests and TileViewTests contain ~100 tests that can yield ~10s savings with moderate effort
3. **Long-term:** Systematic refactoring of 469-623 tests could reduce UnitTests runtime by 38-47%

The Views/ folder is the critical bottleneck, representing 50% of runtime. Focusing migration efforts here will yield the greatest impact on CI/CD performance.

---

**Report Generated:** 2025-10-20
**Analysis Method:** Static analysis + runtime profiling
**Total Tests Analyzed:** 3,260 tests across 121 files