# mb_clouseau Implementation Plan

**Uncovering clues with MusicBee Clouseau**

A comprehensive trace/debug/metrics utility for MusicBee plugin developers with deep introspection capabilities.

---

## Vision

Clouseau goes beyond simple event logging to become a **full diagnostic and introspection platform**:
- **System Metrics**: CPU, memory, disk, network, CLR/.NET internals
- **App Metrics**: MusicBee process health, handles, threads, GC pressure
- **Plugin Metrics**: All loaded plugins, their resource usage, API call interception
- **Deep Dumps**: On-demand state snapshots + continuous light sampling
- **Full Dashboard**: Tabbed UI with charts, graphs, flame graphs (future)

---

## Architecture Overview

```
mb_clouseau/plugin/
├── Plugin.cs                    # Entry point, lifecycle, event routing
├── MusicBeeInterface.cs         # API definitions (copy from TrueShuffle)
├── Core/
│   ├── ClouseauSettings.cs      # Settings manager (loads settings.json)
│   ├── LogManager.cs            # NLog initialization and routing
│   └── StateManager.cs          # Persistent state tracking
├── Logging/
│   ├── EventLogger.cs           # MusicBee event capture
│   ├── LogEntry.cs              # Structured log entry model
│   └── LogBuffer.cs             # Ring buffer for UI display
├── Metrics/
│   ├── MetricsCollector.cs      # Orchestrates all metric collection
│   ├── SystemMetrics.cs         # CPU, memory, disk, network
│   ├── ProcessMetrics.cs        # MusicBee + Clouseau process stats
│   ├── ClrMetrics.cs            # .NET GC, heap, JIT, threading
│   └── AudioMetrics.cs          # Audio device, buffer, latency
├── Introspection/
│   ├── PluginDiscovery.cs       # Find all loaded MusicBee plugins
│   ├── PluginMonitor.cs         # Track plugin resource usage
│   ├── ApiInterceptor.cs        # Hook MusicBee API calls
│   └── StateDumper.cs           # Deep state snapshots
└── UI/
    ├── LogViewerPanel.cs        # Dockable real-time log panel
    ├── SettingsDialog.cs        # Configuration UI
    └── DashboardForm.cs         # Full dashboard window (future)
```

---

## 4 Parallel Implementation Agents

### Agent 1: Core Infrastructure
**Files**: Plugin.cs, MusicBeeInterface.cs, Core/*.cs

**Responsibilities**:
1. Copy MusicBeeInterface.cs from TrueShuffle (API definitions)
2. Implement Plugin.cs with:
   - `Initialise()` - Return PluginInfo with ALL notification flags
   - `ReceiveNotification()` - Route all 40 event types
   - `Configure()` - Configuration panel
   - `OnDockablePanelCreated()` - Panel integration
   - `Close()` / `Uninstall()` - Cleanup
3. Create ClouseauSettings.cs:
   - Load/save settings.json with Newtonsoft.Json
   - Strong typing for all settings sections
   - Default value fallbacks
4. Create LogManager.cs:
   - Initialize NLog from NLog.config
   - Provide named loggers (Events, Metrics, Api, etc.)
5. Create StateManager.cs:
   - Track session state (start time, event counts)
   - Persist state across restarts

**Key Patterns from TrueShuffle**:
- Use `Setting_GetPersistentStoragePath()` for data directory
- Lazy initialization in PluginStartup event
- Menu registration: `MB_AddMenuItem("mnuTools/Clouseau...")`

---

### Agent 2: Event Logging System
**Files**: Logging/*.cs

**Responsibilities**:
1. Create LogEntry.cs - Structured log model:
```csharp
public class LogEntry
{
    public DateTime Timestamp { get; set; }
    public string Category { get; set; }      // Core, Player, Queue, etc.
    public NotificationType EventType { get; set; }
    public string SourceFile { get; set; }
    public PlayState? PlayState { get; set; }
    public Dictionary<string, object> Context { get; set; }
    public long SequenceNumber { get; set; }
}
```

2. Create EventLogger.cs:
   - Handle all 40 NotificationType events
   - Capture rich context for each event type:
     - TrackChanged: artist, title, album, duration, path
     - PlayStateChanged: old state, new state, position
     - VolumeChanged: old level, new level
     - RatingChanged: old rating, new rating, track
   - Write to NLog with structured properties
   - Fire events for UI updates

3. Create LogBuffer.cs:
   - Thread-safe ring buffer (configurable size, default 10000)
   - Support filtering by category/level
   - Support search/regex
   - Provide IEnumerable for UI binding

4. Event categorization mapping:
```csharp
Category Map:
  Core: PluginStartup, ShutdownStarted, MusicBeeStarted
  Player: TrackChanged, PlayStateChanged, Volume*, Shuffle, Repeat, AutoDj*
  Queue: PlayingTracksChanged, PlayingTracksQueueChanged, NowPlayingListEnded
  Library: FileAdded*, FileDeleted, LibrarySwitched
  Tags: TagsChanged, RatingChanged, PlayCountersChanged
  Playlist: PlaylistCreated, PlaylistUpdated, PlaylistDeleted
  Download: DownloadCompleted, SynchCompleted
  UI: EmbedInPanel, ApplicationWindowChanged
  System: ScreenSaverActivating
```

---

### Agent 3: Metrics Collection
**Files**: Metrics/*.cs

**Responsibilities**:
1. Create MetricsCollector.cs - Orchestrator:
   - Background timer (collection every 5s, log every 30s)
   - Aggregate metrics from all collectors
   - Write summary to metrics log
   - Expose current metrics for UI

2. Create SystemMetrics.cs:
   - Use `System.Diagnostics.PerformanceCounter`:
     - `Processor(_Total)\% Processor Time`
     - `Memory\Available MBytes`
     - `Memory\% Committed Bytes In Use`
     - `PhysicalDisk(_Total)\Disk Reads/sec`
     - `Network Interface(*)\Bytes Total/sec`
   - Use `System.Management` (WMI) for additional data
   - CPU per-core breakdown

3. Create ProcessMetrics.cs:
   - Track MusicBee.exe process:
     - Working set, private bytes, virtual bytes
     - Handle count, thread count
     - CPU time (user + kernel)
     - I/O read/write bytes
     - GC heap size (if accessible)
   - Track mb_clouseau.dll metrics:
     - Managed heap allocation
     - GC collections triggered

4. Create ClrMetrics.cs:
   - Use `System.GC` class:
     - `GC.CollectionCount(0/1/2)`
     - `GC.GetTotalMemory()`
     - `GC.GetGCMemoryInfo()` (.NET 5+ features if available)
   - Use CLR performance counters:
     - `.NET CLR Memory\# Gen 0/1/2 Collections`
     - `.NET CLR Memory\Gen 0/1/2 heap size`
     - `.NET CLR Memory\% Time in GC`
     - `.NET CLR Exceptions\# of Exceps Thrown / sec`
     - `.NET CLR LocksAndThreads\Contention Rate / sec`
     - `.NET CLR JIT\% Time in Jit`

5. Create AudioMetrics.cs:
   - Query audio device info via MusicBee API
   - Track `Player_GetOutputDevices()`
   - Monitor buffer underruns (if detectable)
   - Sample rate, bit depth, channel info

**Metrics Output Format**:
```
2026-01-01 14:32:01|METRICS|CPU=15%|MEM=4.2GB/16GB(26%)|MB_WS=245MB|MB_Handles=1847|GC_Gen0=142|GC_Gen1=23|GC_Gen2=3
```

---

### Agent 4: Plugin Introspection & UI
**Files**: Introspection/*.cs, UI/*.cs

**Responsibilities**:

#### Introspection:

1. Create PluginDiscovery.cs:
   - Scan MusicBee Plugins folder
   - Use reflection to inspect loaded assemblies
   - Identify plugin types implementing MusicBee interfaces
   - Extract version, author, description from each
   - Track load order and dependencies

2. Create PluginMonitor.cs:
   - For each discovered plugin:
     - Estimate memory footprint
     - Count threads created
     - Track panel/menu registrations
   - Store historical data for comparison

3. Create ApiInterceptor.cs (Advanced):
   - Hook MusicBee API delegate calls
   - Log which plugins call which APIs
   - Track call frequency and duration
   - Detect API call patterns and anomalies
   - **Implementation**: Wrap API delegates at initialization

4. Create StateDumper.cs:
   - On-demand deep state capture:
     - All loaded assemblies
     - Thread stack traces
     - Open handles
     - Memory regions
     - Current MusicBee state (playing track, queue, settings)
   - Save to timestamped JSON dump file
   - Triggered via menu item or hotkey

#### UI:

5. Create LogViewerPanel.cs (MVP Panel):
   - WinForms UserControl for docking
   - ListView or custom control for log display
   - Category filter dropdown
   - Level filter dropdown
   - Search/filter textbox
   - Auto-scroll toggle
   - Color-coded rows by category
   - Double-click to show details
   - Context menu: Copy, Export selection

6. Create SettingsDialog.cs:
   - Tabbed dialog:
     - General: Log level, enabled categories
     - Metrics: Collection interval, enabled metrics
     - Introspection: Plugin monitoring, API interception
     - Advanced: Buffer size, dump options
   - Apply/OK/Cancel buttons
   - Real-time preview of changes

7. Create DashboardForm.cs (Placeholder for future):
   - Separate modeless form
   - Tabbed interface:
     - Events: Full event log with filtering
     - Metrics: Charts and graphs
     - Plugins: Plugin list and stats
     - Dumps: State dump viewer
   - Menu item to open: "Tools > Clouseau Dashboard"

---

## Implementation Order

**Phase 1 - MVP (All 4 agents parallel)**:
1. Agent 1: Plugin.cs + MusicBeeInterface.cs + Settings
2. Agent 2: EventLogger.cs + LogEntry.cs + LogBuffer.cs
3. Agent 3: MetricsCollector.cs + SystemMetrics.cs + ProcessMetrics.cs
4. Agent 4: LogViewerPanel.cs + PluginDiscovery.cs

**Phase 2 - Enhanced**:
- ClrMetrics.cs, AudioMetrics.cs
- ApiInterceptor.cs, StateDumper.cs
- SettingsDialog.cs

**Phase 3 - Dashboard**:
- DashboardForm.cs with charts
- Export functionality
- Remote streaming (future)

---

## Critical Files Reference

**Patterns from TrueShuffle**:
- `C:\Users\scott\source\repos\MBX\ReferenceCode\mb_TrueShuffle\plugin\Plugin.cs` - Plugin lifecycle
- `C:\Users\scott\source\repos\MBX\ReferenceCode\mb_TrueShuffle\plugin\ShuffleSettings.cs` - Settings pattern
- `C:\Users\scott\source\repos\MBX\ReferenceCode\mb_TrueShuffle\plugin\MusicBeeInterface.cs` - API definitions

**Clouseau Config (already created)**:
- `C:\Users\scott\source\repos\MBX\ReferenceCode\mb_clouseau\plugin\NLog.config` - Logging config
- `C:\Users\scott\source\repos\MBX\ReferenceCode\mb_clouseau\plugin\settings.json` - Full settings
- `C:\Users\scott\source\repos\MBX\ReferenceCode\mb_clouseau\SPEC.md` - Technical spec

---

## Innovative Features

What makes Clouseau unique:

1. **Plugin Spy Mode**: See what other plugins are doing - their API calls, resource usage, errors
2. **API Call Interception**: Hook into MusicBee's API to log all plugin interactions
3. **Deep State Dumps**: Capture complete application state for debugging
4. **CLR Internals**: GC pressure, JIT stats, thread contention - data devs rarely see
5. **Historical Correlation**: Correlate events with metrics to find performance issues
6. **Anomaly Detection**: Alert when patterns deviate from baseline

---

## Success Criteria

MVP Complete when:
- [x] Plugin loads and receives all 40 notification types
- [x] Events logged to file with structured format
- [x] System/process metrics collected every 5 seconds
- [x] Dockable panel shows live event stream
- [x] Plugin discovery lists all loaded plugins
- [x] Build produces working DLL with dependencies

**Phase 2 Complete:**
- [x] ApiInterceptor hooks API calls with timing/anomaly detection
- [x] PluginMonitor tracks plugin registrations
- [x] SettingsDialog with full tabbed UI
- [x] DashboardForm with Overview, Events, Settings, Diagnostics, Plugins tabs
- [x] State dumps via StateDumper

**Phase 3 Complete (v0.2.0):**
- [x] Full context logging to disk (events.jsonl with ALL event data)
- [x] Rich event details dialog on double-click
- [x] Flyout menu under Tools > Clouseau (Dashboard, Settings, Live Events)
- [x] Panel header redesign with FlowLayoutPanel and Settings link
- [x] UI sizing improvements (Settings 900x750, Dashboard 1920x900, Live Log 1350x700)
- [x] ISO 8601 timestamps and sequence numbers in logs
- [x] Multiple log formats: events.log (pipe-delimited), events.jsonl (JSON Lines), clouseau.json (structured)

---

## Dependencies

Already configured in mb_clouseau.csproj:
- NLog 5.2.8 - Logging
- Newtonsoft.Json 13.0.3 - JSON
- System.Diagnostics.PerformanceCounter 8.0.0 - Metrics
- System.Management 8.0.0 - WMI

---

## Backlog / Future Features

### UI Gallery - Refine or Cut (Low Priority)

The UI Gallery feature in Phase 4 did not prove useful in practice. Screenshots were captured manually instead.

**Issues:**
- Auto-capture didn't produce usable results
- Manual screenshot workflow was faster/more reliable
- Feature adds complexity without clear value

**Decision needed:**
- **Refine**: Investigate why auto-capture failed, fix the workflow, make it actually useful for documentation/screenshot automation
- **Cut**: Remove the feature entirely to reduce maintenance burden

**If refining**, consider:
- What's a reusable pattern for UI capture that works across projects?
- Should it integrate with a documentation pipeline?
- Is there a simpler approach (e.g., just hotkey to capture current form)?

*Low priority - revisit when there's a clear use case.*

---

### Deep .NET Reflection Introspection (Phase 4)

Currently not fully utilizing .NET reflection capabilities. This should be added for true "app spy" functionality:

**.NET Reflection gives you:**

1. **All loaded assemblies** - Enumerate everything:
   - MusicBee.exe
   - MusicBeePlugin.dll
   - UI libraries
   - Layout engine assemblies
   - Third-party plugin DLLs

2. **All types inside those assemblies** - Even private ones:
   ```csharp
   foreach (var assembly in AppDomain.CurrentDomain.GetAssemblies())
   {
       foreach (var type in assembly.GetTypes())
       {
           // Discover internal classes, forms, managers
       }
   }
   ```

3. **All fields, properties, and methods** - Even private, internal, protected, static, anonymous, compiler-generated:
   ```csharp
   BindingFlags flags = BindingFlags.Public | BindingFlags.NonPublic |
                        BindingFlags.Instance | BindingFlags.Static;
   var methods = type.GetMethods(flags);
   var fields = type.GetFields(flags);
   ```

4. **Invoke private methods** - This would let you call internal methods like "open Arrange Panels dialog":
   ```csharp
   var method = type.GetMethod("OpenArrangePanels", flags);
   method?.Invoke(instance, null);
   ```

5. **Inspect the UI tree** - Walk the WinForms/WPF control hierarchy:
   - Find hidden controls
   - Inspect event handlers
   - See which methods are bound to which menu items
   - This is the true "app spy" capability

**Implementation Ideas:**

- `ReflectionExplorer.cs` - Browse loaded assemblies and types
- `MethodInvoker.cs` - Safely invoke discovered methods
- `UITreeWalker.cs` - Traverse control hierarchies
- `EventHandlerInspector.cs` - Discover what's wired to what
- Add "Reflection" tab to Dashboard showing assembly/type browser
- "Control Spy" overlay that highlights controls and shows their properties

**Use Cases:**
- Discover how MusicBee's panel system works internally
- Find hidden menu items or features
- Debug why a panel isn't docking correctly
- Understand the event flow through MusicBee's internals
- Invoke internal debug/diagnostic methods if they exist

---

### App Spy Diagnostic Overlay (Phase 4+)

Build a diagnostic overlay that reveals MusicBee's internals:

**1. Menu Item Discovery** - List all menu items + their bound methods:
```
Menu: View -> Arrange Panels
Handler: MusicBee.Layout.LayoutManager.ShowArrangePanelsDialog()
Parameters: (none)
Modifiers: public
```

**2. UI Control Tree** - Every panel, control, hidden element:
```
MainForm (MusicBee.MainForm)
├── MenuStrip (mainMenu)
├── ToolStrip (toolbar)
├── SplitContainer (mainSplit)
│   ├── Panel (leftPanel)
│   │   └── TreeView (libraryTree)
│   └── Panel (rightPanel)
│       ├── TabControl (mainTabs)
│       └── Panel (nowPlayingPanel) [HIDDEN]
└── StatusStrip (statusBar)
```

**3. Event Handler Inspector** - What's wired to what:
```
Button: btnArrangePanels
  Click -> LayoutManager.ShowArrangePanelsDialog
  MouseEnter -> TooltipHandler.Show
  KeyDown -> (none)

Form: MainForm
  Load -> MainForm_Load
  Closing -> MainForm_Closing
  Resize -> LayoutManager.OnResize
```

**4. Private State Reader** - Read internal fields:
```
MusicBee.Layout.LayoutManager:
  _currentLayout = "Default"
  _panelStates = Dictionary<string, PanelState>[12 items]
  _isDirty = false
  _lastArrangeTime = 2026-01-01 14:22:01
```

**5. Method Call Logger** - Hook and log when things happen:
```
14:22:01.123 LayoutManager.ShowArrangePanelsDialog() called
14:22:01.456 ArrangePanelsForm.ShowDialog() returned OK
14:22:01.789 LayoutManager.ApplyLayout("Custom1") called
14:22:02.012 MainForm.PerformLayout() called
```

**Why This Works:**
- .NET metadata is incredibly rich
- Every class, method, field, event stored in structured tables
- Reflection can read all of this at runtime
- Even private methods are fully described
- No decompilation needed - just runtime inspection

**Implementation Components:**
- `AppSpy/MenuInspector.cs` - Enumerate all menus and handlers
- `AppSpy/ControlTreeWalker.cs` - Recursive control enumeration
- `AppSpy/EventHandlerDumper.cs` - Extract event subscriptions
- `AppSpy/PrivateStateReader.cs` - Read private fields
- `AppSpy/MethodCallHooker.cs` - Intercept method invocations
- `AppSpy/SpyOverlayForm.cs` - Visual overlay with crosshairs

**SpyOverlay Features:**
- Hover over any control to see its type, name, properties
- Click to inspect event handlers
- Ctrl+click to show parent chain
- Right-click to invoke methods
- Export full control tree to JSON

---

### Implementation Steps for App Spy

**Step 1: Enumerate all assemblies**
```csharp
var assemblies = AppDomain.CurrentDomain.GetAssemblies();
// Filter for: MusicBee.exe, MusicBeePlugin.dll, UI/layout assemblies
```

**Step 2: Find the main form**
```csharp
var mainForm = Application.OpenForms
    .Cast<Form>()
    .FirstOrDefault(f => f.GetType().Name.Contains("MusicBee"));
```

**Step 3: Inspect private fields and controls**
```csharp
var fields = mainForm.GetType()
    .GetFields(BindingFlags.NonPublic | BindingFlags.Instance);
// Find: menu system, layout manager, panel config, dialog constructors
```

**Step 4: Find the Arrange Panels handler**
```csharp
// Enumerate all private methods, search for ones that:
// - return void
// - take no parameters
// - reference layout or panel configuration
var method = mainForm.GetType()
    .GetMethod("SomePrivateMethodName",
        BindingFlags.NonPublic | BindingFlags.Instance);
```

**Step 5: Invoke the private method**
```csharp
method.Invoke(mainForm, null);  // Bypass menu entirely
```

---

### Three Pillars of App Spy Architecture

**1. Assembly Explorer** - The foundation
- Tree view of all loaded assemblies
- Drill down: assemblies → namespaces → classes → methods/fields/properties
- Browse CLR metadata tables at runtime

**2. UI Tree Inspector** - WinForms control hierarchy
- Walk from MainForm down through all children
- Expose: control type, name, visibility, event handlers, private fields, layout properties
- Discover where dialogs are created and which objects own them

**3. Event & Method Spy** - The fun part
- Inspect event handlers on any control
- See private methods bound to handlers
- View declaring type and method signature
- Log when methods fire
- Invoke methods manually
- "Black box recorder" for MusicBee's UI

---

### End-to-End Lifecycle: Discovering → Inspecting → Invoking

**1. Start at the root: get the main form**
Your plugin is already inside MusicBee's AppDomain, so you can grab the main window directly. That becomes the root of your UI tree.
Once you have the main form, everything else is reachable.

**2. Walk the entire UI tree**
You recursively traverse:
- every child control
- every nested container
- every hidden control
- every owner-drawn custom control
This gives you a complete structural map of the UI.
In your inspector, this populates the Tree View.

**3. Select the View → Arrange Panels menu item**
In the tree, you click the node that corresponds to the menu item.
Your inspector now has a reference to the actual control instance that represents:
"Arrange Panels…"
This is the object whose event handlers you want to inspect.

**4. Extract event handlers from that control**
This is where the magic happens.
You read the private Events field, walk the internal event list, and extract:
- the delegate
- the target object
- the method
- the declaring type
Your inspector now shows something like:
```
Event: Click
  → Method: ShowArrangePanelsDialog
     Declaring Type: MusicBee.Layout.LayoutManager
     Visibility: private
     Parameters: (none)
     Target Instance: 0x12345678
```
This is the moment where you've found the internal entry point.

**5. Inspect the method**
Your Method Panel now displays:
- method name
- declaring type
- visibility
- parameters
- return type
You now understand exactly what the method is and how it behaves.

**6. Invoke the method**
Since you have:
- the MethodInfo
- the target instance
You can safely call:
```csharp
Invoke(target, null)
```
Because the Arrange Panels opener takes no parameters.
This opens the dialog instantly — no keystrokes, no menu navigation.
Your inspector becomes a control surface for MusicBee's internals.

**7. Observe the UI reaction**
Because you're invoking the real internal method:
- the dialog opens exactly as if the user clicked the menu
- all internal state updates correctly
- no hacks, no SendKeys, no Win32 messages
You're calling the same code path MusicBee uses internally.

**8. Repeat for any other UI element**
Once this pipeline exists, you can:
- discover any private method
- inspect any event handler
- invoke any internal action
- map the entire layout engine
- explore hidden dialogs
- understand MusicBee's architecture from the inside
Your plugin becomes a living debugger.
