Friction Logging Pipeline

Overview

The Friction Logging Pipeline acts as the nervous system for recursive self-improvement within an organization-as-platform-architecture. Instead of relying on external feature requests, this architecture uses “friction”—defined as user struggle, system latency, or cognitive confusion—as the primary signal for engineering priorities. The pipeline converts temporary pain into permanent assets through automated detection, diagnosis, and task routing.

Architecture Overview

The friction logging system consists of three layers: Detection, Diagnosis, and Execution. Together, these layers form a closed loop that transforms struggle signals into resolved engineering tasks without requiring manual intervention.

The Sensor-Actuator Model

The pipeline assigns biological roles to organizational tools to prevent data silos:

• Peppermint (The Sensor - Port 8084): Acts as organizational “Memory.” Captures explicit friction where human developers struggle with tools, runbooks, or processes. Replaces ad-hoc notes with structured support tickets.

• Plane (The Actuator - Port 8083): Acts as “Coordination” engine. Routes actionable work to appropriate AI agents. Converts friction points into resolved engineering tasks.

• Knowledge API: Acts as “Long-term Memory.” Stores solutions, patterns, and anti-patterns for future retrieval. Feeds the organizational-brain-api.

Detection Layer: Two Streams of Friction

The architecture captures friction through distinct channels, each optimized for different signal types.

Silent Struggle (Automated Telemetry)

Applications are instrumented to report when they are struggling, even if they don’t crash. This creates a continuous stream of system health signals.

Mechanism:

• Apps send POST /friction or POST /lessons-learned events to Knowledge API
• Payloads include context: operation, latency, timeout status, user action

Example Markers:

Integration Mismatch:

{
  "app": "Finance App",
  "service": "Math & Simulation Engine",
  "error": "ValidationError: precision parameter invalid",
  "frequency": 15,
  "time_window": "24h"
}

Interpretation: API documentation or schema is unclear. Triggers roadmap item to clarify Math Engine documentation.

Adoption Drop-Off (Shadow IT Marker):

• Sudden drop in API calls to core platform (e.g., Identity Service)
• Signals platform has become too bureaucratic or complex
• Triggers audit to find why pods are bypassing the “Golden Path”

User Rage Events:

{
  "app": "Media Refinery",
  "operation": "image_resize",
  "aborted_uploads": 12,
  "avg_latency_ms": 8500,
  "user_patience_threshold": "~5000ms"
}

Interpretation: Resize operation too slow for user patience. Triggers optimization of Media Refinery or offload to background queue.

Explicit Struggle (Human Dogfooding)

Because the organization follows “We are our own first customer” discipline, human pain points are captured immediately through structured workflows.

The 10-Minute Rule:

• In JP Developer IDE, if logic error remains unresolved >10 minutes, system flags it as “Learning Opportunity”
• Triggers Socratic Tutor to convert friction into study item
• Prevents Repetitive Structural Injury (RSI) where same conceptual mistakes repeat

Manual Capture via Peppermint:

• Developer spends >15 minutes debugging
• Classified as “Gotcha” and logged with full context
• Creates permanent record in organizational memory

Diagnosis Layer: Categorizing Resistance

Once captured, friction logs are analyzed to determine why architecture is resisting value flow. Categorization dictates the type of roadmap item created.

Four Critical Telemetry Markers

1. Cognitive Load Markers (Query Patterns)

Measure architectural confusion by analyzing traffic to Knowledge & Patterns API.

Signal: Spikes in specific search queries or decision rationale lookups

What it indicates:

• Tribal Knowledge Gaps: Multiple developers query GET /knowledge/search?q=monte+carlo+precision → Simulation Core API is non-intuitive
• “Why” Inquiries: Frequent hits to GET /knowledge/decision regarding specific standard → Standard is obsolete or poorly justified

Roadmap Action:

• Automate the confusing step
• Rewrite API documentation with examples
• Update “Golden Path” documentation

2. Hardware Saturation Markers (Resource Queuing)

For organizations using local hardware (e.g., Mac Mini M4 for Neural Gateway), friction manifests as physical resource contention.

Signal: Queue depth and wait times in asynchronous task queue (Celery/Redis)

What it indicates:

• Inference Bottlenecks: Neural Gateway shows growing queue of POST /inference tasks → Cognitive Core is overloaded
• Blocking Behavior: JP Developer IDE experiences timeout errors (HTTP 408/504) waiting for Syntax Service → Heavy tasks blocking main thread, violating “Thin Client” principle

Roadmap Action:

• Scale Neural Gateway hardware (e.g., upgrade Mac Mini)
• Offload heavy tasks to cloud
• Implement request throttling or priority queues

3. Integration Friction Markers (Error Patterns)

Signal: High volume of POST /friction events from specific app-to-service integration

What it indicates:

• Schema Mismatch: Finance App logs validation errors calling Math Engine → API contract unclear
• Error Code Shift: Shift from 500s (Server Error) to 400s (Bad Request) → Server stable but difficult to use correctly, indicating documentation failure

Roadmap Action:

• Refine API schema with better validation messages
• Add integration tests to catch contract violations
• Improve error messages in client SDKs

4. Shadow IT Markers (Adoption Drop-off)

In Organization as a Platform model, platform adoption is voluntary. Decline in usage is strongest signal of architectural friction.

Signal: Drop in API calls to core platform relative to number of active pods

What it indicates:

• Bypassing Golden Path: Legal Chat App stops calling central Identity Service and manages own users → Central service became too complex, slow, or bureaucratic
• Pod has “gone off-road” to avoid friction

Roadmap Action:

• Audit platform for bloat
• Interview pod leads to understand why bypass occurred
• Simplify platform interface or reduce mandatory steps

Execution Layer: The Peppermint → Plane Pipeline

The “Integrated Operations Strategy” creates direct link between friction logs and engineering tasks through a structured workflow.

Workflow C: Friction-Driven Improvement

Phase 1: Capture (Peppermint)

1. Developer encounters bug or confusing API while working
2. Logs ticket in Peppermint with full context
3. Creates permanent record of “Silent Struggle”

Phase 2: Escalation (Peppermint → Plane)

1. If friction requires engineering fix (e.g., “CLI hangs on large context”), promoted to Plane issue
2. Ticket includes original context plus categorization

Phase 3: Routing (Plane Labels)

Plane issue assigned specific colored label dictating AI tool routing:

Label	Color	Tool	Purpose
cc-arch	Blue	Claude Code Max	Deep architectural fixes
oc-bulk	Gray	OpenCode	Repetitive cleanup tasks
gem-docs	Yellow	Gemini CLI	Documentation generation
aid-refactor	Green	Aider	Automated refactoring
friction	Pink	Knowledge API	Lesson indexing in organizational brain

Phase 4: Execution (AI Dispatch)

The fetch-and-dispatch.sh script (or future Celery queue) reads Plane label and automatically triggers correct AI agent.

Example:

# Script detects new Plane issue with label "gem-docs"
# Spawns Gemini CLI session with ticket context pre-loaded
gemini-cli --context "$(plane-api get-issue $ISSUE_ID)" \
  --task "Generate API documentation"

Automation Maturity Phases

Phase 1 (Manual - Current):

• Human manually reads Peppermint tickets
• Human creates Plane issues
• Human acts as router

Phase 2 (Scripted Dispatch):

• Bash script queries Plane API for specific labels
• Launches corresponding tool with ticket context pre-loaded
• Human reviews output and merges changes

Phase 3 (Semi-Automated):

• Async queue (Celery/OpenCode background tasks) monitors Plane
• When ticket labeled aid-refactor, automatically spins up Aider session
• Attempts refactor and opens Pull Request for human review
• Human approves or requests changes

Phase 4 (Fully Automated - Future):

• AI agents autonomously resolve friction points
• Human reviews only high-risk changes
• System builds institutional memory without human intervention

Strategic Result: Fix Once, Propagate Everywhere

The ultimate goal of friction-driven roadmap is recursive self-improvement. Because apps are “thin clients” consuming shared utilities within a centralized-utility-api-architecture:

The Mechanism:

1. Friction log reveals bug in Flowchart Renderer
2. Don’t fix in Educational Tool directly
3. Add roadmap item to patch central Diagram & Flow Engine API
4. Fix applied to central service

The Outcome:

• Educational Tool inherits fix instantly
• JP Developer IDE inherits fix instantly
• Ghost Writer app inherits fix instantly
• No changes required in client applications

By treating friction not as nuisance but as “high-fidelity data,” the technical roadmap always solves verified reality rather than hypothetical features.

Memory Organ: Never Make the Mistake Twice

This pipeline implements the “Never Make the Mistake Twice” principle by forcing all friction through structured capture and all fixes through centralized platforms.

Quantification: Track “Friction Score” of every app in EspoCRM

• Apps with high friction scores get priority attention
• Declining friction scores indicate successful platform improvements

Systemic Fixes: AI doesn’t just patch code

• Friction label ensures pattern of error stored in Knowledge API
• Future developers query anti-patterns before making same mistake
• Organizational learning compounds over time

Telemetry Summary

Marker	Friction Type	Remediation Strategy
High POST /friction count	UX/Integration	Refine API schema or improve error messages
Search spike in Knowledge API	Cognitive	Update Golden Path documentation; automate confusing step
Celery queue depth > 0	Infrastructure	Scale Neural Gateway hardware or move heavy tasks to cloud
API usage drop-off	Governance	Audit platform for bloat; interview pod leads

Integration with Organizational Brain

The Friction Logging Pipeline serves as the input layer for the organizational-brain-api. While the brain stores patterns and decisions, the pipeline detects when those patterns are violated or when new patterns need to be created.

Closed Loop:

1. Friction Pipeline detects struggle → Peppermint
2. Struggle categorized and routed → Plane
3. AI agent resolves issue → Code/documentation change
4. Pattern extracted and stored → Knowledge API
5. Future developers query pattern → IDE/tooling
6. Mistake prevented before occurrence

Implementation Considerations

Data Retention

• Raw friction logs retained for 90 days for analysis
• Aggregated metrics retained indefinitely
• Personally identifiable information stripped from logs

Alert Thresholds

• Single high-latency event: Log only
• 3 similar events in 1 hour: Warning alert
• 10 similar events in 24 hours: Critical alert, auto-create Plane ticket

Privacy & Security

• Friction logs must not contain secrets, credentials, or sensitive user data
• Pre-processing pipeline strips PII before storage
• Access controls limit who can query raw friction data

Source Files

1. The Friction-Driven Technical Roadmap - Engineering Recursive Self-Improvement.txt
2. The Recursive Organism - Automating the Friction Logging Pipeline.txt
3. Telemetry Markers for Identifying Architectural Friction.txt