AI Tool Evaluation

Overview

Framework for evaluating AI development tools — criteria, methodology, and decision-making for tool selection in the Aloysius Productions ecosystem.

Evaluation Criteria

• Capability: What tasks can the tool perform reliably?
• Integration: How well does it fit existing workflows?
• Cost: Subscription/usage costs vs. productivity gains
• Reliability: Consistency of output quality across sessions
• Context handling: How much project context can it maintain?

Current Tool Stack

• Claude Code (T1) — primary orchestrator
• Gemini CLI (T2) — research, large context analysis
• Aider (T3) — git-integrated code editing
• Ollama + Qwen (T4) — local inference for mechanical tasks
• GitHub Copilot CLI (T5) — background task execution, independent implementation

Related

• copilot-cli-background-tasks
• coding-agents-testing-patterns
• claude-code-patterns
• simon-willison-jan2026-digest

References

• Aloysius Productions AI tooling audit