AI Advisory Board
Creator & Lead Developer
Public source code
A desktop model council with retrieval, persistent memory, and per-session budgets. Model inference is routed through OpenRouter.
Overview
AI Advisory Board is a native desktop application that runs a 'council' of large language models to produce balanced, multi-perspective answers. Building on Andrej Karpathy's llm-council, it adds persistent memory, retrieval grounding, and cost governance. The system works in three stages: models independently draft responses, anonymously peer-rank each other to reduce bias, and synthesize a final answer with a confidence signal. The desktop application runs a local server and routes model inference through OpenRouter. Local execution does not mean inference stays on-device: prompts and any included context are sent to the selected providers.
My contribution
I architected and built the system end to end: the FastAPI backend, the React frontend, and the desktop packaging. Key work includes the three-stage council protocol, a PageIndex-based retrieval and memory layer, and a per-session cost-governance system that keeps spend predictable.
What I Built
- A three-stage council protocol: independent drafting, anonymous peer ranking to reduce bias, and chairman synthesis with a confidence signal.
- A context-engineering layer using PageIndex-based RAG with persistent cross-session memory, query rewriting, and coreference resolution.
- A per-session cost-governance system with runtime model selection that degrades retrieval context as spend rises and blocks new turns at the budget ceiling.
- A native Windows desktop build (PyWebView wrapper) with a local server and remote model inference through OpenRouter.
Why It Matters
The council is designed to surface disagreement between models for human review. Peer rankings and the synthesis confidence signal are model-generated judgments, not calibrated probabilities or independent evidence of correctness. Per-session budgets constrain usage; local hosting does not make remote inference private.
Technologies & Tools
Deliverables & outcomes
- Shipped a packaged native Windows desktop application built on a three-stage council protocol.
- Routed a configurable registry of 40+ models through OpenRouter with runtime model selection.
- Added persistent cross-session memory and retrieval grounding via a PageIndex RAG engine.
- Implemented per-session cost governance that degrades retrieval context as spend rises and blocks turns at the budget ceiling.