GenAI Gateway & Token Governance Dashboard
Home → Projects → GenAI Gateway & Token Governance Dashboard
GENAI INFRASTRUCTURE · USAGE OBSERVABILITY · COST GOVERNANCE
I built a GenAI gateway and organizational dashboard to help teams route LLM requests and monitor token usage, estimated costs, quotas, rate limits, and model consumption across departments, teams, and individual members.
A full-stack governance workflow for routing requests, recording usage, and making LLM consumption easier to understand.
System
GenAI gateway
Stack
React · TypeScript · Python · PostgreSQL
Focus
Token usage, costs, quotas, rate limits, and model consumption
The Problem
As organizations adopt multiple LLM providers, they need visibility into usage, costs, and consumption across teams. Without centralized monitoring and governance, it becomes harder to understand usage patterns, manage quotas, and maintain accountability.
What I Built
I built a local GenAI gateway that routes requests to a configured OpenAI-compatible chat-completions provider. The backend records returned token usage in PostgreSQL and exposes a dashboard for monitoring consumption, estimated costs, quotas, rate limits, and model usage across departments, teams, and individual members. The application also includes backend API and health monitoring, alerts and audit events, seed-data mode for local development, Docker Compose setup, and Alembic database migrations.
Frontend & API
React and TypeScript with Tailwind CSS; a Python API provides gateway, dashboard, health, alert, and audit workflows.
Database
PostgreSQL stores usage records and analytics data. SQLite is supported for local development.
Infrastructure
Docker Compose provides the local setup, with Alembic handling database migrations.
LLM Gateway
OpenAI-compatible chat completions are configured through environment variables.
Engineering Decisions & Limitations
The project supports local development and seed-data mode. The live gateway requires real provider configuration. AzureMonitorTelemetryProvider exists as an intended integration, but its queries are incomplete and unverified. Models and prices CRUD, Azure connection status, live Azure Monitor data, and the planned Azure handoff/smoke-test workflow remain outside the implementation.
Technology stack: React · TypeScript · Python · PostgreSQL · SQLite · Docker Compose · Alembic · OpenAI-compatible LLM integration
Impact
This project demonstrates GenAI infrastructure design, LLM gateway integration, usage observability, cost governance, database-backed analytics, and full-stack engineering without claiming production deployment or measured performance improvements.
What It Demonstrates
A practical foundation for understanding and governing LLM consumption across organizational boundaries.
React · TypeScript · Python · PostgreSQL · SQLite · Docker Compose · Alembic · OpenAI-compatible LLM integration