RouteJudge
Published:
RouteJudge is a full-stack web application designed for evaluating LLM routing decisions through the lens of diverse user preferences. Unlike conventional benchmarks that optimize for a single metric, RouteJudge accommodates multiple, potentially conflicting user requirements — from cost and latency to accuracy and safety.
Key Contributions:
- Independently developed the full-stack application (frontend + backend)
- Designed evaluation framework supporting pluralistic preference configurations
- Integrated with RoutingToolBox benchmark pipeline
Related Publication: RouteJudge: Preference-Based Evaluation of LLM Routers under Pluralistic User Preferences — accepted at ICML 2026 Workshop
Tech Stack: Python, JavaScript, API integration