role-model



Model routing protocol
Router runtime
Pi package





Protocol and runtime for hybrid local/API model inference. Models are assigned roles based on capabilities, performance, speed and cost; task requests are routed to the best suited models to optimiza the triangle of constraints, cost-speed-accuracy.

Github: https://github.com/try-works/role-model
Docs: https://role-model.dev/



© Try-Works