role-model



Model routing protocol and runtime



Protocol and runtime for hybrid local/API model inference. Models are assigned roles based on capabilities, performance, speed and cost; task requests are routed to the best suited models to optimiza the triangle of constraints, cost-speed-accuracy.


Coming soon.



© Try-Works