Models and pricing
| Model | Capabilities | Context | Status | Prompt weight | Completion weight |
|---|---|---|---|---|---|
Qwen2.5-VL 7B Instructqwen2.5-vl-7b
General-purpose chat model with image input and tool calling (4-bit quantised). Model card |
7Bchatimage inputtool calling | 16k | available | 1.0 | 1.0 |
Model size
A model's size is given by its number of parameters, shown as a grey tag such as 7B, for roughly 7 billion parameters. Larger models are generally more capable, but slower. Some models activate only part of their parameters for each token; a tag such as 35B-A3B means 35 billion parameters in total, of which 3 billion are active at a time. Such a model runs roughly at the speed of its active size, but all the weights are loaded into memory, limiting the total serving capacity of the hardware.
Service levels
Every allocation has a service class. It decides how a request is treated when the service is busy; it does not change what the request costs in standardised tokens.
Priority
Requests are served up to the service's full capacity. Meant for interactive and production use.
Capacity
Under heavy service load, requests wait for the load to go down, and may ultimately fail. Meant for batch and trial use.
Quotas and rate limits
Quotas and rate limits are calculated in standardised tokens: a request's token counts multiplied by model-specific weights, the prompt and completion weights in the table above.