Skip to main content

Starting an inference server for a model

The integrated Playground environments allow for experimenting with available models in a local environment. An intuitive user prompt helps in exploring the capabilities and accuracy of various models and aids in finding the best model for the use case at hand. The Playground interface further allows for parameterizing models to further optimize the settings and attributes of each model.

Prerequisites​

Procedure​

  1. Click the Podman AI Lab icon in the navigation bar
  2. In the Podman AI Lab navigation bar, click Models > Playgrounds menu item.
  3. Click the New Playground button on the top right.
  4. Select the model you want to start an inference server for in the Model list and click the Create playground button.
  5. The playground for the model is being started and after a while, a chat interface is displayed. playground

Verification​

  1. On the chat interface, you can optionally set the system prompt.
  2. Enter text in the prompt field and press the ENTER key.