Starting an inference server for a model

The integrated Playground environments allow for experimenting with available models in a local environment. An intuitive user prompt helps in exploring the capabilities and accuracy of various models and aids in finding the best model for the use case at hand. The Playground interface further allows for parameterizing models to further optimize the settings and attributes of each model.

Prerequisites

• Podman AI Lab installed.
• Model downloaded.
• An inference server created for a model.

Procedure

1. Click the Podman AI Lab icon in the left navigation pane.
2. In the Podman AI Lab navigation bar, click Playgrounds.
3. Click the New Playground button at the top right corner of the page.
4. Select the model for which you want to start an inference server from the Model list.
5. Click the Create playground button.

Verification

1. View the created playground on the same page.
2. Click the name of the playground to experiment with model settings. The Playground page opens.
3. Perform any of the following tasks:
   • Edit the value of the model parameters.
   • Add a system prompt by clicking the Edit system prompt icon.
   • Enter your question in the prompt box and analyze the accuracy of the answer.