configure llm
Last updated
Last updated
The configure llm command configures an LLM job. It is used to deploy an LLM or embeddings model for inference. In the notebook, you will be guided through a template form that will prompt you for various inputs, such as the LLM name, the source type, and the model name.
To create an LLM job:
Select New from the LLM drop-down list. Specify a name for the LLM job in the LLM Name field.
Select the type, which is either llm or embeddings.
Select either huggingface or aizen for the source type.
For aizen fine-tuned LLMs, select the registered Model Name and Version.
For huggingface pretrained LLMs, enter the Model Name and the Hugging Face credentials.
Select Advanced Settings to specify quantization parameters.
Click the Save Configuration button to save the LLM job.
