Running AI Agents

Aizen supports deploying LLM applications, such as AI agents. The AI agents may interact with LLMs, or they may interact with embeddings models served on Aizen or on a non-Aizen provider, such as OpenAI. The AI agents may also interact with vector stores for RAG applications.

To configure an AI agent on Aizen, follow these steps:

1. Log in to the Aizen Jupyter console. See Using the Aizen Jupyter Console.

2. Create an ML project if you have not already done so or set the current working project.

   create project <project name>

   or

   set project <project name>

3. Configure the AI agent by running the configure llmapp command:

   configure llmapp

4. In the notebook, you will be guided through a template form with boxes and drop-down lists that you can complete to configure the AI Agent. Enter the application's name and description. Any text that you enter in the app or tool description boxes will be appended to the system prompt. Select a provider name for the LLM, and select the LLM model name. Select the tools to be added to the AI agent, and specify the tool-specific parameters. Supported tool templates are as follows:

   • RAG Query: a tool that interfaces with a document Store ID from a vector store. Textual queries can be sent to the vector store to retrieve relevant chunks from documents within a Store ID.

   • REST Query: a tool that interfaces with a REST endpoint URL. The REST endpoint must provide an OpenAPI specification that is adequately annotated. Textual queries are used to generate function calls with appropriate parameter values, which are then sent to the REST endpoint via HTTP Get or Post methods.

   • SQL Query: a tool that interfaces with a SQL table. The SQL table must be a configured data source or dataset. Textual queries are used to generate SQL query statements, which are then sent to a SQL engine.

   • Web Search: a tool that searches the web for relevant information. Textual queries can be sent to a web search engine.

   • Custom Functions: a generic Python module with custom Pydantic functions. Functions and parameters must be adequately annotated. Textual queries are used to generate function calls with appropriate parameter values, which result in direct calls to the Python module.

5. Run the AI agent using the start llmapp command. This command will schedule a job to deploy the LLM application. Optionally, you can configure resources for the job by running the configure resource command. If you do not configure resources, default resource settings will be applied.

   configure resource
   start llmapp <llmapp name>

6. Check the status of the LLM application's deployment job and obtain serving URLs:

   status llmapp <llmapp name>

7. The base URL in the status output supports a REST API that lists the AI agents that are currently being served. The endpoint URL in the status output supports AI agent operations, such as creating sessions and chat requests. Both of these URLs provide the FastAPI docs, Redoc and OpenAPI paths.

8. Use the endpoint URL to create sessions and chat with the AI agent.