@@ -0,0 +1,202 @@
+---
+title: Develop application with LlamaIndex and Azure AI studio
+titleSuffix: Azure AI Studio
+description: This article explains how to use LlamaIndex with models deployed in Azure AI studio to build advance intelligent applications.
+manager: nitinme
+ms.service: azure-ai-studio
+ms.topic: how-to
+ms.date: 9/14/2024
+ms.reviewer: fasantia
+ms.author: eur
+author: eric-urban
+---
+
+# Develop applications with LlamaIndex and Azure AI studio
+
+In this article, you learn how to use [LlamaIndex](https://github.com/run-llama/llama_index) with models deployed from the Azure AI model catalog deployed to Azure AI studio.
+
+Models deployed to Azure AI studio can be used with LlamaIndex in two ways:
+
+- **Using the Azure AI model inference API:** All models deployed to Azure AI studio support the [Azure AI model inference API](../../reference/reference-model-inference-api.md), which offers a common set of functionalities that can be used for most of the models in the catalog. The benefit of this API is that, since it's the same for all the models, changing from one to another is as simple as changing the model deployment being use. No further changes are required in the code. When working with LlamaIndex, install the extensions `llama-index-llms-azure-inference` and `llama-index-embeddings-azure-inference`.
+
+- **Using the model's provider specific API:** Some models, like OpenAI, Cohere, or Mistral, offer their own set of APIs and extensions for LlamaIndex. Those extensions may include specific functionalities that the model support and hence are suitable if you want to exploit them. When working with `llama-index`, install the extension specific for the model you want to use, like `llama-index-llms-openai` or `llama-index-llms-cohere`.
+
+In this example, we are working with the **Azure AI model inference API**.
+
+## Prerequisites
+
+To run this tutorial, you need:
+
+1. An [Azure subscription](https://azure.microsoft.com).
+2. An Azure AI hub resource as explained at [How to create and manage an Azure AI Studio hub](../create-azure-ai-resource.md).
+3. A model supporting the [Azure AI model inference API](https://aka.ms/azureai/modelinference) deployed. In this example, we use a `Mistral-Large` deployment, but use any model of your preference. For using embeddings capabilities in LlamaIndex, you need an embedding model like `cohere-embed-v3-multilingual`. 
+
+    * You can follow the instructions at [Deploy models as serverless APIs](../deploy-models-serverless.md).
+
+4. Python 3.8 or later installed, including pip.
+5. LlamaIndex installed. You can do it with:
+
+    ```bash
+    pip install llama-index
+    ```
+
+6. In this example, we are working with the Azure AI model inference API, hence we install the following packages:
+
+    ```bash
+    pip install -U llama-index-llms-azure-inference
+    pip install -U llama-index-embeddings-azure-inference
+    ``` 
+
+## Configure the environment
+
+To use LLMs deployed in Azure AI studio, you need the endpoint and credentials to connect to it. The parameter `model_name` is not required for endpoints serving a single model, like Managed Online Endpoints. Follow these steps to get the information you need from the model you want to use:
+
+1. Go to the [Azure AI studio](https://ai.azure.com/).
+2. Go to deployments and select the model you deployed as indicated in the prerequisites.
+3. Copy the endpoint URL and the key.
+
+    :::image type="content" source="../../media/how-to/inference/serverless-endpoint-url-keys.png" alt-text="Screenshot of the option to copy endpoint URI and keys from an endpoint." lightbox="../../media/how-to/inference/serverless-endpoint-url-keys.png":::
+    
+    > [!TIP]
+    > If your model was deployed with Microsoft Entra ID support, you don't need a key.
+
+In this scenario, we placed both the endpoint URL and key in the following environment variables:
+
+```bash
+export AZURE_INFERENCE_ENDPOINT="<your-model-endpoint-goes-here>"
+export AZURE_INFERENCE_CREDENTIAL="<your-key-goes-here>"
+```
+
+Once configured, create a client to connect to the endpoint:
+
+```python
+import os
+from llama_index.llms.azure_inference import AzureAICompletionsModel
+
+llm = AzureAICompletionsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=os.environ["AZURE_INFERENCE_CREDENTIAL"],
+)
+```
+
+Alternatively, if your endpoint support Microsoft Entra ID, you can use the following code to create the client:
+
+```python
+from azure.identity import DefaultAzureCredential
+
+llm = AzureAICompletionsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=DefaultAzureCredential(),
+)
+```
+
+> [!NOTE]
+> > Note: When using Microsoft Entra ID, make sure that the endpoint was deployed with that authentication method and that you have the required permissions to invoke it.
+
+If you are planning to use asynchronous calling, it's a best practice to use the asynchronous version for the credentials:
+
+```python
+from azure.identity.aio import (
+    DefaultAzureCredential as DefaultAzureCredentialAsync,
+)
+
+llm = AzureAICompletionsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=DefaultAzureCredentialAsync(),
+)
+```
+
+### Inference parameters
+
+You can configure how inference in performed for all the operations that are using this client by setting extra parameters. This helps avoid indicating them on each call you make to the model.
+
+```python
+llm = AzureAICompletionsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=os.environ["AZURE_INFERENCE_CREDENTIAL"],
+    temperature=0.0,
+    model_kwargs={"top_p": 1.0},
+)
+```
+
+Parameters not supported in the Azure AI model inference API ([reference](../../reference/reference-model-inference-chat-completions.md)) but available in the underlying model, you can use the `model_extras` argument. In the following example, the parameter `safe_prompt`, only available for Mistral models, is being passed.
+
+```python
+llm = AzureAICompletionsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=os.environ["AZURE_INFERENCE_CREDENTIAL"],
+    temperature=0.0,
+    model_kwargs={"model_extras": {"safe_prompt": True}},
+)
+```
+
+## Use LLMs models
+
+Use the `chat` endpoint for chat instruction models. The `complete` method is still available for model of type `chat-completions`. On those cases, your input text is converted to a message with `role="user"`.
+
+```python
+from llama_index.core.llms import ChatMessage
+
+messages = [
+    ChatMessage(
+        role="system", content="You are a pirate with colorful personality."
+    ),
+    ChatMessage(role="user", content="Hello"),
+]
+
+response = llm.chat(messages)
+print(response)
+```
+
+You can stream the outputs also:
+
+```python
+response = llm.stream_chat(messages)
+for r in response:
+    print(r.delta, end="")
+```
+
+## Use embeddings models
+
+In the same way you create an LLM client, you can connect to an embedding model. In the following example, we are setting again the environment variable to now point to an embeddings model:
+
+```bash
+export AZURE_INFERENCE_ENDPOINT="<your-model-endpoint-goes-here>"
+export AZURE_INFERENCE_CREDENTIAL="<your-key-goes-here>"
+```
+
+Then create the client:
+
+```python
+from llama_index.embeddings.azure_inference import AzureAIEmbeddingsModel
+
+embed_model = AzureAIEmbeddingsModel(
+    endpoint=os.environ["AZURE_INFERENCE_ENDPOINT"],
+    credential=os.environ['AZURE_INFERENCE_CREDENTIAL'],
+)
+```
+
+## Configure the models used by your code
+
+You can use the LLM or embeddings model client individually in the code you develop with LlamaIndex or you can configure the entire session using the `Settings` options. Configuring the session has the advantage of all your code using the same models for all the operations.
+
+```python
+from llama_index.core import Settings
+
+Settings.llm = llm
+Settings.embed_model = embed_model
+```
+
+However, there are scenarios where you want to use a general model for most of the operations but a specific one for a given task. On those cases, it's useful to set the LLM or embedding model you are using for each LlamaIndex construct. In the following example, we set a specific model:
+
+```python
+from llama_index.core.evaluation import RelevancyEvaluator
+
+relevancy_evaluator = RelevancyEvaluator(llm=llm)
+```
+
+In general, you use a combination of both strategies.
+
+## Related content
+
+* [How to get started with Azure AI SDKs](sdk-overview.md)

@@ -15,7 +15,7 @@ author: eric-urban
 
 # Overview of the Azure AI SDKs
 
-Microsoft offers a variety of packages that you can use for building generative AI applications in the cloud. In most applications, you need to use a combination of packages to manage and use various Azure services that provide AI functionality. We also offer integrations with open-source libraries like LangChain and mlflow for use with Azure. In this article we'll give an overview of the main services and SDKs you can use with Azure AI Studio.
+Microsoft offers a variety of packages that you can use for building generative AI applications in the cloud. In most applications, you need to use a combination of packages to manage and use various Azure services that provide AI functionality. We also offer integrations with open-source libraries like LangChain and MLflow for use with Azure. In this article we'll give an overview of the main services and SDKs you can use with Azure AI Studio.
 
 For building generative AI applications, we recommend using the following services and SDKs:
  * [Azure Machine Learning](/azure/machine-learning/overview-what-is-azure-machine-learning) for the hub and project infrastructure used in AI Studio to organize your work into projects, manage project artifacts (data, evaluation runs, traces), fine-tune & deploy models, and connect to external services and resources.
@@ -54,6 +54,9 @@ Azure AI services
 Prompt flow
  * [Prompt flow SDK](https://microsoft.github.io/promptflow/how-to-guides/quick-start.html)
 
+Agentic frameworks:
+* [LlamaIndex](llama-index.md)
+
 ## Related content
 
 - [Get started building a chat app using the prompt flow SDK](../../quickstarts/get-started-code.md)

@@ -57,10 +57,10 @@ The deployment options and features available for each model vary, as described
 
 Features | Managed compute | Serverless API (pay-as-you-go)
 --|--|--
-Deployment experience and billing | Model weights are deployed to dedicated virtual machines with managed online endpoints. A managed online endpoint, which can have one or more deployments, makes available a REST API for inference. You're billed for the virtual machine core hours that the deployments use. | Access to models is through a deployment that provisions an API to access the model. The API provides access to the model that Microsoft hosts and manages, for inference. You're billed for inputs and outputs to the APIs, typically in tokens. Pricing information is provided before you deploy.
+Deployment experience and billing | Model weights are deployed to dedicated virtual machines with managed compute. A managed compute, which can have one or more deployments, makes available a REST API for inference. You're billed for the virtual machine core hours that the deployments use. | Access to models is through a deployment that provisions an API to access the model. The API provides access to the model that Microsoft hosts and manages, for inference. You're billed for inputs and outputs to the APIs, typically in tokens. Pricing information is provided before you deploy.
 API authentication | Keys and Microsoft Entra authentication. | Keys only.  
 Content safety | Use Azure AI Content Safety service APIs. | Azure AI Content Safety filters are available integrated with inference APIs. Azure AI Content Safety filters are billed separately.
-Network isolation | [Configure managed networks for Azure AI Studio hubs](configure-managed-network.md).  | Endpoints follow your hub's public network access (PNA) flag setting. For more information, see the [Network isolation for models deployed via Serverless APIs](#network-isolation-for-models-deployed-via-serverless-apis) section later in this article.
+Network isolation | [Configure managed networks for Azure AI Studio hubs](configure-managed-network.md).  | Managed compute follow your hub's public network access (PNA) flag setting. For more information, see the [Network isolation for models deployed via Serverless APIs](#network-isolation-for-models-deployed-via-serverless-apis) section later in this article.
 
 Model | Managed compute | Serverless API (pay-as-you-go)
 --|--|--
@@ -74,7 +74,7 @@ Other models | Available | Not available
 
 <!-- docutune:enable -->
 
-:::image type="content" source="../media/explore/platform-service-cycle.png" alt-text="Diagram that shows models as a service and the service cycle of real-time endpoints." lightbox="../media/explore/platform-service-cycle.png":::
+:::image type="content" source="../media/explore/platform-service-cycle.png" alt-text="Diagram that shows models as a service and the service cycle of managed computes." lightbox="../media/explore/platform-service-cycle.png":::
 
 ## Managed compute
 
@@ -94,7 +94,7 @@ The registries build on top of a highly scalable and enterprise-ready infrastruc
 
 ### Deployment of models for inference with managed compute
 
-Models available for deployment to managed compute can be deployed to Azure Machine Learning online endpoints for real-time inference. Deploying to managed compute requires you to have a virtual machine quota in your Azure subscription for the specific products that you need to optimally run the model. Some models allow you to deploy to a [temporarily shared quota for model testing](deploy-models-open.md).
+Models available for deployment to managed compute can be deployed to Azure Machine Learning managed compute for real-time inference. Deploying to managed compute requires you to have a virtual machine quota in your Azure subscription for the specific products that you need to optimally run the model. Some models allow you to deploy to a [temporarily shared quota for model testing](deploy-models-open.md).
 
 Learn more about deploying models:
 
@@ -151,25 +151,25 @@ Pay-as-you-go billing is available only to users whose Azure subscription belong
 
 ### Network isolation for models deployed via serverless APIs
 
-Endpoints for models deployed as serverless APIs follow the PNA flag setting of the AI Studio hub that has the project in which the deployment exists. To help secure your MaaS endpoint, disable the PNA flag on your AI Studio hub. You can help secure inbound communication from a client to your endpoint by using a private endpoint for the hub.
+Managed computes for models deployed as serverless APIs follow the public network access flag setting of the AI Studio hub that has the project in which the deployment exists. To help secure your managed compute, disable the public network access flag on your AI Studio hub. You can help secure inbound communication from a client to your managed compute by using a private endpoint for the hub.
 
-To set the PNA flag for the AI Studio hub:
+To set the public network access flag for the AI Studio hub:
 
 * Go to the [Azure portal](https://ms.portal.azure.com/).
 * Search for the resource group to which the hub belongs, and select your AI Studio hub from the resources listed for this resource group.
 * On the hub overview page, on the left pane, go to **Settings** > **Networking**.
-* On the **Public access** tab, you can configure settings for the PNA flag.
+* On the **Public access** tab, you can configure settings for the public network access flag.
 * Save your changes. Your changes might take up to five minutes to propagate.
 
 #### Limitations
 
-* If you have an AI Studio hub with a private endpoint created before July 11, 2024, new MaaS endpoints added to projects in this hub won't follow the networking configuration of the hub. Instead, you need to create a new private endpoint for the hub and create new serverless API deployments in the project so that the new deployments can follow the hub's networking configuration.
+* If you have an AI Studio hub with a managed compute created before July 11, 2024, managed computes added to projects in this hub won't follow the networking configuration of the hub. Instead, you need to create a new managed compute for the hub and create new serverless API deployments in the project so that the new deployments can follow the hub's networking configuration.
 
-* If you have an AI Studio hub with MaaS deployments created before July 11, 2024, and you enable a private endpoint on this hub, the existing MaaS deployments won't follow the hub's networking configuration. For serverless API deployments in the hub to follow the hub's networking configuration, you need to create the deployments again.
+* If you have an AI Studio hub with MaaS deployments created before July 11, 2024, and you enable a managed compute on this hub, the existing MaaS deployments won't follow the hub's networking configuration. For serverless API deployments in the hub to follow the hub's networking configuration, you need to create the deployments again.
 
-* Currently, [Azure OpenAI On Your Data](/azure/ai-services/openai/concepts/use-your-data) support isn't available for MaaS deployments in private hubs, because private hubs have the PNA flag disabled.
+* Currently, [Azure OpenAI On Your Data](/azure/ai-services/openai/concepts/use-your-data) support isn't available for MaaS deployments in private hubs, because private hubs have the public network access flag disabled.
 
-* Any network configuration change (for example, enabling or disabling the PNA flag) might take up to five minutes to propagate.
+* Any network configuration change (for example, enabling or disabling the public network access flag) might take up to five minutes to propagate.
 
 ## Related content
 

@@ -42,6 +42,7 @@ POST /chat/completions?api-version=2024-04-01-preview
 
 | Name | Required | Type | Description |
 | --- | --- | --- | --- |
+| model |      | string | The model name. This parameter is ignored if the endpoint serves only one model. |
 | messages | True | [ChatCompletionRequestMessage](#chatcompletionrequestmessage) | A list of messages comprising the conversation so far. Returns a 422 error if at least some of the messages can't be understood by the model. |
 | frequency\_penalty |     | number | Helps prevent word repetitions by reducing the chance of a word being selected if it has already been used. The higher the frequency penalty, the less likely the model is to repeat the same words in its output. Return a 422 error if value or parameter is not supported by model. |
 | max\_tokens |     | integer | The maximum number of tokens that can be generated in the chat completion.<br><br>The total length of input tokens and generated tokens is limited by the model's context length. Passing null causes the model to use its max context length. |

@@ -42,6 +42,7 @@ POST /embeddings?api-version=2024-04-01-preview
 
 | Name            | Required | Type                                                | Description                                                                                                                                                             |
 | --------------- | -------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| model |      | string | The model name. This parameter is ignored if the endpoint serves only one model. |
 | input           | True     | string[]                                            | Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays.           |
 | dimensions      |          | integer                                             | The number of dimensions the resulting output embeddings should have. Returns a 422 error if the model doesn't support the value or parameter.                          |
 | encoding_format |          | [EmbeddingEncodingFormat](#embeddingencodingformat) | The format to return the embeddings in. Either base64, float, int8, uint8, binary, or ubinary. Returns a 422 error if the model doesn't support the value or parameter. |

@@ -8,7 +8,7 @@ items:
     href: what-is-ai-studio.md
   - name: Azure AI Studio architecture
     href: concepts/architecture.md
-  - name: "AI Studio or AML: Which should I choose?"
+  - name: AI Studio versus AML studio
     href: /ai/ai-studio-experiences-overview?context=/azure/ai-studio/context/context
   - name: Region support
     href: reference/region-support.md
@@ -65,16 +65,50 @@ items:
     - name: Create a project
       href: how-to/create-projects.md
     - name: Create and manage compute
-      href: how-to/create-manage-compute.md
-    - name: Connect to external resources
+    href: how-to/create-manage-compute.md
+  - name: Connect to services and resources
+    items:
+    - name: Connections overview
+      href: concepts/connections.md
+    - name: Create a connection
+      href: how-to/connections-add.md
+    - name: Create a connection using the Azure Machine Learning SDK
+      href: how-to/develop/connections-add-sdk.md
+      displayName: code
+    - name: Azure AI services connections
       items:
-      - name: Connections overview
-        href: concepts/connections.md
-      - name: Create a connection
-        href: how-to/connections-add.md
-      - name: Create a connection using the Azure Machine Learning SDK
-        href: how-to/develop/connections-add-sdk.md
-        displayName: code
+      - name: What are AI services?
+        href: ../ai-services/what-are-ai-services.md?context=/azure/ai-studio/context/context
+        displayName: cognitive
+      - name: Get started with AI services in AI Studio
+        href: ai-services/get-started.md
+      - name: Connect AI services in AI Studio
+        href: ai-services/connect-ai-services.md
+      - name: Azure OpenAI
+        items:
+        - name: Get started with Assistants and code interpreter in the playground
+          href: ../ai-services/openai/assistants-quickstart.md?context=/azure/ai-studio/context/context
+        - name: Analyze images and video with GPT-4 for Vision in the playground
+          href: quickstarts/multimodal-vision.md
+        - name: Use your image data with Azure OpenAI
+          href: how-to/data-image-add.md
+          displayName: vision, gpt, turbo
+      - name: Azure AI Content Safety
+        items:
+        - name: Content filtering
+          href: concepts/content-filtering.md
+        - name: Prevent input attacks with Prompt Shields
+          href: how-to/prompt-shields.md
+        - name: Detect groundedness in chat responses
+          href: how-to/groundedness.md
+      - name: Speech
+        items:
+        - name: Real-time speech to text
+          href: ../ai-services/speech-service/get-started-speech-to-text.md?context=/azure/ai-studio/context/context
+        - name: Pronunciation assessment
+          href: ../ai-services/speech-service/pronunciation-assessment-tool.md?context=/azure/ai-studio/context/context
+        - name: Hear and speak with chat in the playground
+          href: quickstarts/hear-speak-playground.md
   - name: Select and deploy AI models
     items:
     - name: Model catalog
@@ -218,6 +252,8 @@ items:
         href: how-to/develop/vscode.md
       - name: Start with an AI template
         href: how-to/develop/ai-template-get-started.md
+      - name: Develop with LlamaIndex and Azure AI studio
+        href: how-to/develop/llama-index.md
       - name: Trace your application with prompt flow
         href: how-to/develop/trace-local-sdk.md
         displayName: code,sdk
@@ -261,40 +297,6 @@ items:
       href: how-to/monitor-quality-safety.md
     - name: Troubleshoot deployments and monitoring
       href: how-to/troubleshoot-deploy-and-monitor.md
-  - name: Get started with Azure AI services
-    items:
-    - name: What are AI services?
-      href: ../ai-services/what-are-ai-services.md?context=/azure/ai-studio/context/context
-      displayName: cognitive
-    - name: Get started with AI services in AI Studio
-      href: ai-services/get-started.md
-    - name: Connect AI services in AI Studio
-      href: ai-services/connect-ai-services.md
-    - name: Azure OpenAI
-      items:
-      - name: Get started with Assistants and code interpreter in the playground
-        href: ../ai-services/openai/assistants-quickstart.md?context=/azure/ai-studio/context/context
-      - name: Analyze images and video with GPT-4 for Vision in the playground
-        href: quickstarts/multimodal-vision.md
-      - name: Use your image data with Azure OpenAI
-        href: how-to/data-image-add.md
-        displayName: vision, gpt, turbo
-    - name: Azure AI Content Safety
-      items:
-      - name: Content filtering
-        href: concepts/content-filtering.md
-      - name: Prevent input attacks with Prompt Shields
-        href: how-to/prompt-shields.md
-      - name: Detect groundedness in chat responses
-        href: how-to/groundedness.md
-    - name: Speech
-      items:
-      - name: Real-time speech to text
-        href: ../ai-services/speech-service/get-started-speech-to-text.md?context=/azure/ai-studio/context/context
-      - name: Pronunciation assessment
-        href: ../ai-services/speech-service/pronunciation-assessment-tool.md?context=/azure/ai-studio/context/context
-      - name: Hear and speak with chat in the playground
-        href: quickstarts/hear-speak-playground.md
   - name: Costs and quotas
     items:
     - name: Plan and manage costs