Use unstructured data as a knowledge source for agents in Copilot Studio.

By using Copilot Studio , you can enhance the capabilities of your agents with specialized knowledge supported by the same reliable and familiar data sources you build through the Power Platform connector.

When uploading external content from your device, OneDrive, or SharePoint, you can enrich the capabilities of your agents with contextual knowledge relevant to your business. Microsoft Dataverse securely stores these files and automatically processes them into semantic indexes and embedded vectors. This configuration allows agents to generate more accurate and well-founded responses based on the information you provide.

Files uploaded to Copilot Studio use Microsoft Dataverse to receive raw files, create indexes, and embed vectors. These indexes and embed vectors help provide quality responses to your agents. You can upload these files from your computer or by connecting to OneDrive or SharePoint.

When you upload files as knowledge sources, you enrich the capabilities of agents with additional data, enhancing the language model's knowledge base and helping agents rely on the specific information you provide. You can upload multiple different files; the system will index the semantics as embedded vectors and then use them as knowledge for the agents. You can share the knowledge used in the agents with both authenticated and unauthenticated users of that agent.

To improve the agent's response, the system breaks down uploaded files into smaller parts for faster processing and uses vector indexing to provide semantically relevant results for the user's query. The system securely stores these files in the Dataverse. When a user queries through the agent, Copilot Studio finds the parts that best match the user's query intent and returns the results.

Similarly, Dataverse accepts OneDrive and SharePoint files using the options in the file upload section. It also accepts unstructured content, such as knowledge base articles from Salesforce , ServiceNow, Confluence, and Zendesk, to provide better semantic results for the agent.

Power Platform connectors for unstructured data

The following Power Platform connectors work with unstructured data sources:

OneDrive

Use the Upload files > OneDrive option with the file selection interface to choose the files and folders you want to include. After selection, the system will retrieve the items into the Dataverse and index them for use. The folders you add will include all supported files and subfolders within that folder up to the total file limit.

SharePoint

Use the Upload files > SharePoint option to select files and folders through the file selection interface. After you select these items, the connector will retrieve them into the Dataverse and index them for use. When adding a folder, you will include all supported files and subfolders within that folder up to the total file limit. Currently, the connector does not support Pages.

Note : When using SharePoint as a knowledge source, Copilot Studio retrieves content through SharePoint search indexing, not by directly reading list views like AllItems.aspx. Newly added or updated SharePoint items may not be available to the agent until the search indexing process is complete. Ensure the agent has the necessary permissions, such as Sites.Read.All and Files.Read.All, and that the content is stored in supported file formats.

Steam

The Salesforce connector for unstructured data supports retrieving Knowledge Bases containing knowledge articles. Select a Knowledge Base, and the connector will index all articles within that Knowledge Base. You cannot select individual articles or topics. When querying data, you cannot specify a particular article or knowledge base. The Knowledge list displays a single object for all the knowledge objects you selected when creating the source.

ServiceNow

The ServiceNow connector for unstructured data supports retrieving Knowledge Bases containing knowledge articles. A Knowledge Base contains articles. Select a Knowledge Base, and the connector will index all articles within that Knowledge Base. You cannot select individual articles. When querying data, you cannot specify the knowledge base, directory, or individual articles. The Knowledge list displays a single object for all the knowledge objects you select when creating the source.

Confluence

The Confluence connector for unstructured data supports retrieving spaces containing pages. The connector also supports subfolders. You cannot select individual pages. When querying data, you cannot specify a particular page. The Knowledge list displays a single object for all pages in the workspace.

Zendesk

The Zendesk connector for unstructured data supports retrieving knowledge bases containing knowledge articles. You cannot select individual articles, categories, or sections. When querying data, you cannot specify a particular article, category, or section. The Knowledge list displays a single object for all articles in the knowledge base.

Security

When a user queries an agent using the Power Platform connector, the system performs an access check.

Access connector

When you use a connector-based source for the first time, the system will prompt you to select an existing Power Platform connector or add a new one. This process ensures that you only share data with creators who have the appropriate access rights to the data source.

Access content

When a user makes a query, the system uses their connection information to check the data source and verify that they have permission to view the content. Although the system stores data blocks and indexes locally in the Dataverse, it performs direct checks on queries to ensure the current user has permission to access the data before providing a summary or response.

Note :

1. The system doesn't return results to users if they don't have access to the specific knowledge base files or articles. Instead, they receive a standard message stating "no results found." If users feel they should have results for that source, they need to work with their administrator to ensure they have access to the data they are trying to access. Users need to be assigned an appropriate Dataverse security role, such as the Basic User role.
2. The system does not store content access permission information locally. It performs all permission checks directly with the source to ensure they are up-to-date.

File synchronization and refresh frequency

A scheduled sync task helps keep connected files from OneDrive and SharePoint, as well as unstructured knowledge articles, up-to-date. This task runs automatically in the background, refreshing the contents of files and re-indexing changes to provide accurate results for queries. The refresh not only manages content changes but also ensures that any content removed from the source will no longer appear in any query responses. Currently, you cannot manually trigger the refresh.

Licensing

All knowledge-related requests are charged at Microsoft Copilot's generated reply messaging rates. 

If knowledge sources require data input, the storage of that data and the corresponding indexes for retrieving it must be subject to the storage rights held by the client. 

Limitations and restrictions

When you first enable support for unstructured data, Dataverse may take 5 to 30 minutes to configure and index before processing added files. This time depends on the size of your current Dataverse environment.

Each agent can have up to 500 knowledge objects. These objects can be files, directories, knowledge articles, websites, or other sources.

Currently, an agent can only use 5 different sources at the same time. For example: SharePoint, Dataverse, OneDrive, or other sources.

Note : The Copilot Studio agent requires Dataverse search to use this knowledge source. If you are unable to add Dataverse-enabled files to the agent, ask your administrator to enable Dataverse search in your environment. 

To access OneDrive and SharePoint content stored in the Dataverse, users must have at least a Basic User license for Power Apps or Dynamics 365. Additionally, Basic User permissions must include read access to the following tables and entities:

1. Plug-in Assembly
2. Plug-in Type
3. SDK Message
4. SDK Message Processing Step
5. Sdk Message Processing Step Image

You can configure these permissions in the Power Platform Admin Center or Dynamics 365 Admin Center.