Working with Dataflow in Microsoft Dataverse

Introduction

A dataflow is a group of tables created and managed in Power Apps service environments. Directly from the environment where your dataflow was developed, you may add and change tables, control data refresh schedules, and more.

This blog will show how we can create and use dataflows in Microsoft dataverse.

Dataflows

Dataflows were developed to assist organizations in combining data from many sources and preparing it for use. Using well-known self-service technologies, you may simply develop dataflows to ingest, convert, integrate, and enrich massive data. You must specify the connections to the data sources, the ETL (extract, transform, load) logic, and the location where the output data will be loaded when constructing a dataflow. 

There are mainly three different ways to use dataflow.

1. Author the dataflow in the Power Apps portal: By doing so, you choose the location to load the output data to, and the source to obtain the data from.
    
2. Schedule dataflow runs: You can schedule the frequency after which the dataflow should refresh data.
    
3. Use the data you loaded to the destination storage: Using Azure data services like Azure Data Factory, Azure Databricks, or any other service that supports the Common Data Model folder standard, you may create apps, flows, Power BI reports, and dashboards, or link directly to the dataflow's Common Data Model folder in your organization's lake.

Create dataflow

You can create the dataflow using the steps below:

1. Sign in to Power Apps. Note down the environment you are currently using.

2. From the navigation pane on the left side. Click on dataverse.

3. A menu will appear. Select the dataflow option from there.

4. Select New Dataflow. After that, select Start from blank.

5. A new window will appear. Enter the name of the dataflow and click on Create. Tables are stored in dataverse by default. If you want to store the table in the organization's Azure Data Lake storage account, you can select the Analytical entities only option.

6. On the next screen, you will be asked to select the data source.

7. After selecting the data source. You’ll be prompted to the connection setting page, where you have to set the connection from where the data needs to be fetched.

8. Once the connection has been established. You can select the data for your table. The Power Platform Dataflow service will afterward reconnect to the data source after you select data and a source to maintain the data in your dataflow up to date.

Dataflows and the Common Data Model

Dataflows tables come with new tools that make it simple to convert your company's data to the Common Data Model, add Microsoft as well as outside data to it, and gain streamlined access to machine learning. Utilizing these additional features will give you insightful and useful information about your business data. The Common Data Model has a predefined schema that is used to define standard tables.

Select the Map to Standard transformation in the Edit Queries dialog to open the Common data model with our dataflow. A new window will appear with the tables. select the standard table that you want to map.

All standard columns that are not mapped receive Null values to maintain the Common Data Model standard table.

Selecting the refresh frequency of your dataflow is the next step once you've finished making your selections and finished setting up your table and its data.

Setting the refresh frequency

After your dataflow has been successfully created, you can set the refresh frequency for your data to keep it up to date.

You can follow the steps to set the refresh frequency.

1. Click on the 3-dots ( as shown in the image below ) and select Edit refresh frequency.

2. A new window will appear where you can select Refresh automatically and set the frequency.

3. Set the frequency and click on Save.

Check out Microsoft Interview Experience to learn about their hiring process.