Introduction to Azure Power BI
Power BI is a unified self-service and enterprise analytics solution. It lets you visualize your data and share insights across your organization and embed them in your app or website. Azure Analytics and Power BI together provide insights at scale, allowing you to develop the data-driven culture needed to thrive in a fast-paced, competitive environment.
Objective
In this tutorial we are going to talk about deploying Power BI Reports and Dataset. As per Enterprise Best Practice and standard approach, the deployment process will depict shared dataset deployment along with the report that is built using it.
We are going to utilize FlexDeploy's capability of integrating with Azure DevOps. Power BI Actions extension that is available in Azure DevOps Marketplace, has the ideal support for Dataset and Reports deployment. FlexDeploy will do the entire orchestration to make the deployment happen.
Flow
Azure DevOps Pipelines
DevOps release pipeline will fetch the pbix formatted file for dataset and report and deploy them to the target workspace in PowerBI. As a pre-requisite, the pbix files must be uploaded to SCM repository of user’s choice.
In case, one chooses to deploy one single report pbix file, the pipeline flow will remain same. So, it is flexible enough to support both the scenarios.
Dataset files can grow large in size depending on how much data is wrapped inside it and Azure artifacts feeds become paid service if more than 2GB space is utilized. Hence, we will maintain the dataset pbix files in DevOps build pipeline as part of build artifacts. Release pipeline will consume from there. The benefit is one doesn’t have to much worry about space occupancy in Feed.
FlexDeploy Orchestration
One has the flexibility to selectively choose the dataset and report files for deployment. We are going to brief about the package-based deployment here, which will enable this scope.
The build workflow will take the chosen artifacts and send them to Azure DevOps build pipeline via FlexDeploy’s Azure DevOps plugin.
The deploy workflow will trigger the deployment for those artifacts to target workspace provided by user.
All the user’s choice parameters such as workspace name etc will be set and passed from FlexDeploy.
Setting up Service Principal with necessary privileges
The deployment of datasets and reports to target workspace will be done using a service principal. For this one must ensure right permissions are given to it. We will talk about where exactly we are pointing to this SP from the Azure DevOps Release pipeline in the later sections.
Create a Microsoft Entra app in the Azure portal
Please follow the instructions in Step 2 in this page: https://learn.microsoft.com/en-us/power-bi/developer/embedded/embed-service-principal#create-a-microsoft-entra-app-in-the-azure-portal
For this tutorial we are naming our SP as PowerBICICDApp.
Create a Microsoft Entra security group
Please follow the instructions in Step 2 in this page: https://learn.microsoft.com/en-us/power-bi/developer/embedded/embed-service-principal#step-2---create-a-microsoft-entra-security-group
Enable the Power BI service admin settings
Please follow the instructions in Step 3 in this page: https://learn.microsoft.com/en-us/power-bi/developer/embedded/embed-service-principal#step-3---enable-the-power-bi-service-admin-settings
Add the service principal to your workspace
Please follow the instructions in Step 4 in this page: https://learn.microsoft.com/en-us/power-bi/developer/embedded/embed-service-principal#add-a-service-principal-or-security-group-manually
If there are multiple such target workspaces where the resources will be deployed, then the SP must be given access to every one of them following this step. The other previous 3 steps are one-time work.
In order to continue with our tutorial, we have selected the SP as member type and provided Admin access.
Note: The admin access is given as it is observed to overwrite same resource again and again during multiple deployments attempt, this is necessary.
Preparing the PBIX files for dataset, reports and uploading them to GIT
As SCM repository we are going to take an example of Azure DevOps repository here. But it can be anything of user’s choice, say Github, gitlab etc.
Created a repository named az-powerbi-artifacts under newly created project FD-PowerBIDeploy. As developer creates the dataset and reports, they must save them in the format of .pbix files and store in the repository under respective folders such as dataset, reports. This will help in identifying and choosing easily the respective objects while deploying. Will talk about it in details next.
dataset:
reports:
Azure DevOps Pipeline
Build Pipeline
Navigate to the Pipelines option under the Project FD-PowerBIDeploy and create a new pipeline as below:
Add the variables as highlighted.
Pipeline yaml
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- script: echo Initiating Build!!
displayName: 'Initiated'
- task: UniversalPackages@0
inputs:
command: 'download'
downloadDirectory: '$(System.DefaultWorkingDirectory)/PBIArtifacts'
feedsToUse: 'internal'
vstsFeed: $(feedName)
vstsFeedPackage: $(packageName)
vstsPackageVersion: $(packageVersion)
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/PBIArtifacts'
ArtifactName: $(packageName)
publishLocation: 'Container'
Release Pipeline
Navigate to the Pipelines option under the Project FD-PowerBIDeploy, select Releases and create a new release pipeline as below:
Add the variables as highlighted.
For this tutorial, we are naming it as DeployToPowerBIWS.
We are going to deploy resources to UAT target workspace, hence added the stage UAT. You may add as many stages as applicable to the deployment. Ensure selecting Manual Only bullet option.
Variables
Click on the Variables and add the below ones. We will be using them throughout the pipeline.
Tasks
Download artifacts produced by: Select “Specific build”
Project: FD-PowerBIDeploy
Build pipeline: Select the build pipeline created in previous step
Build version to download: Select “Specific version”
Build: $(buildNumber)
Artifact name: $(buildArtifactName)
Matching pattern: **/*.pbix
Destination directory: $(System.ArtifactsDirectory)
Tick the option “Clean destination folder“
Power BI service connection
We are configuring the service principal that we created earlier, in here.
Click on the Control Options and add the custom condition as: or(eq(variables['resourceType'], 'dataset'), eq(variables['resourceType'], 'all'))
Choose the Action as Upload Power BI report and tick the option Skip report uploading, since we are going to use this task for deploying datasets.
Workspace name: $(datasetWorkspaceName)
Source file: $(System.ArtifactsDirectory)/$(buildArtifactName)/dataset/*.pbix
Tick “Overwrite Power BI file”.
Similarly add the custom condition under Control options as: or(eq(variables['resourceType'], 'reports'), eq(variables['resourceType'], 'all'))
Workspace name: $(reportsWorkspaceName)
Source file: $(System.ArtifactsDirectory)/$(buildArtifactName)/reports/*.pbix
Tick “Overwrite Power BI file”.
For this task we are not selecting “Skip report uploading”
Additional Pipelines
As we have undertaken the shared dataset concept here, hence there would be two more additional pipelines required: 1. Deleting the existing report from the target workspace 2. Rebinding the report to the dataset
Let’s say user has a shared dataset present in the source dataset workspace A, based on which he created a report. Now the same dataset is promoted to higher workspace B. When that report is going to get deployed to Target Workspace, it will still point to the source workspace A for referencing to the dataset, since the pbix file is pointing there. Hence, the rebinding feature is needed in this situation to rebind the target workspace’s report to the dataset present in workspace B. Now when we are continuously revising a report and redeploying it multiple times, in order to make this rebinding work properly, before deploying the report to the target workspace, it should be deleted, so that we don’t end up in having two reports with the same name in the target workspace, one pointing to the workspace A’s dataset and another to B’s.
You don’t need to worry about these two, if you are not following Enterprise architecture of shared dataset and keeping the dataset embedded inside the same pbix file report.
Both the pipelines will have trigger type as “After release”
FlexDeploy Project Configuration
We are going to create generic package-based project in order to have the flexibility to select which dataset and report are going to be deployed.
In this tutorial, we will follow the name as: BuildAndDeployPowerBIArtifacts
Source Control will point to the repository that contains the pbix files as mentioned earlier.
Build Workflow
Purpose
This workflow will pick the selected dataset/report file, upload to Azure artifacts feed for the DevOps build pipeline to continue. The DevOps build pipeline will then store those pbix files as part of build artifacts. This way even if someone deleted the artifacts from the DevOps feed to reclaim space, the deploy workflow will still work. The same pbix files are also stored in FlexDeploy artifact repository.
BuildPowerBIArtifacts
name: BuildPowerBIArtifacts
description: ''
variables:
- code: BuildURL
dataType: String
returnAsOutput: false
constant: false
encrypted: false
scope: LOCAL
- code: BuildNumber
dataType: String
returnAsOutput: true
constant: false
encrypted: false
scope: LOCAL
steps:
- id: '1'
name: copy
type: INVOKE_PLUGIN
data:
pluginName: FlexagonFilePlugin
pluginOperation: copy
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDFILE_INP_FILE_FILTER
value:
value: '*.pbix'
isExpression: false
isEncrypted: false
- name: FDFILE_INP_FILE_FILTER_EXCLUDED
value:
isExpression: false
isEncrypted: false
- name: FDFILE_INP_SOURCE_PATH
value:
value: FD_TEMP_DIR
isExpression: true
isEncrypted: false
- name: FDFILE_INP_TARGET_PATH
value:
value: FD_TEMP_DIR + "/PBIArtifacts"
isExpression: true
isEncrypted: false
- name: FDFILE_INP_CLEAN_DIRECTORY
value:
value: 'true'
isExpression: false
isEncrypted: false
outputs: []
userInputs: []
userOutputs: []
- id: '3'
name: publishArtifacts
type: INVOKE_PLUGIN
data:
pluginName: FlexagonAzurePlugin
pluginOperation: publishArtifacts
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDAZ_DEVOPS_INP_FEED
value:
value: FEED_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PACKAGE_NAME
value:
value: FD_PACKAGE_NAME + "-" + PACKAGE_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PACKAGE_VERSION
value:
value: PACKAGE_VERSION
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PACKAGE_DESCRIPTION
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROJECT_NAME
value:
value: PROJECT_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PACKAGE_PATH
value:
value: FD_TEMP_DIR + "/PBIArtifacts"
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
value:
isExpression: false
isEncrypted: false
outputs:
- output: FDAZ_DEVOPS_OUT_ERR
- output: FDAZ_DEVOPS_OUT_RESP
userInputs: []
userOutputs: []
- id: '4'
name: buildPipeline
type: INVOKE_PLUGIN
data:
pluginName: FlexagonAzurePlugin
pluginOperation: buildPipeline
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDAZ_DEVOPS_INP_BUILD_DEFINITION_ID
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_DEFINITION_NAME
value:
value: BUILD_DEFINITION_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_QUEUE_ID
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_COMMIT_ID
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_BRANCH_NAME
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROJECT_NAME
value:
value: PROJECT_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_VARIABLES_LIST
value:
value: VARIABLES_LIST
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PACKAGE_NAME
value:
value: FD_PACKAGE_NAME + "-" + PACKAGE_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_ADDITIONAL_PARAMS
value:
isExpression: false
isEncrypted: false
outputs:
- output: FDAZ_DEVOPS_OUT_ERR
- output: FDAZ_DEVOPS_OUT_RESP
variable: ''
- output: FDAZ_DEVOPS_OUT_BUILD_PIPELINE_WEB_URL
variable: BuildURL
userInputs: []
userOutputs: []
- id: '5'
name: Assign
type: ASSIGN
data:
from: BuildURL.substring((BuildURL.indexOf("buildId=")+8))
to: BuildNumber
- id: '6'
name: saveArtifacts
type: INVOKE_PLUGIN
data:
pluginName: FlexagonFilePlugin
pluginOperation: saveArtifacts
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: true
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDFILE_INP_SOURCE_SUBFOLDER
value:
value: PBIArtifacts
isExpression: false
isEncrypted: false
- name: FDFILE_INP_FILE_FILTER
value:
isExpression: false
isEncrypted: false
- name: FDFILE_INP_FILE_FILTER_EXCLUDED
value:
isExpression: false
isEncrypted: false
- name: FDFILE_INP_TARGET_SUBFOLDER
value:
isExpression: false
isEncrypted: false
outputs: []
userInputs: []
userOutputs: []
Deploy Workflow
Purpose
Deploy workflow will have an optional step for deleting the report from the target workspace, performing the deployment of the report/dataset as chosen by user and rebinding the report to the dataset.
We will consider singular value for the simplicity purpose of this tutorial while specifying the deletion and rebinding step. One might reconfigure them to handle multiple values as needed. If those two steps are not needed at all then one may delete them altogether as well.
DeployPowerBIArtifacts
name: DeployPowerBIArtifacts
description: ''
inputs:
- code: DELETE
dataType: String
subDataType: TextField
defaultValue: 'NO'
defaultValueExpression: false
listData: YES,NO
listDataExpression: false
multiselect: false
required: false
encrypted: false
variables:
- code: ENV_DETAILS
dataType: String
returnAsOutput: true
constant: false
encrypted: false
scope: LOCAL
- code: RELEASE_ID
dataType: String
returnAsOutput: true
constant: false
encrypted: false
scope: LOCAL
- code: ENVIRONMENT_ID
dataType: String
returnAsOutput: true
constant: false
encrypted: false
scope: LOCAL
- code: RELEASE_URL
dataType: String
returnAsOutput: true
constant: false
encrypted: false
scope: LOCAL
steps:
- id: '1'
name: If
type: IF
data:
condition: DELETE == "YES"
elseIfBlock: []
steps:
- id: '1.1'
name: DeleteReportFromWS
type: INVOKE_PLUGIN
data:
pluginName: FlexagonAzurePlugin
pluginOperation: createRelease
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDAZ_DEVOPS_INP_RELEASE_DEFINITION_ID
value:
value: '2'
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_ARTIFACTS_PAYLOAD
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROJECT_NAME
value:
value: PROJECT_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_VARIABLES_LIST
value:
value: REPORTSTOBEDELETED
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROPERTIES
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_MANUAL_ENVIRONMENTS
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_RELEASE_STATUS_CHECK
value:
value: 'true'
isExpression: false
isEncrypted: false
outputs:
- output: FDAZ_DEVOPS_OUT_RESP
- output: FDAZ_DEVOPS_OUT_RELEASE_WEB_URL
- output: FDAZ_DEVOPS_OUT_REL_ENV_STAGE_DETAILS
- output: FDAZ_DEVOPS_OUT_RELEASE_ID
userInputs: []
userOutputs: []
- id: '2'
name: createRelease
type: INVOKE_PLUGIN
data:
pluginName: FlexagonAzurePlugin
pluginOperation: createRelease
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDAZ_DEVOPS_INP_RELEASE_DEFINITION_ID
value:
value: RELEASE_DEFINITION_ID
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_ARTIFACTS_PAYLOAD
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROJECT_NAME
value:
value: PROJECT_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_VARIABLES_LIST
value:
value: '"buildArtifactName:" + FD_PACKAGE_NAME + "-" + PACKAGE_NAME + "##buildNumber:"
+ BuildNumber + "##resourceType:" + POWERBIRESOURCETYPE'
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROPERTIES
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_MANUAL_ENVIRONMENTS
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_RELEASE_STATUS_CHECK
value:
value: 'false'
isExpression: false
isEncrypted: false
outputs:
- output: FDAZ_DEVOPS_OUT_RESP
- output: FDAZ_DEVOPS_OUT_RELEASE_WEB_URL
- output: FDAZ_DEVOPS_OUT_REL_ENV_STAGE_DETAILS
variable: ENV_DETAILS
- output: FDAZ_DEVOPS_OUT_RELEASE_ID
variable: RELEASE_ID
userInputs: []
userOutputs: []
- id: '3'
name: Parse Environment specific Id
type: INVOKE_PLUGIN
data:
pluginName: FlexagonXPathPlugin
pluginOperation: parseJson
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDP_SOURCE
value:
value: ENV_DETAILS
isExpression: true
isEncrypted: false
- name: FDP_XPATH
value:
value: '''$..[?(@.name==\''''+FD_ENVIRONMENT_NAME+''\'')
].id'''
isExpression: true
isEncrypted: false
outputs:
- output: FDP_VALUE
variable: ENVIRONMENT_ID
userInputs: []
userOutputs: []
- id: '4'
name: updateReleaseEnvStatus
type: INVOKE_PLUGIN
data:
pluginName: FlexagonAzurePlugin
pluginOperation: updateReleaseEnvStatus
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDAZ_DEVOPS_INP_RELEASE_ID
value:
value: RELEASE_ID
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_RELEASE_ENVIRONMENT_ID
value:
value: ENVIRONMENT_ID.substring(1,ENVIRONMENT_ID.length()-1)
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROJECT_NAME
value:
value: PROJECT_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_RELEASE_ENVIRONMENT_STATUS
value:
value: InProgress
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_VARIABLES_LIST
value:
value: ENVIRONMENT_VARIABLES_LIST
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_RELEASE_COMMENT
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_SCHEDULED_DEPLOYMENT_TIME
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_REL_ENV_STATUS_CHECK
value:
value: 'true'
isExpression: false
isEncrypted: false
outputs:
- output: FDAZ_DEVOPS_OUT_RESP
- output: FDAZ_DEVOPS_OUT_REL_ENV_WEB_URL
variable: RELEASE_URL
userInputs: []
userOutputs: []
- id: '5'
name: RebindReports
type: INVOKE_PLUGIN
data:
pluginName: FlexagonAzurePlugin
pluginOperation: createRelease
endpointInstanceOverride:
isExpression: false
consumesArtifacts: false
producesArtifacts: false
endpointSelection:
choice: All
endpointExecution:
choice: Any
stopOnError: false
inputs:
- name: FDAZ_DEVOPS_INP_RELEASE_DEFINITION_ID
value:
value: '3'
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_ARTIFACTS_PAYLOAD
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROJECT_NAME
value:
value: PROJECT_NAME
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_VARIABLES_LIST
value:
value: REBINDREPORTS
isExpression: true
isEncrypted: false
- name: FDAZ_DEVOPS_INP_PROPERTIES
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_MANUAL_ENVIRONMENTS
value:
isExpression: false
isEncrypted: false
- name: FDAZ_DEVOPS_INP_RELEASE_STATUS_CHECK
value:
value: 'true'
isExpression: false
isEncrypted: false
outputs:
- output: FDAZ_DEVOPS_OUT_RESP
- output: FDAZ_DEVOPS_OUT_RELEASE_WEB_URL
- output: FDAZ_DEVOPS_OUT_REL_ENV_STAGE_DETAILS
- output: FDAZ_DEVOPS_OUT_RELEASE_ID
userInputs: []
userOutputs: []
Execution
Discover the files from SCM and create package selecting the intended ones.
Upon submitting the Build, the build ID from Devops is returned, which denotes the Id against which the resources are stored. As report points to dataset in our case, hence we chose the strategy of deplying dataset before report.
This value gets saved as Output as well, so that can be utilized in the Deploy workflow.
Once the dataset and report is deployed to respective workspaces, in Power BI console it would look like: