More information from Microsoft here:
https://learn.microsoft.com/en-us/power-bi/developer/projects/projects-overview
There's been so many releases and announcements from Microsoft over the last 6 months or so that it's easy for some of them to fly under the radar. One that I missed was the announcement of a new file format in Power BI which I can't wait to try out.
I have always cringed when I see people saving copies of a file, throwing different versions of a file at each other in Outlook. We've all seen Sales_Figures.xlsx, Sales_Figures(1).xlsx, Copy of Sales_Figures.xlsx, Copy of Copy of Sales_Figures.xlsx... you get the idea. No one really knows which one is the reliable version, let alone what changes have been made since the last copy was saved. And what happens if someone makes a mistake and deletes crucial data? It can often be very difficult to recover such lost data once it's gone. This lack of history means you have no way of knowing what changes were made and who made them.
What is 'Version Control'?
Version control solves this through the "versioning" of files.
History Tracking - Every time you make a change to a file or document, version control software records it. It keeps a detailed history of every modification, including who made the change and when.
Branching & Merging - Version control allows you to create different "branches" of your project. Think of these as separate copies of your work. You can experiment with new features or fix bugs in these branches without affecting the main project. Once you're confident in your changes, you can "merge" them back into the main project.
Collaboration - Multiple people can work on the same project simultaneously without interfering with each other's work. Version control keeps everything organised and ensures that everyone's contributions are tracked.
Rollback - If you make a mistake or encounter issues with recent changes, version control allows you to easily revert to a previous, stable version of the file. This minimises the risk of losing important work.
This way of tracking file changes became more mainstream after the explosion of GitHub a few years ago which now has over 100 million users. 'Git' is an open-source version control system that is now the main standard of handling version control on files.
Power BI & Git Integration
Up until now (well, June 2023) there was no real way of utilising Git with Power BI. Your only option was deployment pipelines but this requires a costly Premium Power BI license. Others resorted to uploading .pbix files to SharePoint but this was limited in it's version control features.
In June 2023, Microsoft launched a new file format (.pbip) which enables 3 key features that can be used independantly or in conjunction with each other:
- Data models and reports are seen as separate objects, stored as text files and folders
- Native Git integration in a local file system repository
- Git integration and file synchronisation for a published workspace
Feature number 2 is what we're most interested in here as this is where version control can be fully realised. Once a folder is set up as a Git repository locally, you can then publish this to an online Git repository like Azure DevOps. This allows multiple, regionally disconnected developers to work on the same project remotely and have all changes tracked and recorded.
The PBIP File
Instead of a single .pbix file being saved, a series of files saved in a few folders is created.
The PBIP File: This contains a pointer to the report folder. When you click on this, it opens the report and the model inside Power BI
The Report folder: This contains the report, pages, model diagrams, and report visuals information. Each file contains text that can be edited, even with programs like Notepad. Most files here are stored in JSON format.
The Dataset folder: This contains all the data model components
When changes are made to a Power BI report, they're temporarily stored in unappliedChanges.json. When you click "Apply Changes" witih Power BI, the list of changes from this file are written to the following:
- model.bim contains a Tabular Model Scripting Language (TMSL) Database object definition of the project model;
- definition.pbidataset contains the overall definition of a dataset and core settings;
- json contains diagram metadata that defines the structure of the dataset associated with the report;
- config.json identifies the folder as a source control representation of a service item;
- metadata.json contains attributes that define the item.
The great feature of having these aspects of a Power BI report held in their own individual files means that:
- We can use Power BI Desktop to author reports and datasets files better
- We can better collaborate, use source control integration, enable version history tracking, as well as compare different revisions and even revert to previous versions;
- CI/CD workflows can be put in place to enforce quality gates prior to reaching production environments;
By default, in Power BI, the main project folder is set up to become a local Git repository, often called a "repo." You can easily create a new repository using GitHub Desktop or a similar tool by specifying the project folder. This process quietly sets up the repository and adds some hidden files. Then, you can copy the repository and make different versions to keep your project in sync with an online project repository.
Developers can now manage the data model and report separately using a shared Git repository. They can check-out and branch versions of a dataset or report, make separate edits, compare changes, and then merge them back into a single branch. This approach was previously not possible with a standard PBIX file. However, now it functions similarly to most dev projects, with separate code modules and classes.
Enable the preview feature
You need to have the June 2023 update for Power BI from the Microsoft Store. Go to File -> Options, and within the 'Preview Features' section, you'll find "Power BI Project (.pbip) save option".]
