Data governance is a framework that provides data management governance. It’s the process of structuring data so it can be governed, managed and used more effectively. Data governance framework forms the key aspect of data analytics strategy. This blog post will discuss key functions of a standard data governance framework and can be taken as a template or example to help you get started with setting up your data governance program.
What is Data Governance Framework?
Data governance can be defined as enterprise-wide management of data from availability, usability, security and integrity standpoint. The data governance framework is intended to put some structure around how data can be managed and used in an organization based on well-defined rules and processes around a variety of data related operations and decisions. Data governance framework is important to data-reliant organizations because it provides a structure for data management and usage. Some of the key aspects of data governance includes the following:
- Data management (data lineage, data cataloging, data classification)
- Data discovery (Discovering data based on metadata, dat lineage, etc)
- Data proliferation
- Privacy and compliance
Key Functions / Components of a Data Governance Framework
Here are four different key components / functions of a data governance framework:
- Establishing & maintaining standard
- Establishing accountability for data
- Managing & communicating data development
- Providing information about the data environment
Establishing & maintaining standards
The primary role of data governance is to establish and maintain standards around data. This can be achieved in different manners such as the following:
- Identifying what data sources are preferred for each type of data or metric used in an organization. There’s an concept called Master Data Management, or MDM, that identifies the most critical data within an organization and ensures there is a clear understanding of where that data should come from and where it should be stored.
- Ensuring that the reference data is complete and accurate. Reference data provides sets of allowable values for certain data attributes, or provides additional descriptive information about key ideas in the company’s data environment. Reference data also helps data consumers by providing data descriptions and metadata.
- Establish common data definitions and rules / calculations. This will provide data quality and accuracy.
- Monitor / manage data access and compliance. A data governance process helps to manage and monitor who should have access to data under what circumstances. It is often applied in support of more general SOX controls and data privacy concerns.
Establishing Accountability for Data
The second major role of data governance is to establish and maintain accountability for data. Data governance program or framework assigns responsibility for specific data domains to individuals called data stewards. Data stewards are generally accountable for ensuring that their area has the correct definitions and are responsible for the overall state of their data domain. Governance can also help identify who is responsible for addressing various types of data quality issues.
Managing & Communicating Data Development
The third key aspect of data governance framework is to help manage the overall process of data development and to communicate changes to the data environment. The following are two key activities:
- Prioritize data projects: Data is required by many teams. And, the challenge is that the teams require data in different forms based on some modifications relating to the business demands. However, this is very cumbersome to meet everyone’s demand and there needs to be some way of prioritizing the work that needs to get done. This is where data governance framework lays down the process for assessing, and prioritizing which data projects are undertaken, usually by rationalizing those projects against the overall business priorities of the enterprise.
- Communication around data projects & environments: In addition, data governance framework also ensures the communication regarding the evolution of data environments (change or improvement) and letting the users of the data know when new data is added. Having a well structured data governance approach can facilitate communication about data and make sure everyone is informed and aware of the changes.
Providing Information about the Data Environment
The fourth major function of the data governance framework is providing information about the data environments to the stakeholders at regular intervals. The key aspect is providing information about data also called as metadata. This process can also be termed as metadata management. Given that we’ve gone through all the trouble of creating standard definitions and calculations, it’s generally useful to formally document them and provide that documentation to the enterprise. Metadata can speak to the what and where of the data environment, but it can also indicate how good the information is. The following represents the key activities of metadata management:
- Provide information on data lineage & metrics: The information about the lineage of data and metrics is provided. This helps trace where data elements come from. Or keep a history of changes that have been made to a data environment. All of these would fall under metadata management. Given that organizations are moving on to data lake, data lakehouses, etc., there is a danger of loss of opportunity based on lack of capability on discovering different aspects of data such as the origin and lineage of data, data assets available in the data lake and information stored in these data assets. Thus, it becomes of utmost importance to manage and publish different aspects of data discovery while providing some form of utility which allows consumers to know and access the data for data-driven decision making.
- Provide information on data quality: You can also provide information about the quality of one or more data domains or metrics. Likewise, governance can help keep track of the who, including tracking who data stewards are and who may be involved in other data governance functions. Users can consult this information to determine who to contact with questions or concerns about the data. Managing data quality includes some of the following aspects:
- Creating controls for validation
- Enabling quality monitoring and reporting
- Tracking data incident and supporting the triage process for assessing the level of incident severity
- Root cause analysis
- Remediation of data quality issues
Data Governance Framework Template / Example
Based on the previous section explaining the key functions of data governance framework, the following can be formed as a template of your data governance framework. You could create one or more excel spreadsheets to capture / track the following:
- Document data sources against different kind of data
- Document reference data (allowable attributes of different dataset)
- Document common data definitions
- Document data access permissions; Monitor the access permissions from time-to-time.
- Document different data domains and related data stewards
- Assess, prioritize and document data projects
- Schedule communication on data projects & environment. Capture information about the same including communications’ logs
- Document data quality framework; Capture data quality logs
- Document data lineage and metrics information
Data Governance Framework is a data-driven approach to data management. By implementing the framework, data can be managed in an efficient and effective manner without compromising on quality or accuracy. The framework provides for standard definitions of data domains along with access permissions which ensures that user’s needs are met while data security remains intact. Implementing this type of strategy will help you manage your data more effectively and meet organizational needs while avoiding information overload among staff members who use it on a day-to-day basis. For more information about Data Governance Strategies, please feel free to reach out. In the next blog, we will look into how to go about implementing data governance program.
Check out my latest book titled as First Principles Thinking: Building winning products using first principles thinking. Check out my other blog, Revive-n-Thrive.com
- MSE vs RMSE vs MAE vs MAPE vs R-Squared: When to Use? - April 25, 2024
- Gradient Descent in Machine Learning: Python Examples - April 22, 2024
- Loss Function vs Cost Function vs Objective Function: Examples - April 19, 2024
Very practical and concise content. Thanks for sharing.