How to Master Data Fabric and Data Mesh

While the data industry in general has trended toward a centralized data management architecture, this approach can cause numerous challenges. These challenges are often at odds with what many organizations want - increased scalability, greater data agility, better data quality, enhanced data observability, faster access, and improved data interoperability.

Using data mesh and fabric architectural approaches, as defined in Table 1 and depicted in Figure 2, can help an organization meet its data management goals faster.

Table 1

Figure 2

This distributed domain-centered approach to data management provides many benefits, including:

• Ensuring ownership sits with domain knowledge. Meaning, those with business knowledge of the data better understand data quality issues and integration challenges or nuances
• Allowing domain teams to react faster to new requirements from data consumers or emerging quality issues
• Making it easier to scale the number of sources and consumers
• Empowering local data management leads while aligning infrastructure and metadata under a CDO
• Delivering a unified environment for applying master reference data standards, as well as attribute-based access control and asset categorization
• Providing the ability to apply role-based access control for zero trust uniformly to identities at the discovery layer
• This article provides actionable steps (Figure 3) you can follow to take advantage of data mesh or fabric

Figure 3

Identify Need

Many data issues caused by using a centralized data management architecture can be addressed by implementing data mesh or fabric. These broad issues include a lack of data source knowledge, limitations in adapting to evolving business information needs, difficulty in scaling to meet increased data volumes and types of data, and resistance from data stewards in providing access to data assets for ingestion, resulting in organizational fiefdoms with little enterprise alignment.

Emily Robertson-Yeingst, product marketing leader at TIBCO, describes common pain points that push organizations toward data fabric, such as, “...needing access to the right data at the right time, needing to trust data, and needing to know I can find the data I need.” She notes that data mesh needs are a little different than a centralized data architecture and include, “...getting the right data to the right end user, consistent and clean information, and obtaining an infrastructure that is agile and scalable.”

Marcus Gilg, the Chief Information Officer at Deutsche Bahn Regio Bus in Germany, recently transformed his organization’s data management architecture by implementing a data fabric and mesh concept, recognizing, “...there was a need to develop synergy and increase data usage,” to best utilize data for decision-making.

Both the experts note that data fabric and mesh address challenges organizations experience with their current data management architectures. Data leaders should also ensure there are well-defined information needs that a data fabric and mesh approach aims to enable. This will result in more tangible benefits to data consumers and executives investing in these efforts.

Develop Strategy

As you cultivate your strategy and consider prospective timelines to implement data mesh and fabric, understand that this will be a multi-year plan. As Gilg observes, data mesh and fabric are ‘long-term journeys with long-term benefits.’

Below are some questions to consider:

• How mature is your data catalog/metadata management program to support data discoverability
• What changes will be required to your architecture and technology stack
• Are data domains defined and do they adequately reflect the business
• What distributed data governance exists — including data stewardship — and how mature are the associated governance policies and technology
• Considering that data mesh is an operating model concept and data fabric is a tool, how do these integrate into the existing organization and architecture
• Do stakeholders impacted by these changes have ample data acumen to manage and govern data in a distributed fashion
• What value to the organization is expected
• What initial projects are recommended
• How are you going to manage the changes

Establish Key Pillars

Once your strategy is developed, the foundation of data fabric and mesh can be implemented. The key pillars will vary based on the maturity of data management in your organization. Some of them are:

Unified Data Catalog and Metadata Management — This provides a centralized view of all data assets, enabling teams to discover and understand data, improve data quality, and promote collaboration.

Data Governance —  It delivers structure via decentralized data governance. Governance is pushed out to the domain teams and these teams are responsible for creating and enforcing data governance policies — including data privacy and security policies, data quality policies, and data access policies — within their domain.

Data Domains — Logically organize and prioritize data assets and identify data sources associated with each data domain to inform the rollout of the data fabric. Data domains are important to data mesh because they align data ownership with business domains, promote an independent evolution of data products, and break down data silos.

Data Orchestration — This provides a means to manage the flow and usage of data, ensuring that data is consistent and accurate.

Data Quality Program — This supplies insights into the health of data and raises awareness of quality issues. Such programs are instrumental in allowing domain owners to provide access to trusted data products.

Start Small

As opposed to implementing data mesh and fabric across all your data domains, consider starting with one or two cases to understand the process and impact. Demonstrating quick wins and establishing credibility are critical to sustaining momentum and increasing the velocity of your strategy.

According to Gilg, in implementing data fabric and mesh at Deutsche Bahn, he “...started with small use cases with the business-critical data, and ultimately improved/added …different use cases.”

Socialize and Enable

Once you have demonstrated wins in your data fabric and mesh approach, you will need leadership support for broader implementation. As Gilg explains, “...before training people, it is important to create the data culture. Obtain support from top management first to get buy-in for a data-driven organization.”

With leadership support in hand, it is time to create a socialization and enablement plan that includes personnel training. Below are some questions your plan should consider:

• Who are the impacted stakeholders
• What are their roles and responsibilities
• What artifacts (e.g., guides) would accelerate training
• What are some incentives to change behavior

Summary

Like any emerging technology or approach, data mesh and fabric are still being evaluated in real-world contexts. However, there are many reasons to believe that data mesh and fabric approaches have the potential to offer significant benefits to organizations that adopt them. If data mesh and fabric are a fit for your organization, you should follow the five steps outlined above to understand your organization’s data challenges and how data mesh and fabric can help systematically address them.

Co-authored by Bob Audet, Kyle Armstrong, Selene Bainum, and Rich Hernandez with TechTarget.