Increasingly, the market is asking about the relationship between Master Data Management (MDM) and big data. Big data is defined as the increasing volume and variety of data, created at a faster velocity than before. These new sources of big data may contain new insights but are often hard to analyze quickly and cost-efficiently; these sources include external data such as social media, third party data, internet data, and internal data sources such as unstructured content and transaction data among others.
Long-standing MDM clients are asking how big data can be added to their MDM implementation, and new MDM clients are wondering whether big data provides new opportunities for leverage.
There should be a symbiotic relationship between MDM and big data; big data technology can feed insights to MDM, and MDM can feed master data definitions to big data. In some ways, this is similar to the relationship between MDM and the data warehouse – MDM will both receive and feed that system.
One of the most common use cases for big data technology is social media analytics. Recently I visited a client whose CEO publicly declared that they would analyze social media to understand customers and sell more products to them. As you can imagine, there was a frantic scramble from the IT department to figure out how to do this. Their conclusion was telling – they could employ big data technology to mine social media and understand intent, and that would unearth potential new ‘customers. ’ But what if those same prospects were already customers? And what if they already knew, or should have known, their intentions? The company concluded that, in order to make a targeted and purposeful analysis of big data, they needed a starting point – and that starting point should be understanding their existing customers through MDM.
This highlights the first aspect of the big data and MDM relationship – MDM feeds big data. MDM can provide master definitions of customer, household, relationship, product, hierarchies to big data. When your requirement moves from aggregate analysis (e.g., general market sentiment towards your company) to specific analysis (e.g., which customers have an intent to purchase product X), that is when you require master data to guide big data analysis.
Many other clients are looking to big data to augment or fulfill the “360 degree view” of MDM. They utilize MDM as a starting point to define a customer or a product, and then analyze big data sources (often unstructured content) for new data to bring into MDM – often relationships, hierarchies, intent, sentiment, etc.
There are many cases where big data can feed MDM, or even be a starting point for an MDM implementation. Big data technology can process and analyze unstructured data sources, for example PDF documents, to determine unique identities and relationships among master data entities. One organization was interested in analyzing third party data (unstructured PDF documents on company financials and ownership) to help determine organization parties and hierarchies.
Clearly there is a bi-directional relationship between MDM and big data. Big data technology can benefit from master data as a starting point for analysis, and it can also help augment or feed new insights and facts into an MDM system. I will be speaking about this at the Gartner MDM Summit in Los Angeles tomorrow, April 4, at 3:45 PM. Next week I’ll go a little further into the MDM readiness for big data and social media, and the requirements for an MDM system.
