digg this | del.icio.us | Reddit | Stumble It!

I had a very interesting briefing the other day by Johnnie Konstantas, the VP of Marketing at Varonis.

Varonis is a software company that focuses on governing unstructured data. Johnnie’s perspective was pretty illuminating:

• A 2007 IDC study found unstructured data accounts for over 80% of all business data
• In 2008, a study by the Ponemon Institute found that 84% of organizations believe their unstructured data is accessible by people with no clear business need and 32% have experienced an unstructured data breach
• The IDC study also found that data grows at a rate of 57% per year

There have been high profile stories lately about unauthorized people snooping in presidential candidates’ passport files, the theft of 94 million credit cards from TJX, and the exposure by an investment firm of data on 2,000 clients, including Supreme Court Justice Stephen Breyer.

And in June, The Identity Theft Resource Center reported that nearly 16 percent of breaches this year came from insiders, up from 6 percent in 2007.

Given that many of the people I talk to or work with are building Master Data Management solutions for their companies, or putting together a Data Governance program, I had to stop and ask myself “maybe we’re all missing the forest for the trees here”.

Granted, the picture on the structured data side of things needs improvement too. Companies still struggle to pull together the “Single View of the Customer”. Islands of data still exist, and artificial silos still cost companies money and hurt productivity.

But I think we ignore the unstructured data problem at our peril. I believe savvy business owners will eventually expect an integrated approach to governing both structured and unstructured data. Even though the technology tools might be quite different, a common organization and policies addressing both types of data will be necessary.

It’s not enough to lock up the structured data, when over 80% of the information in the company is unstructured and is not adequately protected or managed.

I haven’t had a chance to thoroughly research Varonis and its products yet, but it looks like a unique way to govern the unstructured data on file systems, and I’m impressed by the company’s approach.

digg this | del.icio.us | Reddit | Stumble It!

We’ve been writing this blog for a year now, with a total of 83 posts so far.

It’s been a very positive experience.  We’ve had clients tell us “the reason we hired your firm was because of your blog”. And we’ve gotten lots of great feedback from our partners (Oracle, IBM, SAP, Initiate Systems and Siperian).

What we’ve tried to do is to write for people who are new to Master Data Management (MDM) and looking for basic information (like “Useful Definitions for MDM”,“Five Essential Elements of MDM” and “Ten Best Practices for Master Data Management”).

But we’ve also tried to cover more advanced topics too (such as “Master Data Management and the Art of Politics”, “The Key Requirement in Choosing a Product MDM Hub”, and “Data Governance Critical to MDM Success”).

We thought that by presenting a mix of basic and advanced topics, and highlighting key milestones in the development of the firm, we could keep your interest, and hopefully keep you coming back.

The numbers tell a good story.  We’ve had a total of 8,100 hits in the past year, with an average of 32 hits per day (over the last 30 days), 200 hits per week and 835 per month (over the last 6 months).

Our “Top 10″ posts have been:

1. Ten Best Practices for Master Data Management
2. Our MDM Partnership Strategy
3. How Master Data Management is Similar to ERP
4. Different Styles of MDM Hub
5. Metadata and Master Data Management
6. Five Essential Elements of MDM
7. Critical Data Quality Questions
8. The Key Requirement in Choosing a Product MDM Hub
9. Master Data Management and the Art of Politics
10. MDM Business Case Creation & ROI Analysis

We get most of our traffic from our web site at www.hubdesigns.com (there’s a prominent “Blog” link there), and from the “Master Data Management” and “Customer Data Integration” tags at WordPress.com.  We also get a fair amount from Google Reader, My Yahoo, and my LinkedIn profile.

Our Top 10 search terms that people are using to get to the blog are: “Hub Solution Designs”, “Dan Power”, “Gaurav Arora”, “data quality questions”, “MDM vendors”, “Master Data Management best practices”, “critical to quality”, “Oracle MDM”, “ERP and MDM” and “Master Data best practices”.

We’ve tried to keep the blog vendor-neutral, and have resisted the temptation (so far at least) to accept any form of advertising.

In the coming year, we’re looking forward to more in-depth coverage of the leading MDM and data quality platforms, more insights gleaned from working with our clients, more pointers to other places where our writing appears (like my monthly column in the online edition of DM Review), and continuing to try to break new ground and be thought leaders on MDM.

If there’s anything in particular you’d like to see us cover here, please let us know via a comment. It’s been an honor to write for you over the past year, and we’ll work hard to make this a useful resource for you in the coming year.

digg this | del.icio.us | Reddit | Stumble It!

Here’s a quick excerpt from my latest “MDM Insights” column in DM Review.

It’s a long journey from the first efforts of “customer cleanup” to a full-fledged data governance program. But that’s where many companies start. They gradually accept that there are issues with their customer data such as:

• A lack of consistently applied standards and controls,
• Problems arising from conversion of customer data from acquired companies,
• Lack of ownership of customer data,
• Invalid addresses leading to undelivered and returned mail or
• Customer service problems caused by large numbers of duplicate and inaccurate records.

So they form a committee, hire a consulting firm, and involve their internal IT folks. That’s a great start, but it’s important to realize that this is not a once-and-done project.

Click on “From Customer Cleanup to Data Governance” to continue reading.

And please let us know your thoughts by commenting here …

digg this | del.icio.us | Reddit | Stumble It!

DM Review magazine has an excellent e-book on “Best Practices in Master Data Management”, with a cover story on “Project Management Challenges with a Changing Landscape” by Hub Solution Designs.

You can access it at http://sm.ebookhost.net/dmr/mdm/1/ or http://sm.ebookhost.net/dmr/mdm/1/ebook/1/allcontent.pdf.

Our article begins on page 4.

Please let us know what you think by commenting here.

digg this | del.icio.us | Reddit | Stumble It!

Many companies are still deploying Enterprise Application Integration (EAI), with proprietary adapters and integration servers.  However, for a Master Data Management (MDM) solution, we recommend a Service-Oriented Architecture (SOA) approach for integration between the hub and source systems using web services.

A typical web server provides Hypertext Transfer Protocol (HTTP), so Web browsers can receive pages from a web site.

Application servers provide the Simple Object Access Protocol (SOAP) interface and host the web services. The web services also provide object components, which provide the business service layer above the applications.

The development time for SOA-based MDM integration will depend on the number of business entities to be exchanged, the availability of a vendor-supplied Software Development Kit for the Web Services Definition Language (WSDL), the complexity of the applications to be integrated and the number of Web services to ultimately be deployed.

Some guidelines for developing an SOA integration for an MDM hub are:

• Use XML (eXtensible Markup Language) for all data exchange (XML is a language that provides a standard way of representing data and information).
• Use UDDI (Universal Description, Discovery, and Integration) for listing and locating applications. UDDI is a directory standard that is provided by some application tools as a built-in service to use during integration.
• The WSDL (Web Services Description Language) file should be obtained from the source system to which data needs to be sent or retrieved. WSDL is a “descriptor standard” that an application uses to describe its interface and interaction rules to other applications. WSDL is a document written in XML which describes a Web service. It specifies the location of the service and the operations (or methods) the service provides.
• WSDL should be leveraged with the help of proprietary tools provided for each application to generate the XML message required to meet that data structure.

Currently, some of the Master Data Management platforms (such as Siperian, Initiate Systems, Oracle and IBM) provide excellent SOA libraries of web services.

With some work by the end customer, these products can provide a standard set of data services at the application level. We believe this approach ultimately will give you more flexibility and adaptability than EAI-based integration.

digg this | del.icio.us | Reddit | Stumble It!

I’m often asked how many data stewards a company needs.

A rule of thumb I’ve used for quite a while is “one data steward per $1 billion in annual revenue”, but of course that can’t be a hard and fast rule.

Depending on the company, the degree of automation, what data quality tools are available, and the major processes around the enterprise, that number is going to vary.

Here are some thoughts to keep in mind as you build your data governance organization:

1. Build the business case and secure your funding.  Don’t forget this is a multi-year effort.  Several times, I’ve seen data governance teams and MDM initiatives that never thought beyond their first year’s funding, and who were surprised when funding for Year 2 and beyond wasn’t forthcoming.
2. You’ll probably need a multi-layer, cross-functional approach.  Start with a high level group that sets data policy, usually called a Data Governance Steering Committee or Data Governance Council (DGC).  Your executive sponsor(s) should be involved with the DGC. Then you’ll probably have a small group of data stewards which handle day-to-day data management, which reports to the Data Governance Council.
3. Focus on your data governance processes, including relevant metrics that show the impact to the business, and on communicating those metrics and their impact to the rest of the enterprise. Create a communications and education strategy to get the word out.
4. Look for IT people who “speak business” and business people who “speak IT”.  These liaison-type people are going to be crucial to your data governance organization.  Although I believe that data governance should be driven by the business, there’s no question that you’re going to need a lot of support from IT. So look for the “bi-lingual” folks in your company - they’re out there and they can be incredibly valuable in understanding the business impact of data issues, and in providing the technology support that the data stewards will need.
5. Don’t forget about metadata.  In your travels as a data governance organization, you’re going to run into a lot of metadata (information about information).  Develop a disciplined approach to cataloging and maintaining the metadata you come across.  In many ways, that will probably be one of the most useful outputs of your data governance initiative.

We’ll have more to say on data governance in this blog in the future, but let me leave you with one idea.  Instead of describing your MDM initiative as (hopefully) including data governance, think of it as a data governance initiative that has an MDM component.

You’ll be doing data governance in your enterprise long after your current MDM hub implementation is over.

If you don’t agree, or you have a perspective on data governance you’d like to share, please comment here on this blog.

digg this | del.icio.us | Reddit | Stumble It!

In a recent article, we covered the importance of data integration to Master Data Management (MDM).

This article elaborates on that in the context of today’s software market, and talks about how the visionary data integration vendors are scrambling to acquire and offer data quality as part of their integration suites.

Historically, data quality initiatives have been rolled out only on a project-by-project basis, while integration initiatives have been point-to-point application integration projects.

But more recently, as MDM programs have started gaining momentum, there is a need being felt in the marketplace for a single platform that can integrate data from multiple sources across the enterprise (web, data warehouse, ERP, CRM, legacy systems and systems from acquired companies) as well as run sophisticated data quality processes on these sources.

This requirement allows for a single administration console and metadata repository, as well as common transformations, user interface and a unified developer workbench, all on a single, combined integration and data quality platform.

This vision is consistent with moving towards a Service-Oriented Architecture as well, and is very conducive to providing a single environment for “data as a service” that is trusted and consistent across sources.

The customer tends to love it, since an integrated platform like this generally implies lower Total Cost of Ownership and smaller IT costs than standalone integration and data quality investments, and more rapid software development cycles.

The MDM vendor, if not providing such a platform itself, loves it since it can focus on what it does best, i.e. matching, merging and building data hierarchy.

Recognizing the above need, California-based Informatica Corporation acquired identity resolution vendor Identity Systems, while Massachusetts-based Trillium Software acquired address cleansing vendor Global Address. These are just two examples of recent data integration and data quality market convergence.

And the acquisition of Group 1 Software by Pitney Bowes provides more evidence of this shift.  In the process, niche data quality players are finding it more difficult to compete in such a dynamic marketplace.

If you’re considering acquiring a data quality tool for a corporate initiative, consider the above dynamics. And we’d love to hear your thoughts via comments on this article.

digg this | del.icio.us | Reddit | Stumble It!

I wrote an article appearing today on the Identity Resolution Daily blog.

Here’s a quick excerpt:

In a recent post, I wrote about the importance of data integration to Master Data Management (MDM). Today, I’d like to dig into the importance of identity resolution. The question of identity resolution, or matching, is critical to a successful MDM initiative. There’s a good article by Roger Wolter and Kirk Haselden of Microsoft which says “the matching accuracy of MDM tools is one of the most important purchase criteria.”

Click on “The Importance of Identity Resolution to MDM” to continue reading.

And please let me know what you think by commenting here …

digg this | del.icio.us | Reddit | Stumble It!

Here’s our latest “MDM Insights” column in the online edition of DM Review.

The technology industry is the mother lode of acronyms, and in the field of master data management (MDM), there are more than a few. We have the subsets of MDM known as customer data integration (CDI) and product information management (PIM). And in the closely related fields of middleware and integration, we’ve got enterprise information integration (EII); enterprise application integration (EAI); extract, transform and load (ETL); service-oriented architecture (SOA), among others. You get the point. Today, the acronym I’d like to focus on is business process management (BPM).

Click on “Master Data Management and Business Process Management” to continue reading.

And please let us know your thoughts by commenting here …

digg this | del.icio.us | Reddit | Stumble It!

We’ve discussed many other topics on this blog, such as data quality, best practices for Master Data Management, the five essential elements of MDM, and Master Data Management and the art of politics.

But one topic I don’t think we’ve given enough “airtime” to is integration. There are many different types of integration technology available today, and a veritable alphabet soup of acronyms to go with them.

There’s Extract-Transform-Load (ETL), Enterprise Application Integration (EAI), Enteprise Information Integration (EII) and Business Process Management (BPM).

In an upcoming piece in the online edition of DM Review magazine which I wrote last week, I go into more detail on the different types of integration and why I think Business Process Management offers some real advantages, due to the close fit with Service Oriented Architecture (SOA) and its flexibility and ability to model complex business processes that span multiple application silos.

But as I said, that’s a topic for another piece, and we’ll post a link to that here when it becomes available later this month.

In today’s piece, I want to urge MDM project team leaders, program managers, and “data champions” to think about the importance of integration itself, and the existence of certain typical requirements.

While integration doesn’t always have to be real-time, if you find yourself thinking solely in a batch-oriented mode, take a step back and ask yourself, “what will we be giving up by not providing for any real-time capabilities?”

And while straightforward, point-to-point XML data exchange may be all you need, ask yourself if you’ll be giving up anything important by not being able to model, deploy and manage business processes.

And try not to limit yourself by planning only for one-way integration into the hub. As hard as it is to convince the business owners to subscribe to the data quality improvements and external content that you typically do in an MDM hub project, you’ll make it much harder to achieve your expected ROI for the MDM initiative if the source system business owners don’t receive any of those improvements.

What are some of the integration “sticking points” you’ve run into on your MDM projects?

digg this | del.icio.us | Reddit | Stumble It!

TechTarget recently had an article that quoted heavily from my presentation at the recent Oracle Applications Users Group (OAUG) conference, COLLABORATE 08.

The article discusses the Master Data Management program at Tektronix, Inc. and segues into an overview of MDM based on my presentation at the conference.

Here’s the full link:

http://searchoracle.techtarget.com/news/article/0,289142,sid41_gci1312492,00.html

Please let us know what you think of the article by commenting.

digg this | del.icio.us | Reddit | Stumble It!

I’ve been building up a Favorites folder of web sites relating to Master Data Management for several years now, so I thought I’d share some of them with you today.

First, I should mention our own site, www.hubdesigns.com. It has some great MDM resources, and we’re updating the Publications page to include our MDM Summit, OAUG Collaborate 08 and DIG 2008 conference presentations.

Another site I find very helpful is www.dmreview.com. And my monthly column, “MDM Insights”, started in their Online Edition last month, discussing Master Data Management and Service-Oriented Architecture.

I’ve always found Aaron Zornes’ analysis of the MDM market and its players very insightful, and I still visit The MDM Institute and the MDM Summit sites pretty often, particularly their MDM product reviews.

On the topic of Data Governance, an area close to my heart, there’s a good blog called the Data Governance Blog, which includes a couple of useful definitions of Data Governance.

A blog I read regularly is Jill Dyché’s Blog. It’s written in a very engaging first person style, but still contains some great nuggets of wisdom about Customer Data Integration, Business Intelligence and Master Data Management.

Another blog that I like is by Steve Sarsfield at Trillium. It has some good content on Data Governance and Data Quality. You can find it at Data Quality Insider.

One of my favorite blog’s is written by Andy Hayler, an industry expert and the founder of Kalido, who has written quite a few good pieces on MDM at Andy on Enterprise Software.

I’ll continue this in “My Favorite MDM-Related Web Sites (Part 2)”.

digg this | del.icio.us | Reddit | Stumble It!

Here’s a great video that illustrates the difficulties in tracking and managing corporate hierarchies:

As the video shows, in today’s acquisition-heavy environment, it’s pretty tough to keep track of all the M&A activity in your customer base. As they say at the ballpark, “you can’t tell the players without a scorecard”.

Having worked for D&B in my previous life, I know they do a good job in maintaining the most important corporate hierarchies, with over 8.7 million company records linked to the appropriate family tree. D&B’s not perfect, but they do try pretty hard. If corporate hierarchies are important to your MDM initiative, think about including D&B early in your planning process.

And thanks to my friends Jack Dally from Transitions [2], and Mani Kumar Manda from Rhapsody Technologies for pointing me to this video.

digg this | del.icio.us | Reddit | Stumble It!

Here’s my new monthly column, “MDM Insights”, in the online edition of DM Review.

One of the many “hidden benefits” of a successful master data management (MDM) strategy is the contribution it can make to your enterprise’s move toward a service-oriented architecture (SOA).

Click on “Master Data Management and Service-Oriented Architecture” to continue reading.

Please let me know what you think with your comments here …

digg this | del.icio.us | Reddit | Stumble It!

I’m really looking forward to speaking at next week’s Decisions, Information and Governance conference in Las Vegas, sponsored by The Palladium Group.

And I spoke earlier this week at the New England Oracle SOA Users Group, talking about Master Data Management as a foundation for Service-Oriented Architecture.

MDM initiatives seem to be getting linked to Service-Oriented Architecture or to advanced analytics and business intelligence programs.

I think there can be a problem (but also an opportunity) for MDM in inserting itself between two things that used to talk directly from one to the other (an ERP system to a data warehouse) or (b) asserting itself as a predecessor task to ensure a better outcome (for example, when MDM is used to consolidate and improve the quality of enterprise data before people try to use it in analytics or business intelligence).

While I think it’s true that MDM is in fact needed at most large organizations, having to coordinate with an already-underway SOA initiative, or step back from a planned BI initiative and first tackle MDM, does complicate things a bit. So that’s the “problem” part.

The “opportunity” part is that, for organizations that have the foresight or the luck to tackle MDM first, it makes implementing SOA or achieving business intelligence success that much easier. There’s already a centralized repository of information on customers or products (or whatever domains have been mastered), and that information is proactively managed so that it’s trusted to be accurate, complete, timely and consistent.

Whichever situation your organization is in (tackling MDM first or building it into something else like SOA or advanced analytics), spend the time to develop a workable MDM strategy, using a holistic approach that addresses people, process, technology and information. By all means include an MDM hub in your planning, but make sure you also plan for business process management or sophisticated integration, as well as built-in or bolted-on data quality and enrichment capabilities. And be sure to build a data governance framework around your MDM initiative.

I’ll write a trip report after next week’s DIG conference, to let you know what I thought of the conference itself and whether I got lucky at the tables!