Data Integration in a Nutshell: Four Essential Guidelines

Dashboard Insight published a post by Philip Russom, senior manager of TDWI Research, called Data Integration in a Nutshell: Four Essential Guidelines, where he compiled a list of four points that keep coming up in conversations, interviews, and consulting about data integration. He thinks of these points as guidelines in a nutshell that can shape how people fundamentally think of DI, as well as how people measure the quality, modernity, and maintainability of DI solutions. He hopes these nutshell guidelines can help DI specialists and the people who work with them to see a more future-facing vision of what DI can and should be.

Guideline #1: Data integration is a family of techniques and best practices

The unfortunate knee-jerk reaction of many data warehouse professionals is that the term data integration is synonymous with ETL (extract, transform, and load) simply because ETL is the most common form of data integration found in data warehousing. However, there are other techniques (and best practices to go with them), including data federation, database replication, and data synchronization. Different techniques have different capabilities and prominent use cases, so it behooves a data integration specialist to know and apply them all.

Guideline #2: Data integration practices reach across both analytics and operations

In Analytic DI, one or more DI techniques are applied in the context of business intelligence (BI) or data warehousing (DW). Operational DI applies DI techniques outside BI/DW, typically for the migration or consolidation of operational databases, synchronizing operational databases, or exchanging data in a business-to-business context. Analytic DI and operational DI are both growing practice areas, and both are progressively staffed from a common competency center or similar organization.

Guideline #3: Data integration is an autonomous data management practice

In some old-fashioned organizations, DI is considered a mere subset of DW. It can be that, but it can also be independent. For example, the existence of operational DI proves DI’s independence from DW. Furthermore, hundreds of DI competency centers have sprung up in the last ten years or so as a shared-service organization for staffing all DI work -- not just DI for DW.

Guideline #4: A data integration solution should have architecture

After all, other types of IT solutions have architecture. DI architecture helps you with DI development standards, the reuse of DI objects, and the maintenance of solutions. The preferred architecture among integration technologies -- whether for data or application integration -- is the hub-and-spoke. For this reason, most DI tools today lend themselves to hub-and-spoke. However, there are many variations of it, so you need to actively design an architecture for your DI solutions.