13
Architect  ·  Module 13 of 14

Enterprise
Case Studies

Theory crystallises in the real world. This module walks through four enterprise-scale Data Vault implementations across Banking, Retail, Healthcare, and legacy migration scenarios — with the specific design decisions, regulatory constraints, and performance trade-offs that define architect-level thinking.

6 ConceptsCore topics
~70 minReading time
ArchitectLevel
Modules 01–12Prerequisites
Module Progress 0%

Concept 01

Banking & Finance Case Study

Scenario: Global Retail Bank — Regulatory Reporting Unification

A global retail bank operates across 22 countries with 14 core banking systems (Temenos, Flexcube, FIS), a separate CRM (Salesforce), a risk engine (Moody’s), and three legacy mainframe systems dating to the 1980s. The regulatory team produces BCBS 239, Basel III, and IFRS 9 reports — all requiring the same entity data reconciled across every source.

Problem: each reporting team maintained its own reconciliation queries. CUSTOMER_ID in the core banking system was a 12-digit numeric; in Salesforce it was an alphanumeric prefixed with “CRM—”; on the mainframe it was a 6-digit legacy code. Three different keys for the same person. Risk models and regulatory reports produced different customer counts depending on which source they used.

DV2 Solution: Same-As Link (LINK_CUSTOMER_SAME_AS)

A Same-As Link is a Data Vault pattern that records that two different business keys represent the same real-world entity. It links two Hub rows in the same Hub table — connecting the core banking CUSTOMER_HK to the Salesforce CUSTOMER_HK to the mainframe CUSTOMER_HK via a Golden Record satellite.

SQL — Same-As Link DDL (Banking MDM Pattern)
-- Same-As Link: two Hub rows that represent the same real-world customer
-- Used to reconcile customer IDs across 14 core banking systems
CREATE TABLE LINK_CUSTOMER_SAME_AS (
    LINK_CUSTOMER_SAME_AS_HK  CHAR(32)     NOT NULL,  -- hash of both HKs concatenated
    HUB_CUSTOMER_HK_MASTER    CHAR(32)     NOT NULL,  -- the surviving / golden record HK
    HUB_CUSTOMER_HK_DUPLICATE CHAR(32)     NOT NULL,  -- the matched / subordinate HK
    LOAD_DATE                  TIMESTAMP    NOT NULL,
    RECORD_SOURCE              VARCHAR(100) NOT NULL,  -- MDM system that determined match
    CONSTRAINT pk_lnk_cust_same PRIMARY KEY (LINK_CUSTOMER_SAME_AS_HK)
);

-- Satellite on Same-As Link: match quality metadata
CREATE TABLE SAT_L_CUSTOMER_SAME_AS_MATCH_DETAIL (
    LINK_CUSTOMER_SAME_AS_HK  CHAR(32)    NOT NULL,
    LOAD_DATE                  TIMESTAMP   NOT NULL,
    HASH_DIFF                  CHAR(32)    NOT NULL,
    RECORD_SOURCE              VARCHAR(100)NOT NULL,
    MATCH_SCORE                DECIMAL(5,4),           -- 0.0000 to 1.0000
    MATCH_METHOD               VARCHAR(50),            -- EXACT_SSN, FUZZY_NAME_DOB, etc.
    MATCH_CONFIDENCE           VARCHAR(20),            -- HIGH / MEDIUM / LOW
    REVIEWED_BY                VARCHAR(100),           -- human reviewer ID if manual
    REVIEW_DATE                TIMESTAMP,
    CONSTRAINT pk_sat_match PRIMARY KEY (LINK_CUSTOMER_SAME_AS_HK, LOAD_DATE)
);

Key Regulatory Outcomes

  • BCBS 239 compliance: single lineage-traceable customer entity across all 14 source systems. Regulators can trace any aggregate figure back to individual source rows.
  • Basel III LCR reporting: the Link between ACCOUNT and CUSTOMER with Effectivity Satellites enables a point-in-time view of which accounts were associated with which customers on any reporting date — critical for High-Quality Liquid Assets calculations.
  • GDPR right-to-erasure: because all customer attributes are in Satellites, a Data Erasure satellite can be inserted to flag that a customer’s PII must be masked in mart queries — without deleting any raw vault rows (preserving regulatory audit trail).
HUB CUSTOMER NATIONAL_ID CRM_ID CORE_BANKING_ID ACCOUNT ACCOUNT_NUMBER IBAN TRANSACTION TRANSACTION_REF CLEARING_ID SAT: KYC / AML / CreditRisk SAT: Balance / Status SAT: Amount / FX / Category SAME-AS LINK MDM reconciliation
💡

Architect lesson: in banking, the audit trail is a first-class regulatory requirement, not a nice-to-have. Data Vault’s insert-only design means every regulatory examiner question about historical data states can be answered by querying the Raw Vault directly. The vault IS the audit log — there is no separate audit table to maintain.

Concept 02

Retail & E-Commerce Case Study

Scenario: Global Fashion Retailer — 360° Customer & Inventory Platform

A global fashion retailer operates 450 physical stores, a global e-commerce site, a mobile app, and a wholesale division. Customer data arrives from: POS systems (in-store), Salesforce Commerce Cloud (online), a loyalty platform, a marketing attribution engine, and a returns management system. Each uses a different customer identifier. Inventory is tracked across 12 ERP instances by region.

Business problem: “What did customer X buy, return, and click on in the last 90 days, across all channels?” was a 4-day manual data exercise. Inventory analysts had no single view of stock levels across regions.

DV2 Design Decisions

  • Hub: CUSTOMER — business key is the email address (lowest common denominator across all source systems). CRM ID, loyalty card number, and device fingerprint are all business key candidates modelled as a cross-reference satellite.
  • Hub: PRODUCT — business key is the Global Trade Item Number (GTIN). Style code and SKU are alternate keys in a satellite. The same physical product sold in different channels has the same GTIN Hub row.
  • Hub: STORE and Hub: CHANNEL — separate Hubs for physical stores and digital channels. A Hierarchy Link connects STORE to REGION to COUNTRY for rollup reporting.
  • Link: CUSTOMER + PRODUCT + CHANNEL + DATE (Transactional Link) — every purchase event. The Link Satellite holds quantity, price, currency, promotion code, and payment method.
  • Effectivity Satellite on CUSTOMER-STORE Link — tracks which store a customer is “associated with” (e.g., their preferred store for loyalty purposes). Changes when they move or switch stores.
SQL — Retail: 90-Day Customer 360 Query via PIT + Bridge
-- Customer 360: all purchase and return events in the last 90 days
-- Bridge table pre-joins CUSTOMER to all related ORDER and RETURN links
SELECT
    c.EMAIL                         AS CUSTOMER_EMAIL,
    p.PRODUCT_NAME, p.GTIN,
    ch.CHANNEL_NAME,
    ev.EVENT_TYPE,                  -- PURCHASE or RETURN
    ev.EVENT_DATE,
    ev.QUANTITY, ev.NET_AMOUNT_USD

FROM  BIZ_VAULT.BRIDGE_CUSTOMER_EVENTS  br
JOIN  MART.DIM_CUSTOMER                 c   ON c.CUSTOMER_HK  = br.HUB_CUSTOMER_HK
JOIN  MART.DIM_PRODUCT                  p   ON p.PRODUCT_HK   = br.HUB_PRODUCT_HK
JOIN  MART.DIM_CHANNEL                  ch  ON ch.CHANNEL_HK  = br.HUB_CHANNEL_HK
JOIN  SAT_L_CUSTOMER_EVENTS_DETAIL      ev  ON ev.LINK_HK     = br.LINK_HK
                                                AND ev.LOAD_END_DATE IS NULL
WHERE c.EMAIL = 'jane.doe@email.com'
  AND ev.EVENT_DATE >= DATEADD('day', -90, CURRENT_DATE())
ORDER BY ev.EVENT_DATE DESC;

Outcomes

  • Customer 360 query latency reduced from 4 days (manual) to 2 seconds (pre-computed Bridge table).
  • Inventory visibility achieved across all 12 ERP instances via a single HUB_PRODUCT + SAT_PRODUCT_INVENTORY_POSITION satellite loaded from each region’s ERP independently.
  • A/B test attribution now uses Effectivity Satellites to track which experiment variant a customer was in during any specific date range — enabling accurate incremental revenue measurement.

Concept 03

Healthcare (HIPAA) Case Study

Scenario: Multi-Hospital Network — Clinical Data Integration

A 12-hospital network operates across 3 states with separate EHR systems (Epic, Cerner, Meditech). Patients frequently visit different hospitals in the network. Clinical outcomes research requires linking patient encounters, diagnoses, medications, and lab results across facilities — but HIPAA mandates strict data access controls and audit logging of every PHI access.

DV2 + HIPAA: The Data Vault Advantage

  • PHI isolation in dedicated Satellites. All Protected Health Information (name, SSN, date of birth, medical record number) is separated into SAT_PATIENT_PHI. All clinical data (diagnoses, medications, lab values) is in separate satellites. This enables column-level access control: clinical researchers get the clinical satellites without PHI, while identity resolution gets the PHI satellite without clinical payload.
  • Hash Key as de-identifier. The HUB_PATIENT_HK is the MD5 of the Master Patient Index number — a non-reversible identifier. Clinical research datasets use only the HK, never the source patient ID. Re-identification requires access to the HUB table (separately controlled).
  • Full audit trail of PHI access. An AUDIT_SATELLITE pattern records every query that touches PHI satellites, including the requesting user, timestamp, and purpose code. This satisfies HIPAA’s minimum-necessary-access audit requirement without additional tooling.
  • Right-to-access response time. When a patient requests their complete record under HIPAA right-of-access, a single query against all satellites for their HK retrieves the full history in seconds — previously a 3–5 day manual process across 3 EHR systems.
SQL — HIPAA: PHI-isolated Satellite with Row-Level Security
-- PHI Satellite: access controlled to identity team only
-- Clinical research views NEVER join to this table
CREATE TABLE SAT_PATIENT_PHI (
    HUB_PATIENT_HK  CHAR(32)     NOT NULL,
    LOAD_DATE        TIMESTAMP    NOT NULL,
    HASH_DIFF        CHAR(32)     NOT NULL,
    RECORD_SOURCE    VARCHAR(100) NOT NULL,
    -- PHI columns: role-secured; masked in non-prod via dynamic masking policy
    FIRST_NAME       VARCHAR(200),
    LAST_NAME        VARCHAR(200),
    DATE_OF_BIRTH    DATE,
    SSN_HASH         CHAR(64),   -- SHA-256 of SSN — not stored in plaintext
    MRN              VARCHAR(50), -- Medical Record Number per facility
    FACILITY_CODE    CHAR(10),
    CONSTRAINT pk_sat_phi PRIMARY KEY (HUB_PATIENT_HK, LOAD_DATE)
);
-- Snowflake Dynamic Data Masking: clinical role sees masked name
CREATE OR REPLACE MASKING POLICY phi_mask_policy AS
    (val VARCHAR) RETURNS VARCHAR ->
    CASE WHEN CURRENT_ROLE() IN ('PHI_ADMIN', 'IDENTITY_TEAM') THEN val
         ELSE '***MASKED***'
    END;
ALTER TABLE SAT_PATIENT_PHI MODIFY COLUMN FIRST_NAME
    SET MASKING POLICY phi_mask_policy;

-- Clinical satellite: no PHI — safe for research teams
CREATE TABLE SAT_PATIENT_CLINICAL_ENCOUNTER (
    HUB_PATIENT_HK   CHAR(32)     NOT NULL,
    LOAD_DATE         TIMESTAMP    NOT NULL,
    HASH_DIFF         CHAR(32)     NOT NULL,
    RECORD_SOURCE     VARCHAR(100) NOT NULL,
    ENCOUNTER_DATE    DATE         NOT NULL,
    ICD10_CODE        VARCHAR(20),  -- diagnosis code
    ENCOUNTER_TYPE    VARCHAR(50),  -- INPATIENT, OUTPATIENT, EMERGENCY
    FACILITY_CODE     CHAR(10),
    ATTENDING_PROVIDER_HK CHAR(32),-- FK to HUB_PROVIDER (de-identified)
    CONSTRAINT pk_sat_encounter PRIMARY KEY (HUB_PATIENT_HK, LOAD_DATE)
);

Concept 04

Legacy DW Migration Playbook

🧠 ELI5 — Explain Like I’m Five

Migrating from a legacy data warehouse to Data Vault is not a “big bang” cutover — it’s a parallel run with incremental cutover by subject area. You keep the old DW running for reports that haven’t migrated yet, while the new Data Vault starts ingesting the same sources and delivering mart outputs for migrated subject areas. This way, the business never sees a day with zero reporting.

The Six-Phase Migration Playbook

Phase Activity Duration (typical)
1. Discovery Profile source systems, map existing ETL logic, catalogue business rules in legacy transforms 4–6 weeks
2. Business key analysis Identify all business keys across sources; resolve duplicates; design Hub taxonomy 2–4 weeks
3. Raw Vault build & historical load Build Hub/Link/SAT DDL; ingest historical data from legacy sources (backfill) 6–12 weeks
4. Business Vault & mart build Build PIT/Bridge tables; rebuild first subject area as Information Mart 4–8 weeks
5. Parallel run & reconciliation Run legacy DW and new DV mart simultaneously; reconcile row counts and aggregate values daily 4–6 weeks
6. Cutover & decommission Switch BI tools to new mart; sunset legacy ETL; decommission old DW per subject area 2–4 weeks per area
⚠

The #1 migration failure mode: migrating business rules from ETL into the Raw Vault. Legacy ETL often contains decades of accumulated business logic (cleansing, derivations, source prioritisation) embedded in the transformation code. When migrating, document each business rule from the legacy ETL, then explicitly put it in either: (a) the Business Vault Computed Satellite layer, or (b) the Information Mart query. Never let it slip into the Raw Vault staging code.

SQL — Reconciliation Query: Legacy DW vs DV Mart (Daily Check)
-- Daily reconciliation: compare legacy DW FACT_SALES to new MART.FACT_SALES
-- Run as a pipeline check during parallel run phase
WITH legacy_totals AS (
    SELECT
        DATE_TRUNC('day', TRANSACTION_DATE)  AS SALE_DATE,
        COUNT(*)                               AS ROW_COUNT,
        SUM(NET_AMOUNT)                        AS TOTAL_SALES
    FROM LEGACY_DW.FACT_SALES
    WHERE TRANSACTION_DATE = CURRENT_DATE() - 1
    GROUP BY 1
),
dv_totals AS (
    SELECT
        DATE_TRUNC('day', ORDER_DATE)         AS SALE_DATE,
        COUNT(*)                               AS ROW_COUNT,
        SUM(NET_SALES_AMOUNT)                  AS TOTAL_SALES
    FROM MART.FACT_SALES
    WHERE ORDER_DATE = CURRENT_DATE() - 1
    GROUP BY 1
)
SELECT
    l.SALE_DATE,
    l.ROW_COUNT          AS LEGACY_ROWS,
    d.ROW_COUNT          AS DV_ROWS,
    l.ROW_COUNT - d.ROW_COUNT              AS ROW_DELTA,
    l.TOTAL_SALES        AS LEGACY_SALES,
    d.TOTAL_SALES        AS DV_SALES,
    l.TOTAL_SALES - d.TOTAL_SALES           AS SALES_DELTA,
    ABS(l.TOTAL_SALES - d.TOTAL_SALES) /
        NULLIF(l.TOTAL_SALES, 0) * 100       AS PCT_VARIANCE
FROM legacy_totals l
FULL OUTER JOIN dv_totals d USING (SALE_DATE)
WHERE ABS(l.TOTAL_SALES - d.TOTAL_SALES) / NULLIF(l.TOTAL_SALES, 0) * 100 > 0.01;
-- Alert if variance exceeds 0.01% — investigate root cause before cutover

Concept 05

Data Governance with Data Vault

🧠 ELI5 — Explain Like I’m Five

Data Governance answers: “Who owns this data? Where did it come from? What does it mean? Who can access it? How accurate is it?” Data Vault’s architecture makes governance easier than any other pattern because every row carries its provenance (RECORD_SOURCE + LOAD_DATE), nothing is ever deleted, and business rules are explicit and documented in the Business Vault layer — not hidden in opaque ETL scripts.

How Data Vault Supports Each Governance Domain

Governance Domain DV Mechanism
Data Lineage RECORD_SOURCE on every row traces data to its source system. Combined with LOAD_DATE, you can reconstruct exactly which extract batch a value came from.
Data Ownership Each Hub is owned by a domain team. Satellites are owned by the team responsible for the source system that feeds them. Bridge/PIT ownership lies with the Business Vault team.
Data Quality Error Mart (Concept 04 of Module 06) captures all DQ violations per source, per batch. DQ metrics are queryable, trendable, and SLA-able.
Data Retention / GDPR Erasure requests are handled by inserting a “tombstone” record in a dedicated Erasure Satellite. Mart queries filter this out. Raw Vault retains the technical audit trail without PII exposure in reporting.
Business Glossary Hub names and business key definitions should map 1:1 to the organisation’s data catalog. The Hub name IS the canonical business entity name.
Access Control PHI, PCI, or confidential satellites are secured at the table level. Mart views use dynamic data masking. Bridge tables control which entity combinations are visible per role.

GDPR Right to Erasure: The Tombstone Pattern

SQL — GDPR Erasure: Tombstone Satellite Insert + Mart Filter
-- GDPR erasure satellite: records that a subject has exercised right-to-erasure
-- Raw Vault rows are NEVER deleted — tombstone controls mart visibility
CREATE TABLE SAT_CUSTOMER_GDPR_ERASURE (
    HUB_CUSTOMER_HK  CHAR(32)     NOT NULL,
    LOAD_DATE        TIMESTAMP    NOT NULL,
    HASH_DIFF        CHAR(32)     NOT NULL,
    RECORD_SOURCE    VARCHAR(100) NOT NULL,
    ERASURE_DATE     TIMESTAMP    NOT NULL,
    ERASURE_REQUEST_ID VARCHAR(100),         -- ticket/case reference
    ERASURE_SCOPE    VARCHAR(50)  DEFAULT 'FULL',
    CONSTRAINT pk_sat_gdpr PRIMARY KEY (HUB_CUSTOMER_HK, LOAD_DATE)
);

-- Mart view: automatically excludes erased customers
CREATE OR REPLACE VIEW MART.DIM_CUSTOMER_GDPR_SAFE AS
SELECT c.*
FROM MART.DIM_CUSTOMER c
WHERE NOT EXISTS (
    SELECT 1 FROM SAT_CUSTOMER_GDPR_ERASURE er
    WHERE er.HUB_CUSTOMER_HK = c.HUB_CUSTOMER_HK
      AND er.LOAD_END_DATE IS NULL
      AND er.ERASURE_DATE <= CURRENT_TIMESTAMP()
);

Concept 06

Interview Q&A: Enterprise Scenarios

How does Data Vault handle GDPR right-to-erasure without violating the insert-only principle? ▶

GDPR requires that PII be erased upon a valid data subject request. Data Vault’s insert-only principle means we cannot DELETE or UPDATE existing rows. The reconciliation is done through the Tombstone pattern:

(1) Raw Vault preservation. The raw data rows are never deleted. They remain as an audit trail of what was held. This satisfies the regulatory need to prove you once held this data (some regulations require proof of deletion, not proof of never having held data).

(2) Erasure Satellite insert. A new row is inserted into SAT_CUSTOMER_GDPR_ERASURE marking the customer as erased with a timestamp and case reference. This is an INSERT, not a DELETE — preserving the DV principle.

(3) Mart-layer filtering. All Information Mart views and Data Products add a NOT EXISTS (SELECT 1 FROM SAT_CUSTOMER_GDPR_ERASURE WHERE ...) filter. Erased customers are invisible to all downstream consumers immediately after the tombstone insert.

(4) Physical deletion where required. For regulatory regimes that require physical deletion (not just visibility suppression), a separate GDPR purge process runs nightly to DELETE the PHI-containing satellite rows (SAT_CUSTOMER_PHI, SAT_CUSTOMER_CONTACT) for tombstoned customers, while leaving non-PHI satellites and Hub rows intact for referential integrity and aggregate audit purposes.

What is a Same-As Link and when do you use it instead of a surrogate master key? ▶

A Same-As Link is a self-referencing Link on a single Hub, recording that two different Hub rows represent the same real-world entity. It is used when the same entity appears in multiple source systems with different business keys and you cannot create a single canonical master key at load time.

When to use Same-As Link:

(1) Business key equivalence is discovered over time. You load 1M customer records from CRM and 900K from ERP. The matching process (MDM, probabilistic matching) runs separately and identifies that 850K CRM rows = 850K ERP rows for the same person. The Same-As Link records these matches without modifying any existing Hub rows.

(2) Match quality is uncertain and revocable. The Same-As Link Satellite carries a MATCH_SCORE and MATCH_METHOD. If a match is later found to be wrong, you insert a new EFF SAT row closing the Same-As relationship — the false match is preserved in history, the correction is applied going forward.

When to use a surrogate master key instead:

If you control the master key at load time — for example, an MDM system assigns a canonical GLOBAL_CUSTOMER_ID before data reaches the vault — use that as the Hub business key from day one. The Same-As Link is for post-hoc identity resolution when the master key is not available at source extract time.

What is the most important thing to document during a legacy DW migration, and why? ▶

The business rules embedded in the legacy ETL code.

Legacy ETL systems typically contain 10–20 years of accumulated transformations: source priority rules (“if CRM says X and ERP says Y, prefer CRM”), data cleansing logic (“if COUNTRY_CODE = ‘GB’, normalise to ‘GBR’”), imputation rules (“if customer birthdate is NULL, set to 1900-01-01”), and derived metrics (“LIFETIME_VALUE = sum of all orders minus returns”). These rules are often undocumented and only exist as SQL transformations in SSIS packages, Informatica mappings, or hand-written Python scripts.

If these rules are not extracted and explicitly placed in the new DV architecture, two things happen: (1) they are silently re-implemented in the Raw Vault staging code, violating the zero-business-rules principle, or (2) they are lost entirely, causing mart outputs to diverge from legacy DW outputs during reconciliation — with no clear explanation of why.

Document each business rule: what it does, which source column it transforms, why it exists (the business justification), and where it should live in the new architecture (Business Vault Computed Satellite or Information Mart). This documentation is also the basis for the reconciliation test suite used in the parallel run phase.

✅ Module 13 Key Takeaways

  • Banking: the Same-As Link resolves multi-system identity without modifying existing Hub rows. Regulatory requirements (BCBS 239, GDPR) are naturally satisfied by DV’s insert-only audit trail and RECORD_SOURCE lineage on every row.
  • Retail: the Bridge table pre-computes the Customer-to-Event relationship across channels, reducing a 4-day manual 360° query to a 2-second mart query. Effectivity Satellites track experiment cohort membership for accurate A/B attribution.
  • Healthcare: isolate PHI in dedicated Satellites with dynamic data masking. Use the Hash Key as the de-identifier for clinical research. The Satellite history enables HIPAA right-of-access responses in seconds, not days.
  • Legacy migration: use a 6-phase playbook with parallel run and daily reconciliation. The #1 failure mode is migrating business rules into the Raw Vault — document every ETL rule and explicitly assign it to Business Vault or mart layers.
  • Data Governance: DV provides natural lineage (RECORD_SOURCE + LOAD_DATE), ownership by domain (Hub = entity = owner team), and GDPR erasure via the Tombstone pattern — never deleting Raw Vault history, but filtering erased subjects from all mart outputs.