Paridity Documentation

Description	Verified data. Trusted outcomes. — CDC-Aware Reconciliation Platform
Copyright	Created by Evil Genius™ · © 2026 Evil Genius MindWorks LLC. All rights reserved. Customer Confidential.

Paridity Documentation¶

Paridity proves that data arriving in your analytical systems is complete and correct after replication — and produces evidence an auditor can check independently.

This site is the rendered form of the customer-facing documentation set. Every page here is generated from the same Markdown the engineering team maintains, so it cannot drift from the source it was written against.

Start here¶

If you want to…	Go to
Reach a first successful reconciliation	Getting started
Understand what the product does	Capability overview
Look up a term	Glossary
Find a specific setting	Configuration reference

The four sections¶

User guide — how to operate the platform, screen by screen: systems, replication profiles, mappings, field configuration, watermarks, schedules, runs and evidence.

Proof & credibility — how the platform's central claims are substantiated, and how to check them yourself rather than take them on trust: accuracy, performance, audit & compliance, architecture, how we compare, and a proof-of-value guide.

Deployment — what gets deployed into your cloud account, what your team owns versus what Paridity owns, and the prerequisites to collect before implementation begins.

Security & trust — the security whitepaper, data-flow descriptions, continuity posture, open-source bill of materials, and standing answers to vendor-risk questionnaires.

A note on honesty. These documents mark each capability with how you can reach it — UI, API, Ops (set at deployment), or Derived — and say plainly where the product does not yet have a control rather than implying a screen that does not exist. If you find a page claiming something the application does not do, that is a bug in the page and we want to hear about it.

Getting started

Getting started — your first reconciliation¶

Goal: from an empty tenant to a run that concludes pass_clean, with evidence you can inspect. Budget about 20 minutes.

This guide is deliberately concrete. If you are evaluating Paridity, doing this once teaches you more than reading the rest of the documentation.

Before you start¶

You need:

A role that can configure — platform_admin, tenant_admin, or operator. Auditors and analysts can see results but not create mappings.
Two systems to compare, and credentials for each stored in your secrets backend. Paridity never stores credentials; you give it a pointer.
Network reachability from the Paridity workers to both systems.
One table that exists on both sides. For a first run, pick the boring one: a modest row count, a single-column integer or string primary key, no BLOBs, and no rows changing while you test.

Most common first-run mistake: choosing a table that is actively being written. Paridity will correctly report a difference, and you will spend twenty minutes proving it was right. Start with something static.

The two paths¶

Open Setup → Onboarding Wizard (/onboarding). The first screen asks how you want to begin:

Choice	Use when
Start with tables	You know which tables you want compared. This is the normal evaluation path.
Start with a replication task	Your replication tool already defines the set of replicated tables, and you want Paridity to inherit that list. Requires a replication profile whose tool supports task discovery.

The rest of this guide follows Start with tables.

Step 1 — Systems (source and target)¶

Register the two systems you are comparing, or pick existing ones.

The important field is connection_ref, and it is worth understanding before you type anything: it is a URI pointer into your secrets backend, not a credential. Paridity resolves it at run time and never persists the secret.

aws-secrets://arn:aws:secretsmanager:us-east-1:123:secret/name
azure-keyvault://vault-name/secrets/secret-name
gcp-secretmanager://projects/proj/secrets/name/versions/latest
infisical://environment/path/SECRET_NAME
vault://secret/path/to/credential

Each system also declares a max concurrency, which acts as a hard ceiling: no mapping can ask that database for more parallel queries than you declared here, regardless of what the mapping says. See Systems and connections.

Use Test Connection before continuing. A failure here is far cheaper to diagnose than a failure three steps later.

Step 2 — Select tables¶

Paridity introspects the source and lists what it can see. Choose the table(s) you want compared.

System schemas (pg_catalog, information_schema, Oracle's SYS*, and so on) are hidden by default; there is a toggle to show them. For systems whose "schemas" are really user namespaces — Qlik spaces, object-store prefixes, folders — nothing is hidden, because those are your data.

Step 3 — Matches¶

Paridity proposes source→target pairs by name, using exact, case-insensitive, and prefix/suffix-stripped matching. Review these. Auto-matching is a convenience, not an authority — a wrong pair produces a confidently wrong comparison. Unmatched tables are listed so nothing disappears silently.

Step 4 — Configure¶

Field-level rules are inferred from column types: numeric scale from the column's scale, timestamp precision from its fractional digits, and BLOB/CLOB columns default to content-exact hashing.

For a first run, accept the defaults. When you later want to control comparison semantics per column, see Field configuration.

Step 5 — Preview¶

The proposals in full, before anything is created.

Paridity never auto-deploys a mapping. Auto-configuration produces proposals; a human confirms. This is deliberate — an incorrectly inferred key column silently changes what "matching" means.

Step 6 — Apply¶

Mappings are created in a single transaction: either all of them are created or none are. A partial batch would leave you guessing which half exists.

Step 7 — Schedule¶

Set how often the mapping runs. You can skip this for an evaluation and trigger runs manually.

Schedules are interval-based, not cron: an interval value and unit, optional days of the week, an optional daily time window, and optional calendar bounds. That shape exists because quarter-end and year-end scenarios are calendar-bounded, which cron expresses badly. See Schedules and close mode.

Run it¶

Go to Monitor → Runs (/runs) and trigger a run for your new mapping, or wait for the schedule.

What happens, in order¶

Readiness — if a replication profile is attached, Paridity checks the replication tool's state and the watermarks on both sides. If the target is still catching up, the run defers rather than comparing — this is the feature, not a delay.
Config audit — the resolved configuration is recorded with its checksum before any data is compared, so an auditor can prove which policy was in force.
Proof — row counts, then one aggregate fingerprint per side. If they match, the run concludes pass_clean and stops. No row data was moved.
Diff — only if proof failed: identify which rows differ (by per-row fingerprint), then which columns within them.

Reading the result¶

Status	Meaning
`pass_clean`	Source and target are identical for the compared window. This is the goal.
`no_new_data`	Nothing new since the last successful run — nothing to compare. Not a failure.
`fail_count`	Row counts differ. Usually missing or duplicated rows.
`fail_hash`	Counts match but the aggregate fingerprints differ — content differs somewhere.
`fail_field`	The diff phase identified specific columns that differ. This is the actionable one.
`pass_resolved_lag`	Initially mismatched rows resolved themselves on re-check - replication was simply behind, or one side had been read as of the readiness instant and both sides agree at latest.
`schema_drift_error`	A column changed in a way that halts comparison. Requires acknowledgement.
`system_error`	Something failed in the platform or a connector, not in your data.

The full vocabulary, including what to do about each, is in Runs.

If your first run fails¶

Work through these in order — the first three explain most first-run failures:

Is the table actively being written? Re-run against static data.
Is the window right? Without a watermark, Paridity compares a time window that may not mean what you expect on both sides. See Watermarks and windows.
Are the key columns correct? A wrong key makes unrelated rows look like mismatched pairs.
Open the evidence. fail_field tells you the exact column and both values. Frequently it is a type or precision difference — a timestamp stored at second precision on one side and microsecond on the other, or a numeric with different scale. Those are field configuration settings, not data defects.

What to look at next, as an evaluator¶

Trigger a deliberate failure. Change one value on the target and re-run. You should get fail_field naming that exact column, with both values shown (subject to your masking policy). That round trip is the product.
Look at the evidence trail. Evidence shows what an auditor would see, including the immutable comparison timestamp and the configuration checksum the run was pinned to.
Attach a replication profile and watch a run defer because the target is behind. See Replication profiles. This is the behaviour that distinguishes Paridity from a table-diff tool.
Read the configuration reference to see how much of the comparison you can control, and at which layer.

Paridity - Capability Overview¶

CDC-Aware Reconciliation Platform

Verified data. Trusted outcomes.

What Is Paridity¶

Paridity is a data reconciliation platform that proves information arrives complete and correct when a company copies it from its operational systems of record - SAP, Oracle, DB2, PostgreSQL, MySQL, SQL Server, Salesforce, and others - into the warehouses, lakehouses, applications, files, and AI-facing systems that depend on it, such as Snowflake, Databricks, BigQuery, or Redshift.

It acts as an independent checker on that replication process: rather than naively comparing tables and raising false alarms while data is still in transit, it lines both sides up to the same point in time, then creates a reliable fingerprint of each dataset and compares them. When the fingerprints match, the business has proof the copy is correct for that window; when they don't, Paridity drills down to pinpoint the exact rows and fields that differ, so issues can be fixed fast.

Running on demand or on schedule and built on a vendor-neutral, audit-grade foundation, it is engineered around four non-negotiable principles - accuracy (a false pass is treated as a compliance failure), efficiency (viable at hundreds of millions to billions of rows), visibility, and trust - so teams know the data behind their dashboards, reports, AI tools, and compliance processes is complete, current, and correct.

Paridity is the trust layer between the system of record and every copied version of that data.

Why It Matters¶

Most companies now run the business from copied data. Finance reports from the warehouse. Operations runs on replicated dashboards. Compliance relies on extracts and evidence. And increasingly, AI agents, MCP tools, and automated workflows act on data that has been moved out of the original system of record.

The catch is that data movement success is not the same as data correctness. A pipeline can finish, a dashboard can refresh, and an AI tool can confidently answer - while the target is quietly missing records, carrying extra ones, using stale values, or reflecting a schema change no one reviewed. Nothing fails loudly; the bad number simply flows downstream into a report, a filing, or an automated decision before anyone notices.

That gap creates real cost:

Silent errors are the dangerous kind. A pipeline that fails loudly gets fixed. One that quietly drops a fraction of its rows looks healthy on every monitor while steadily corrupting the numbers people trust.
AI raises the stakes. When agents and automation act on copied data at scale, a small replication error no longer causes one bad report - it propagates through every downstream output and decision. Verified data is the precondition for trustworthy AI.
Auditors require proof, not assurances. For SOX, ICFR, and similar controls, "we believe it's accurate" isn't enough. Paridity produces an immutable, audit-grade record that an external reviewer can independently verify.
Spot-checking doesn't scale. At hundreds of millions to billions of rows, manual sampling catches almost nothing. Paridity reconciles entire datasets, on demand or on schedule, so coverage is complete rather than anecdotal.
Trust, once lost, is expensive to rebuild. The first time a stakeholder catches a number that doesn't tie out, confidence in the whole data platform erodes - and teams start debating whose numbers are right instead of acting on them.

Paridity matters because it replaces all of that doubt with a simple, provable answer:

"Yes - this copied data matches the source." Or: "No, it does not - and here is exactly where it differs."

It changes the conversation from "the pipeline ran" to "the target data was proven correct, and the evidence is available."

Strategic Use Cases¶

Paridity isn't only a day-to-day operational check - it underwrites several initiatives the business treats as high-stakes. The same core capability (prove the target matches the source) becomes strategically valuable in these scenarios:

Cloud & data-platform migration. Moving off legacy systems (SAP, Oracle, DB2) into a modern platform (Snowflake, Databricks, BigQuery, Redshift) is one of the riskiest data projects a company undertakes - and "the load finished" is not proof the migration was faithful. Paridity validates that every dataset landed complete and correct before cutover, turning a high-anxiety go/no-go decision into an evidence-based one and giving leadership the confidence to retire the source.
Continuous replication assurance. Replication isn't a one-time event; pipelines run forever and drift over time as volumes grow, schemas change, and tools fall behind. Paridity monitors live CDC pipelines on an ongoing basis - with native state adapters for Debezium, Airbyte, Fivetran, Oracle GoldenGate, and Qlik Talend Cloud, and generic watermark-based readiness for other tools - catching silent loss, duplication, and staleness in steady-state production, before they reach a report.
AI & automation enablement. Before letting AI agents, MCP tools, and automated workflows act on copied data, the business needs assurance that data is trustworthy. Paridity provides the verification layer that makes scaling AI defensible - so automation acts on proven facts, not silently corrupted copies, and a single bad copy doesn't propagate through every downstream decision.
Financial close, compliance & audit. Quarter-end and year-end reporting depend on figures that must tie back to the system of record, and SOX/ICFR controls demand verifiable evidence that those controls actually ran. Paridity produces an immutable, audit-grade trail on demand - shortening close cycles, reducing audit prep, and giving external reviewers independent proof of data integrity at a point in time.
Legacy decommissioning & system consolidation. Retiring an old system or collapsing redundant ones is only safe once you can prove the surviving target holds a complete, correct copy of everything that mattered. Paridity provides that proof, removing the "are we sure we can turn it off?" risk that stalls consolidation programs and prolongs costly dual-running.

Benefits¶

Across these use cases, Paridity delivers a consistent set of benefits - the durable results an organization keeps no matter which scenario brought it in. Where a use case is the occasion, these are the returns:

User Trust. Data consumers can rely on the dashboards, reports, extracts, data products, and applications they use every day, because the critical data behind them is proactively proven against its source - not assumed to be correct.
AI, MCP & Agentic Readiness. AI tools, MCP servers, copilots, and automated actions operate on verified target data rather than unverified copies. Paridity is the proof layer that tells teams whether the data feeding their AI and automation actually matches the source - so a single bad copy can't quietly propagate through every downstream decision.
Compliance & Audit Evidence. Every reconciliation produces a reviewable record of what was checked, which configuration was used, what changed, who acted, and what result was produced. This immutable, audit-grade trail shortens close cycles, reduces audit preparation, and gives external reviewers independent proof that data controls actually ran.
Operational Confidence. Issues are surfaced proactively - the moment source and target diverge - and resolved fast, because Paridity pinpoints the exact rows and fields that differ. Teams spend less time hunting across pipelines and arguing over whose numbers are right, and more time acting on data they trust.
Reduced Risk Exposure. Errors are caught before they reach a filing, a financial close, a customer-facing system, or an automated decision - turning potential restatements, penalties, and rework into non-events.
Enterprise Scale & Consistency. The same proof of correctness applies across many systems, large tables, and complex, mixed data estates - with consistent rules, access, masking, and audit records across teams. One control spans the whole landscape instead of a patchwork of manual, per-platform checks.

Capabilities¶

Status definitions¶

Status	Meaning
Verified	Implemented and validated end-to-end (live integration and byte-identity proof against the reference engine).
Supported	Implemented and operational; broader live verification continues to expand.
Roadmap	Planned; adapter contract defined, not yet production-validated.
Evaluating	Candidate under consideration; no committed implementation.

Reconciliation & Data Trust - Verified¶

Source-to-target correctness proof; full-table, incremental, CDC-aware, and snapshot reconciliation; partitioned reconciliation for very large tables; proof-only checks; row-level mismatch detection; field-level difference detection; missing, extra, and changed-row detection; open-issue carry-forward; no-new-data handling; pass-after-in-flight-recheck handling.

Large & Complex Data - Verified¶

Time-based, key-range, list-value, and composite partition comparison; partition-level results, evidence, and concurrency controls (per-partition SQL scoping renders today for time partitions on PostgreSQL; other partition kinds are compared correctly but without per-partition scoping, and the product names that condition on the mapping rather than leaving it implied); source and target row filters; cross-name column mapping; BLOB, LOB, and binary handling; encrypted-field comparison; numeric-precision, timestamp and time-zone, and null and empty-string controls; diff sampling for large mismatch sets; SCD2 and bitemporal scoped comparison - an operator-declared, versioned current-rows scope on the history-bearing side (a flag column, a far-future end-date sentinel, or a NULL end date), never inferred, with per-run invariant checks that a scope's own premise holds (exactly one current row per business key; optionally no overlapping validity intervals). Scope rendering is live-verified on PostgreSQL and Snowflake; any other engine rejects the declaration rather than applying it on faith.

Business Windows & Replication Awareness - Verified¶

CDC readiness checks; watermark, control-marker, ingest-timestamp, and partition-max readiness; LSN and sequence-based windows; full and auto/incremental window modes; fixed-interval and watermark-range windows; in-flight recheck tracking; advisory and strict CDC modes; replication profiles with replication-tool context.

Schema-Change & Drift Control - Verified¶

Schema-drift detection; added and dropped-column detection; type, nullability, and collation-change detection where the engine exposes it; full physical schema baseline with a separate reconciled field set; advisory and halt-worthy drift handling; drift severity classification; drift-event evidence and status filtering; schema refresh and baseline self-healing; config version history; config checksums; mapping contracts and contract-drift checks; always-halt protection for the columns a declared SCD scope depends on - dropped or breaking-changed, they halt regardless of policy, because continuing would compare an unscoped superset.

Evidence, Audit & Compliance - Verified¶

Append-only evidence; proof, mismatch, field-difference, schema-drift, and in-flight recheck evidence; run-phase attempt records; watermark observation history; terminal-status audit logs; configuration audit records; signed evidence bundles verifiable offline with the shipped fail-closed verifier - asymmetric (AWS KMS, Azure Key Vault, GCP KMS, or local ECDSA) so a third party verifies with the public key alone, or HMAC with a customer-held key; signed period-export containers; audit-ready exports; raw-value and UI masking; field classification; actor and IP audit context; period evidence bundles persisted and downloadable through the product, optionally into the customer's own S3, Azure Blob or GCS bucket; write-once (WORM) anchoring of period containers, attested only when the store itself confirms immutability and verified by full read-back; evidence-bundle retention (bundle_retention_days) enforced by a built-in, default-on, dry-run-first purge schedule, with every deletion audited before it happens; approval-gated legal holds that freeze a tenant, mapping or bundle against that purge.

Security, Access & Governance - Verified¶

OIDC/JWT bearer authentication; role-based access control; scoped grants; business-domain scoping as an enforcement boundary (a domain-scoped principal can read or act on mappings, systems, schedules, replication profiles, runs, evidence and exports only inside its domain subtree; tenant-scoped roles are unaffected; identity and grant metadata has no domain owner and stays tenant-readable for the roles whose bundles include it); user, service-account, and API-key administration; tenant-aware authorization; sensitive-evidence access controls; session-expiry handling; SAML single sign-on - the interactive browser login - across Okta and Microsoft Entra ID (both live-verified); per-tenant OIDC bearer-token validation against a registered issuer, for API and service-account access (there is no OIDC click-to-sign-in path); SCIM user and group provisioning; SSO identity-link audit; access review; elevation requests and approvals.

Operations & Monitoring - Verified¶

Manual and scheduled runs; interval-based schedules with active-time windows, effective-date ranges, and day-of-week controls; schedule tags; bulk enable and disable; mapping groups with group health and run filtering; run cancellation; operator trigger notes; high-risk run acknowledgement; guarded destructive actions; notifications across Slack, email, webhook, PagerDuty, Teams, and ServiceNow; ticketing paths where configured.

Discovery, Onboarding & Setup - Verified¶

System and connector catalogs; connectivity testing; schema, object, column, key, and watermark-candidate discovery - schema discovery on every SQL-capable connector (eleven SQL engines, plus Qlik Cloud spaces), so the onboarding picker lists the schemas the connection can see rather than asking for free text; auto-configuration preview and apply; column-mapping suggestions with accept and dismiss workflows; guided onboarding; replicated-table discovery by profile.

Dual-Path Configuration - Verified¶

Two equally-supported configuration paths that converge on the same operator-reviewed result before anything is created. Table-first onboarding discovers source and target objects, matches pairs, introspects keys, types, and watermark candidates, and proposes mappings. Replication-task-first onboarding selects an existing task from any active replication tool that supports task discovery - Airbyte, Debezium, Fivetran, Oracle GoldenGate, and Qlik Talend Cloud today - and adopts the metadata that tool exposes as high-confidence defaults: table set, source-to-target table and column mappings, sync mode, key columns, watermark and cursor field, and column filters, to the depth each tool provides. Wizard mode choice with a dedicated replication-task step; one replication task maps to one mapping group; provenance tagging on every proposed default; cached task metadata for onboarding when the tool is briefly offline. Nothing is auto-deployed - operator review is always required and the replication tool is user-declared.

AI, MCP & Agent Readiness - Supported¶

Native read-only MCP server exposing triage, failed-run, and onboarding-match tools so external agents and copilots can query proof results directly; proof that AI-facing target data matches its source; run-triage copilot with evidence-referenced findings and specific next actions, advisory only (AI never auto-confirms operator actions); workflow explanations for run failure, auto-config and column matching, skipped objects, and watermark selection, plus plain-language risk narration; schema-validated, actionable output rather than free-form text; privacy by design - per-tenant object-name masking (absolute on prompts Paridity sends to a model provider; best-effort on the MCP surface, which is at parity with the REST API and where the customer's own agent and model are the disclosing party), prompt-context scrubbing, and data-residency provider restriction; configurable multi-provider AI with administrator approval; tenant AI settings; quota-aware usage metering; deterministic fallback when AI is unavailable.

Correctness itself is always deterministic - established by aligned time windows, dataset fingerprints, row-level hashes, and field-level diffs. AI assists with triage and explanation; it never determines whether data matches.

Commercial & Deployment Readiness - Verified¶

Connector-tier visibility; subscription status; usage reporting; Mappings Run per Month tracking; Active Mapping Cap limits; soft (meter-never-block) run-meter enforcement; customer-VPC deployment posture.

Commercial & Deployment Readiness - Roadmap¶

On-premises and air-gapped deployment (enterprise-custom). Single-tenant deployment entirely within the customer VPC is the architecture today - the whole platform runs in the customer's own cloud account.

System and Data Platform Coverage¶

System / Platform	Status
PostgreSQL	Verified
MySQL	Verified
SQL Server	Verified
Snowflake	Verified
Databricks	Verified
Oracle	Verified
Redshift	Verified
DB2 LUW	Verified
SAP HANA	Verified
BigQuery	Verified
Trino / Hive	Verified
ClickHouse	Verified
File datasets	Verified
Object stores (inventory-level)	Verified
DuckDB (reference engine)	Verified
Salesforce	Verified
Qlik Cloud (QVDs via QIX engine)	Verified
SAP application-level source	Roadmap
Kafka CDC signals	Roadmap
MongoDB	Evaluating
Iceberg / Delta / Hudi native pushdown	Evaluating
Additional SaaS connectors	Evaluating

Note: Kafka CDC is a replication-readiness signal, not a dataset-proof connector.

Replication Tool Coverage¶

Replication tool support is separate from system support. Paridity compares the data itself through its system connectors; replication tool adapters add extra context such as task health, latency, replicated tables, and pending changes.

Replication Tool	Status
Debezium	Verified
Airbyte	Verified
Fivetran	Verified
Oracle GoldenGate	Verified (MA REST 21c+ and Classic heartbeat modes; live-verified)
Qlik Talend Cloud	Verified (SaaS Data Integration REST; live-verified)
Native direct (no replication tool)	Verified
Custom replication profile	Verified
SAP SLT	Roadmap
SAP BODS	Roadmap
Qlik Replicate	Roadmap
Informatica CDC	Roadmap
AWS DMS	Evaluating
Google Datastream	Evaluating
Azure Data Factory	Evaluating
Microsoft Fabric	Evaluating
Kafka Connect	Evaluating
SQL Server Replication	Evaluating
SAP Datasphere	Evaluating
SAP HANA SDI	Evaluating
IBM InfoSphere	Evaluating
Striim	Evaluating
Fivetran HVR	Evaluating

Notes: Debezium, Airbyte, Fivetran, Oracle GoldenGate and Qlik Talend Cloud are production (Tier-1) integrations. Roadmap tools have a defined adapter contract and are pending design-partner validation. Evaluating tools have no committed implementation. Fivetran HVR is a separate product from Fivetran and remains Evaluating. Qlik Talend Cloud is the SaaS Data Integration product - distinct from Qlik Replicate, which remains Roadmap.

What Makes Paridity Different¶

Most data tools help move, observe, transform, catalog, or describe data. Paridity does something more fundamental: it proves whether the copied data is actually right.

That is a different category of trust.

A pipeline can say the job completed. A dashboard can show numbers. A catalog can describe a table. A monitoring tool can say systems look healthy. Paridity answers the question the business really cares about:

"Does the target data match the source?"

Existing Tool Type	What It Does	What Paridity Adds
Replication and CDC	Moves data	Independent correctness proof
ETL and ELT	Loads and transforms data	Source-to-target validation
Data quality	Tests rules	Equality proof against the source
Data observability	Watches health signals	Reconciliation evidence
BI review	Shows results	Proof that the underlying data matches
AI governance	Manages model behavior	Proof that agent-facing data is trusted
Manual SQL	One-off checks	Repeatable, scheduled, audited controls

And it answers the question the right way. Rather than naively comparing tables and raising false alarms while data is still in flight, Paridity aligns both sides to the same point in time first, then proves the match - so it flags real differences, not timing artifacts. Behind that sits an uncompromising standard: a false pass - telling you the data matches when it doesn't - is treated as a compliance failure, never an acceptable trade-off. That is what makes the proof trustworthy enough to stake an audit on.

This makes Paridity the independent proof layer for modern data operations. It sits after data movement and before business reliance, AI action, audit review, and executive decision-making.

And it is smart and safe by design. Paridity runs entirely inside your own cloud and stores no customer data of its own; it never holds a credential (connections are pointers resolved from your own secrets vault at run time); and every result is backed by a tamper-evident, hash-chained audit log and signed evidence bundles your auditors can verify offline with your own signing key - without trusting the platform's dashboards. Whole categories of vendor risk simply don't exist in its design - which is why a small, focused platform can sit next to your systems of record with confidence. (See the Trust & Security overview.)

Paridity addresses one of the most under-controlled risks in modern data operations: important decisions being made from unverified copies of source data. It brings together proof, investigation, schema-change awareness, evidence, security, automation, AI readiness, and compliance into one control plane - applied consistently across a mixed data estate, from operational systems of record to every modern warehouse and lakehouse. That combination is what makes it different.

It is not just another data tool. It is the system of proof for copied business data.

Paridity glossary & canonical terminology¶

The single source of truth for Paridity terminology. Every doc, tooltip, and screen uses these terms exactly. When a doc introduces one of these terms, it links here. The Paridity style guide (§3, internal: docs/governance/style_guide.md) enforces conformance. Getting metering terms wrong is a correctness defect, not a style choice — it misprices the product (#644).

Canonical commercial units (use these exactly)¶

These three are the highest-stakes terms in the product. The retired terms below were ambiguous and caused a mispricing defect.

Canonical term	Definition	Source of truth	Never write
Mapping	The configured source→target reconciliation unit: one source dataset compared to one target dataset under a resolved config. The fundamental object operators create, schedule, and run.	`cfg.mappings`	~~Validation Pair~~, ~~recon pair~~, ~~table pair~~
Mappings Run per Month	The headline usage meter: distinct Mappings run within a UTC calendar month. Drives the commercial tier. Meter-never-block — exceeding it never blocks a run; it meters for expansion.	`db/seeds/subscription_tiers.sql` `base_mappings_run_per_month`	~~Validation Execution~~, ~~validation run~~ (as a meter)
Active Mapping Cap	The entitlement limit: how many Mappings may be concurrently active (configured/enabled) on a tier. Hard on every tier that carries one — Trial, Team, Department, Division and Enterprise all enforce it identically; only Global Enterprise is uncapped. Enforced as published; once a tier's single growth addon is active the cap becomes the greater of the published cap and 1.5 × the effective run allowance, rounded up (the published cap is itself 1.5 × the base allowance; #3421). Blocks Mapping creation (HTTP 402 `MAPPING_LIMIT_REACHED`), never a run.	`db/seeds/subscription_tiers.sql` `max_active_mappings` + `entitlement_service.get_entitlements`	~~pair limit~~, ~~mapping limit~~, ~~seat cap~~

Important — "Mappings Run per Month" is a meter (measures usage, never blocks). "Active Mapping Cap" is a limit (bounds concurrent configuration). They are different dimensions; conflating or swapping them misprices a deal.

Retired terms — do not use anywhere: Validation Pair, Validation Execution, validation run (as a metered noun). Replace on sight with the canonical unit above.

Core reconciliation concepts¶

Term	Definition
Reconciliation	Proving a target dataset is complete and correct versus its source, within a deterministically aligned time window. Paridity is a replication correctness assurance platform, not a data-quality or observability tool.
Source	The system of record the data is copied from (e.g. SAP ECC on Oracle/DB2).
Target	The analytical/downstream system the data is copied to (e.g. Snowflake, Databricks, ADLS via Trino).
Proof phase	The fast O(1) pass/fail gate: compares aggregate dataset fingerprints (`hash_total`) plus row counts. Proves an entire dataset identical without pulling rows.
Diff phase	The detailed evidence phase that runs only when proof fails: identifies which rows differ (`row_hash`), then compares them column-by-column (`field_diff`).
Proof-only mode	Per-mapping/schedule mode that stops after the proof phase regardless of outcome — for cost/egress-sensitive high-frequency checks.
CDC-aware	Paridity aligns source and target to the same point in time using CDC tool state before comparing, avoiding false positives from in-flight replication. The category descriptor is CDC-Aware Reconciliation Platform (locked phrase).
Key identity (exact-v1)	The rule for when a source row and a target row are the same entity: their key values, as each database returns them, are identical. No case folding, accent folding, Unicode normalisation or trimming is applied to keys. `SMITH` and `smith` are two keys (one missing, one extra), not a match with a field difference. See the key identity contract.
Declared SCD scope	An operator-declared, versioned scope limiting the comparison to the current rows of an SCD2/bitemporal side. Never inferred; stored in the checksummed config snapshot; supported only where scope rendering is live-verified (PostgreSQL and Snowflake today). Watermark queries are never scoped by it.
SCD invariant	A per-run assertion that a declared scope's own premise holds on the scoped side: exactly one current row per business key (always checked) and, opt-in, no overlapping validity intervals. A violation is a qualification, never a new terminal status: the run keeps its data verdict, is reclassified SCD Invariant Violated, and is excluded from pass counts.

Fingerprints & hashing (never confuse these)¶

Term	Definition
`hash_total`	Aggregate dataset fingerprint (proof phase). Proves an entire dataset identical. A fast pass/fail signal — not a checksum.
`row_hash`	Per-row SHA-256 fingerprint (key-extraction phase). Identifies which specific rows differ.
`field_diff`	Column-by-column comparison output (diff phase). The actual reconciliation deliverable.
`config_checksum`	SHA-256 of the resolved config JSON. Detects policy changes between runs. Concerns config, not data.
`hash_method`	The named hashing algorithm used on a side (e.g. `xor_sum_sha256`). Source and target must match; if they differ, both fall back to the DuckDB reference implementation.
Canonicalization	Normalizing every column value to a fixed string form (NULL token, fixed-point numerics, UTC ISO timestamps, etc.) before hashing, so identical data hashes identically across engines.
Reference engine (DuckDB)	The ground-truth hash implementation. When engines can't match `hash_method`, comparison falls back to DuckDB.

Configuration & policy¶

Term	Definition
Config resolution (7 layers)	Config is resolved at run time from L1 platform defaults → L7 run overrides, via the `ConfigResolver`. Most settings: lower layer wins; special rules for `max_concurrency` (MIN), `store_raw_diff_values` (AND), `priority` (MAX), and `mask_in_ui` (field level only — there is no tenant-wide mask).
Resolved config snapshot	The frozen config a run uses, stored in `cfg.config_versions`. A run always uses the snapshot active when it started.
`store_raw_diff_values`	AND-chain across tenant (L2), mapping (L4), field (L5). Any `FALSE` suppresses raw diff storage and sets `values_masked`.
`mask_in_ui`	Field-level (L5) only. `TRUE` masks the field in the UI; a tenant-wide mask policy is not implemented.
Field Config	Per-field settings on a Mapping (`cfg.mapping_fields`): hash column order, numeric scale, timestamp precision, blob compare mode, decrypt expression, masking.
Replication Profile	First-class `cfg` entity capturing how data flows source→target via a specific replication tool. Sits between Systems and Mappings; provides default watermark templates and known-replicated-table lists. One per source+target+tool.
Replication tool	The CDC/replication technology (Debezium, Airbyte, SLT, GoldenGate, …). User-declared — the platform does not auto-detect it.

Windows, watermarks & readiness¶

Term	Definition
Watermark	The signal used to align source and target to the same point in time. Strategies in preference order: `control_marker` → `delta_version`/CDC offset → `ingest_ts` → `partition_max` → `timestamp` → `custom_sql` → `snapshot`.
`watermark_domain`	The comparability space of a watermark: `time_utc` (e.g. SCN, ingest_ts) or `sequence` (LSN, delta_version). Never mix domains within a Mapping.
Readiness	The pre-proof evaluation: CDC probe → watermark fetch → latency evaluation → stability → advancement. Determines whether a window is safe to compare.
In-flight recheck	In-workflow re-check loop for rows that differed only because replication was still in flight when they were compared (`lag_retry_pending` internally). Does not create a new run; `compare_started_at` never changes across rechecks.
`compare_started_at`	The immutable SOX/ICFR audit timestamp, set once when a run starts. Never updated - not on in-flight rechecks, not on reruns.
Window mode	`auto` (incremental window, default) or `full` (full-table comparison, e.g. quarter-end audits).

Evidence, audit & schema¶

Term	Definition
Evidence	Append-only records in the `ev` schema: `hash_total`, `row_hash`, `field_diff`, metrics, in-flight rechecks, anomalies, signed bundles. Immutable except retention purge and defined lifecycle exceptions.
Evidence bundle	A signed, content-hash-chained package of a run's evidence, verifiable by an external auditor.
Period bundle	A signed container over the individually signed per-run evidence bundles for one mapping across a date range. Persisted and downloadable through the product; optionally stored in the customer's own object store.
Write-once anchor	The attestation that a period bundle's container sits in a store that itself refuses deletion (S3 Object Lock, Azure Blob immutability, GCS retention lock). Written only when the store confirms protection, and verified by reading the object back in full.
Legal hold	A scoped freeze — tenant, mapping or single bundle — that makes the retention purge skip the held evidence. Approval-gated (`evidence.legal_hold`) and audited on placement, release, and every skip.
Schema drift	A change in either side's columns/types versus the config snapshot's `known_column_set`, detected at the start of each proof phase. Added columns are advisory and never halt. A lost or breaking-changed reconciled column follows `schema_drift_policy` (`halt` or `continue_excluding`; default `continue_excluding`) — a `continue_excluding` run records reduced coverage and can never display or count as a clean pass. Losing a key column, or a column a declared SCD scope depends on, always halts to `schema_drift_error` regardless of policy.
Config audit	The `ev.ev_config_audit` record (with `config_checksum`) written before proof begins, proving the config was stable during the run.
Audit log	`cfg.audit_log` — terminal status transitions, admin actions, and lifecycle events with actor, entity, action, IP, timestamp.

Access & tenancy¶

Term	Definition
RBAC roles	`platform_admin`, `tenant_admin`, `operator`, `dba`, `steward`, `analyst`, `auditor`, `viewer` — eight roles. `platform_admin`, `tenant_admin`, `operator` and `auditor` share a common read set (mappings, systems, runs, masked evidence, metrics, the audit log); `dba`, `steward`, `analyst` and `viewer` carry only their own narrower permission sets. Enforced in the API layer only.
Grant / scope	A scoped permission binding a principal to a set of resources (e.g. a business domain).
Tenant	An isolated customer boundary; row-level security (RLS) is live as defense-in-depth.
SSO / SCIM	Single sign-on and SCIM user/group provisioning. The interactive browser login is SAML (Okta and Microsoft Entra ID, both live-verified); OIDC is bearer-token validation against a registered per-tenant issuer — for API and service-account access, not a sign-in redirect.
Trust Center	The curated bundle of security, architecture, and compliance docs a customer's vendor-risk team consumes in one place.

Capability status vocabulary¶

Used in every capability/coverage table (style guide §5). Do not invent synonyms.

Status	Meaning
Verified	Implemented and validated end-to-end (live integration + byte-identity proof vs the reference engine).
Supported	Implemented and operational; broader live verification still expanding.
Roadmap	Planned; adapter/contract defined, not yet production-validated.
Evaluating	Candidate under consideration; no committed implementation.

Locked brand phrases (verbatim)¶

Verified data. Trusted outcomes. (tagline)
CDC-Aware Reconciliation Platform (category descriptor)
Evil Genius MindWorks LLC (legal entity — text only, where legally required)

See the Paridity brand kit and style guide (internal: docs/governance/brand/brand_kit.md, docs/governance/style_guide.md).

User guide

Paridity User Guide¶

What Paridity is¶

Paridity proves that data arriving in your analytical systems is complete and correct after replication — and produces the evidence an auditor can check independently.

It is not a data-quality tool and not an observability monitor. Those tell you about the data you have. Paridity answers a narrower, harder question: does the target match the source, right now, for a window both sides have finished delivering?

Three properties make that answer trustworthy:

It compares a deterministically aligned window. Most tools compare tables and then explain away the differences caused by replication still being in flight. Paridity uses the replication tool's own state to decide when a time window is safe to compare, and defers if it isn't. That is the difference between a real failure and a false alarm.
It proves equality before it explains inequality. A single aggregate fingerprint per side (hash_total) is a fast pass/fail. Only when that fails does the platform spend money identifying which rows differ, and then which columns within them.
It records immutable evidence. Every run pins the exact configuration it ran under, timestamps every phase, and writes append-only evidence. The comparison timestamp is never rewritten, even across retries.

If you are evaluating Paridity¶

Read in this order:

Getting started — from an empty tenant to a first successful reconciliation. Start here; it takes about 20 minutes and makes everything else concrete.
Glossary — the eight terms that carry the whole product. hash_total vs row_hash vs field_diff is the one distinction worth learning before anything else.
Configuration reference — every setting, which of the seven layers it lives at, how conflicts resolve, and whether you set it in the UI, the API, or at deployment.
The feature guide for whatever you are assessing.

A note on honesty. These guides mark each option with how you can reach it — UI, API, Ops (set at deployment), or Derived. Where the product does not yet have a control, the guide says so instead of implying a screen that does not exist. If you find a guide claiming something the application does not do, that is a bug in the guide and we want to hear about it.

The application, section by section¶

The left sidebar groups screens by what you are trying to do.

Monitor — is my data correct right now?¶

Screen	Path	What it answers
Dashboard	`/`	Overall reconciliation health across everything you own.
Mapping Performance	`/observability`	Throughput, duration and transfer-volume trends per mapping.
Runs	`/runs`	What executed, what it concluded, and why.
Anomalies	`/anomalies`	Statistical deviations the analyzers raised, tenant-wide; acknowledge and resolve.
Evidence	`/evidence`	The proof behind a conclusion — mismatched keys and field-level differences.
Recommendations	`/recommendations`	Suggested configuration and coverage improvements.

Setup — get started quickly¶

Screen	Path	What it does
Onboarding Wizard	`/onboarding`	Discovers tables, matches source to target, proposes mappings, and creates them after you confirm. The fastest path from nothing to a first run.

Configure — define what gets compared¶

Screen	Path	Guide
Systems	`/systems`	Systems and connections
Replication Profiles	`/replication-profiles`	Replication profiles
Mappings	`/mappings`	Mappings and Field configuration
Mapping Groups	`/mapping-groups`	Grouping for bulk operations.
Schedules	`/schedules`	Schedules and close mode
Notifications	`/notifications`	Where alerts go.

Governance — who did what, and under what authority¶

Screen	Path	Guide
Audit Log	`/audit`	Access, roles and audit
Elevations	`/elevations`	Break-glass requests and approvals.

Administration¶

Screen	Path	Notes
Users & Access	`/administration/users`	Access, roles and audit
Business Domains	`/administration/business-domains`	Organisational grouping for mappings.
Security Posture	`/administration/posture`	Access, roles and audit - every unsafe or unverified deployment setting, evaluated on any environment. `platform_admin` / `tenant_admin`.
Reconciliation Consumption	`/administration/cost-usage`	Reconciliation Consumption — provider units per run, mapping, system and period; no currency anywhere. `platform_admin` / `tenant_admin`.
Authentication	`/settings/authentication`	Identity provider configuration. `platform_admin` / `tenant_admin`; an OIDC provider's issuer, client secret reference and JWKS base URL are `platform_admin` only.
Subscription	`/settings/subscription`	Entitlements and tier.
Platform Defaults	`/settings/platform`	`platform_admin` only. See the configuration reference for what is settable here versus at deployment.
AI Settings	`/settings/ai`	`platform_admin` only.
Legal Holds	`/settings/legal-holds`	Freeze a tenant, mapping or bundle against the retention purge. Approval-gated and audited. See Evidence.
Retention Schedule	`/settings/retention`	The built-in retention purge: cadence, dry-run window, paused state, last fire — and where an admin pauses or resumes it.

Screens are role-gated: if a colleague cannot see something described here, the most likely reason is their role, not a missing feature. See Access, roles and audit.

Guides¶

Guide	Read it when you want to…
Getting started	Reach a first successful reconciliation.
Configuration reference	Understand any setting, its layer, and its resolution rule.
Systems and connections	Register a source or target and understand credential handling.
ClickHouse	Provision a ClickHouse read user, know the supported versions, and choose a read mode for ReplacingMergeTree and ClickPipes tables.
Replication profiles	Tell Paridity which replication tool moves your data, so readiness works.
Mappings	Define what is compared to what, and the run-level policy.
Field configuration	Control comparison semantics per column.
Key identity contract (exact-v1)	Know exactly when a source row and a target row are the same entity, and which engines have proven it live.
Watermarks and windows	Choose how incremental windows are bounded.
Lakehouse tables - Iceberg-aware reads	Reconcile Iceberg or Delta tables at a pinned commit through Trino, Databricks or Snowflake.
Delta Lake outside Databricks	Read a Delta table through Trino, Snowflake Delta Direct or the engine-less file dataset reader, and know each reader's pin and limits.
Schedules and close mode	Control when runs happen, including quarter-end.
Runs	Interpret a run's status and act on a failure.
Troubleshooting	Work back from a symptom - a held run, a refusal, a stale worker, a sign-out - to its cause, what to check, and when to report it.
Anomalies	Triage the deviations the analyzers raised; acknowledge and resolve.
Evidence	Read the proof, and export it for an auditor.
Access, roles and audit	Understand roles, grants, break-glass and the audit trail.
Report a problem	Send support a privacy-safe Diagnostic Bundle with your email, and know what it does and never contains.
Error codes	Every error code the API returns, with its HTTP status, message and what to do - generated from the code.
Known limitations	An index of the documented limitations, each linked to the guide section that describes it.
Glossary	Look up a term.
Float column handling	Handle floating-point columns correctly.
Repair posture	Answer "now fix it" — why Paridity generates remediation but never executes it, and where the evidence stops.

Systems and connections¶

Navigate: Configure → Systems (/systems).

A system is one database, warehouse, lake or application that Paridity reads from. Mappings compare an object in one system against an object in another.

Credentials never enter Paridity¶

connection_ref is a URI pointer into your secrets backend, resolved at run time by the worker. The platform stores the pointer, never the secret.

aws-secrets://arn:aws:secretsmanager:us-east-1:123:secret/name
azure-keyvault://vault-name/secrets/secret-name
gcp-secretmanager://projects/proj/secrets/name/versions/latest
infisical://environment/path/SECRET_NAME
vault://secret/path/to/credential

The form offers shortcut chips for the common schemes plus a dev-only env://. vault:// is accepted but has no chip — type it. A value that doesn't start with a supported scheme is rejected with an explicit list of what is allowed.

When you edit a system whose secret is already stored, the field is replaced by two choices — Leave unchanged or Replace with new value — so you never have to re-enter a pointer you aren't changing. Test entered credential (testing a replacement value before you save it) needs the right to change the system's connection reference (system.update, or system.create); without it the button is disabled, and Test connection on the stored value still works for anyone who may run discovery (discovery.execute).

Connectors¶

Choose a connector first; the rest of the form adapts to it. Cards are grouped as SQL Database, Data Warehouse, Lake / Files, Application and Streaming / CDC, with an Active only filter.

Status	Meaning
(no badge)	Production-ready.
Beta	Usable, with narrower verification. (None currently.)
Future Release	Visible but disabled — cannot be configured. SAP Application, Kafka CDC Stream.

Sixteen production connectors: Oracle, SQL Server, PostgreSQL, MySQL/MariaDB, IBM Db2 LUW, SAP HANA, Snowflake, Google BigQuery, Amazon Redshift, Databricks, Trino/Hive, ClickHouse, File Dataset, Qlik Cloud, Salesforce, Object Store (inventory-level — it verifies your landing zone's object inventory against a manifest or mirror; file contents are File Dataset's job). ClickHouse has its own page: ClickHouse (provisioning, versions, read modes).

Most offer a Deployment Flavor — for example Oracle covers on-prem, Amazon RDS, Exadata and RAC; PostgreSQL covers self-managed, RDS, Aurora, AlloyDB and Azure; File Dataset covers Parquet, CSV, JSONL, Avro, ORC, fixed-width and QVD. The flavor sets sensible defaults and, for some, changes the guidance shown.

Looking for Iceberg or Delta Lake? They are table formats, not systems, so they are not connector cards. The picker shows a Table formats row (and finds them when you search "iceberg" or "delta"): choose a format, then the engine that reads it - Trino, Databricks, Snowflake or File Dataset - and the normal system form opens for that engine. On a Trino system, Test Connection reports the catalog in play. See Lakehouse tables (Iceberg) and Delta Lake outside Databricks.

Configuring a system¶

Field	Notes
Name	Required, unique per tenant. Used in mappings and replication profiles.
Business Domain	Required when creating. Drives access scoping.
Description	Optional operator note. Not a secret.
Deployment Flavor	See above.
Connection Reference	The secrets pointer. Required.
Enabled	Edit mode only. A disabled system cannot be selected for new mappings.

Advanced¶

Field	Range	What it does
Max Concurrency	1–20 in the form (the platform supports up to 50)	A hard ceiling. No mapping can exceed this for this system, regardless of its own setting. The recommended value for the chosen connector is shown.
Query timeout (seconds)	10–3600	Overrides the platform default for this system.
Default source / target watermark type	free text	Pre-fills new mappings.
Default schema	text	Scopes discovery and browsing. Leave blank to expose everything.
Cost per TB scanned / egress (USD)	≥ 0	Feeds Cost & Usage. Scan bytes are priced at each system's own scan rate, and egress is priced per side: bytes pulled from the source are billed at the source system's egress rate, bytes pulled from the target at the target system's rate (#2413). Runs recorded before per-side egress accounting have only a combined byte count and stay priced at the source rate; the Cost & Usage page footnotes them when any fall in the window. Only engines that report bytes scanned (Snowflake, Databricks, BigQuery, file datasets, Qlik Cloud, Trino/Hive, and — harvested from the engine catalogue — Oracle, SQL Server, Redshift (#2456) and Db2 LUW (#2457)) produce scan volume; for every other engine, PostgreSQL included by design (#2457), Cost & Usage says not measured rather than $0. On the harvested engines the bytes are a calibration denominator, not a vendor-metered unit: they bill by instance/RPU/cluster hours or per core. Leave blank to inherit the tenant default.
Scheduled rate changes	edit mode	Set a future effective date for renegotiated rates. Historical cost is never rewritten.
Tags	key/value	Keys accept `[a-zA-Z0-9_:.-]`, 1–64 characters.

Connector-specific fields¶

Qlik Cloud — Scratch space (a shared space where the Paridity identity can create ephemeral reload apps; engine reads fail until it is set) and Reload timeout (30–3600s). Reload compute runs on your Qlik capacity, so keep concurrency at 2–4.
Oracle — Server OS timezone. Required for SCN-based watermark windowing when the server clock is not fixed UTC, because Oracle resolves SCN↔time in the server's frame. Validated live on connect.
PostgreSQL / MySQL — Transport encryption (TLS). Connections are encrypted by default and refused otherwise — there is no silent plaintext fallback. The default, Require encryption, negotiates TLS with no certificate setup needed; Verify CA / Verify identity additionally authenticate the server against a CA certificate (supply it via sslrootcert= in the DSN inside your secret for PostgreSQL, or the ssl_ca key in the secret for MySQL) and are recommended for any hop crossing a network segment you do not fully control. Disable is an explicit opt-out for a genuinely trusted, isolated segment: the form shows a warning, the change is recorded in the audit log, and every connection made with a weakened mode is logged. A directive inside the secret payload (sslmode/ssl_mode) — or, for PostgreSQL, embedded in the DSN — is also honored; the System-form setting wins when both are present. For MySQL the enforcement is proven server-side on every connect (the session's own negotiated cipher is checked), because the MySQL driver cannot refuse plaintext on its own. Note: MySQL 8 servers enable TLS out of the box; MariaDB does not — a MariaDB system needs TLS configured on the server or an explicit Disable decision.

Testing a connection¶

Test Connection appears on the edit form and reports success with a latency, or the failure reason - its first line, at most 200 characters (a timeout reads timeout); the full driver message is in the server log under the request's reference (X-Request-ID). When you are replacing a stored secret the button becomes Test Entered Credential, so you can validate the new pointer before saving.

For Qlik Cloud it additionally renders a space access matrix showing, per space, whether the identity is assigned (engine-readable) or merely visible — with remediation text. That distinction is the usual cause of "it lists but won't read".

Test Connection is available while creating a system too, so you can validate a secrets pointer before committing anything. Nothing is saved by the test — it resolves the pointer through your secrets backend, dials, and reports the result. Because it takes a pointer rather than credentials, it can only reach secrets your deployment already holds.

If a PostgreSQL or MySQL test fails mentioning TLS or SSL: the server could not provide an encrypted transport and Paridity refused to proceed unencrypted (the fail-closed default). Either enable SSL on the database server — the right fix for anything reaching production — or, if this hop is genuinely on a trusted, isolated network, set Transport encryption to Disable on the system. Paridity never falls back to an unencrypted connection on its own.

Browsing a system¶

The folder icon on any row opens Browse schemas & tables: schema counts, table and view counts, a filter, and per-table columns with types, nullability and detected primary/unique keys — with a suggested PK for reconciliation.

If nothing appears, that is expected for connectors with no schema concept (file datasets, object stores, Salesforce), or where credentials are unavailable in that environment. Trino does introspect — it lists the schemas and tables in the connection's catalog.

Deleting¶

A system cannot be deleted while anything still points at it — mappings or replication profiles. The message tells you how many of each, so you can clear them in one pass rather than discovering the second blocker after removing the first.

A delete that fails leaves the reason on screen and the button available, so you can fix the cause — usually a mapping still referencing the system — and retry without reopening the dialog.

ClickHouse¶

Paridity reads ClickHouse natively. The aggregate proof (hash_total) is computed inside ClickHouse, so only fingerprints leave the server; when the proof fails, Paridity extracts keys and row fingerprints, then fetches only the rows that differ. ClickHouse can be the source or the target of a mapping, paired with any other supported system.


Readiness	Active (since 2026-10-08)
Proof	Native SHA-256 pushdown with a real XOR aggregate (`xor_sum_sha256`), the same fingerprint every other engine computes
Supported servers	ClickHouse 25.8 LTS or newer
Tested servers	25.8.33.6 and 26.8.20.9, in continuous integration
Interface	HTTPS (port 8443) with certificate verification, by default
Driver	clickhouse-connect 1.10.0 (Apache-2.0), shipped in the worker image
Cross-engine pairs	Verified live against every other SQL engine Paridity supports

Supported versions¶

Paridity supports ClickHouse 25.8 LTS and newer. Continuous integration tests every change against 25.8.33.6 (the oldest supported version) and 26.8.20.9.

A server older than 25.8 is refused when Paridity connects, with an error that names the server version and the 25.8 floor. Paridity does not fall back to a slower comparison on an older server: the query settings Paridity relies on (see Why every query carries a SETTINGS clause) and several rendering functions differ or do not exist before 25.8, and a result that cannot be verified is not reported.

Provisioning the read user¶

Paridity connects as a dedicated user that can read and can change query settings, but cannot write.

Create the user with readonly = 2. readonly = 2 lets a query set its own settings and still refuses every write. readonly = 1 is not enough: Paridity sets settings on every query, so a readonly = 1 user fails at connect with a message telling you to use readonly = 2. readonly = 0 would allow writes; Paridity never needs them.
Grant read access to the databases you reconcile, and to the system and information-schema tables Paridity uses for introspection and discovery.
Use a settings profile with no row, byte or time caps for this user (for example max_rows_to_read, max_bytes_to_read, max_execution_time). Paridity pins every *_overflow_mode setting to throw, so a cap can never cut a result short silently. If a cap is reached, the run fails loudly (ClickHouse error code 158) rather than comparing a partial result. Paridity does not override a cap your administrators set; give the read user a profile without one.
Do not put a CONST constraint on any setting Paridity pins (the list is below). If a profile makes one of them CONST, ClickHouse refuses Paridity's query (error code 452, SETTING_CONSTRAINT_VIOLATION) and the connection test fails with that message. This is deliberate: the alternative is a query that runs with a setting that can change the answer.

The grant set that Paridity is tested with:

CREATE USER recon IDENTIFIED BY '<password>' SETTINGS readonly = 2;

GRANT SELECT ON <your_database>.* TO recon;      -- one line per reconciled database
GRANT SELECT ON system.* TO recon;               -- tables, columns, databases, settings
GRANT SELECT ON information_schema.* TO recon;
GRANT SELECT ON INFORMATION_SCHEMA.* TO recon;

A narrower system grant may work but is not tested.

The connection secret¶

Like every other system, the connection details live in your secrets manager; the system record holds only a pointer (connection_ref) to the secret. The secret is a JSON object:

Key	Required	Default	Meaning
`host`	Yes		Server host name
`port`	No	`8443` (or `8123` when `secure` is false)	HTTP(S) interface port
`database`	No	`default`	Default database for unqualified names
`user`	No	`default`	The read user (`username` is accepted too)
`password`	No	empty	The read user's password
`secure`	No	`true`	TLS. Set `false` only for a local test server; a plaintext connection is logged as a warning on every connect
`ca_cert_pem`	No	none (the worker's system trust store)	The CA certificate that signed the server's certificate, as PEM text (the `-----BEGIN CERTIFICATE-----` block itself, not a file path). Needed when the server uses a private or self-signed CA. Ignored when `secure` is false
`verify`	No	`true`	Certificate verification. Set `false` only for a local test server; the server's identity is then not checked, and a warning is logged on every connect. Prefer `ca_cert_pem`

Why every query carries a SETTINGS clause¶

Some ClickHouse settings change the answer a query returns without raising an error. During onboarding we reproduced each of these on a real server:

a user-profile limit = 1000 returned 1,000 of 50,025 rows with no error;
read_overflow_mode = 'break' with a row cap returned a partial count and a different fingerprint;
timeout_overflow_mode = 'break' returned a count of 0;
additional_table_filters silently filtered the table.

Setting the correct values through the driver is not enough: the driver drops a setting that a profile marks CONST without telling anyone, and reading the setting back (getSetting(), system.settings) reports the requested value even while the server is truncating. So Paridity appends an inline SETTINGS clause to every statement it sends. Inline settings are enforced by the server, and a constraint that forbids one makes the server refuse the query loudly instead of answering it differently.

The pinned settings:

Setting	Pinned to	Why
`session_timezone`	`'UTC'`	Timestamps render in UTC, and window bounds parse in UTC
`formatdatetime_f_prints_scale_number_of_digits`, `formatdatetime_f_prints_single_zero`	`0`	Fractional seconds render with a fixed width
`limit`, `offset`	`0`	No profile-level truncation
`read_overflow_mode`, `result_overflow_mode`, `timeout_overflow_mode`, `group_by_overflow_mode`, `sort_overflow_mode`, `join_overflow_mode`, `set_overflow_mode`, `distinct_overflow_mode`, `transfer_overflow_mode`	`'throw'`	A limit fails the query instead of returning part of the answer
`additional_table_filters`, `additional_result_filter`	empty	No hidden filters
`final`	`0`	`FINAL` comes only from the read mode you declare on the mapping
`do_not_merge_across_partitions_select_final`, `optimize_move_to_prewhere_if_final`	`0`	Both can make a `FINAL` read return row versions that should have been merged away
`use_query_cache`	`0`	A cached fingerprint would be a stale verdict
`apply_patch_parts`	`1`	Lightweight updates are visible to the comparison

Read modes: tables that hold unmerged row versions¶

ReplacingMergeTree and the other merge-dependent engines (Collapsing, VersionedCollapsing, Summing, Aggregating, including their Replicated and Shared forms) store an update as a new row version and remove the old version later, when parts merge. Until then a plain read sees both versions. A replica that is exactly right can therefore show more rows than the source and fail the comparison.

You tell Paridity how to read such a table with the mapping's Table Semantics setting (mapping form, Watermark & CDC step; API field cdc_table_semantics). It is set per mapping only and is never overridden by a schedule or a run.

Table Semantics	Use it when	What Paridity reads
None (default)	The ClickHouse table is a plain MergeTree, or a merge-dependent table you have fully merged (`OPTIMIZE ... FINAL`)	The table as stored
ClickHouse: read through FINAL (ReplacingMergeTree) (`replacing_merge_tree_final`)	The table is a ReplacingMergeTree (or another merge-dependent engine) loaded by your own pipeline	`FROM table FINAL` on every read
ClickHouse: ClickPipes / PeerDB (FINAL, excluding deleted rows) (`clickpipes_peerdb`)	The table is a ClickPipes (PeerDB) CDC destination: deletes arrive as new versions with `_peerdb_is_deleted` set	`FROM table FINAL` plus `_peerdb_is_deleted = 0` on every read

The two ClickHouse options appear only when one side of the mapping is a ClickHouse system; the API refuses them otherwise (HTTP 422, CDC_TABLE_SEMANTICS_SYSTEM_MISMATCH).

Why both parts matter for ClickPipes tables. On a ClickPipes-shaped table with 50,000 ids, 5,000 unmerged updates and 1,000 unmerged deletes, the true row count is 49,000. A plain read counts 56,000; FINAL alone counts 50,000 (deleted ids survive as their latest version); only FINAL with the delete filter gives 49,000, and its fingerprint equals an independent deduplication of the same table.

What Paridity checks. Before reading, Paridity confirms the table fits the mode you declared and refuses the run with the reason if it does not: a FINAL mode on a table whose engine is not merge-dependent, or clickpipes_peerdb on a table without the _peerdb_is_deleted column.

Suggestions, never automatic. When you onboard tables, the preview suggests clickpipes_peerdb for a ReplacingMergeTree that has the three _peerdb_* columns, and replacing_merge_tree_final for other merge-dependent tables. A suggestion is never applied unless you accept it.

Left at None on a merge-dependent table. The run proceeds and reads the table as stored. The mapping and the run narrative carry an advisory that unmerged versions are counted. The failure this can cause is a visible false failure (extra rows), never a false pass.

The advisory knows the table engine. Refresh Schema (and the first run of a mapping that has never been refreshed) records the ClickHouse table engine with the schema baseline. Once recorded, the advisory names the engine when it is merge-dependent ("The ClickHouse side is a ReplacingMergeTree table read as-is ..."), and is not shown at all for a plain MergeTree. Until it is recorded the advisory keeps its conditional wording and asks you to Refresh Schema. Paridity never queries ClickHouse to render the mapping page; the engine is read from the baseline that introspection wrote.

Cost. A FINAL read merges at query time. In our onboarding measurement it cost about 1.27 times a plain read on 2.2 million input rows (indicative, one test server).

Watermarks do not need FINAL. The newest value of the replication progress column (for ClickPipes, _peerdb_synced_at) over all row versions is the replication mark, merged or not, so watermark reads never use FINAL.

About ClickPipes and PeerDB. ClickPipes is ClickHouse's managed CDC service, built on the open-source PeerDB project. Paridity only reads the tables they write. PeerDB is licensed AGPL-3.0; Paridity never bundles, imports or links it.

Keys and watermarks¶

Keys. ClickHouse has no primary-key or unique constraint; ORDER BY does not guarantee uniqueness. Discovery proposes a key only for a Replacing- or Collapsing-family table whose sorting key is a list of plain columns (the deduplication key under FINAL). For a plain MergeTree it proposes none: set the key columns on the mapping yourself.

Key columns of these types are extracted natively: 8- to 64-bit integers, String, Decimal, Date, Date32, DateTime, DateTime64 up to microsecond precision, UUID and Bool. A key column of another type (for example a 128- or 256-bit integer, or DateTime64(9)) may be extracted through Paridity's reference engine instead: slower, equally exact.

Watermarks. Supported strategies: timestamp, ingest_ts, control_marker and custom_sql. Discovery suggests _peerdb_synced_at as an ingest_ts watermark (high confidence) and other DateTime columns as timestamp watermarks. It never suggests the virtual columns _part, _block_number or _part_data_version.

Known limits¶

Dates before 1900 on ClickHouse 25.8. On 25.8, Date32 and DateTime64 store only 1900-01-01 through 2299-12-31; an older value is silently stored as 1900-01-01 when it is written. (Date holds 1970-01-01 to 2149-06-06 and DateTime 1970 to 2106; values outside those ranges are also stored at the nearest end of the range.) If your source holds values outside the range of the ClickHouse column, the replica does not hold the source value, and Paridity reports that row as a real difference. That is correct: the data in ClickHouse is not the data in the source.
Types compared through the reference engine only. Array, Map, Tuple, Nested, JSON / Object, Variant, Dynamic, IPv4 / IPv6 and the aggregate-function types have no native rendering. A mapping that compares such a column takes the slower reference path for the whole comparison. No cross-engine verification has been recorded for these types; exclude such columns from the comparison, or verify them on your own data before relying on a pass. A container (Array, Map or Tuple) whose element is a Date or DateTime type is refused on the reference path as well: the read has no verified conversion for it, so the run fails with a clear error instead of comparing it. Exclude such a column.
Settings that fall back to the reference path. A rounding mode other than half-up, a numeric tolerance, a float compare mode, a canonical type override, timestamp rounding, a non-UTC timezone for naive timestamps, and transform chains other than null_to_empty, empty_to_null and decimal_separator_normalize are computed by the reference engine, not in ClickHouse.
Encrypted columns. An in-SQL decrypt_expression is not supported on ClickHouse; the run refuses it. Use a Python-side decryption algorithm (decrypt_algo) for the field instead.
Distributed tables under FINAL are not supported. For a Distributed table whose sharding key differs from its ORDER BY, FINAL merges each shard separately, so duplicate versions on different shards may survive. We have not tested this on a multi-shard cluster, so we document it as unsupported rather than claim either way.
TLS is verified with a private CA, not yet against ClickHouse Cloud. HTTPS with certificate verification on, against a server whose certificate is signed by a private CA supplied as ca_cert_pem, runs in continuous integration (step "ClickHouse gate (#2953) - TLS 8443 self-signed ENFORCED"), together with checks that a wrong CA, a plaintext port and a missing CA are each refused with a configuration error. Mutual TLS (client certificates) is not supported.
ClickHouse Cloud has not been verified live yet. The connector speaks the same HTTPS interface, but no end-to-end run against ClickHouse Cloud has been recorded.
Hierarchical narrowing (segment filters) is not available on ClickHouse in this version.
Strings must be valid UTF-8. A String column holding bytes that are not valid UTF-8 stops the run with an error naming the column. Set that field's compare mode to hash (compared as hex) to reconcile it.

Evidence¶

Every claim on this page traces to a recorded run:

Byte identity. ClickHouse's rendering of every value, its row fingerprint and its hash_total are proven identical to Paridity's reference implementation on 25.8.33.6 and 26.8.20.9, over the shared cross-engine test vectors, the hash_total vectors, and LOB values up to 1.5 MB. The same tests run in continuous integration on both servers, plus a check that a 22.8 server is refused.
End-to-end pairings. Live runs against every other SQL engine passed on matching data and failed on an injected difference, with the proof computed natively on both sides. The run ids are in the pairing matrix (docs/pairing_matrix.md, "ClickHouse").
TLS. The HTTPS transport with certificate verification connects and computes a hash_total identical to the reference over a private-CA server in continuous integration; a wrong CA, a plaintext port and a missing CA each fail with a configuration error.
Read modes. The ClickPipes-shaped pairing passes under clickpipes_peerdb, and the same table read with Table Semantics = None does not pass.

The full record, including each finding from onboarding and how it was checked, is in the verification record (cross_engine_hash_audit.md, Issues 62 to 65).

Reconciliation Consumption¶

Paridity is a consumption reporter. It records what your systems consumed while running Paridity's own comparison statements, in each provider's own units, and shows that consumption per run, per mapping, per system and per period. It never converts consumption to money: there is no currency, no rate, no cost estimate and no cost budget anywhere in the product. If you need a monetary figure, multiply the reported units by your own contract rate — the numbers here are the ones your provider meters, so that multiplication is yours to make and to check.

Governing design: docs/specs/plan_reconciliation_consumption_v1.md.

The units¶

Unit	Provider	What it is
bytes scanned	any engine that reports it	Bytes the engine read for Paridity's statements
TiB billed	BigQuery on-demand	The billed-bytes figure BigQuery records per job, in binary TiB (2^40 bytes)
slot-hours	BigQuery reserved capacity	Slot time BigQuery records per job
credits	Snowflake	Provider-reported execution time × the warehouse size's documented credits per hour
DBUs	Databricks	Allocated from provider-reported execution time × the SQL warehouse's DBUs per hour; warehouse startup and idle time are not included. Rates are verified for Azure, and for AWS serverless 2X-Small; any other combination shows Not estimated
bytes transferred	worker-side	Bytes Paridity pulled out of a system during fallback comparison, key extraction or field diff

Units are never added together. A page or card that lists several units lists each one on its own; there is no combined total, because a credit and a DBU and a byte are not the same thing.

What is recorded for a run¶

A run's ledger is in Evidence > Process steps, in the Resources used section at the bottom of the run's process-step timeline. It is collapsed until you open it, and nothing is fetched until then.

The section is grouped by side, then by phase. Each side's header names the system; a side whose engine has no provider meter (for example Trino, PostgreSQL or Redshift, which report bytes but no credits, DBUs or billed TiB) is labelled usage only, no provider meter. Under each phase (proof, field diff, in-flight recheck) the section lists the amounts in the provider's own units with the basis of each figure, and a query IDs disclosure that opens to the provider query or job IDs, in monospace with a copy button, so any figure can be checked against the provider's own query history. It also shows:

for a side with a credit or DBU amount, the compute arithmetic: the provider-reported execution time multiplied by the warehouse size's documented units per hour, so the figure can be checked by hand;
for a BigQuery side, billed bytes, slot time and whether the job ran on-demand or on reserved capacity;
Typical for this mapping: per unit, this run's figure beside the median (p50) and 95th percentile (p95) of comparable runs.

An amount the provider did not report is shown as unmeasured. It is never converted to zero. A measured zero remains zero, for example when a query result cache reports no scanned bytes. A metered side whose unit could not be calculated for this run lists the unit as unmeasured with the reason recorded.

The run detail page carries no consumption card. Bytes and rows are on its Data volume card; below it, a single Resources used line appears only when a provider-metered unit (credits, DBUs, billed TiB or slot-hours) has a value for the run, and links to the Evidence section with the run preselected. A run on engines without a provider meter shows no such line. The full ledger for the run, including bytes, is in the Evidence section.

Measurement support depends on the connector, query path, provider metadata and the permissions available to the connection. Read the section as the record of what was available for that run rather than as a promise of uniform measurement coverage. A phase whose provider telemetry was incomplete is reported as unmeasured for that unit rather than as a partial number.

The consumption record for a run is written once when the run reaches a terminal state and is append-only evidence. It is never recomputed.

Metered vs usage¶

Every unit above is one of two kinds, decided by what the engine declares it can report, never by a list kept in the page:

Metered - the engine reports a unit the provider itself bills or allocates: credits (Snowflake), DBUs (Databricks), TiB billed or slot-hours (BigQuery).
Usage only - the engine reports volume but no provider meter: bytes scanned, and bytes transferred by Paridity's own workers. Trino, PostgreSQL and Redshift are of this kind.

Self-hosted engines (for example PostgreSQL, MySQL, Oracle, SQL Server and DB2 LUW) have no provider meter at all, because there is no provider billing the statements. Their systems are labelled no provider meter and report usage only where the engine exposes it. A usage figure is real evidence of volume, but it is not a provider-billed amount and is never converted into one.

Reading the Reconciliation Consumption page¶

/administration/cost-usage is available to platform and tenant admins. It summarises compared runs - runs that actually compared data; runs that found no new data are excluded - for a chosen period:

Period	Window
Month to date	The calendar month so far
Last 30 days	Rolling 30 days ending now
Last 90 days	Rolling 90 days ending now

The page is system-first: one card per system, in that system's own units. By default only metered systems are listed. Turn on Show systems without a provider meter to add the usage-only systems, each labelled no provider meter. Nothing on the page is added across systems or across units, and there is no tenant-wide figure.

Each card shows:

one block per unit the system reports, with the period total and its coverage: how many compared runs recorded that unit and how many did not;
a trend per unit - one point per day (month to date, 30 days) or per ISO week (90 days). A bucket with no compared run, or with runs that did not record the unit, is a gap in the line, never a zero;
Top mappings on that system, ranked in the card's primary unit, with proof-only and full-window runs split out, and See the full ranking to page through every mapping;
a link to Recommendations, where the Cost advisor view lives.

In the mapping rows, not reported means the engine has no such unit at all; unmeasured means the engine could have reported it and this run did not. Neither is ever shown as zero. A row where some runs recorded the unit and some did not is marked partial.

The coverage line states how many compared runs recorded any consumption at all and names the engines that report none, so an estate whose engines report nothing is never read as consuming nothing.

The pre-run estimate¶

Before a single run, Paridity can show an estimate for that mapping. It uses comparable compared runs from recent history, matching full or automatic window mode and proof-only mode. For each unit it reports the median (p50) and the 95th percentile (p95) of those runs' consumption, with the number of comparable runs and how many of them were unmeasured for that unit. Duration, row and byte estimates use medians of the same history.

A unit with no measured history reads not measured. Paridity does not invent a figure. The estimate is for one run; the Schedules page multiplies the per-unit median by the schedule's runs per day to show a daily figure per unit, and the bulk-run dialog sums the per-unit medians across the selected mappings — each unit on its own, never across units.

The consumption advisor¶

The AI consumption advisor proposes changes (a lower cadence, proof-only mode, a narrower window) and states each proposal's monthly saving as a vector of units — for example ~3.6 credits, 360.0 GiB scanned/mo. Materiality is judged per unit against that unit's own monthly consumption. A saving is never expressed in a currency.

What Paridity does not do¶

It does not price consumption, store a rate, or accept a currency.
It does not enforce or warn on a cost budget.
It does not reproduce or reconcile a provider invoice. Consumption excludes startup and resume time, idle time, auto-suspend minimums, committed-capacity adjustments, account allowances, licensing and the effect of other workloads sharing the same warehouse or instance.
It does not project a period total or offer what-if modelling. Use the recorded history for external planning if you need those analyses.

Replication profiles¶

Navigate: Configure → Replication Profiles (/replication-profiles).

A replication profile tells Paridity which tool moves your data from a source system to a target system. That single fact is what turns a table-diff into a CDC-aware reconciliation: knowing the tool lets the platform read its state and refuse to compare a window replication hasn't finished delivering.

Without a profile, a mapping still works — but readiness short-circuits, CDC integration is forced to disabled, and latency/quiescence controls are unavailable. You are then comparing on trust rather than on evidence that the window is safe.

Which tools are usable today¶

The picker lists 22 tools. Be aware of what the badges mean, because most are not yet selectable.

Selectable now: Debezium, Airbyte, Fivetran, Oracle GoldenGate (Microservices REST 21c+, plus a Classic-architecture heartbeat-table mode), Qlik Talend Cloud (SaaS Data Integration — a distinct product from Qlik Replicate), Native / Direct Replication, Custom / Other.

Everything else shows "Future Release" and is disabled — including SAP SLT, SAP BODS, Qlik Replicate, Informatica PowerExchange CDC, IBM InfoSphere, Striim, SQL Server Transactional Replication, AWS DMS, Google Datastream, Azure Data Factory, Microsoft Fabric Mirroring, Kafka Connect and Fivetran HVR.

Fivetran HVR is a different product from Fivetran

Fivetran's managed SaaS connectors are supported. Fivetran HVR, the self-hosted high-volume replication product, is a separate entry in the picker and is not yet available.

If your stack uses one of the future-release tools, choose Custom / Other. The mapping works and comparison is fully accurate; what you lose is tool-aware readiness, so set max_lag_seconds and quiescence_seconds to match your pipeline's real behaviour. The UI warns you about exactly this when you pick Custom.

The legacy none value resolves automatically to Native / Direct.

Creating a profile¶

A profile is a route: one source system, one target system, one tool. The combination must be unique — creating a second profile for the same trio is rejected.

Field	Notes
Name	Optional but recommended, e.g. ECC FI/CO → Snowflake (SLT).
Business Domain	Required by the form.
Description	Optional.
Source System	Fixed after creation.
Target System	Fixed after creation. The source is excluded from the list.

Source, target and tool cannot be changed after creation. To re-route, create a new profile.

Debezium: replication lag (optional)¶

A Debezium profile's tool configuration has an optional Metrics endpoint: the Prometheus URL of the JMX exporter running on the Kafka Connect worker (for example http://kafka-connect:9404/metrics). When it is set, every readiness check reads the connector's streaming MilliSecondsBehindSource and the run's Replication tool line shows it as the tool's own lag. Leave it empty and Debezium lag stays unmeasured - shown as "did not report a lag figure", never as zero. Kafka Connect's own REST API does not carry a lag figure, so there is no other source for it.

The value must be an http:// or https:// URL; anything else is refused when you save.
The figure is the delay of the last change the connector processed. On a quiet source it keeps its last value until the next change arrives.
A connector that has not streamed a change since it started reports no figure yet (Debezium publishes -1 until then).
If the endpoint is down or slow (the read gives up after 2 seconds), the run records why and readiness carries on exactly as it would have.
The Kafka Connect credential is never sent to the metrics endpoint.

Debezium: sink connector (optional)¶

Connector name is the Debezium source connector, the one that captures changes into Kafka. A Kafka Connect sink connector usually applies those changes to the target, and readiness cannot see it unless you name it. Set the optional Sink connector name and readiness also checks that this sink connector is running: the tool's state reads as running only when both connectors are running with no failed task.

Leave it empty and only the source connector is checked, exactly as before.
When set, it must be a connector name; a blank value is refused when you save.
If the sink is paused, stopped or failed, the run's readiness record says the sink connector was the cause. Connector names are not written to the run's evidence - only which side it was and its state.
What a not-running connector does to a run depends on the mapping:
- Snapshot mappings (no watermark) are held - readiness status Replication not running - and re-checked on the normal readiness backoff, because a snapshot comparison has nothing else to show whether the target is up to date. This applies in advisory CDC mode too.
- Mappings with a watermark follow the mapping's CDC mode as before: advisory carries on with the watermark latency check, strict stops.
- A tool that cannot be reached at all (timeout, connection refused) is not a definite answer: advisory carries on and the run is marked degraded; strict stops.

Watermark defaults¶

These pre-populate new mappings created under the profile; every mapping can override them.

Field	Options
Default Watermark Strategy	`control_marker (preferred)`, `delta_version (CDC offset)`, `ingest_ts (load completion)`, `partition_max (ADLS/Hive)`, `timestamp (max field)`, `custom (last resort)`
Watermark Domain	`time_utc` or `sequence`

The form pre-selects what the chosen tool actually produces — Debezium defaults to delta_version in the sequence domain, Airbyte to ingest_ts in time_utc — and shows the recommendation inline.

See Watermarks and windows for what each strategy means.

Comparison defaults¶

These are inherited by every field under the profile unless a field overrides them. They exist to declare how your replication pipeline behaves, so Paridity canonicalizes both sides the same way.

Field	Purpose
Default Numeric Rounding Mode	How the pipeline rounds when decimal scale is reduced. Applies to DECIMAL/NUMERIC only. Unset behaves as half-away-from-zero.
Default Timestamp Rounding Mode	How the pipeline rounds when sub-second precision is reduced — e.g. a microsecond source loaded into `DATETIME2(3)`. Unset behaves as truncate.
Default Dual Compare Mode	How Qlik dual values (numeric + display text) are compared. Only affects QVD file datasets.

Selecting "None / inherit …" clears a saved value and returns the setting to inheriting. Picking the explicit equivalent (Half up …, Truncate (default), Automatic (default)) instead stores that concrete value.

Replicated tables¶

On an existing profile, Refresh asks the tool which tables it is actively replicating. Auto-configuration can then propose mappings only for tables the tool genuinely covers.

Not every tool supports this. When it doesn't, you get a clear message that live discovery isn't available for that tool and that the profile is otherwise fully functional — only automated listing is missing.

Associated mappings¶

A read-only list of mappings using this profile, each showing whether it is enabled. Useful before changing defaults.

Changing the business domain¶

The Business Domain can be changed on an existing profile. Because the domain controls who can see the profile, you can only move it to a domain you also have access to — an attempt to move it outside your scope is refused, and the change is written to the audit log.

Mappings¶

Navigate: Configure → Mappings (/mappings) → New Mapping, or open a mapping and choose Edit Settings.

A mapping is one source object compared against one target object, plus the policy that governs the comparison. It is the central object in Paridity.

The four steps¶

Step	Covers
1. Source & Target	Systems, objects, key columns, replication profile
2. Column Mapping	Source↔target column pairing (creation only)
3. Watermark & CDC	See Watermarks and windows
4. Advanced	Filters, compute, diff limits, evidence, drift, ownership, partitioning

Step 2 is hidden when editing, so the sidebar reads 1, 3, 4. Column mappings for an existing mapping are edited in the dedicated column-mapping editor.

Step 1 — Source & Target¶

Field	Notes
Name	Required, unique per tenant.
Source / Target System	Fixed after creation.
Source / Target Table	Type a reference or pick from discovery. File-based systems use a path (`bucket/path/file.parquet`); everything else uses `SCHEMA.TABLE`. Fixed after creation — shown read-only when editing.
Key Columns	Comma-separated. Creation only. Auto-filled from the detected primary key — the label turns green to say so, and clears the moment you type.
Replication Profile	Optional but strongly recommended. Only profiles whose route matches your two systems are listed. Changeable after creation — see below.

Why the tables and key columns are fixed. They define what the mapping is and what its comparison covers. A mapping keeps its identity, its history and its open mismatch keys across an edit, so repointing it at a different table would leave every earlier run's evidence describing a dataset the mapping no longer reads. To change them, create a new mapping. To pick up added or changed columns on the same tables, use Refresh Schema on the mapping detail page. Refresh Schema only applies a schema it could actually read from both systems: if either side cannot be reached, the table is missing, or the connection lacks catalog permissions, the refresh is refused with a message naming the side and the reason, and your field configuration is left exactly as it was. Fix the connection or object reference and refresh again.

Changing the replication profile. You can move a mapping to a different profile, or clear the profile entirely, from this step — and the change now takes effect (before #2217 the form reported success and the profile never moved). Only profiles connecting the same two systems are accepted. If your watermark settings are still exactly the ones the old profile's template supplied, they follow the new profile; anything you changed yourself is left alone. Clearing the profile is refused while latency or quiescence controls are still set — clear them in the same save.

Selecting a source table pre-populates step 3. It sets the watermark strategy and field on both sides, and for file-based sources it also guesses a manifest path. Once a watermark field is set, changing the table again leaves your configuration alone — auto-fill only runs while the field is empty. Check step 3 before saving regardless: the guess is a starting point, not a decision.

The object picker tells you whether discovery succeeded — "N objects discovered" — or that the connector was unreachable and you should type the reference manually. An unreachable connector does not block mapping creation.

Step 4 — Advanced¶

Filters¶

Source / Target filter SQL are WHERE-clause fragments applied before comparison. Omit the WHERE keyword itself.

They are deliberately restricted: no statement terminators, no comments, no DDL/DML keywords, 2000 characters maximum. The form shows a soft warning for suspicious patterns; the server enforces the rules and rejects violations with a specific reason.

SCD2 / bitemporal tables — declared scope¶

An SCD2 (slowly-changing-dimension) or bitemporal table legitimately keeps closed history rows the other side no longer shows. Compared whole, such a pair fails forever on data that is actually correct. Declare SCD scope (compare current rows only) — in the mapping editor's Advanced step — scopes the comparison to the current rows of the side that holds history:

Scoped side — which side (source or target) holds the history.
How current rows are marked — one of three typed markers: a flag column equal to a value (current_flag = 'Y'), an end-date column equal to a far-future sentinel (valid_to = 9999-12-31), or an end-date column that is NULL while the row is current. You pick columns and values — you never write SQL, and the platform never infers any of this: an undeclared scope is a false-pass machine, so the scope is always an explicit, versioned, checksummed declaration.
Validity interval columns (optional) — valid_from / valid_to, required if you opt into the interval-overlap check below.

The SCD bookkeeping columns (flag, validity dates, version numbers) are simply not mapped as comparison fields — the compare covers the business columns you map, and the scope decides which physical rows carry them. A scope column cannot also be a mapped comparison field; the editor blocks that combination.

Every run under a declared scope also checks the scope's own premise (invariants, evaluated on the scoped side before the proof):

Exactly one current row per business key — always on; a scope with duplicate current rows has no honest row identity.
No overlapping validity intervals (opt-in, needs the interval columns; costs an extra self-join per run) — catches history corruption even when the current rows still match.

A violated invariant never silently passes or fails the run: the run keeps its data verdict but is reclassified SCD invariant violated, excluded from pass counts, and fires a warning notification — see Runs.

Runs under a declared scope carry a “Scoped: SCD current rows” badge on the run detail and evidence pages: a scoped pass proves the current rows, not the whole table, and the display never hides that. Scope declarations are supported on live-verified engines only (currently Postgres and Snowflake); declaring a scope on any other engine is rejected outright rather than applied on faith.

One deliberate asymmetry: the scope applies to the comparison, never to watermark queries — see Watermarks for why.

Compute¶

Field	Range	Notes
Compute Mode	`pushdown_only`, `hybrid`, `in_memory_only`	Where SQL executes. `hybrid` falls back to the reference implementation for unsupported operations; `in_memory_only` is the most portable and the slowest.
Max Concurrency	1–50	Capped by each system's own ceiling.
Max Diff Rows	≥ 1000	Cap on rows handed to the diff engine.
Query Timeout	10–3600s	Overrides both systems for this mapping. Size it for the proof query at full volume.
Max Partition Concurrency	1–20	Parallel partition workers.

Diff safety limits¶

Caps on how much diff evidence one run may produce. Each resolves as the minimum across platform, tenant, mapping and schedule — a run override can lower them but never raise them.

Egress Row Limit (rows pulled back for diffing — the main egress control) · Per-Column Diff Limit · Global Diff Cell Limit · Fail-Fast Threshold (%) (skip the diff phase entirely when this share of rows mismatch — the run is already conclusive) · Diff Truncation Method (First N, Random sample, Stratified; schedules and runs may only choose the first two).

Evidence¶

Coverage mode — full field diff, limit to N rows, or a percentage. Sampled diffs are labelled is_sampled = true, and count and hash proof stay exhaustive regardless: sampling reduces the evidence you keep, never the rigour of the pass/fail decision.

Store Raw Diff Values is an AND-chain across tenant, mapping and field — any FALSE suppresses storage and sets values_masked. Retention for summaries, mismatch keys and field diffs cannot go below the tenant minimums.

Schema drift¶

Fail on Schema Drift, plus a Lost Reconciled Column policy (halt, or continue_excluding to drop the column and keep going) and separate handling for additive and compatible drift (warn / halt / auto-accept).

Losing a key column always halts, whatever the policy says. Without the key there is no row identity to reconcile on, so the run stops rather than comparing what is left.

Other breaking changes — type narrowing, NOT NULL added, a collation change — are classified as breaking and then follow the policy. Under continue_excluding the affected column is dropped from the comparison and the run is reported as Schema drift — Available data passed: it is counted as an issue, never as a clean pass, so reduced coverage is always visible.

Unverifiable Schema policy (schema_unverified_policy; API-configurable, UI pending): what happens when the platform cannot verify a side's schema at run time — introspection failed (permission loss, catalog error) or the engine cannot introspect at all.

proceed (default): the data is still compared — but the run displays Pass · Schema unverified (amber; or the calmer Schema not verifiable for engines declared non-introspecting), records which side(s) were unverified and why, and is excluded from pass counts.
halt: the run stops (Schema unverified · Not compared) rather than comparing against a schema that could not be verified — for high-assurance mappings.

The policy only controls the halt. The labeling, the evidence record, and the pass-count exclusion are absolute — no setting renders an unverified schema as verified. A tenant-level default exists in the database only (no API/UI yet), like the tenant term of the drift policy.

Ownership, close mode, partitioning¶

Data steward / technical owner / business owner / DBA group and business criticality (Regulatory, Business Critical, Operational, Dev / Test). Close Mode Eligible and Tags control inclusion in bulk close-mode toggles. Auto-ticket opens a ServiceNow or Jira ticket on terminal failure, with a deep link to the signed evidence bundle. Switching auto-ticket off also un-assigns the ticket channel, and a saved edit that clears the channel now takes effect (before #2208 the form reported success and the channel stayed assigned). A mapping cannot auto-ticket to no channel: leaving the toggle on with no channel selected is rejected on save. You pick the ticket channel by name from your enabled ServiceNow and Jira channels. If none exist yet, the form says so and links to creating one. Disabled channels are listed by name so you can see why they are missing from the choices. A saved channel that has since been disabled or deleted is flagged, because no ticket opens until you enable it or pick another.

Partition Strategy splits a large window into independently proven and diffed chunks: Time (with a grain and lookback), Key Range, List, or Composite (a two-dimensional grid). A live estimate warns when the count is large; above 256 it suggests a coarser granularity. Per-partition SQL scoping is decided per side and per (engine, partition kind, column-type family) cell — see the live-verified matrix in the configuration reference. On a cell that is not live-verified the partition is still proven and diffed correctly — coverage is unaffected — but its proof runs over the whole window on that side rather than its own slice, so the speed benefit you configured is not realised there. The mapping detail page shows the pre-run prediction per component and side (a time partition under a non-UTC timezone reads "classified at run time" until a run records its frame), and the run page shows what each run actually recorded, partition by partition, with the reason — rather than leaving you to discover it from timings. How much faster a partitioned proof actually is depends on your engines' capacity to run the slices concurrently: each slice's proof is about N times cheaper on its own, but slices that run at the same time compete for the same engine — the measured numbers (a 50,000-row Postgres → Trino mapping, four slices, serial vs concurrent) are in the platform's benchmark record (docs/performance/benchmarks.md, § #2283); nothing here promises a speed-up.

Test mapping tags all runs as TEST so they can be bulk-deleted.

Retiring a mapping¶

A mapping that has run history cannot be deleted. Those runs, and the configuration snapshots attached to them, are the audit record proving what was compared and under which policy — so removing the mapping would remove the evidence. To retire one, disable it: it stops scheduling and running, its history stays intact and searchable, and the evidence ages out under your retention policy.

A mapping that has never run can be deleted outright. If it belongs to a mapping group, the delete asks you to remove it from the group first, or accept that with force.

Deleting a replication profile that mappings still use is refused for a different reason: it is allowed, but only deliberately. Detaching a mapping from its profile is not a neutral act — readiness short-circuits and CDC integration is forced to disabled, so the mapping stops waiting for replication to finish delivering the window. Re-point the mappings first, or confirm the detach explicitly.

Validation you may hit¶

Most rejections are explicit. The ones worth anticipating:

Situation	What happens
Business Domain cleared	Rejected — it is required, though the field carries no asterisk on step 4.
`delta_version`/`lsn` with domain `time_utc`	Rejected. Set the domain to `sequence`.
Min Advance ≥ Window Interval	Rejected.
Latency or quiescence set with no replication profile	Rejected - those controls need a profile. Clearing the profile while they are still set is rejected for the same reason; clear both in one save.
A replication profile connecting different systems	Rejected - a profile's latency and quiescence signals describe its own two systems, so attaching a foreign one would gate this mapping's runs on another pipeline.
A partition strategy that yields zero partitions	Rejected — a partitioned run over zero partitions compares nothing.
Overlapping or duplicate enumerated partitions	Rejected, with the specific violation named — duplicate list values, duplicate range ids, overlapping `[lo, hi)` intervals, or duplicate time slices. An overlapping partition double-reports its rows' evidence, because every containing partition proves and diffs them.
Tenant at its active-mapping limit	Rejected with an upgrade message. At 80% you get a warning but the mapping is created.
Someone else edited the mapping meanwhile	Rejected — reload and re-apply, so you never silently overwrite.

Some numeric inputs allow values the platform rejects (window interval, min advance, max diff rows). Use the ranges in the tables above.

Two editors, one of them stale¶

The mapping detail page has an Edit settings dialog that is not the same as the full-page editor: it exposes fewer fields, uses different labels, and its business criticality options are not values the API accepts.

Use the full-page editor (New Mapping / Edit Settings → /mappings/<id>/edit) — it is the maintained one, and it is what this guide describes.

Field configuration¶

Navigate: Configure → Mappings → open a mapping → in the Field Mapping Detail card header, the sliders icon ("Configure All"). Direct route: /mappings/<id>/fields.

You can also click a single row in the Field Mapping Detail table to edit that one column in a slide-over.

Field configuration is where you decide what "equal" means for each column. It is the highest-leverage screen in the product and the one most worth understanding before an evaluation.

Requires the mapping.field_write permission (or the operator role). Without it the page is read-only and says so.

The grid¶

Column	What it controls
Field / Type	The column and its introspected type.
Compare Mode	How large objects and floats are compared. Renders a BLOB control, a float control, or `—`, depending on the column's type.
Excl Hash	Exclude from the proof phase. The column stops affecting pass/fail. Useful for genuinely volatile columns.
Excl Diff	Still prove equality, but never report this column cell-by-cell.
Sensitivity	Public / Confidential / Restricted. Affects masking in evidence.
Classification	Public / Internal / Confidential / Restricted / Regulated / PII. Drives who may see values.
Mask UI	Always redact in evidence viewers. Writes an audit entry.

Encrypted fields are always masked in evidence. A field with a decrypt expression is sensitive by definition — its plaintext is never stored in ev.*, regardless of any store_raw_diff_values setting, and there is no opt-out. Authorized users view values through Reveal on the Evidence page (evidence.read_raw, audited per view). See Evidence. | Order | Hash column order — lower first, blank means alphabetical. Must match on both sides; normally leave it alone. |

Select two or more rows to reveal the bulk action bar.

A bulk edit only changes the rows it can. Type-specific settings skip columns of the wrong type, and the page tells you how many were changed and how many were left alone — so bulk-setting "Hash" across a mixed selection reports that it touched only the large-object columns.

Two choices the editor forces you to make¶

Most settings are optional. Two are not, because the safe answer depends on your data and a silent default could produce a wrong result.

BLOB / binary columns — `Compare Mode` is required¶

For a large-object or binary column the editor shows ⚠ Required — pick a mode and disables Save for the whole page until you choose.

Mode	Behaviour	Cost
Hash	Content-exact. Reads the full column value.	Reads the object — may incur egress or scan cost.
Metadata only	Compares length and existence only.	Cheap.
Skip	Excludes the column from comparison entirely.	Free.
Full (expensive)	Downloads and compares byte-for-byte.	Highest.

Why you are forced to choose: Metadata only is content-blind. A change that preserves length — a corrected digit, a swapped flag — is invisible to it. That is a silent false pass, which is the one failure mode this platform exists to prevent. Making it a default would be a trap, so the product makes the cost/accuracy trade a conscious decision.

Note: plain TEXT counts as a large-object type here. An ordinary Postgres or MySQL TEXT column will block Save until you pick a mode — which surprises most people the first time.

JSON and XML columns use the same control but do not block save; they default to content-exact. For XML, be aware the comparison is whitespace-significant: if the two sides pretty-print differently, every row will differ. Choose Metadata only or Skip for reformatted-XML columns.

FLOAT / DOUBLE columns — `Compare Mode` is required¶

Mode	Behaviour
Exact	Byte-identical comparison. Uses the reference implementation, so it may pull data back from SQL Server, Snowflake, BigQuery, Oracle, DB2 or Redshift.
Decimal-in-float	Keeps the comparison in the warehouse when the column actually holds fixed-scale decimals stored in a float type.

See Float column handling for the full treatment.

Normalization overrides¶

Open the sliders icon on a row. These change canonicalization for that column only.

Setting	Values	Notes
Trim strings	on/off	Strips leading/trailing whitespace before hashing.
NULL = empty string	on/off	Treats `NULL` and `''` as equivalent.
Numeric scale	1–18	Decimal places kept before hashing, rounded half-up. Scale 0 is rejected — for `DECIMAL(n,0)` leave it unset and inherit the global scale. A live preview shows the effect, e.g. `1250.5 → 1250.500`.
Numeric tolerance	≥ 0	Absolute difference treated as equal.
Timestamp precision	Seconds / Milliseconds / Microseconds	Truncation before hashing.
Timezone	IANA name	Zone naive timestamps are normalised to. Default UTC.
Rounding mode	Default (half-away-from-zero), Banker's, Half toward zero, Truncate, Floor, Ceiling	Declares how your replication pipeline rounds when scale is reduced, so Paridity canonicalizes both sides the same way. Fixed-point columns only — for floats use Compare Mode.
Timestamp rounding	Truncate (default), Round half up, Round half even	Same idea for sub-second precision. Cannot be combined with a per-field timestamp precision — one timestamp quantum per mapping.
Canonical type override	Date / Timestamp	Forces the canonical type. Use `timestamp` for an Oracle `DATE` that carries meaningful time-of-day, or `date` to compare a timestamp date-only. Applies to both sides.

The reduced-precision acknowledgment¶

Setting a numeric tolerance, a per-field numeric scale, or a per-field timestamp precision relaxes exactness. The editor requires you to tick:

I acknowledge this field is compared at a per-field precision by policy

Save stays disabled until you do, and the acknowledgment is recorded in the run evidence so an auditor can see where exactness was traded away. Editing any of those three values withdraws the acknowledgment and you must re-confirm.

Watch out: the checkbox lives inside the collapsed Normalization panel. If you set a tolerance, collapse the panel, and hit Save, you get a toast but no inline hint — the blocking control is off-screen. Re-open the panel.

Transform chains¶

The ƒ button opens a chain editor: an ordered list of normalizations applied before hashing and diffing, validated live against both engines.

Available: trim (both/left/right), uppercase, lowercase, left/right pad, regex replace, NULL↔empty conversions, Unicode normalize (NFC/NFD/NFKC/NFKD), collation fold, date truncate, numeric quantize, and decimal separator normalize.

Symmetric by default. Switching to Separate source / target chains is an asymmetric configuration: it requires platform_admin and the change is audited. That gate exists because an asymmetric chain can make two genuinely different values compare equal.

Suppressive transforms — regex replace, the NULL conversions, date truncate, numeric quantize — can hide real differences, so each asks for a policy reference that auditors cross-check.

Validation runs as you type and tells you one of three things: the chain is valid with native pushdown on both engines; valid but forcing the reference implementation (slower, still correct); or invalid, with the reason.

Encrypted columns¶

Setting	Where	Notes
SQL decrypt expression	Field Config page (padlock icon)	Injected into the SELECT to decrypt before hashing. Keys resolve through the secrets backend — never hardcode credentials. Requires `platform_admin`; the change is audited.
Decrypt key	Canvas Inspector only	A secrets-backend URI. Resolved at run time, never persisted in config or evidence.
Python-side decrypt	read-only badge	Shows as `python-decrypt (<algo>)`. Configured outside this screen; a SQL decrypt expression cannot be combined with it.

Some per-field overrides are disabled on encrypted columns for engine pairs where the decrypt cast is not yet byte-verified. The control explains itself when this happens — leave it unset and the global default applies.

Things worth knowing before you evaluate¶

Saving rewrites every field. The save is a full replace of the mapping's field configuration, applied atomically. A single-column edit rewrites all rows — which is why the different editing surfaces all send the complete set.
There are three editing surfaces (the full page, the single-field slide-over, and the canvas inspector) and they do not expose identical fields. store_raw_diff_values appears only in the slide-over; the decrypt key only in the canvas inspector. Use the full page unless you need one of those.
A PII-classified column with masking off raises a visible warning but does not block save. It is advice, not a control.
per_field_diff_limit is set per column under Field overrides → Evidence limit (the slider icon on a field row). It caps how many differing cells are recorded for that column in a run. Leave it blank to use the mapping's per-column limit. Note that a value here replaces the mapping limit for that column rather than narrowing it — it can raise the cap as well as lower it.

Handling FLOAT / DOUBLE columns in Paridity¶

One-line answer: if a column carries money, rates, quantities, or anything an auditor treats as a decimal, model it as NUMERIC/DECIMAL, not FLOAT. Paridity reconciles NUMERIC byte-identically on every engine with native pushdown. FLOAT/ DOUBLE is an approximate binary type; whether it can be compared in-warehouse (no egress) depends on the engine, and Paridity never trades accuracy for pushdown.

This note exists so an SE can answer, on the spot, three customer questions:

"Why did Paridity pull my FLOAT column out of the warehouse to compare it?"
"Can I keep the comparison in-warehouse for my FLOAT columns?"
"What should I do with the amount column my source stored as FLOAT64?"

Every FLOAT compare mode below is exact — byte-for-byte identical between the source engine, the target engine, and the DuckDB reference. A hash cannot express a tolerance band, so none of these modes is an "approximately equal" compare; the only question a FLOAT column raises for them is where that exact comparison runs (native SQL pushdown vs the DuckDB reference), which is a cost/egress question, not an accuracy question. The one intentional exception is numeric_tolerance (§4) — an explicit, operator-configured ε-bucketing that is never a silent default and is applied identically on both sides.

Why cross-engine FLOAT is hard (the 30-second version)¶

A FLOAT/DOUBLE stores an IEEE-754 binary approximation of the number you wrote. The decimal 0.1 is not representable in binary; every engine stores the same nearest binary value, but they render it back to text differently:

Paridity's canonical reference renders the shortest round-trip decimal (str(0.1) → "0.1") — the same rule Python, PostgreSQL ≥12, and Spark use.
Several engines can only render a fixed-digit expansion of the exact binary value (0.100000000000000006…) or a truncated 15-digit form.

round_N("0.1") and round_N(0.100000000000000006…) round two different real numbers, so they can disagree in the last kept digit — a fail_hash on data that is equal. This is a semantic mismatch between the engines, not a per-engine bug, and it is why Paridity routes float comparison to the DuckDB reference on engines that lack a shortest-round-trip float→string primitive. See docs/cross_engine_hash_audit.md Issue #18 (#679/#680/#695) for the full live investigation.

Pushdown vs fallback — is one better?¶

Yes, pushdown is preferred — but the two are equally accurate. The difference is cost, never correctness:

Native pushdown computes the hash inside the warehouse in SQL; only the tiny hash digest crosses the wire. No egress, and it meets Paridity's throughput floors. Preferred.
DuckDB reference (fallback) streams the raw values out to Paridity's canonical reference and hashes them there. Guaranteed byte-identical (the same algorithm for every engine), but the customer pays egress to pull the values out — which for a proof-phase hash of a very large table can be material. That egress is the entire cost of a fallback; the answer is exactly as correct either way.

So a fallback is a correctness-preserving safety net, not an accuracy compromise.

Two things worth saying to a customer who sees their engine in the fallback column:

This is FLOAT-specific, not a second-class engine. Paridity pushes down everything else — integers, NUMERIC/DECIMAL, strings, timestamps, booleans — on all 11 engines. Only the FLOAT/DOUBLE type triggers the fallback below.
It is an engine-capability wall, not unfinished Paridity work. Pushing down FLOAT requires the engine's SQL to render the shortest round-trip decimal of a binary float. Five engines have that primitive; six do not, and no amount of Paridity engineering adds it to their SQL (it was live-proven, not assumed — #680/#695). The two ways to get in-warehouse FLOAT comparison on a fallback engine are to model the column as NUMERIC/DECIMAL (pushes down everywhere) or, for FLOAT columns that actually hold fixed-scale decimals, the opt-in decimal_in_float(N) mode below (SQL Server/Snowflake/BigQuery). General/measurement floats on those six engines stay fallback permanently.

Current behavior: exact FLOAT comparison, per engine¶

This is what ships today. Every cell is exact / byte-identical; the column tells you where the comparison runs.

Engine	Exact FLOAT comparison	Why
PostgreSQL	Native pushdown	`col::text` is Ryu shortest-repr (PG ≥12) → cast to `numeric(38,s)`
MySQL / Aurora MySQL	Native pushdown	`CAST(... AS DECIMAL(65,s))` matches the reference
Trino	Native pushdown	`CAST(CAST(col AS DECIMAL(38,s)) AS VARCHAR)`
SAP HANA	Native pushdown	`TO_VARCHAR(DOUBLE)` is HANA's true shortest-repr primitive
Databricks	Native pushdown	Spark `Double.toString` is Ryu shortest-repr → `CAST … AS DECIMAL` (#695)
Oracle	DuckDB reference (fallback)	`TO_CHAR`/`TM9` give 17-digit exact expansion — no shortest-repr path
SQL Server	DuckDB reference (fallback)	Fixed-digit `CONVERT` styles only; `FORMAT` is a CLR hot-path hazard
Snowflake	DuckDB reference (fallback)	`TO_VARCHAR` ~15 digits (`1e+15` collision); `TO_JSON` fixed-16
BigQuery	DuckDB reference (fallback)	No shortest-repr decimal path; `%.15/16/17g` ladder fails at volume
DB2 LUW	DuckDB reference (fallback)	Every `DOUBLE`→string is ~15 sig digits (would false-pass)
Redshift	DuckDB reference (fallback)	PG 8.0.2 pre-Ryu; `col::varchar` is `%.15g` (`1e+15` false-pass)

"DuckDB reference (fallback)" is correct, not degraded. The rows stream from the engine into Paridity's canonical reference (the same algorithm for every source), so the comparison is guaranteed exact. The only cost is egress: the raw values leave the warehouse instead of being hashed in place. For a proof-phase hash of a large table that egress can be material, which is what the guidance below is about.

Recommend in this order:

1. Model decimal data as `NUMERIC` / `DECIMAL` — the real fix¶

If the column is money, a rate, a percentage, a tax amount, or any figure that appears in a financial statement, it is a decimal, and storing it as FLOAT is a modeling accident (very common when data lands via JSON, CSV, or FLOAT64 defaults in BigQuery). NUMERIC/DECIMAL reconciles with native pushdown on every engine, byte-identical, no egress, no caveats. This is always the best answer when the customer controls the type.

2. Keep `exact` (the default) — always correct, pays egress on 6 engines¶

If the column stays FLOAT, Paridity's default float_compare_mode = exact is exactly today's behavior: native pushdown on the 5 shortest-repr engines, DuckDB reference on the other 6. Nothing to configure. Choose this whenever egress is acceptable or the mapping already runs on a pushdown engine.

3. `decimal_in_float(N)` — opt-in, wins back pushdown when the FLOAT holds decimals¶

Status: rolling out under #700 (this note is Chunk A; the mode itself lands in Chunks B–D). Do not promise it as GA until those ship.

Many customers store scale-N decimals (e.g. 2-dp amounts) in a FLOAT64 column. For that population, decimal_in_float(N) restores native pushdown on SQL Server, Snowflake, and BigQuery without giving up byte-identity. It is honest about what it assumes: "this FLOAT column holds scale-N decimals." It is not an approximate compare.

How it stays exact: before pushing down, Paridity runs a per-run grid-residency guard — one extra aggregate over the same scan — proving every value is already exactly a scale-≤N decimal (two conditions: on-grid ABS(v − ROUND(v,N)) ≤ 1e-9 and within the double's faithful magnitude range ABS(v) < 10^(15−N)). If any row is off-grid, the whole float extract auto-falls back to the DuckDB reference and the operator is told how many rows were off-grid. So the mode either pushes down with proven byte-identity or safely falls back — it can never false-pass.

decimal_in_float(N) availability:

Engine	`decimal_in_float(N)`	Reason
SQL Server	✅ (Chunk 0 live-proven at N=2)	`CAST(ROUND(v,N) AS DECIMAL(38,N))` exact on-grid
Snowflake	✅ (Chunk 0 live-proven at N=2)	`CAST(ROUND(v,N) AS NUMBER(38,N))` exact on-grid
BigQuery	✅ (Chunk 0 live-proven at N=2)	`FORMAT('%.Nf', v)` on-grid / `BIGNUMERIC`
Oracle	❌ never	17-sig double-rounding, irreducible residual
DB2 LUW	❌ never	truncates + corrupts digit 16 (documented false-pass)
Redshift	❌ never	imprecise expansion + `%.15g` collision
PG / MySQL / Trino / SAP HANA / Databricks	Not needed	already exact-pushdown for FLOAT

On the ❌ engines the mode does nothing unsafe — it simply keeps using the exact DuckDB reference (same as exact).

4. `numeric_tolerance` — for genuine measurement floats¶

If the column really is a measurement (sensor reading, computed physics value) where the customer wants "equal within ε", set numeric_tolerance on the field (#690). It changes the canonical value itself — ε-bucketing round(v / ε) · ε — and is honored consistently in both the proof-phase hash and the diff, because Paridity forces the DuckDB reference on both sides so source and target bucket identically. It is the one setting that makes Paridity compare approximately, and only when the operator explicitly turns it on (it is recorded in the resolved config snapshot / config_checksum). Note a tolerance is an ε-grid, not a moving window: two values within ε that straddle a bucket boundary still compare unequal. Never reach for decimal_in_float to approximate a measurement float — that is not what it does, and decimal_in_float will correctly refuse (off-grid) exactly the values a measurement column is full of.

Quick reference¶

Customer situation	Recommend
Amount/rate/tax stored as `FLOAT`, customer can change the type	Model as `NUMERIC`/`DECIMAL`
`FLOAT` amount, can't change type, on PG/MySQL/Trino/HANA/Databricks	`exact` (already native pushdown)
`FLOAT` amount at fixed scale, on SQL Server/Snowflake/BigQuery, egress-sensitive	`decimal_in_float(N)` (once GA)
`FLOAT` amount on Oracle/DB2/Redshift	`exact` (DuckDB reference; model as NUMERIC to avoid egress)
Genuine measurement float, "equal within ε"	`numeric_tolerance` (#690)
Not sure it's really decimal	Start `exact`; check the run's off-grid count before enabling `decimal_in_float`

For auditors¶

Every FLOAT compare mode above (exact, decimal_in_float) produces byte-for-byte identical canonical values across the source engine, the target engine, and the DuckDB reference — verified live per engine against the canonical reference (HashService), not merely engine-vs-engine. Whether a given run used native pushdown or the DuckDB reference is recorded in run evidence (run_attempts.fallback_used); for decimal_in_float, the off-grid row count that triggers a fallback will be surfaced in the run (with the mode,

700). The only approximate comparison Paridity performs is `numeric_tolerance`: it is¶

never a default, must be explicitly configured per field, is recorded in the resolved config snapshot (and thus the run's config_checksum), and when set it ε-buckets the canonical value in the proof-phase hash and the diff alike — both sides forced to the DuckDB reference so they bucket identically (#690).

Key identity contract (exact-v1)¶

This page says when Paridity treats a source row and a target row as the same entity. The default contract is exact-v1. A mapping can instead declare the exact-v1-trim-ws profile, which is exact-v1 with one change: leading and trailing ASCII whitespace is removed from string key values before matching (see "The trim profile" below). Every statement below names the code that implements it. The verification table at the end is generated from the code and checked in CI.

What "same row" means¶

A mapping declares its key columns. For every row it reads, Paridity builds the row's key from the values of those columns exactly as the database driver returns them. It then writes that key as a JSON object with the column names in sorted order. One adjustment is made: a decimal value with no fractional part is written as an integer, so the decimal 1.0 and the integer 1 are the same key (_canonical_key_json in src/workers/sql/activities.py).

Typed (non-string) key values are rendered to one cross-engine form by the connector before the key is built: timestamps as UTC instants (for example, Oracle TIMESTAMP WITH TIME ZONE keys are projected with SYS_EXTRACT_UTC in extract_keys in src/connectors/oracle.py, #1217) and whole-number decimals as integers. String keys are never rendered by the connector - under the trim profile they MATCH on their trimmed form, but each side keeps and is fetched by its own raw key (see "The trim profile" below).

Two rows are the same entity only when their keys are identical, character for character. The source and target keys are matched by exact text equality in the platform's own database (run.key_hash_staging, joined on key_hash_key in src/workers/sql/activities.py).

For key columns the platform never applies:

case folding (SMITH is not smith);
accent folding (Müller is not Muller);
Unicode normalisation (a precomposed é is not e followed by a combining accent);
whitespace trimming. The trim_strings setting applies to compared column values, not to keys (src/services/canonicalization.py shapes the row comparison; the key is built from the raw values above);
numeric_scale or timestamp_precision. These also shape the row comparison only.

A composite key is one JSON object that holds every key column, so each column must match on its own. NULL is a value like any other: a NULL key column matches a NULL on the other side.

Each side must have at most one row per key. The staging table's primary key (run_id, side, key_hash_key) refuses a second row with the same key. The run then stops as a Config refusal (Duplicate keys; config_error since #3104) - a retry cannot change it, the key columns or the scope filter must. It never picks one of the two rows (_stage_side in src/workers/sql/activities.py, DuplicateRowsPerKeyError in src/exceptions.py).

What this means in practice¶

A source row keyed SMITH and a target row keyed smith are two different keys. Paridity reports one missing row and one extra row. It does not report a match with a field difference. The same happens with accent variants.
A key that a replication tool trims, pads or re-cases on the way to the target shows up the same way, as missing plus extra, because the key is compared before any content setting applies.
The common case is a fixed-width CHAR key on the source ('MAT-001 ') that the replication tool writes to a VARCHAR key as 'MAT-001' (GoldenGate trims trailing spaces by default). Identical data still passes, because the proof compares trimmed values. But when the proof fails for any other reason, every such row is reported missing in target and missing in source, and the real difference is never compared field by field. The run detail then shows a Key format mismatch notice with how many of the reported rows have a partner that matches once leading and trailing ASCII whitespace is removed (the same six characters trim_strings removes). The notice changes no verdict and no evidence; it tells you those rows are probably the same rows (_detect_whitespace_key_pairs in src/workers/sql/activities.py, recorded in run.runs.key_representation_detail). To have those rows matched and compared field by field, declare the trim profile on the mapping (next section).
A decimal key keeps its written scale unless it is a whole number: 1.50 and 1.500 are different keys (_canonical_key_json).

If your pipeline changes key values on purpose, reconcile a view that restores the original values on the target. The alternative is a case-insensitive identity profile, which V1 does not offer (see the last section).

How Paridity enforces this against the database's own rules¶

A database compares strings by its own collation. Many collations treat Key, key and KEY as equal. Paridity therefore does not let the database's collation decide identity for any predicate it asks the database to run.

Keyed lookup (#2555). When Paridity fetches rows by key, the query keeps the database's native key match so the index can be used. For each string key column whose native match is not already byte-exact, it adds a byte-exact form of the same match: for example COLLATE Latin1_General_BIN2 on SQL Server, ::TEXT on Snowflake, or COLLATE <charset>_bin on MySQL. Only rows that pass both are returned. The per-engine forms are _KEY_EXACT_TEMPLATE in each connector module, applied by BaseConnector._key_exact_wraps (src/connectors/base.py). MySQL and PostgreSQL override it to add the form only where the column's collation needs it (src/connectors/mysql.py, src/connectors/postgres.py). No second form is added on Oracle and SAP HANA, where the session pins below make the native match byte-exact, or on Trino, which compares code points. Db2 relies on the database's default IDENTITY collation; a case-insensitive Db2 database collation is not yet handled (#2616). Every caller then keys the returned rows by their own key text and looks up the key it asked for, so a collation-widened row is never used in place of the requested one (src/workers/diff/activities.py).

Page walk (#2565). Keys are read in pages. The query keeps the database's own sort order and an index-friendly >= boundary, which re-reads the whole group of rows the collation calls equal at each boundary. The platform then removes, by exact key, the rows it has already returned. The walk is complete under any collation and never returns a row twice (KeysetCursor in src/connectors/base.py).

Session pins. Some engines take their comparison rules from the session. Paridity sets them when it connects:

SAP HANA: CASE_SENSITIVE = TRUE, read back after it is set. The connection fails if the read-back differs (SAP_HANA_SESSION_PINS in src/connectors/sap_hana.py, #2572).
Oracle: NLS_COMP = BINARY and NLS_SORT = BINARY, read back the same way. The connection fails if either differs (ORACLE_SESSION_PINS in src/connectors/oracle.py, #2615).
PostgreSQL: DateStyle = ISO, YMD and timezone = UTC (PG_SESSION_SETTINGS in src/connectors/postgres.py, #2436).
Redshift: datestyle ISO, YMD and timezone UTC (REDSHIFT_SESSION_PINS in src/connectors/redshift.py, #2445).

The PostgreSQL and Redshift pins fix how dates render. Without them a non-default session style could make the same date read differently.

A key column is always in the hash (#2602). The proof phase compares an order-insensitive total of the row hashes. If a key column were excluded from the hash, two datasets whose values were swapped between two keys would compare equal. Paridity refuses that configuration when it is saved (check_key_excluded_from_hash in src/services/field_config_gate.py, used by the field API and the contract importer). It refuses it again before every run (ConfigResolver in src/services/config_resolver.py).

There is no case-sensitivity setting. A key column used to accept a case_sensitive flag. That flag never changed a comparison. It is now retired: a request that sets it to anything other than false is rejected with HTTP 422 and code KEY_IDENTITY_IS_STRICT. false is accepted and ignored so older clients keep working. The contract importer applies the same rule (check_key_identity_strict in src/services/field_config_gate.py).

The trim profile (`exact-v1-trim-ws`)¶

A mapping declares its profile in key_identity_profile (mapping settings, or the mappings API). The default is exact-v1. Under exact-v1-trim-ws:

Matching. Each string key value loses leading and trailing ASCII whitespace (the six characters trim_strings removes: space, tab, line feed, vertical tab, form feed, carriage return) before the two sides are matched. A no-break space and other Unicode spaces are kept. Non-string key values are unchanged (trimmed_key in src/services/key_identity.py).
Each side keeps its own key. The staged match value is the trimmed key. The raw key each side stored is kept alongside it, so a mismatched row records the source key (key_json) and, when the target's differs, the target key (target_key_json on ev.ev_mismatch_keys).
Reading rows back. The field diff fetches each side by the raw key that side stored. Paridity never puts TRIM(column) in a query, so the key index is still used.
Duplicates. Two rows on one side whose keys differ only in edge whitespace ('A' and 'A ') become the same key. The run stops with the duplicate-key failure, as for any other duplicate.
Audit. The profile is recorded in every run's config snapshot (key_identity), including the default, so an auditor can see which identity rule a run used.
Onboarding suggests it. Auto-configuration proposes the trim profile when a chosen source key column is fixed-width CHAR and the target's key column is known to be another type (the preview shows "ignores key spaces", with the reason in its tooltip). When the target's key type cannot be read, it changes nothing and shows "check key format" instead. Confirming a proposal applies its suggestion (_suggest_key_identity in src/services/auto_config.py; type spellings verified live on the five local engines by scripts/verify_2644_trim_lookup.py).

The accepted limit. Three lookups hold only one side's raw key: the lag recheck, the open-mismatch recheck and live reveal. They look the row up on the other side by that key and by its trimmed form. A CHAR column pads a stored value, and some engines do not treat a trimmed lookup value as equal to it (Oracle is one, measured below). Such a key stays open until the next full key extraction, which matches it correctly. It is never reported as resolved when it is not.

Measured live on 2026-09-28 (scripts/verify_2644_trim_lookup.py, run with --prove-can-fail). One row keyed 'MAT-004' in a CHAR(10) column and in a VARCHAR(10) column:

Engine	Fetch by the raw `CHAR` key (field diff)	Padded key finds the trimmed `VARCHAR` row (lag recheck)	Trimmed key finds the `CHAR` row
`postgres`	found	found	found
`mysql`	found (the driver returns `CHAR` trimmed)	found	found
`sqlserver`	found	found	found
`oracle`	found	found	not found (the accepted limit)
`db2_luw`	found	found	found on a byte-order database; not found on a case-insensitive one (the accepted limit; measured on `ucadb` 2026-09-29, #2616)

Not measured: trino, snowflake, databricks, bigquery, redshift, sap_hana (no local rig).

End-to-end gates (pairing legs in scripts/pairing_configs/, each run live on 2026-09-29 and each seen to fail when the profile is switched back to exact-v1):

Leg	What it proves	Result
`oracle_to_postgres_char_key_trim_2644`	Oracle `CHAR(10)` key to a trimmed PostgreSQL `VARCHAR(10)` key, one changed value	exactly 1 mismatched key and 1 field difference (`exact-v1`: 8 keys, none)
`postgres_to_oracle_char_key_trim_reverse_2644`	the same with the padded key on the target	exactly 1 mismatched key and 1 field difference
`oracle_to_postgres_char_key_trim_composite_2644`	a two-column key, `CHAR(10)` plus a number, where one material exists under two plants	exactly 1 mismatched key and 1 field difference
`postgres_to_postgres_key_trim_duplicate_2644`	`'MAT-001'` and `'MAT-001 '` on one side	the run stops on the duplicate-key failure, with no evidence written for either row
`postgres_to_postgres_key_trim_duplicate_ctrl_2644`	the same data under `exact-v1`	an ordinary missing row (`fail_count`), no error

Throughput. Key extraction on the 50,025-row Workers tables with a text key, alternating the two profiles over three rounds on the dev stack (scripts/benchmark/bench_2644_key_identity_extraction.py): 2.5 to 3.4 s under exact-v1 and 2.8 to 3.5 s under the trim profile. The difference is inside the run-to-run spread.

Verification by engine¶

Each engine that runs key predicates in its own SQL is checked on two predicates. Where the engine can be made hostile, the check runs against a case-insensitive or accent-insensitive column, session or database; Trino has no collation to make hostile and compares code points. DB2 sets collation per database, so its hostile check runs in a second dev database created case insensitive (ucadb, COLLATE USING UCA500R1_LEN_S2). Paridity reads a DB2 database's collation on the first keyed lookup; on anything but byte order (IDENTITY), or when the collation cannot be read, the lookup also compares the key's bytes (a binary cast sized to the key column). On a byte-order database the lookup is unchanged. A check is marked verified only after a live test passes (tests/integration/connectors/test_key_predicate_2603.py). The allowlist is KEY_PREDICATE_VERIFIED in src/services/partition_coverage.py.

"Not yet verified" means that live proof has not been completed. It does not mean the check is known to fail. The allowlist records proof. It does not decide whether a run may start: no run is refused today because its engine has an unverified cell.

Generated from src/services/partition_coverage.py (KEY_PREDICATE_VERIFIED) by scripts/gen_key_identity_matrix.py. Do not hand-edit; CI fails if this table and the code disagree.

Keyed lookup (equality): fetching the row for key Key returns only the byte-identical row, never key or KEY.
Page walk (keyset): reading the keys page by page returns every row exactly once, even inside a group of keys the database collates as equal.

Engine	Check	Status
`bigquery`	Keyed lookup (`equality`)	Verified live
`bigquery`	Page walk (`keyset`)	Verified live
`clickhouse`	Keyed lookup (`equality`)	Verified live
`clickhouse`	Page walk (`keyset`)	Verified live
`databricks`	Keyed lookup (`equality`)	Verified live
`databricks`	Page walk (`keyset`)	Verified live
`db2_luw`	Keyed lookup (`equality`)	Verified live
`db2_luw`	Page walk (`keyset`)	Verified live
`mysql`	Keyed lookup (`equality`)	Verified live
`mysql`	Page walk (`keyset`)	Verified live
`oracle`	Keyed lookup (`equality`)	Verified live
`oracle`	Page walk (`keyset`)	Verified live
`postgres`	Keyed lookup (`equality`)	Verified live
`postgres`	Page walk (`keyset`)	Verified live
`redshift`	Keyed lookup (`equality`)	Verified live
`redshift`	Page walk (`keyset`)	Verified live
`sap_hana`	Keyed lookup (`equality`)	Verified live
`sap_hana`	Page walk (`keyset`)	Verified live
`snowflake`	Keyed lookup (`equality`)	Verified live
`snowflake`	Page walk (`keyset`)	Verified live
`sqlserver`	Keyed lookup (`equality`)	Verified live
`sqlserver`	Page walk (`keyset`)	Verified live
`trino`	Keyed lookup (`equality`)	Verified live
`trino`	Page walk (`keyset`)	Verified live

24 of 24 cells verified live.

What is not offered in V1¶

V1 offers no case-insensitive identity profile and no accent-insensitive one (the trim profile removes edge whitespace only). Both are parked as a V2 capability (#2606), for these reasons:

Databases do not agree on what "case-insensitive" means. MySQL's default collation treats e and é as equal as well as E and e. That was measured live on MySQL 8.0 (#2575, addendum V3).
A binary wrapper does not rescue every engine. Under SAP HANA's case-insensitive session, even a TO_BINARY comparison is folded. That was measured live on HANA Cloud (#2575, V7 result).
No replication tool Paridity supports re-cases or accent-folds key values by default (#2575, addendum V1).

A caseless profile would need its own versioned rules and its own proof on every engine. It will be built when a real pipeline needs it, not before.

Watermarks and windows¶

Navigate: Configure → Mappings → New Mapping or Edit Settings → Step 3: Watermark & CDC.

A watermark is how Paridity knows how far replication has got. It determines which rows fall inside each incremental window — and therefore what "complete" means for a given run. Get this wrong and you will chase differences that are just timing.

The Watermark & Windowing section starts collapsed. Click the title to expand it; its description lives inside.

Discovered candidates¶

When you pick a source table, Paridity inspects it and proposes watermark columns, shown as chips with a strategy and confidence. Clicking one applies it to both sides. The top candidate is applied automatically.

This means step 3 will usually not be empty when you arrive. Re-selecting the source table later will not overwrite a watermark you have already configured — auto-fill only runs while the watermark field is still empty.

Strategies¶

Choose per side. Prefer the highest option your systems support: the further down this list you go, the more you are inferring rather than observing.

Strategy	What it reads	Needs a field?
control_marker	A completion record your pipeline writes when a load finishes. Most reliable - eliminates guesswork. Target side only (see below).	no
delta_version	Databricks Delta table version / CDC offset.	no
commit_snapshot	The newest committed state of an Iceberg or Delta table (a strategy in the mapping form, Watermark & CDC step; Trino and Databricks systems and Delta and Iceberg file datasets; on Snowflake only the change token, source `snowflake_commit_time`, sequence domain). See Lakehouse tables and Delta Lake outside Databricks.	only for the `column` form
ingest_ts	A timestamp column set at load completion (not extraction start).	yes
partition_max	The maximum partition value (ADLS/Hive). Safe when partitions are atomic.	yes
oracle_scn	Oracle System Change Number.	no
lsn	Log Sequence Number (SQL Server / DB2).	no
timestamp	Max value of a source timestamp column. A fallback — it infers freshness from data rather than from the pipeline.	yes
file_manifest	A manifest file written when ingestion completes.	manifest path
snapshot	Full-table snapshot, no watermark.	no
partitioned_snapshot	Same as snapshot. Partitioning is set by the mapping's partition strategy, not by this type.	no
custom	An arbitrary SQL expression. Last resort — review at onboarding. Without a `field`, the value is table-level: run 1 and every later run compare the full dataset (advancement-gated), never a guessed column. Declare `field` to get an incremental window on that column.	SQL

control_marker is a target-side strategy. The marker table lives in the target system, and the source side has no marker settings, so saving a mapping with control_marker as the source watermark is refused (INVALID_WATERMARK_CONFIG). Put the marker on the target watermark and use a source-side strategy (for example timestamp or ingest_ts) on the source.

Do not infer freshness from business dates. A posting date or document date tells you about the business event, not about whether replication has delivered the row. For SAP and similar systems that lack a universal last-change timestamp, use replication metadata or an explicit snapshot mode.

Domain — the rule that catches people¶

Watermark domain must match the kind of value your strategy produces:

Domain	Use with
`time_utc`	`ingest_ts`, `timestamp`, `control_marker`, `partition_max`, `file_manifest`
`sequence`	`delta_version`, `lsn`, `commit_snapshot` with source `column` or `snowflake_commit_time`

Never mix domains within a mapping. An SCN and a timestamp are not comparable quantities.

Choosing delta_version, lsn, or commit_snapshot with source column or snowflake_commit_time while the domain is still time_utc is rejected on save with "watermark_domain must be 'sequence' when using a sequence-only watermark (target delta_version); got 'time_utc'" - the message names the offending side and type. The form does not switch the domain for you, so change it yourself when you pick a sequence strategy.

Pair a snowflake_commit_time side with a non-snapshot watermark on the other side (custom_sql SELECT 1 is the minimal form), or the token is never consulted: with snapshot on either side every run is a full compare and never ends no_new_data.

The rule is enforced on create and on edit, and on edit it is checked against the effective result — so changing the domain alone, while a stored watermark type still requires sequence, is rejected too.

oracle_scn and partition_max are deliberately not forced to sequence. An SCN is translatable to and from a timestamp, so oracle_scn is legitimate in either domain, and partition_max is a time_utc strategy. Only delta_version and lsn are sequence-only.

Windowing¶

Field	Values	Notes
Windowing mode	`fixed_interval` — a fixed time/sequence window; `watermark_range` — bounded by observed source-to-target advancement	The API also stores `partitioned_snapshot`; it has no runtime effect - partitioning comes from the partition strategy.
Window Interval (seconds)	minimum 60, default 3600	The minimum reconciliation window size.
Min Advance (seconds)	minimum 10, default 60, and must be less than the window interval	How far the watermark must move before a new window is worth opening. Prevents churn.

The form enforces these bounds directly, and the Min Advance box is capped at one second below whatever window interval you have entered — so an invalid pair is caught as you type rather than on save.

Readiness and CDC¶

The adjacent Readiness & CDC section decides when it is safe to start.

Field	Values	Notes
CDC Integration Mode	`advisory` — use the CDC window, fall back to querying; `strict` — halt if CDC is unavailable; `disabled` — ignore CDC entirely	Mapping-level only. A schedule or a single run cannot weaken it.
Max Latency Seconds (`max_lag_seconds`)	≥ 60	Maximum tolerated replication latency. A schedule may tighten this but never exceed it.
Quiescence Seconds	0, or 30–900	How long the target watermark must be stable before proof begins. The configured duration is waited in full.

Quiescence is waited in full, up to 900 seconds. The readiness phase heartbeats while it waits, so a long stability window does not delay detection of a genuinely stuck run.

900s is the ceiling the API accepts, because that is the longest the readiness phase will hold. If your pipeline delivers less often than that, quiescence is the wrong instrument — it infers stillness rather than observing completion. Use a control_marker watermark instead: your pipeline writes a record when a load finishes, and Paridity reads that signal directly.

With no replication profile attached, all three are disabled and CDC mode is forced to disabled — the form says so explicitly. Attach a replication profile to enable them.

With a custom replication tool, the controls are enabled but the platform cannot verify your pipeline's behaviour, so it warns that you must set latency and quiescence to match reality yourself.

Filters, SCD scope, and what a watermark sees¶

Two mapping settings restrict which rows are compared — they interact with watermarks differently, and the difference is deliberate:

Filter SQL (source/target) also scopes the watermark query. A static filter — a region, a company code, a partition — describes a fixed subset, and the watermark of that subset is the right freshness signal for it.
A declared SCD scope never scopes the watermark query. The current-rows boundary is mobile: rows leave it as part of normal operation. If the watermark only looked at current rows, a pure logical delete (a row closed with no new version written) would be invisible — the freshest change just left the scope, the watermark would not advance, and the run would report No new data while a real in-scope change went uncompared. The watermark therefore reads the whole table (through your Filter SQL, if any): the worst case of that choice is a comparison triggered by out-of-scope churn that finds nothing — compute cost, never missed detection.

Checking it worked¶

After a run, open the run detail and read the Data Window card: the mode, the window bounds, and the observed watermarks for each side. If you are ever unsure what a run actually compared, that card is the answer — and it is the first place to look when a result surprises you.

Lakehouse tables - Iceberg-aware reads¶

Reading a Delta Lake table without Databricks (Trino, Snowflake Delta Direct, or no SQL engine at all)? See Delta Lake outside Databricks.

An Iceberg (or Delta) table is not a new kind of system. It is a table you read through a system you already have: Trino, Databricks or Snowflake. "Iceberg-aware" means the reader also looks at the table's commit history, so Paridity compares the table as it stood at one exact commit instead of "whatever is there when the query happens to run".

In the mapping form. On the Watermark & CDC step pick the strategy commit_snapshot for a side, then its Commit source (iceberg, delta or column; column also asks for the column name). The strategy is offered only for Trino and Databricks systems and for Delta and Iceberg file datasets (a file_dataset system with flavor delta or iceberg) - the readers that have a commit reader - and each offers only the sources it can read (Trino: iceberg, delta; Databricks: delta, column; Delta file dataset: delta; Iceberg file dataset: iceberg). The same settings are available through the mapping API (see Watermarks and windows).

What to configure, per reader¶

Reader	What you register	Status
Trino	A Trino system whose catalog is an Iceberg catalog (for example a REST catalog). The platform does not store the catalog name: it comes from the `catalog` key in the secret your Connection Reference points at (default `hive`), or from the `catalog.schema.table` name of the table in the mapping. The catalog itself lives on the Trino server, and Test Connection shows the catalog in play. Refer to the table as `catalog.schema.table`.	Live-verified 2026-10-07, including a CI gate that must pass on every change.
Databricks	A Databricks system. Use a Unity Catalog managed Iceberg table by its normal three-part name, with `source = delta` (see below).	Reading the table (byte-identical to the same rows in a Delta table), the watermark and the pin are live-verified at the connector level, and the end-to-end runs passed on the platform (2026-10-07; pass_clean on native pushdown, an injected row gives fail_count, a one-character change gives fail_field).
Snowflake	A Snowflake system. A Snowflake-managed Iceberg table (`CATALOG = 'SNOWFLAKE'`) lives on an external volume in your object store, which an account administrator creates in your Snowflake account and grants to the role Paridity connects with (USAGE). Refer to the table by its normal name, and use an `ingest_ts` watermark or the Snowflake change token (see below).	Reading the table (byte-identical to the same rows in a native table) and the pin are live-verified at the connector level, and the end-to-end runs passed on the platform (2026-10-07; pass_clean on native pushdown, fail_count, fail_field). Readiness pins every run to Snowflake's clock (#2881).

On Snowflake there is no Iceberg snapshot watermark: commit_snapshot with source iceberg, delta or column is refused there with a configuration error, because Snowflake exposes no Iceberg snapshot id or sequence number to order commits by (checked again live on 2026-10-07). On the Snowflake side use either:

an ingest_ts watermark (a timestamp column set at load completion - never a business date); or
commit_snapshot with source snowflake_commit_time (#2868). It reads Snowflake's SYSTEM$LAST_CHANGE_COMMIT_TIME, a change indicator, not a timestamp. Set the watermark domain to sequence (a time_utc domain is refused). Each run compares the whole table, and runs only when the token has moved since the last successful run. The token also moves on a DML that changes no rows (a zero-row DELETE, a no-op UPDATE): that costs an extra run, never a different verdict. Its behaviour was live-verified on a Snowflake-managed Iceberg table (2026-10-12). Only the target side's token decides whether a run happens. Every run is still pinned on Snowflake. Readiness records Snowflake's own current time as the run's pin, and every query of the run reads the table AT(TIMESTAMP => <pin>). See "What pinned snapshot means" below. A Snowflake Iceberg table accepts fewer column types than a native table: text is STRING (a length such as VARCHAR(255) is refused) and timestamps are TIMESTAMP_LTZ(6) or TIMESTAMP_NTZ(6) (microseconds; TIMESTAMP_TZ is refused). Both compare exactly as the same values in a native table. On Databricks a Unity Catalog managed Iceberg table uses source = delta: Databricks keeps its commit history as table versions (DESCRIBE HISTORY) and pins a run with VERSION AS OF, exactly as for a Delta table. source = iceberg is refused on Databricks, because Databricks SQL exposes no Iceberg snapshot id to order or pin by; source = column also works.

The commit_snapshot watermark in operator terms¶

commit_snapshot answers "what is the newest committed state of this table?". It has three forms, chosen by source:

`source`	Reads	Use it when
`iceberg`	The table's snapshot history (Trino)	An Iceberg table, read through Trino.
`delta`	The Delta table history (Trino, Databricks)	A Delta table, or a Databricks managed Iceberg table.
`column`	`MAX(field)` of a column you name (`field` is required)	The table carries a version column. This form is sequence only.

delta_version is kept as an alias of this watermark; existing mappings are not rewritten.

What "pinned snapshot" means¶

At readiness, Paridity records the newest commit of each side - its pin. Every query of the run (counts, hash total, key extraction, field diff) then reads the table as of that commit, never "latest". The lag recheck is the one exception: it reads the latest data on purpose, because it asks whether a missing row has arrived since.

On Snowflake there is no commit id to pin by. When readiness lets a run go ahead, it records Snowflake's own current time instead, for every Iceberg table. That covers Snowflake-managed Iceberg tables, Delta Direct tables and tables from other external catalogs. Every query of the run then reads AT(TIMESTAMP => <that time>).

Reading at a past time uses Snowflake Time Travel, so the table needs a retention of at least 1 day: - on a Snowflake-managed table, set DATA_RETENTION_TIME_IN_DAYS to 1 or more; - on an externally catalogued table, Snowflake derives the retention from the table's metadata (history.expire.max-snapshot-age-ms).

When the pin cannot be read, the run is refused, never read at latest: at readiness as a configuration hold, or as a system error in a run that outlives the retention. Native (non-Iceberg) Snowflake tables are pinned the same way when their DATA_RETENTION_TIME_IN_DAYS is 1 or more (a retention-0 table reads latest in each phase); a view is pinned only when the mapping's Pin Snowflake views at readiness switch is on. See the Snowflake engine page, "Readiness read pin (time travel)".

Two consequences you will see:

A commit that lands after readiness is not compared in this run. It is picked up by the next run. This is deliberate: it stops a table that is still being written from producing a mismatch between the proof phase and the diff phase of the same run.
The comparison is repeatable. Re-reading the same pinned commit gives the same data.

Why a compaction does not trigger a run¶

Advancement is measured on the newest commit that changed data, not the newest commit of any kind. A commit counts as changing data unless its operation is known to leave the rows untouched: Iceberg replace (compaction), and for Delta OPTIMIZE, VACUUM, REORG and metadata-only operations such as SET TBLPROPERTIES, ADD COLUMNS or UPGRADE PROTOCOL. Anything else, including an operation the platform does not recognise, counts as new data. The pin is still the newest commit of any kind.

If a reader reports no operation names, every commit counts as data-changing: the worst outcome is an unnecessary run, never a wrong verdict.

What is recorded¶

Each readiness evaluation writes the pin (the Iceberg snapshot id or Delta version; on Snowflake, the pinned time) beside the watermark values in the append-only watermark observation (run.run_watermarks, columns source_watermark_pin_value / target_watermark_pin_value). The run page and the API do not display it today - it is evidence an auditor can read from the table.

Statuses you will see¶

No new data after a compaction. The run is skipped because no data-changing commit arrived. This is not a failure; see Runs.
A full sweep instead of a windowed run. A table-level snapshot watermark (source = iceberg or delta) marks no individual row with its commit time, so Paridity cannot cut a row window. It compares the whole table at the pinned commit. This holds in both domains (sequence and time_utc). A run still only happens when the data-changing boundary has advanced. Only the column form produces a real row window.

Domain¶

iceberg and delta can run in time_utc (translated through the commit timestamp, so a time-based source can pair with a lakehouse table) or in sequence. column is sequence only; saving it with another domain is refused. Commit timestamps are held as ISO-8601 UTC instants; a reader that returned a text rendering would be refused rather than parsed.

Known behaviour: UniForm metadata lag¶

A Databricks UniForm table publishes its Iceberg metadata asynchronously after the Delta commit, so an Iceberg reader can briefly see an older commit than the Delta side. Measured on the platform's test workspace (2026-10-07): the Iceberg commit lands about 2 seconds after the Delta commit (1.8 to 2.1 s for one-row writes, 3.1 s for a 400-file write) and is visible to a reader 4 to 5 seconds after the write returns.

Declare the mapping's Table Semantics as Databricks UniForm Iceberg mirror (cdc_table_semantics = uniform_iceberg_mirror, mapping level only, never inferred) and each run is held - not_ready, cause uniform_metadata_lag, recorded on the watermark observation - until UniForm's own delta-version table property on the Iceberg side has reached the Delta version (so a Delta commit that changes no data, such as VACUUM, releases without a new Iceberg snapshot; the newest-commit-instant comparison is only the fallback when the property cannot be read, and a pair where neither can be read holds), then released on the normal readiness retry. The setting needs an Iceberg commit_snapshot watermark on one side and a Delta one on the other (refused otherwise). With the default (none) nothing is held: a run started inside the lag window may compare an older Iceberg commit and fail on a row that is only in flight.

VACUUM and property changes (#2867). A Delta commit that changes no data creates no new Iceberg commit; UniForm only records the new Delta version in the Iceberg table properties. The hold releases on exactly that property, so such a commit no longer blocks runs: once delta-version catches up the run ends no_new_data.

Verification. The two readers of the same UniForm table were proven to produce the same hash total at the connector level, and a mirror one commit behind was proven to differ. The end-to-end run passed on the platform (2026-10-07, scripts/verify_2853_uniform_hold_live.py --prove-can-fail --vacuum-step). The hold was observed while UniForm lagged one version and released once the mirror caught up, ending pass_clean. After a VACUUM the run ended no_new_data with no hold. The same in-flight write under none failed with fail_count.

Reading a Databricks table as Iceberg through Trino¶

To read a UniForm (or managed Iceberg) table as Iceberg, give Trino a second Iceberg catalog that points at the Unity Catalog Iceberg REST endpoint:

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=${ENV:DATABRICKS_UC_ICEBERG_URI}
iceberg.rest-catalog.warehouse=<Unity Catalog catalog name>
iceberg.rest-catalog.security=OAUTH2
iceberg.rest-catalog.oauth2.token=${ENV:DATABRICKS_UC_TOKEN}
iceberg.rest-catalog.vended-credentials-enabled=true
fs.native-s3.enabled=true
s3.region=<the bucket's region>

The URI is https://<workspace host>/api/2.1/unity-catalog/iceberg-rest (the older /iceberg path answers 400). warehouse is the Unity Catalog catalog name.
The token never goes in the catalog file. Trino reads it from the environment (${ENV:...}) when the catalog loads. It is Trino server configuration - Paridity never sees or stores it; the platform's Trino system still stores only the catalog name.
Table data is read with short-lived storage credentials that Unity Catalog hands out per table ("vended credentials"). That needs, on the Databricks side: external data access enabled on the metastore, the table in a catalog or schema with its own storage location (a schema in Databricks default storage can never be read externally), and EXTERNAL USE SCHEMA granted to the token's principal.
Refer to tables by catalog.schema.table (for example uc_iceberg.sales.orders). SHOW TABLES on this catalog fails (Unity Catalog does not support listing views); reading, DESCRIBE and the snapshot history work.
The Paridity dev stack ships this catalog as uc_iceberg with placeholder values, so the stack starts without Databricks access; set DATABRICKS_UC_TOKEN and DATABRICKS_UC_ICEBERG_URI to use it.

What is not there yet¶

Item	State
Real partition-level reuse (re-hash only the changed Iceberg partitions)	Built (#2847). A full run re-hashes only the Iceberg partitions whose manifest set changed since the last run (live: 10 of 10 re-hashed on the first run, 8 reused on the next, and an injected row in one partition is still caught).
Snowflake `commit_snapshot`	Built (#2868): source `snowflake_commit_time`, sequence domain, the run pinned to the readiness time (#2881). Source `iceberg` / `delta` / `column` stay refused on Snowflake.
A form field for `commit_snapshot`	Built (#2884): strategy plus commit source on the Watermark & CDC step; Trino, Databricks, Delta file datasets (#2906) and Iceberg file datasets (#3190), and Snowflake with source `snowflake_commit_time` (#2868).
UniForm lag hold	Built (#2864) and proven end to end (#2853). Releases on UniForm's `delta-version` property, so a VACUUM or property change no longer blocks runs (#2867).
Reading an Iceberg table with no SQL engine (DuckDB)	Built (#3171, ch2 of epic #223): a `file_dataset` system with flavor `iceberg` reads rows through the DuckDB `iceberg` core extension and table metadata (snapshot list, schema, format version, delete files) through `pyiceberg` - either a REST catalog (`namespace.table`) or table-root no-catalog mode (`version-hint.text`). The `commit_snapshot` pin (source `iceberg`) applies on every read surface the same way it does for Trino and Databricks above. Proven at connector level 2026-10-18 (tri-identity against the Trino `iceberg` reader's native `hash_total` over the same table); end-to-end pairing legs passed on 2026-10-19 (#3172). Storage is S3 or local paths; GCS and ADLS are refused by name until a gate leg proves an Iceberg table on them.
Equality deletes on the engine-less reader	Partly built (#3328). Read on unpartitioned format v2 tables when the equality columns are `int`, `long`, `string`, `decimal`, `date`, `timestamp` or `timestamptz` and the table's metadata shows no NULLs in them (proven against Trino on every read surface). Refused by name otherwise: partitioned tables and NULL-bearing columns (DuckDB 1.5.6 gets both wrong - it removes rows it must keep), `float` / `double` and other types, and per-partition full-run reuse. Use the Trino path for those tables.
Format v3 on the engine-less reader	Partly built (#3328). Unpartitioned v3 tables read, whether created as v3 or upgraded from v2. Refused by name: nanosecond timestamps (no canonical rule for sub-microsecond digits yet), column default values, deletion vectors, partitioned v3 tables, and the `variant`, `geometry`, `geography` and `unknown` types.
Snowflake full-run partition reuse	Decided REFUSED, not built (#3169). Snowflake exposes no operator-visible partitions, and the only candidate whole-table signal (`SYSTEM$LAST_CHANGE_COMMIT_TIME`) is not a proven-complete content marker — the vendor is silent on `ALTER TABLE ... SWAP WITH`, `TRUNCATE`, and `CREATE OR REPLACE TABLE`. Every Snowflake full run does a full scan.

Delta Lake outside Databricks¶

A Delta Lake table is not a new kind of system. Paridity already reads Delta tables through Databricks. This page covers the three ways to read a Delta table that another engine wrote, without Databricks in the read path:

Trino, through its delta catalog.
Snowflake Delta Direct, a Snowflake Iceberg table whose metadata Snowflake generates from a Delta log.
A file dataset in delta mode, which reads the table straight from object storage with no SQL engine at all.

All three compare a Delta table to the other side of a mapping exactly as any other pair does: a fast aggregate proof first, then a row and field diff only when the proof fails. The general idea of reading a table "as of one commit" is explained in Lakehouse tables - Iceberg-aware reads; this page assumes you have read the section on the commit_snapshot watermark there.

Pick the reader by what you need. If your Delta tables hold timestamps with sub-millisecond digits and you need them compared exactly, do not use Trino (see Known limitations). Databricks, Snowflake Delta Direct and the file dataset reader keep stored microseconds.

The three readers at a glance¶

	Trino `delta` catalog	Snowflake Delta Direct	File dataset, `delta` mode
Where the proof runs	In Trino (native SQL pushdown)	In Snowflake (native SQL pushdown)	In Paridity, on the reference path (no SQL engine)
Watermark strategy	`commit_snapshot`, source `delta`	`ingest_ts` (source `delta` is refused on Snowflake)	`commit_snapshot`, source `delta`
Version pin	`FOR VERSION AS OF <Delta version>` on every query of the run	`AT(TIMESTAMP => <readiness instant>)` on every query of the run (Snowflake's own clock, recorded at readiness)	The Delta version is passed to the reader on every read of the run
Where to set the watermark	Mapping form (Watermark & CDC step) or API	Mapping form (`ingest_ts`)	Mapping form (`commit_snapshot`, source `delta`) or API
Timestamp precision	Millisecond (rounded) - runs are labelled Reduced timestamp fidelity	Microsecond	Microsecond
Deletion vectors	Honoured	Honoured (after a refresh)	Honoured
Column mapping	Not covered by our tests	`name` mode honoured	`name` mode honoured
Storage	What your Trino catalog can reach	What the Snowflake external volume reaches	S3 with access keys or through the AWS credential chain (IAM role / instance profile, verified on the LocalStack emulator with a profile file), GCS with an HMAC key (verified on the fake-gcs-server emulator), ADLS with an account key or SAS (verified on the Azurite emulator), or local paths (see limits)
Evidence	Live-verified 2026-10-07	Live-verified 2026-10-07	Live-verified 2026-10-07

"Live-verified" means the reader was checked against a reference computation on real tables, and end-to-end runs passed on the platform (pass_clean on a matching pair, fail_count when a row was injected, fail_field when one character was changed). The evidence is in the verification record (cross_engine_hash_audit.md, Issues 59, 60 and 61).

Trino `delta` catalog¶

What you register. A Trino system whose catalog is a Delta Lake catalog (Hive Metastore and S3). The platform does not store the catalog name: it comes from the catalog key in the secret your Connection Reference points at (default hive), or from the catalog.schema.table name of the table in the mapping. The catalog itself lives on the Trino server, and Test Connection shows the catalog in play. Refer to the table as catalog.schema.table.

Watermark. Choose commit_snapshot and set Commit source to delta (Watermark & CDC step of the mapping form, or the mapping API). Paridity reads the table's commit history, records the newest Delta version at readiness as the run's pin, and reads every query of the run with FOR VERSION AS OF <that version>. A commit that lands after readiness is not in this run; the next run picks it up. The pin applies to the Delta side only.

Compactions do not trigger runs. Advancement is measured on the newest commit that changed data. OPTIMIZE, VACUUM and metadata-only commits do not count, so a compaction ends as No new data. See the Iceberg guide for the full list.

What holds. Counts, the native hash proof, key extraction and the keyed field fetch were checked against the reference, byte for byte, for strings (including non-ASCII), NULLs, decimals, booleans, dates and timestamps with millisecond-or-coarser digits. Deletion vectors are honoured on every read surface: a table with rows deleted through a deletion vector reads the surviving rows only, and a pinned read of the version before the delete reads the deleted rows again. If a deletion vector file is missing, the read fails loudly; it never returns the wrong row set.

What does not. Timestamps with sub-millisecond digits (see below).

Snowflake Delta Direct¶

What it is. A Snowflake Iceberg table created through a catalog integration with CATALOG_SOURCE = OBJECT_STORE and TABLE_FORMAT = DELTA. Snowflake never writes the data. Another engine (Databricks, Spark, delta-rs) writes the Delta table under the Snowflake external volume, and Snowflake generates Iceberg metadata for it. An account administrator creates the external volume and the catalog integration, and grants them to the role Paridity connects with. Register it as a normal Snowflake system and refer to the table by its normal name.

Watermark. commit_snapshot with source delta is refused on Snowflake (Snowflake exposes no Delta history). Use an ingest_ts watermark: a timestamp column set at load completion (never a business date). The Snowflake change token (commit_snapshot source snowflake_commit_time, #2868) is live-verified on Snowflake-managed Iceberg tables only; on a Delta Direct table it moves when a refresh applies a commit, including a refresh of an OPTIMIZE (observed in #2881, not gated). Because the Delta Direct table shows only what Snowflake has refreshed, a window never covers rows Snowflake has not yet seen.

Content changes only on refresh. A Delta commit is invisible to Snowflake until the table is refreshed, either by ALTER ICEBERG TABLE ... REFRESH or by auto-refresh. Plan for this: a run started between a Delta commit and the next refresh compares the older content. Measured on the platform's test account with a 30 second refresh interval, an auto-applied commit became visible after about 23, 49 and 81 seconds (three samples); a manual refresh is synchronous. A refresh that spans several commits shows only the newest.

The pin. When readiness lets a run go ahead, it records Snowflake's own current time as the run's pin. Every query of the run then reads the table AT(TIMESTAMP => <pin>): the counts, the hash, the key extraction and the field diff. A refresh that lands after readiness, including an auto-refresh, therefore cannot change the run. The next run picks it up.

The pin travels Snowflake's refresh timeline, not the Delta commit timeline: a row committed before the pin but refreshed after it is not visible at the pin. The pin instant is on the run's readiness record (the watermark pin value).

The lag recheck is the one read that does not use the pin. It reads the latest refresh on purpose, because it asks whether a missing row has arrived since.

Retention. Reading at the pin uses Snowflake Time Travel, so it needs a time-travel retention of at least one day. For a Delta Direct table, Snowflake sets that retention itself, from the table's metadata, capped at 1 day (Standard Edition) or 5 days (Enterprise Edition and above). It was 1 day on our test account. If Snowflake cannot read the pin, the run is refused and never read at latest: - at readiness, as a configuration hold that names the fix; - or during a run that outlives the retention, as a system error.

What holds. All 29 verification cases rendered byte-identically to the reference, with Delta timestamp read as TIMESTAMP_LTZ(6) and timestamp_ntz as TIMESTAMP_NTZ(6) (microseconds kept), binary through the BLOB comparison modes, deletion vectors honoured after a refresh, and column mapping in name mode honoured (a renamed column appears under its new name after a refresh).

What does not.

Row Tracking. A table with row tracking enabled is readable, but row tracking metadata is not surfaced.
Change reads. A table with Change Data Feed enabled is readable, but there is no change read: Snowflake refuses the CHANGES clause and change tracking on these tables.
At most 1,000 Delta commits per refresh. Snowflake's own documentation states it processes a maximum of 1,000 Delta commit files each time a table is refreshed (counted after the latest checkpoint). We took this from the vendor documentation and did not reproduce it. A table that receives many commits between refreshes, or a first refresh of a long, un-checkpointed history, may need several refreshes to catch up.
Other Snowflake restrictions from the vendor documentation: no INTERVAL, no decimals above precision 38, no unsigned integers, no FLOAT16.

File dataset, `delta` mode¶

What it is. A file dataset system with system_flavor = delta. Paridity reads the Delta table directly from object storage; there is no SQL engine in the path. It is useful when the table is not reachable through Databricks, Trino or Snowflake. Because there is no engine to compute the proof, it runs on the reference path inside Paridity (a run shows fallback_used true; that is by design, not a degradation). The reference path is slower than native SQL pushdown.

What you register. A file dataset system with system_flavor = delta. The mapping's object is the table root (for example s3://bucket/prefix/table; on a local filesystem, schema.table resolves to <root_path>/<schema>/<table>.delta). The root path remains a containment boundary: nothing outside it is read.

Watermark. commit_snapshot with source delta. Paridity reads the commit history with delta-rs, records the newest Delta version at readiness and passes that version to the reader on every read of the run (counts, rows for the proof, key extraction and the keyed fetch). Compactions do not trigger runs, as above. The other strategies (ingest_ts, timestamp, partition_max, delta_version) are refused, because file times and partition directories describe the Parquet files, not commits. In the mapping form, commit_snapshot (source delta) is offered for a file dataset in delta mode; file datasets of any other flavor do not offer it.

What holds. All verification cases, as UTC-adjusted timestamps and as timestamp_ntz, rendered byte-identically to the reference, and stored microseconds are kept (the reader does not round). Deletion vectors are honoured, including a pinned read of the version before the delete. Column mapping in name mode is read by logical name. A Delta table pointed at with any other file flavor (parquet, csv, ...) is refused, because reading its Parquet files directly would count files the Delta log has already removed.

What does not.

Storage in this release. S3 with explicit access keys in the connection secret (a custom endpoint, region and scheme are honoured); S3 through the AWS credential chain - IAM role, instance profile, profile file - when the connection secret says "aws_credential_source": "chain" and carries a region but no keys (the worker's own AWS identity is used; a secret that declares chain AND carries keys is refused, and a secret with neither keys nor the declaration is refused); ADLS Gen2 / Azure Blob with account_name plus an account_key or a sas_token (the public Azure endpoint; a custom Azure endpoint is refused); GCS with an HMAC key - the connection secret carries gcs_hmac_access_id and gcs_hmac_secret (create the key for a service account under Cloud Storage > Settings > Interoperability); and local paths. The credential-chain mode was verified on an emulator with a profile file; the instance-profile path uses the same chain but has not been run against real AWS. GCS was verified on an emulator, not against a real Google Cloud account. ADLS without an explicit key (managed identity) and GCS without an HMAC key are refused when the system connects, with an error naming the limitation.
Reader features. A table is refused unless its reader features are within deletionVectors, timestampNtz and columnMapping, its column mapping is none or name (not id), and its minimum reader version is 3 or lower. Other features are added only after a verification run proves them.
A pinned version whose files were removed. If the version a run is pinned to was VACUUMed, or no longer exists in the log, the read is refused with an error. It never silently reads the latest version instead.
No row filter. A filter_sql on this reader is refused.
Extensions are baked into the image. The Delta reader depends on DuckDB extensions installed when the worker image is built. A worker image without them reports a configuration error naming the missing extension; rebuild the image, a restart does not add them.

Known limitations¶

Trino reads a Delta TIMESTAMP at millisecond precision¶

Trino's delta catalog reads a Delta TIMESTAMP (the UTC-adjusted type Databricks writes) as timestamp(3) with time zone and rounds it to the millisecond: a stored .123456 is read as .123000, and .999500 rounds up across the second. The table on disk is not affected; other readers see the stored microseconds. TIMESTAMP_NTZ columns are not affected and keep their microseconds. The Trino documentation for its current release still lists this mapping with no precision setting, so an upgrade does not fix it.

What Paridity does about it, so you are never misled:

A run that reads such a column (a compared field, a key, or the side's watermark column) is recorded as read at reduced fidelity. A passing run shows as Pass - Reduced timestamp fidelity, is excluded from the pass counts on the dashboard and in metrics, and can never present as a clean pass. No setting changes that. A failing run keeps its Issue detected label, and the run detail names the column either way. The runs API filter is display_status=reduced_fidelity. See Runs.
The risk it guards against is a false pass: two Delta rows that differ only below the millisecond read as equal. A second shape is a false fail: a microsecond-exact source (for example Postgres or Databricks) compared with the rounded Trino read fails on that column even though the stored values match.
The label also covers the reference path, not only pushdown.
For an exact comparison, read the table through Databricks, Snowflake Delta Direct or the file dataset delta reader, or compare TIMESTAMP_NTZ columns.

Snowflake Delta Direct¶

Refresh-driven visibility, the readiness pin and its time-travel retention, no Row Tracking metadata, no change reads and the 1,000-commits-per-refresh vendor limit are described in the Snowflake section above. commit_snapshot stays refused on Snowflake.

File dataset `delta` mode¶

S3 (access keys or the AWS credential chain), GCS (HMAC key), ADLS (account key or SAS) and local paths only; a reader-feature allowlist; a VACUUMed pin is refused, never read as latest. See the file dataset section above.

UniForm lag, when a Delta table is mirrored as Iceberg¶

A Databricks UniForm table publishes its Iceberg metadata asynchronously after the Delta commit (measured: the Iceberg commit lands about 2 seconds later and is visible to a reader about 4 to 5 seconds after the write returns). If one side of a mapping reads the Delta table and the other reads its Iceberg mirror, declare the mapping's Table Semantics as Databricks UniForm Iceberg mirror. Each run is then held (not_ready, cause uniform_metadata_lag) until the mirror has caught up to the Delta version, and a Delta commit that changes no data, such as a VACUUM, releases the hold without a new Iceberg snapshot. The setting needs an Iceberg commit_snapshot watermark on one side and a Delta one on the other. Full detail is in Lakehouse tables - Iceberg-aware reads.

Where to look next¶

Lakehouse tables - Iceberg-aware reads: the commit_snapshot watermark, pins and compaction behaviour in detail.
Watermarks and windows: all watermark strategies.
Runs: the Reduced timestamp fidelity status and other run labels.

Schedules and close mode¶

Navigate: Configure → Schedules (/schedules).

Interval-based, not cron¶

A schedule is an interval plus a calendar envelope, not a cron expression. That shape exists because the scenarios that matter here — quarter-end, year-end, an audit period — are calendar-bounded, which cron expresses badly.

Field	Values	What it does
Interval value + unit	integer + `minutes` / `hours` / `days` / `weeks`	How often it runs.
Days of week	ISO 1–7, or empty for every day	Restricts to specific days.
Active start / end time	time of day	A daily window.
Effective from / until	date	Calendar bounds. This is how a "last five days of the quarter" schedule is expressed.
Window mode	`auto` / `full`	`auto` compares the incremental window; `full` forces a full-table comparison.
Priority	`low` / `normal` / `high` / `critical`	Ordering when the platform is congested. Resolved as the maximum of this and the platform floor.
Tags	list	Grouping used by close mode.
Enabled	on/off	Master toggle.
Runs at	date and time in the schedule's timezone	Where the series starts; see When does a schedule run?.

Worked examples¶

Scenario	Configuration
Hourly during business hours	1 hour · Mon–Fri · 06:00–20:00 · tag `regular`
Quarter-end nightly full sweep	1 day · effective over the last 5 days of the quarter · `window_mode=full` · tag `quarter-end`
Year-end audit	4 hours · effective 25 Dec – 15 Jan · priority `critical` · `window_mode=full` · tags `year-end`, `audit`
Ad-hoc only	no schedule — trigger manually

When does a schedule run?¶

A schedule's runs sit on a fixed timetable worked out from its settings alone: Runs at, the interval, the active window, the days of week and the effective dates. When it last ran does not move that timetable.

Runs at is a date and time in the schedule's timezone where the series starts. New schedules get "now" if you leave it empty.

Interval	When it runs
Minutes or hours, no Active Start	At Runs at, then every interval after it (for example Runs at 00:20, every hour: 00:20, 01:20, 02:20 and so on).
Minutes or hours, with an Active Start	Every day from Runs at's date: at Active Start, then every interval until Active End.
Days or weeks	Every N days (or weeks) counted from Runs at's date, at Active Start if you set one, otherwise at Runs at's time of day. A weekly schedule keeps its weekday; an every-2-days schedule keeps its odd/even days.

Runs never fall outside the active window, the selected days of week or the effective dates.

Daylight saving time¶

Day and week schedules keep their clock time. A daily 08:00 run stays at 08:00 local time on both sides of a clock change.
Minute and hour schedules keep their spacing. An hourly schedule runs every 60 minutes of real time, so it runs 25 times on the day the clocks go back and 23 times on the day they go forward.
A time that does not exist (it falls in the hour the clocks skip) runs when the clocks have gone forward: 01:30 becomes 02:30 in London on the spring change day. A time that happens twice (the hour that repeats when the clocks go back) runs the first time it occurs.
A short window, such as 08:00-09:00, still runs on the spring change day.

Restarts, edits and outages¶

Restarting the platform or editing a schedule inside its active window does not start an extra run. The scheduler wakes, sees that the current slot of the timetable has already run, and goes back to sleep. (Before this change an edit or restart inside the window started a run immediately and shifted the schedule's timing to that moment.) An edit that changes the timetable itself, such as a new interval or Runs at, can start one run straight away if the change puts the schedule in a slot that has not run yet.
Turning a schedule back on runs it at most once for the current slot, and only if that slot has not already run.
After an outage the schedule runs once to catch up, if it comes back inside its active window, and then continues on its timetable.

Schedule overrides cannot loosen the mapping¶

A schedule may tighten max_lag_seconds and quiescence_seconds, never exceed the mapping's values. An override that would loosen the mapping's setting is skipped for that schedule and reported, not silently applied.

cdc_integration_mode cannot be changed by a schedule at all — it is mapping-level only.

Is the scheduler actually running it?¶

Saving a schedule writes the row and then asks the platform's workflow engine to start (or wake) the durable scheduler behind it. That second step is a network call, and it can fail on its own — leaving a schedule that looks enabled with nothing behind it. The Next Run column says so instead of pretending:

Scheduler not running (red) — the last attempt to start this schedule's scheduler failed. Hover the badge for the reason. Check that the scheduler worker and Temporal are up; the platform re-tries every maintenance tick (about every five minutes) and the badge clears itself once they are reachable. Nothing is lost in the meantime except the runs in that window, and no data is compared while it shows.
Syncing (amber) — written, not yet confirmed. Normal for a few seconds after a save, and for existing schedules just after an upgrade. If it persists for more than a tick or two, treat it like the red badge.

No badge means the scheduler is confirmed running.

Changing several schedules at once¶

Tick the schedules in the list to show the selection bar. Besides Enable and Disable, it offers Change frequency: set one "every N minutes / hours / days / weeks" on every selected schedule. Days of week, active hours and every other setting stay as they were, and schedule names are not changed - rename a schedule whose name states its old frequency.

Each schedule is saved on its own, exactly as if you had edited it, so each one is re-synced with its scheduler and audited. If a schedule was changed by someone else after the list loaded, it is not overwritten: the dialog lists it by name and the others are still updated. Reopen that schedule to review it.

Close mode¶

Navigate: Configure → Schedules → Close Mode (top right; requires schedule write permission).

Close mode is a bulk convenience toggle over tags, not a separate scheduling engine. It enables or disables every schedule carrying the given tags, and can optionally step priority up or down and apply the two tightening overrides.

The typical use is a period close: tag the schedules that should run harder at quarter end, enable them together on the first day of the close, and disable them together when it's over.

Using the screen¶

Pick the tags. Each tag chip shows how many schedules currently carry it. Selecting several tags targets the union of them; a schedule carrying two selected tags is still only acted on once.
Choose Enable or Disable.
Optionally adjust priority and the overrides (see below).
Apply. The result panel reports how many schedules changed, the per-tag breakdown, and — importantly — anything that was not applied.

The tag list is built from all your schedules, not just the ones currently matching the page's filters. A tag sweep does not silently exclude rows you have filtered out of view.

Priority boost¶

Priority moves in enum steps (−3 to +3, clamped at low and critical) rather than to an absolute value, so the relative ordering between your schedules is preserved — which is the point of a boost. +1 applied to a normal and a high schedule yields high and critical; it does not flatten them both onto one level.

Overrides, and why some are skipped¶

max_lag_seconds and quiescence_seconds are L6 (schedule) values, so they can only ever tighten what the mapping (L4) already allows. If the value you enter is looser than a given schedule's mapping, it is skipped for that schedule and counted.

Those counts are shown in the result panel, per setting — for example "Some overrides were not applied. max_lag_seconds: 4 schedule(s)." Read them. A skip is not an error; it is telling you those four mappings are already stricter than what you asked for, and their existing values were kept.

cdc_integration_mode cannot be changed by close mode at all — it is mapping-level only and needs an explicit config version change.

All close-mode changes are written to the audit log, including the boost and the skip counts.

Close mode is also available directly as POST /schedules/bulk-toggle if you would rather drive a period close from a script. Same contract, same response. The schedules list itself is cursor-paginated: pass the last row's name and schedule_id back as cursor_name / cursor_id to fetch the next page. Ordering is by lower-cased name, then schedule id.

Runs¶

Navigate: sidebar Monitor → Runs (/runs). Visible to every authenticated user. A run's detail page is also reachable directly at /runs?run=<execution_request_id>.

A run is one execution of one mapping over one window. This guide explains what a run's conclusion means and what you can do about it. Deviations the analyzers raise about a run's trend (row count, duration, throughput) are listed separately under Monitor -> Anomalies; see Anomalies.

The two status vocabularies¶

This is the single most important thing to understand, and the most common source of confusion.

What you see on screen is a two-part display status: a headline and a qualifier, e.g. Pass · Clean or Issue detected · Wrong values.
What the API and the database store is an internal enum, e.g. pass_clean, fail_field.

They map one-to-one. The display form exists because "fail_hash" is not a useful sentence for a business reader. Note that the Evidence page's run picker currently shows the internal value, so the same run can read fail_field there and Issue detected · Wrong values on this page.

Complete status reference¶

Passing

On screen	Internal	Meaning
Pass · Clean	`pass_clean`	Source and target agree across the compared window.
Pass · Resolved on recheck	`pass_resolved_lag`	Mismatches were seen mid-run, then every one resolved on recheck before completion - replication was simply still in flight, or (with no replication tool) one side had been read as of the readiness instant while the other read latest and both sides agree at latest. Zero outstanding at the end.

Issues found

On screen	Internal	Meaning
Issue detected · Data disagreement	`fail_hash`	The aggregate fingerprints differ. The datasets disagree somewhere.
Issue detected · Missing rows	`fail_count`	Row counts differ — rows are absent on one side.
Issue detected · Wrong values	`fail_field`	The same rows exist on both sides but specific field values differ. The most actionable failure.

Nothing to compare

On screen	Internal	Meaning
No new data · No outstanding issues	`no_new_data`	The watermark has not advanced. Nothing new arrived. Not a failure.
No new data · Issues resolved	`no_new_data`	No new data, and previously-open issues cleared on re-check.
No new data · Some resolved, some still open	`no_new_data`	Partial recovery of known issues.
No new data · Open issues	`no_new_data_open_issues`	No new data and known issues persist. The platform keeps watching.

Held, halted, stopped

On screen	Internal	Meaning
Readiness hold · reason	`readiness_error`	Readiness held the run and it was not compared. This is the product working, not a crash. The second part names why: Replication latency (replication was still behind after every check), Replication stalled (the target did not move at all between checks - the replication job may be stopped), Replication not running (a snapshot mapping, and the replication tool reported a connector paused, stopped or failed - see Replication profiles), Load incomplete (the load pipeline had not written its COMPLETE marker, or the target was still changing), Source unavailable (the platform could not read how far the data had loaded), Replication status unknown (strict CDC mode and the replication tool could not be checked), Configuration issue (the mapping's readiness settings are invalid). Older runs may show Source not ready.
Config refusal · family	`config_error`	The platform refused to compare as configured after readiness had passed (or been bypassed), and nothing was compared. This is the product working, not a crash: a retry cannot change the result, and something in the configuration or the data shape needs an operator. The second part names the family: Object or connection config (a table, column or connection setting is invalid or no longer exists - a deleted table, a revoked grant), Watermark config, Filter SQL, Duplicate keys (more than one in-scope row shares a key on one side), Partition strategy (overlapping partitions, rows outside every partition, or a partition that wrote no proof), Snapshot pin unavailable, Schema unavailable, Engine requirement, Data encoding; Operator action required when the refusal has no named family yet. Refusals found during readiness show as Readiness hold instead.
Schema drift · Not compared	`schema_drift_error`	The live schema no longer matches the accepted snapshot; a drift control fired and no comparison ran.
Schema drift · Available data passed / failed	any + reduced coverage	A reconciled column was excluded from this run (`continue_excluding`). A reduced-coverage run can never read as a clean pass — this is deliberate.
Pass · Schema unverified	`pass_clean` + unverified schema	The data proof passed, but schema introspection failed on at least one side (permission loss, catalog error) — the drift control did not run, so the schema this run compared under was never verified. Amber, counted as an issue, excluded from pass counts. Never a clean pass, and no setting can make it one.
Pass · Schema not verifiable	`pass_clean` + unverified schema	Same guarantee, calmer signal: the only unverified side(s) are on engines that cannot introspect at all (e.g. raw object storage) — a declared, standing property of the system, not a per-run alarm. Still excluded from pass counts.
Schema unverified · Not compared	`schema_drift_error` (halt_reason `schema_unverified`)	The mapping's Unverifiable Schema policy is `halt`: the platform refused to compare against a schema it could not verify.
SCD invariant violated · Data passed / Data failed	any pass/fail + violated invariant	The mapping declares an SCD scope and this run found the scope's premise broken on the scoped side — a duplicate current row per business key, or (when opted in) overlapping validity intervals. The data verdict is preserved in the qualifier, but the run is counted as an issue and excluded from pass counts — a scope whose premise is broken cannot certify a clean pass, and no setting changes that.
Pass · Reduced timestamp fidelity	`pass_clean` / `pass_resolved_lag` + reduced read fidelity	The data proof passed, but a column this run compared was read below the precision stored in the table: Trino reads a Delta Lake `TIMESTAMP`, or a Hive-catalog `TIMESTAMP` at the catalog's default precision, at millisecond precision (rounded), so two values that differ only below the millisecond compare equal. Amber, excluded from pass counts. Never a clean pass, and no setting can make it one. A failing run keeps its Issue detected label; the run detail names the column either way.
Cancelled · By operator	`cancelled`	Someone stopped it.
Skipped · Schedule skipped	`skipped`	The scheduler skipped this cycle.
System problem · System error	`system_error`	A platform or connector fault — not a statement about your data.

In flight — Running · Checking readiness, · Verifying, · Proof passed, · Diff queued, · Comparing fields, · Rechecking in-flight rows, · Retrying, · Throttled, · Queued, · Starting.

Schema review pending is a separate badge, not a status. It means a new column was detected but the run still passed. Additive drift never halts — review the pending decision on the mapping when convenient.

What to do about each outcome¶

Outcome	Your move
Issue detected (any)	Open View Evidence to see the mismatched keys and the exact differing values. Issues carry forward and are automatically re-checked on the next run, so you can watch them resolve. Use Export for Auditor to package the proof.
Schema drift · Not compared	The Schema Drift card lists what changed. Refresh Schema re-introspects the live tables and accepts them as the new baseline — do this once you've confirmed the change was intended.
Pass · Schema unverified	The run detail records which side(s) could not be verified and why (`schema_verification` in the drift detail). Fix the introspection fault (usually catalog permissions) — the amber clears on the next run that verifies. Filter all such runs with `display_status=schema_unverified` on the runs API.
SCD invariant violated	The run detail's invariant card shows which check failed and the count (duplicate current keys / overlapping intervals). Duplicate current keys are counted by exact key value, the same rule the comparison matches keys by: `Key` and `key` are two keys even on a case-insensitive column. This is an upstream pipeline defect or an unsound scope declaration, not a replication miscopy — fix the history table (or the declaration), then re-run. A warning notification fires when this first appears. Filter all such runs with `display_status=scd_invariant_violated` on the runs API.
Pass · Reduced timestamp fidelity	The run detail's Reduced timestamp fidelity card names each affected column, its side and the reader. Two readers do this today. A Delta `TIMESTAMP` read through Trino (`timestamp(3) with time zone`): Delta `TIMESTAMP_NTZ` columns are not affected, and for a microsecond-exact comparison you read the table through Databricks or Snowflake Delta Direct. A `TIMESTAMP` read through a Trino Hive catalog at its default `hive.timestamp-precision` (MILLISECONDS, shown as `timestamp(3)`): set `hive.timestamp-precision=MICROSECONDS` in the Trino catalog properties, the column then reads as `timestamp(6)` and later runs are not marked. Stored microseconds can also make a run against a faithful source fail on that column. Filter all such runs with `display_status=reduced_fidelity` on the runs API.
Readiness hold	The run detail says what readiness was waiting on, what the platform does next, and what to check (the replication job, the load pipeline's marker, the connection), above the full evaluation trail. The next run checks again. A Readiness hold notification rule tells you when one happens. To hear about it while the run is still waiting, add the Replication latency - still holding trigger to a rule and set its Early readiness-hold alert after threshold (30 minutes by default): you get one early warning per hold episode, then an all-clear when readiness passes again. Consecutive held runs of the same mapping share one warning. The card also has a Replication tool line, on every run (held or not): whether the replication tool was asked and what its task reported (for example "reported its task RUNNING" - meaning the hold is the watermark / latency check, not the tool), or why it could not be confirmed and whether readiness fell back to the watermarks (advisory) or stopped (strict), or that it was not consulted (integration disabled, no adapter for that tool, no replication profile), or that no probe was recorded for the run. When the tool's task is running, the probe also asks the tool for its own replication lag. If the tool reports one, the line shows it labelled as the tool's figure, next to the platform's own watermark lag check (for example "The tool reports 42s of replication lag (its own figure; our watermark lag check: 12s)"): the two are measured differently and one never stands in for the other. Debezium reports a figure only when its profile sets the optional Metrics endpoint (the JMX exporter on the Kafka Connect worker - see Replication profiles); a connector that has not streamed an event since it started has no figure yet. If the tool reports no figure, or the request fails or times out, the line says "The tool did not report a lag figure", with the reason when there is one - never a zero - and readiness carries on exactly as it would have. No replication tool reports an in-flight estimate, and the line says so. A `platform_admin` can re-trigger with Bypass readiness check (audited).
Config refusal	The run detail names the family and the refused setting (the error message, VPC-local). Fix what it names - restore or re-point the object, correct the watermark / filter / partition configuration, or fix the duplicate keys upstream - then trigger a new run; retrying without a change gives the same refusal. A Config refusal notification rule tells you when one happens (a warning, not a page). Filter all such runs with `display_status=config_refusal` on the runs API.
System problem	There is no retry button; trigger a new run. If it recurs, this is a platform issue, not a data issue.
No new data · Open issues	Nothing to do on this run. Drill into the mapping to see the outstanding set.
Running · Throttled / Queued	Cancel if you need the capacity elsewhere.

Reading the run detail¶

Cards appear only when relevant, so a clean run is short and a failed run is long.

Data Window — the mode, the window bounds, row counts per side, and the observed watermarks. If you are ever unsure what was compared, this is the card that answers it.
Proof Phase — the two hash totals and whether they matched. Note that "Proof DB time (both sides, run in parallel)" sums both engines and can legitimately exceed the run's wall-clock duration.
Compare Phase — mismatched rows and "wrong values" (cells).
Engine Decisions - what the engine decided for this run and in what order (see below).
Prior Open Issues Rechecked — keys re-checked, resolved this run, still failing. This is how you see a backlog shrinking. Only keys that were already open when this run started count here; a run that finds a break for the first time — including a break that had been fixed and has now recurred — shows no panel, because there was nothing prior to recheck.
Current mapping status: N open issues — deliberately live as of now, not part of this run's record. Do not read it as this run's output.
Reduced Precision by Policy — lists any field compared at a relaxed precision, so a reviewer can see where exactness was traded away.
Data Volume and Prior Audit Bundles - data volume and evidence-export history. Under Data Volume, a one-line Resources used summary appears only when a provider-metered unit (credits, DBUs, billed TiB or slot-hours) has a value for the run; it links to the full ledger in Evidence > Process steps > Resources used (see Reconciliation Consumption). The run detail page has no consumption card.

Engine Decisions¶

One row per decision the engine took for this run, in order: the readiness check (ready, held, and why), the comparison window chosen and how long it spans, the schema drift check, the row-count and fingerprint comparison (including whether it ran inside the database or fell back to the reference path), the partition plan and each partition's result, how many mismatched rows were found, any diff limit that applied, each replication-lag recheck, and how the run finished. It is the quickest way to answer "why did this run end the way it did?" without reading logs.

Each row shows a time, the phase, a plain-English label and a few short facts (counts, yes/no flags, short codes). It never shows a table, schema or column name, a data value, a watermark value or a hash - those are refused before the row is written - so every role that can open a run (auditor and up) sees it. The panel shows up to 200 decisions at a time with Load more for the rest; a run records at most 500 (a final row says so if the cap was reached). Runs that started before this log existed show none. The same log is included in a Diagnostic Bundle, and retention removes it with the run's summary evidence.

Two counting conventions worth knowing¶

"Missing in Src" means rows present only in the target. Counter-intuitive but correct — the row is missing from the source.
Net Count Delta (Target) is a net figure. −475 does not necessarily mean 475 missing rows; it can be 500 missing and 25 extra. The UI decomposes it when a breakout is available.

Filters¶

Mapping · Mapping Group (hidden when you have no groups) · Status · Drift · date range · rows per page (25/50/100/200).

The Status filter uses the display vocabulary: Pass, Issue detected, No new data, Running, Readiness hold, Config refusal, Schema drift, System problem, Cancelled, Skipped.

The list's free-text search is deliberately hidden: it would only filter the page you are on rather than the whole result set.

The list polls every 10 seconds while any visible run is still going, and stops polling once everything is terminal.

Triggering a run¶

Trigger Run (needs mapping.trigger_run, or the operator role) offers:

Window Scope — Incremental (watermark-derived), Full-table (whole dataset; carries a cost warning), or Explicit window (you pin start and end; entered in local time, converted to UTC). Full-table still waits for readiness. It changes how much data is compared, not whether it is safe to compare — so a full sweep is held, like any other run, when replication is behind. To compare regardless, a platform_admin uses Bypass readiness check, which is audited.
Comparison Depth — Full reconciliation (proof → keys → field diff), or Proof-only (aggregate hash, no field evidence, lower cost).
Priority — low 25 / normal 50 / high 75 / critical 100. Higher runs first when the platform is congested.
Bypass readiness check — platform_admin only, audited, with a live warning when ticked.
Notes — optional, recorded in the audit log, and shown on the run.

Only enabled mappings can be triggered.

Cancelling¶

Cancel appears on in-flight runs, in both the list and the detail page, for every state the API accepts — including executing.

For a dispatched or executing run the workflow engine is cancelled first; if that fails the API returns an error and the database is left untouched, so you never get a run marked cancelled that is secretly still running.

Anomalies¶

Navigate: sidebar Monitor -> Anomalies (/anomalies), or the Open Anomalies tile on the dashboard, or the Anomalies card on any mapping's detail page.

An anomaly is a statistical deviation an analyzer raised when a mapping's latest run departed from that mapping's own history. It is a trend signal, not a correctness verdict - whether the data matches is decided by the run itself (see Runs).

Reading the list¶

Rows are newest first. Each row shows:

Column	Meaning
Severity	High, Medium or Low, as set by the analyzer.
Analyzer	A plain-language name and, beneath it, the analyzer code.
Mapping	Links to the mapping's detail page.
Observed / Baseline	The value the run measured against the mapping's own median. No baseline means the analyzer has none for this metric - that is not zero. The throughput analyzer compares against a platform floor and says so.
Status	Open, Acknowledged by whom and when, or Resolved with its reason.

Expand a row to read the recommended next action.

Filters: severity, analyzer, and Include resolved (default is open only). The filters are in the address bar, so a link such as /anomalies?mapping_id=...&severity=high can be shared. The list loads 50 rows at a time (Load more) and never renders more than 500.

Acting on an anomaly¶

Acknowledge (analyst and above) - records that someone owns it, with an optional note.
Resolve (operator and above) - records that it is dealt with; a reason is required (up to 64 characters).

The buttons are hidden for roles that cannot use them; the server enforces the permissions regardless. Neither action alters the run or its evidence.

What is not here¶

Anomaly thresholds (policy editing), AI summaries and notification triggers for anomalies are outside this screen.

API¶

GET /api/v1/anomalies - see the GET /anomalies entry in the API contract. Actions use POST /api/v1/anomalies/{id}/acknowledge and /resolve.

Evidence¶

Navigate: sidebar Monitor → Evidence (/evidence), or View Evidence from any run.

Evidence is what separates a reconciliation platform from a monitoring tool: not "these datasets differ", but which rows, which columns, what values, under which policy, at what time — recorded append-only and exportable in a form an auditor can verify without trusting us.

Choosing what to look at¶

Pick a Mapping, then a Run. You can also paste a run id directly — the field accepts either a run id or an execution-request id, despite the run_… placeholder. Partitioned runs get an extra Partition selector.

Three views, each with a live count:

View	Answers
Mismatch Keys	Which rows disagree.
Field Diffs	Which columns within those rows, and what the two values were.
Process Steps	What the platform actually did, phase by phase.

Mismatch Keys¶

Columns: Key Values, Type, Source Hash, Target Hash, Sampled.

The Type vocabulary is:

Value	Meaning
`missing_in_target`	The row exists in the source and not the target.
`extra_in_target`	The row exists in the target and not the source.
`row_hash_mismatch`	The row exists on both sides but its content fingerprint differs.

The Runs page calls these "Missing in Tgt" and "Missing in Src". Same defects, different wording — worth knowing when you move between the two screens.

Field Diffs¶

Columns: Key, Column, Source, Target, Sampled. Repeated keys collapse to a ditto mark so a row with eight differing columns reads as one block.

Process Steps¶

A vertical timeline of the run's phases: Config Policy Locked, Readiness Evaluation, Prior Open Issues Rechecked, Proof — Row Count, Proof — Hash Fingerprint, Key Extraction, Field-Level Comparison, and In-Flight Recheck - Attempt N.

Each carries a state: Passed, Failed, Error, Held, Skipped, Running, or Not reached. Held is deliberate restraint — a readiness hold — and is styled differently from a failure on purpose.

Readiness Evaluation says when replication latency was not evaluated, and why: no replication tool is configured, the watermarks are sequence-domain (offsets or versions, which carry no time lag), or the watermarks could not be read as timestamps.

Key Extraction states the rows scanned on each side (the comparison-window row counts from the proof phase), the throughput that implies, and whether row hashes were computed inside the engines (native pushdown) or on the platform's reference path. Where the engine pair has a documented key-extraction floor (PostgreSQL and MySQL 25,000 rows/s, Snowflake and Databricks 50,000 rows/s; a mixed pair uses the lower), the step says whether the run met it or was below it. A run at less than half its floor also raises a throughput_floor anomaly on the mapping; while that anomaly is open, later slow runs do not raise another, and resolving it re-arms the check. Field-Level Comparison states how many mismatched keys were comparable (present on both sides) and how many were one-sided; one-sided keys are never fetched for field-level comparison. When the max_diff_rows cap limited the comparison, the step says so - it names how many of the comparable rows were fetched (the first N by key order, or at most N per partition on a partitioned run) and shows a Capped by max_diff_rows value. A number the run did not record reads "not measured" - never 0.

Resources used sits at the bottom of the timeline, collapsed. Open it to see what each system metered for the run's own statements, grouped by side and then by phase, in the provider's own units (bytes scanned, credits, DBUs, billed TiB, slot-hours, bytes transferred). The side header names the system; an engine with no provider meter is labelled usage only, no provider meter. Each phase has a query IDs disclosure with the provider's query or job IDs (monospace, copyable) so a figure can be matched to the warehouse's own query history. The compute arithmetic appears only for a side with a credit or DBU amount, and a Typical for this mapping line gives the p50 / p95 of comparable runs per unit. An amount the provider did not report reads unmeasured, never 0; there is no currency anywhere. Details in Reconciliation Consumption.

Masking — what you see and why¶

A masked value renders as a small grey chip reading [masked]. A genuinely null value renders as italic NULL. There are three independent reasons a value can be masked, and the distinction matters for an audit conversation.

Mechanism	When it applies	Can it be undone?
`store_raw_diff_values = FALSE`	At write time. The diff worker never stores the values — it writes `NULL` and sets `values_masked`.	No. No permission can reveal what was never recorded.
`mask_in_ui = TRUE`	At read time, per query.	Yes — it is policy, and it applies retroactively to historical evidence the moment you set it.
Data classification	At read time, by role. `internal`/`confidential` need `evidence.read_raw`; `restricted` needs `audit.read`; `regulated` needs `evidence.export_restricted` (auditor only).	Yes, with the right role.

Two honest caveats:

The UI does not currently distinguish "never stored" from "stored but you may not see it" — both show [masked]. For an audit conversation that distinction matters; check the mapping's store_raw_diff_values setting to tell them apart.
A value suppressed purely by classification currently renders as NULL rather than [masked], which is indistinguishable from a genuinely null value. Do not rely on the chip alone when classification gating is in play.

Mismatch-key hashes are never masked — they identify rows, they do not reveal contents.

Exporting¶

Export CSV exports the entire result set, not just the page you can see — it walks every server page. Files are written with a UTF-8 byte-order mark so Excel opens them correctly, and masked cells contain the literal [masked].

Requires the evidence.export permission. Without it the button is visible but disabled.

Evidence bundles — the auditor deliverable¶

From a terminal run's detail page, Export for Auditor produces a signed ZIP. This is the artifact you hand to someone who does not have access to the platform and should not have to trust it.

What's inside¶

Folder	Contents
(root)	`manifest.json`, `README.txt`
`run/`	Run summary, a human-readable execution narrative, run attempts, watermark observations, in-flight rechecks
`config/`	The resolved config snapshot, its checksum, and the known column set — the exact policy this run was pinned to
`evidence/`	Proof results, mismatch keys, field diffs, schema-drift events, run metrics, diff limits
`audit/`	The audit-log entries for the run and the status-transition chain
`verify/`	`verify_bundle.py` and, for asymmetric (KMS / ECDSA) signing, `public_key.pem`

Why it is tamper-evident¶

The manifest records a SHA-256 for every file, a hash_chain over all of them, and a previous_bundle_hash linking to the prior bundle for the same mapping — that last field means a removed bundle is as detectable as an altered one. A detached manifest.json.sig signs SHA-256 of the manifest, so one signature covers exactly the bytes that ship. Which key signs is the tenant's Bundle signing key (Settings → Configuration defaults): a KMS key (awskms://…, azurekv://…, gcpkms://…) or a secret holding an ECDSA private key signs asymmetrically (ECDSA-P256) and the bundle embeds the public key and its fingerprint; a secret holding an HMAC key signs symmetrically. The Audit page shows which is in force and, for asymmetric signing, the public key and fingerprint to publish to auditors. Period exports are signed containers over individually signed per-run bundles.

Period bundles (a date range of runs for one mapping) are requested from the mapping page — Mappings → mapping → Runs tab → Period audit bundles: pick the range, Generate period bundle, and the row shows Generating… until the evidence worker has built and stored the container, then Download. The download is the exact signed container: the platform refuses to keep bytes that do not hash to the recorded content_hash, re-checks the hash as it streams, and the browser discards a file whose SHA-256 disagrees with the server's X-Content-SHA256 header. A container larger than the retention cap (evidence_bundle_max_persist_bytes, default 512 MiB) keeps its signed metadata but is labelled Too large to retain rather than offered as a download — unless an object store is configured. Per-run bundles are streamed at export and are not retained — re-export them from the run.

Object store for containers (#2026). Set Evidence bundle object store (Settings → Configuration defaults → Audit bundles; evidence_bundle_store_ref, tenant value with a platform fallback) to an S3 bucket (s3worm://<bucket>[/<prefix>]), an Azure Blob container (azureworm://<account>/<container>[/<prefix>]) or a GCS bucket (gcpworm://<bucket>[/<prefix>]) — the same destinations and cloud identity as the audit-chain anchors; the value is a destination, never a credential. From then on every period container is written to that bucket (create-once, the SHA-256 recorded in the object's metadata) and the platform database keeps only its location and hash, so the size cap no longer applies and the database and its backups stop carrying container bytes. The row shows Ready (object store) and Download streams the container from the bucket through the same integrity check — the platform serves the exact object it recorded (by version where the store keeps versions) and aborts the download if the bytes do not hash to the recorded content_hash. If the object has been removed from the bucket the download says so (object missing); if the bucket cannot be reached it says that instead — never "no bytes". A store that is configured but cannot be written fails the bundle request loudly rather than quietly falling back to the database. GET /evidence-bundles/store states the store in force for scripts. The deployment setting evidence_bundle_store_threshold_bytes (default 0 = all) can keep small containers in the database and send only larger ones out.

Retention applies to stored objects too: the purge deletes the object with the record and writes the store's answer per object on the audit entry — deleted, retained by store (the bucket's Object Lock / immutability / retention policy, or an identity without delete rights, refused; the entry carries the lock facts) or already absent. A refused delete is recorded as exactly that, never reported as freed.

Write-once anchoring (#2027). If the object store you name is a write-once store — an S3 bucket with Object Lock, an Azure container with an immutability policy, a GCS bucket with a retention policy — the container the platform writes there is your write-once copy, and the platform attests it: alongside the bundle's generation entry it writes a bundle.anchored audit entry naming the store, the exact object (bucket, key, version), the store's own statement of the protection, the date the protection lapses, and the container's SHA-256. (S3 and GCS state that date per object; an Azure container-level immutability policy does not expose a per-object date, so for Azure the entry carries the policy statement and no date rather than an invented one.) This is written only when the store confirmed the protection on that object — a bucket without a lock shows Store unprotected on the row and is never presented as an anchor. The row shows Anchored (hover for the lock mode and the protected-until date). Verify anchor asks the server to read the anchored object back from the store in full and compare it with the record: verified means the bytes match and the store still reports the protection; invalid means the write-once copy and the platform record disagree (one of them was altered); absent means the copy is gone; unverifiable means the store could not be reached — reported as exactly that, never as a pass. Scripts get the same from POST /evidence-bundles/{id}/verify (anchor_valid, anchor.state). When retention later removes the bundle record, the store's lock keeps the anchored copy; the purge's audit entry records how many anchored copies were kept and until when (anchored_copy_retained_until), and the retention line on this card says the same. This is the posture SEC 17a-4 / FINRA 4511 ask for: the record is on write-once storage from the moment it exists, the platform can prove it, and deleting it is refused by the store, not by application code.

Retention. Bundle records — and the stored period containers behind them — are kept for Bundle retention (days) (Settings → Configuration defaults; default 2555 = 7 years). The tenant value cannot go below the platform value: the platform row is a floor, not just a fallback. The built-in retention purge schedule (a durable Temporal singleton, on by default and dry-run-first for the first days after install — see Retention schedule below) that ages out run evidence enforces it: on its next run, every bundle generated longer ago than the retention window is deleted, its stored container goes with it (the purge checks the cascade and aborts if a container survives), and one audit entry per mapping is written before the delete — bundle and run ids, rows removed from both tables, bytes freed, the cutoff and the policy that applied. The mapping's Period audit bundles card states the policy in force, what is waiting for the next purge run, and the last purge on record (GET /evidence-bundles/retention is the same statement for scripts). A bundle ages from its own generation time, not from its run's start.

Retention schedule (#2025). The purge runs automatically on a cadence (daily by default), so your retention policy is enforced without anyone remembering to run it — the difference between a policy and an intention. It is safe to run automatically because a legal hold always freezes evidence you must preserve, and it is dry-run-first after install so a new deployment attests a summary before it ever deletes. Each fire — dry-run, execute, or a paused no-op — writes one audit row, so an auditor can list every scheduled run. An operator can pause the schedule (an approved, audited act): a paused schedule still fires but deletes nothing, which is itself a visible compliance fact. Settings → Retention Schedule shows the state (enabled, cadence, dry-run window, paused, last fire) and is where operators pause and resume.

The hash chain across a purge. The first surviving bundle still names the purged predecessor in previous_bundle_hash. That is retention, not tampering, and the server-side verification says so: the purge's audit entry records the newest hash it removed for the mapping, so verify_evidence_bundle reports the link as CHAIN_HEAD_PURGED — with the purge date and the policy applied — and chain_link_valid stays true. A predecessor missing with no purge entry naming it is BROKEN. The offline verifier prints the link for the record; it never judges it (it has no view of the predecessor) and it is never a reason on its RESULT line.

Legal holds — freezing evidence against the purge. When evidence must be preserved for litigation, a regulatory inquiry, or an active audit, place a legal hold. A hold freezes a scope — a whole tenant, one mapping, or a single bundle — and the retention purge skips it: a held record past its cutoff is never deleted. Placing and releasing a hold needs an approved evidence.legal_hold elevation (a second person approves, the same separation of duties as an ad-hoc purge) and both acts are audited. Settings → Legal Holds lists what is held and is where you place and release holds; a held run or bundle carries a Held label with the reason, and the Period audit bundles / retention card shows active holds and what they hold back (nothing under a hold is ever counted as "eligible" for the next purge). When the purge skips a held record it records the skip — which hold, how many rows, which table — so the trail shows the hold working, not a silent gap. Release the hold when the matter closes and the scope ages out under the normal policy again; a hold given an expiry stops protecting once it passes.

The bundle row and its audit entry are written before the ZIP is released. If either write fails, you get an error and no bundle — evidence is never handed out unrecorded.

Verifying — the part that matters to an auditor¶

# asymmetric (KMS / ECDSA) signing — anyone can verify with the PUBLIC key:
python verify/verify_bundle.py <path-to-this-zip> --public-key public_key.pem
python verify/verify_bundle.py <path-to-this-zip> --public-key-fingerprint <sha256-hex>

# HMAC signing — needs the tenant's signing key from its key custodian:
python verify/verify_bundle.py <path-to-this-zip> --key-from-env PARIDITY_BUNDLE_HMAC_KEY

It recomputes every file hash and the hash chain, then checks the detached signature. Exit code 0 means integrity and authenticity verified; 1 means tampered, unsigned, wrong key, or the wrong kind of key — the RESULT line names the reason. It needs only Python 3.9+ (no packages — ECDSA verification is built into the single file) and it does not contact Paridity — that independence is the point.

Where the public key comes from matters. The copy embedded at verify/public_key.pem is a convenience: a bundle rebuilt by an attacker would carry the attacker's key. The assurance path is a key — or just its SHA-256 fingerprint — obtained out of band from the data owner's KMS or key custodian (aws kms get-public-key, az keyvault key download, gcloud kms keys versions get-public-key). The verifier never trusts the embedded key unpinned; --public-key-fingerprint pins it to what you were given.

A server-side verification endpoint also exists for round-tripping a downloaded bundle, though there is no button for it in the UI today.

Masking modes¶

Mode	Effect
`masked` (default)	Applies the `store_raw_diff_values` chain and each row's stored masking.
`metadata_only`	Strips every value and every row hash — counts and structure only.
`full`	Raw values where policy permits. `platform_admin` only — a request from anyone else is silently downgraded to `masked`, and the downgrade itself is audited.

One documentation caveat: the README.txt inside each bundle states that fields with mask_in_ui are labelled <MASKED>. No code emits that label. Field-level mask_in_ui reaches the bundle only indirectly, when the write-time gate had already suppressed the value. Treat the README's wording as aspirational until it is corrected.

Finding earlier bundles¶

A terminal run's detail page lists Prior Audit Bundles with the generation date, who requested it, whether it was signed, and the file count and size. The list is read-only — re-downloading an earlier bundle is not available in the UI.

Masked values and Reveal¶

A masked cell shows [masked] because policy suppressed the raw value — either the store_raw_diff_values chain, or automatically because the field is encrypted at rest (a field with a decrypt expression never stores plaintext in evidence; there is no setting that overrides this).

Masked does not mean lost. For runs on tenants with an evidence key configured, the platform stores each side's value encrypted at run time, plus a keyed fingerprint. The fingerprint alone answers most triage questions — has the source changed again since the last run, do these rows share one wrong value — without anyone seeing content.

Reveal appears on masked rows for users holding evidence.read_raw (stewards, auditors, admins — the same permission that has always separated value evaluation from triage). It offers two modes, and the difference matters:

As of run (default) decrypts the stored values as the run compared them — a point-in-time record, not a re-read of the live systems — which is what impact assessment needs when a decision or report relied on the run.
Now (live) re-reads the current values directly from the source and target systems (it can take up to ~30 seconds while both systems are queried). Each side is labelled unchanged since run or changed since run, verified against the fingerprints the run recorded — so you know immediately whether the row moved after the comparison, including when it was deleted outright ("row no longer exists"). Use it to check whether a discrepancy has since been corrected before acting on it.

Two things to know before clicking either:

Every reveal writes an audit entry — who, which run, which keys and columns, and which mode. Viewing a sensitive value is itself an audited event, and the dialog says so up front.
Runs from before the evidence key was configured have no stored envelopes; Reveal explains this rather than failing. Only their fingerprints exist (and live mode reports "no fingerprint to compare" for those rows).

Enabling Reveal. Reveal depends on one tenant setting: the evidence key (evidence_kek_ref), a pointer into your secrets backend — never the key itself. A platform admin sets it on Settings → Platform & Tenant Defaults → Evidence key (secrets pointer); the same screen shows Masked-value reveal: available or not configured, and GET /tenant/config-defaults reports it as masked_evidence_reveal_available. Until it is set, the platform does not offer Reveal and then apologise: the Reveal control on masked rows is disabled and the Field Diffs view says why —

"Masked-value reveal is not configured for this tenant — set the evidence key under Settings" when the tenant has no key, or
"This run was recorded before the evidence key was configured; only fingerprints exist" when the tenant has one now but this run's pinned configuration did not (runs pin their configuration at start; the key takes effect on each mapping's next run).

Each run's detail carries reveal_available_as_of_run, reveal_available_live and masked_evidence_reveal_configured so an integration can tell the two cases apart the same way. Development stacks ship with a dev-only key already configured (env://RECON_DEV_EVIDENCE_KEK); a real deployment points at AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, Infisical or Vault. See the configuration reference for the option itself.

Our repair posture¶

One-line answer: Paridity will tell you exactly what differs and can hand you the remediation to apply — but it will never apply it itself. Every operation it performs against your production systems is a read, and we ask you to grant it read-only credentials so that stays true regardless of our code. That is a deliberate decision, not a missing feature.

This note exists so that everyone gives the same answer to the question every operator asks about ninety seconds after their first failed run: "fine — now fix it."

The position¶

Three sentences, in the order a buyer needs them:

Paridity never writes to a dataset it reconciles. Every operation the connector layer performs against your data is a read: introspect, count, fingerprint, extract, fetch-by-key. Two narrow exceptions exist and are named below — neither touches reconciled data.
Paridity will generate the remediation for you — handing you those artifacts is on the roadmap as an evidence export, not as an execution capability. The architecture is specified in plan_repair_generation_v1.md (internal engineering spec) (#1532). Note it generates from a fresh read, not from stored evidence, for the reason given in limit 2 below.
You review and apply it through your own change-control process. That is where a production data change belongs.

We are not neutral about this. An audit product that can silently mutate the data it audits is a weaker control, not a richer one.

Be precise with a security reviewer¶

No connector contains write DDL or DML, and no caller anywhere in the platform asks a connector to write — the entire invoked connector surface is reads, describes and connection lifecycle. But do not claim the platform "cannot" write. Three things a reviewer will find, which you should raise first:

1. The generic SQL door. The connector interface exposes execute_scalar(sql), used for liveness probes (SELECT 1) and the platform's own read queries. Nothing in the product hands it a write statement, but the method would execute one if something did.

2. Operator-authored SQL is DML-blocklisted on both of its surfaces. source_filter_sql / target_filter_sql are validated against a keyword blocklist (DROP, INSERT, UPDATE, DELETE, MERGE, EXEC, …) at the API boundary and re-validated at run time. Since #1546, custom watermark SQL — the statement fields (watermark_sql, max_ts_sql, max_partition_sql, lsn_sql, the delta_version version_sql override (gated since #3049), the custom_sql override) and the marker_filter_sql fragment — passes the same two-layer gate, and the statement fields additionally must start with SELECT/WITH, so the read-only property there is a checked shape, not just a blocklist. A value persisted before the gate existed blocks the run at config resolution rather than reaching the engine. The honest residual: a keyword blocklist plus a SELECT-prefix check is not a SQL parser — an engine-specific writeable construct inside a SELECT (a function with side effects the operator's own grants permit) is not statically detectable. Read-only credentials remain the control that makes this structural.

3. Qlik Cloud creates and deletes a scratch app. On the Qlik Cloud engine path, Paridity creates an ephemeral app in an operator-designated scratch space, loads data into it to read, and deletes it. That is a genuine write into the customer's Qlik tenant — of an ephemeral compute artifact, in a space the operator names. It never writes to a reconciled dataset.

Therefore: grant Paridity read-only credentials. That is the control that makes the guarantee independent of our code, and it is what we recommend in every deployment. Tell a reviewer the platform never writes to their data and that their own grant is what enforces it — not that it is incapable. The stronger claim collapses under five minutes of scrutiny and takes the rest of your credibility with it.

Why we do not execute repair¶

The evidence and the fix cannot come from the same authority. Paridity's output is used as a control record — evidence that a dataset was proven complete and correct at a point in time. If the same system could also change the data that record describes, the record becomes an assertion about something the asserting party could edit. Separating the two is the whole reason an independent reconciliation tool is worth buying instead of trusting the replication vendor's own self-assessment.

Read-only is a materially easier security approval. A platform that needs write privileges on production ERP and warehouse systems is a different risk conversation from one that asks only to read. Keeping the capability absent — rather than present and restricted — is what makes that conversation short, and it is why we would rather not build repair than build it and gate it.

An automated repair on a moving target is unsafe. See coverage limits below: the evidence describes a window that has closed, while replication keeps running. A fix generated from that evidence is a proposal about a past state. A human applying it under change control can account for that; an automated writer cannot.

What we offer instead¶

Everything below ships today.

All paths are under /api/v1.

You want to…	Use
See which rows differ	`GET /api/v1/runs/{id}/mismatch-keys` — the differing keys and their per-row fingerprints (hashes only, no row payload)
See which columns within them differ	`GET /api/v1/runs/{id}/field-diffs` — column-by-column source vs target
See the pass/fail proof itself	`GET /api/v1/runs/{id}/proof` and `.../summary`
Hand an auditor something they can check without trusting us	`POST /api/v1/runs/{id}/evidence-bundle`, then `POST /api/v1/evidence-bundles/{id}/verify` — a signed, hash-chained zip shipping its own offline verifier
Read a sensitive value that was masked	`POST /api/v1/runs/{id}/field-diffs/reveal` — see the limits below
Raise it where the work happens	Jira / ServiceNow / PagerDuty / Slack / Teams / webhook / email notification channels
Know whether your fix actually worked	The open-mismatch loop, below

The open-mismatch loop is the part people miss, and it is the strongest answer to "but you don't fix it." Failing keys persist across runs in run.open_mismatch_keys, and every subsequent reconciliation rechecks the still-open keys before it is allowed to conclude pass_clean (src/services/open_mismatch_service.py). So when you apply a correction through your own change control, the next run confirms it — and if it did not work, the platform will not let the mapping report clean. You get closed-loop verification of a repair you performed, which is a stronger control than an automated repair nobody re-verified.

Coverage limits — say these out loud¶

A generated remediation is only as complete as the evidence behind it. There are four reasons the evidence may not cover everything, and an SE should raise them before a customer discovers them.

1. Sensitive columns may carry no values at all. Raw diff values are stored only when an AND-chain permits it: tenant (L2) AND mapping (L4) AND field (L5) must all allow it. Any FALSE stores NULL and sets values_masked instead (config_resolver.py:594-600, diff/activities.py:441-446 and :858-860). Separately, any field configured for decryption is masked unconditionally, regardless of the chain. These are usually exactly the columns a customer most wants fixed — and the correct value is not in the evidence to build a statement from.

There is a bounded escape hatch, not a bulk one: if the tenant evidence encryption key is configured, the diff worker also stores an encrypted point-in-time envelope, and a user holding evidence.read_raw can reveal at most 50 diffs per call, as-of-run only, with one audit entry per call. If that key is not configured, masked values degrade to plain NULL and the values are genuinely unrecoverable from evidence. Either way this is a reviewed, audited lookup — not a pipeline you can generate a mass repair from.

2. What we store is the comparison rendering, not the original value — and this one applies everywhere, not just to masked columns. source_value and target_value hold the canonical form the comparison used: strings trimmed, numbers quantized to the configured numeric_scale, timestamps truncated to timestamp_precision, per-field transforms applied. For a CLOB or BLOB in hash mode it is a SHA-256 digest or hex content; in metadata_only mode it is a length marker such as BLOB:1024 (or SEMI:<code points> for a JSON / XML column, #2996).

This is the strongest reason not to auto-execute a repair, and it holds even when nothing is masked and nothing is truncated. A statement built naively from source_value could write a trimmed string, a rounded number, a truncated timestamp, or the literal text BLOB:1024 into a production system. The evidence is a faithful record of what was compared; it is not a value of record to write back.

3. Caps can truncate the diff set — or skip it entirely — on large failures. Several independent limits apply, and a big failure is exactly when they fire:

fail_fast_threshold_pct — if the mismatch rate reaches this (50% by default), the entire field-diff phase is skipped. The run terminates fail_count if the row counts differ, otherwise fail_hash - never fail_field, because no column was compared - with zero field diffs, recording fail_fast_triggered and the observed percentage.
max_diff_rows - a hard cap (500,000 by default) on the comparable mismatch keys loaded (keys present on both sides), taken in key order and applied before sampling and the limits below; on a partitioned run it applies per partition. When it fires, the run records it in limits_applied (as __max_diff_rows__), and the run's Process Steps say how many of the comparable rows were actually fetched.
egress_row_limit, per_column_diff_limit, global_diff_cell_limit (migration 060) — each stops diff production when hit.

The run records which limit fired in proj.diff_run_limits, and the API exposes the result as diff_truncated — so incompleteness is visible rather than silent. But "visible" is not "complete". Note one asymmetry worth knowing: only egress_row_limit marks individual rows truncated; a per-column or global-cell cap leaves the surviving rows unflagged, and the fact of truncation lives at run level.

Sampling belongs on this list too (live since #1543). diff_sampling_mode sample_n / sample_pct with a diff_sample_size diffs a deterministic hash-selected subset of mismatched rows; sampled diffs are flagged is_sampled = TRUE and the total mismatch count is still recorded. (Before #1543 the size never reached the resolved config snapshot and sampling silently did nothing — runs under a config version minted before that fix still diff 100%.) For repair generation it is one more declared coverage bound: a sampled run's evidence names only the sampled keys.

4. Evidence is a point in time; replication is not. A run's evidence describes the window it compared, anchored to an immutable compare_started_at. Replication keeps moving afterwards. A remediation derived from that evidence is a statement about a state that has already passed, which is the single strongest argument against executing it automatically — and the reason a human applying it under change control is the right design, not a compromise.

The honest summary for a customer: we can tell you precisely what was wrong at a proven moment, and we will not pretend that is the same thing as knowing what is wrong right now. Where evidence is masked, sampled or truncated, we say so in the run record rather than emitting a confident-looking fix built on a gap.

Competitive context — the honest version¶

Most of the replication-adjacent field ships some form of repair, and buyers arriving from those products will ask:

Product	Repair capability
Oracle GoldenGate Veridata	Generates Repair SQL; 26c extends it to Databricks/MongoDB/SingleStore
Fivetran HVR 6 Compare/Repair	Compare-and-repair inside the replication channel; can refresh target data
AWS DMS validation	Data resync
Striim Validata	Repair SQL
Syniti	Repair
Datafold Data Diff	No general repair product documented
QuerySurge	No automated repair documented

Do not say "every competitor ships repair." It is not true, and it concedes ground we do not need to concede. The accurate framing is that the tools bundled with a replication channel repair through that channel — which is exactly the independence problem we sell against. The dedicated comparison tools, which are the products closest to what Paridity is, largely do not.

Note also what repair means for the competitors that have it: HVR repairs through HVR, and DMS resyncs through DMS. Their repair works because they own the pipe. Paridity is deliberately independent of the movement channel, so "just repair it" was never available to us on the same terms — and the independence is the thing customers are buying.

Sources: Veridata repair, HVR Compare, AWS DMS validation. Full scan: the Paridity industry position assessment, 2026-08-04 (internal research).

Answering the objection¶

"So I still have to fix it myself." Yes — and you would have had to review an automated fix before trusting it anyway. What Paridity removes is the expensive part: finding out precisely which rows and which columns are wrong, and being able to prove it afterwards.

"Veridata will just do it." Through the GoldenGate channel it controls. If your control objective is independent assurance, a tool that both moves the data and repairs it is the thing being assured, not the assurance.

"Is this on the roadmap?" Generating remediation artifacts from evidence is. Executing them is not, and is not a sequencing decision we expect to revisit — it is the posture.

Classification: Customer Confidential. The position stated here is deliberate; if a deal appears to require executed repair, escalate rather than implying it is coming.

Access, roles and audit¶

Navigate: Administration → Users & Access (/administration/users), Governance → Elevations (/elevations), Governance → Audit Log (/audit).

Paridity's access model has three moving parts: a role says what you can do, a grant says where you can do it, and an elevation grants a role temporarily with a second person's approval.

Roles¶

Eight roles. Four of them — platform_admin, tenant_admin, operator, and auditor — carry a shared read set covering mappings, systems, runs, masked evidence, metrics, and the audit log; dba, steward, analyst, and viewer hold only their role-specific grants.

Role	What it is for	Notably can	Notably cannot
Platform Admin	The installation owner	Everything, including granting `tenant_admin`, setting platform defaults, and the five privileged sub-gates (configure decryption, asymmetric transforms, broad anomaly policy, a notification channel's credential and destination, an identity provider's issuer / client secret reference / JWKS base)	—
Tenant Admin	Full authority within one tenant	Everything a platform admin can, within the tenant: register and manage SAML sign-on, edit an identity provider's name, enabled state, audience, client id and email domains	Grant `tenant_admin`; change platform settings; the five sub-gates - so a tenant admin cannot add an OIDC provider or change which secret it uses or which host the sign-in talks to (those pick a deployment-held secret and where its value is sent)
Operator	Day-to-day reconciliation work	Create/edit mappings, trigger and cancel runs, schema refresh, field config, schedules, export evidence	See raw evidence values; manage users or grants
DBA	Connection custodian	Read and update systems and their connections, run discovery	Touch mappings, evidence values, or governance
Steward	Data owner for a domain	Update mappings and classifications, read raw evidence, acknowledge and resolve anomalies	Read the audit log — which is why `restricted` classification is closed to stewards
Analyst	Reads results	Read masked evidence and metrics, acknowledge anomalies, export	Read raw values; read the audit log
Auditor	Independent review	Read-only everything, raw evidence, restricted/regulated exports, audit export	Any write at all
Viewer	Dashboard only	Metrics	Everything else

The audit log is broader than you might expect. operator inherits audit read access through the shared read set. If your control design assumes only admins and auditors can read it, verify that against your role assignments.

Permissions that are allowed but held¶

Seven permissions resolve to "allowed, pending approval" rather than a straight yes: mapping.bypass_readiness, mapping.mass_delete, evidence.retention_purge, evidence.legal_hold, evidence.export_restricted, principal.create_service_account_broad, and grant.manage_tenant_admin.

Holding the role is not enough — an active approved elevation must clear the gate. This is the break-glass boundary.

Grants — where a role applies¶

A grant binds a role to a scope. Resolution, in order:

A tenant-scope grant covers everything in the tenant.
An exact scope match covers that resource.
Business domains inherit: a grant on a parent domain covers every descendant domain.
A business-domain grant narrows list endpoints instead of denying them, and covers an individual resource only when that resource's own domain is at-or-under the granted domain. This is enforced: opening a mapping, system, schedule, replication profile, mapping group, run, evidence page or evidence bundle outside your domain by direct link returns "outside your business-domain scope" rather than the object. Creating a mapping, system or replication profile in a domain you do not hold is refused the same way. The Evidence page's own mismatch-key and field-diff tables, and the evidence-bundle list, are scoped the same way — the list shows only bundles for mappings in your domains.

Two things a business-domain grant does not confine, by design: identity and grant metadata (principals, grants and business-domain records have no domain owner, so they stay tenant-readable for roles whose bundle includes them), and auto-config apply, which creates its mappings in the tenant's root domain.

Grants come from three places — assigned directly, inherited via group membership, or synthesised from an active elevation. The Users page labels the source, and only direct grants can be revoked there.

Disabled principals get nothing, and expired grants stop applying immediately. Authorization decisions are recorded with the permission, scope, matched grant and actor IP.

Every decision is recorded on its own, so the record does not depend on what happened to the request afterwards: a refusal that stops the request at the door — the 403 you get for opening something outside your domain — leaves its deny row, and a request that was allowed and then failed for some unrelated reason still leaves its allow row. Read the decision log as a complete list of the access checks the platform performed.

Upgrading from an older release? Before this was fixed, a refusal that raised the 403 lost its row with the rolled-back request session, so decision history recorded before the upgrade under-reports denials. Only checks that ran inside a request which still returned a response were kept.

What is recorded is what was enforced, not what the grant alone would have allowed. Each decision carries a reason:

Reason	Recorded as	Means
`grant`	allow	a matched grant permitted it
`no_grant`	deny	no grant covered the request
`approval_required`	deny	a grant matched, but the permission is approval-gated and no active elevation cleared it — the request was refused with `APPROVAL_REQUIRED`. The matched grant is still recorded, so you can see the base entitlement behind the refusal
`elevation`	allow	an active approved elevation cleared the gate; that elevation's id is recorded alongside

Decisions written before this was introduced have an empty reason — they are left as they were rather than reclassified after the fact.

Assigning a role¶

Users & Access → expand a user → Assign Role. Choose the role, then the scope: Entire tenant, Specific business domain, or Specific system. For platform_admin and tenant_admin the scope picker disappears — those are always tenant-wide — and a reason is mandatory for the SOX trail.

You can only hand out what you hold. The role list offers only roles at or below your own highest tenant-wide role, and the API enforces the same rule:

a role above yours is refused with 403 GRANT_ROLE_ABOVE_CALLER. Only a platform_admin can grant (or revoke) platform_admin; a tenant_admin can grant every other role. The same ceiling applies to revoking another principal's grant, so a tenant_admin cannot remove a platform_admin's access. Removing one of your own grants is always allowed.
you cannot grant yourself, or a group you belong to, a role you do not already hold: 403 GRANT_SELF_ESCALATION. Ask another administrator.
an active break-glass elevation does not raise this ceiling - it is time-boxed, and grants are not.

Every refusal is written to the audit log as grant.refused (warning), with who asked, for whom, which role and why.

Three things to know before you rely on this screen:

Granting tenant_admin needs a second administrator. It records a pending approval (HTTP 202) and creates no grant yet; the dialog says so and stays open until you close it. The request appears in the Pending approvals card on this page, where a different administrator holding approval.decide approves or rejects it, with an optional comment. Approving creates the grant (Granted by = the approver). You cannot approve your own request (403 SELF_APPROVAL_FORBIDDEN - the button is disabled), and the role ceiling above applies to the approver too: a tenant_admin held only through a break-glass elevation cannot approve, and nobody approves their own promotion. You can Cancel a request you raised while it is pending. Requests expire after 7 days and then show as Expired; raise the grant again. Each decision is audited (approval.approved, approval.rejected, approval.cancelled); "Show decided" lists past requests.
Grants created in the UI never expire. The API supports an expiry; the dialog does not expose it, so the Expires column will read "Never".
The API supports six scope types (tenant, business domain, system, replication profile, mapping group, mapping); the UI offers three.

Users and service accounts¶

Users are created automatically on first SSO login, or pre-registered here so their roles are ready before they arrive — matching is by the email their identity provider asserts.

Add Service Account issues an API key shown exactly once. Copy it then; it cannot be retrieved later. Keys look like dak_<prefix>_<secret>; new keys carry a 12-character prefix, and keys issued earlier — with an 8-character prefix — keep working with no deadline and nothing to rotate. Issuing and revoking a key are both recorded in the audit trail alongside the grant events.

Send the key as an X-API-Key header. It authenticates as the service account itself, so what it can do is decided by the grants you give that account — a service account with no grants can sign in and read nothing.

Two notes for anyone upgrading. Service accounts created before this release were never fully registered, so their keys were refused on every call; the upgrade repairs them in place and their existing keys start working, with nothing to re-issue. And a service account's name must be unique within your organisation — creating a second one with a name already in use is refused rather than silently accepted.

Disable user prevents login while preserving the audit trail — prefer it to deletion. Mark break-glass lets an account sign in with local credentials during an SSO outage; those logins are audited.

Reset identity binding is the recovery path after an identity-provider migration. Paridity links an account to the provider that first authenticated it; a later sign-in for the same email from a different provider is refused rather than silently re-bound, because a silent re-bind is how an account gets taken over. The refused user sees "Sign-in refused" and is told to contact you — retrying will not clear it.

Resetting unbinds the account so the user re-links on their next single sign-on. The account, its roles and its history are all preserved. A reason is required and is written to the audit log. Two refusals worth recognising: the account has no email address to key the placeholder on (set one first), or another account already holds that placeholder (merge or disable it first).

Elevations — temporary privilege with a second pair of eyes¶

Navigate: Governance → Elevations. Everyone can see this page, but non-approvers see only their own requests.

The flow¶

1. Request. Choose a role, give a reason of at least 10 characters. The elevatable roles are Operator, Tenant Admin, DBA, Data Steward, Auditor — platform_admin is deliberately excluded, because that path requires explicit grant management rather than break-glass. Requests raised from a mapping's detail page are scoped to that mapping; requests raised here are tenant-wide, and the dialog warns you so.

2. Approve. Only platform_admin and tenant_admin hold elevation.approve. The reviewer sets a duration in hours — default 4, maximum 72.

Separation of duties is enforced. The requester cannot approve their own request, even though tenant_admin holds both permissions. Attempting it returns a clear 403, and the rule is also embedded in the database update so it cannot be bypassed by calling the service directly.

3. In force. While approved and unexpired, the platform synthesises the granted role — tenant-scope, or mapping-scope if a mapping was named — and it can clear the approval-gated permissions listed above. Your session reflects an elevation within about 15 seconds.

4. End. Either it expires (evaluated at check time — the row keeps showing approved with a past date rather than flipping to expired), or someone revokes it. Revocation is limited to the original requester or a holder of elevation.approve.

Audit log¶

Navigate: Governance → Audit Log. Readable by platform admin, tenant admin, operator and auditor. Append-only — nothing on this page mutates it.

Every entry carries who, what, when, where, why and outcome: actor with IP and source, entity type/name/id, action, category, severity, a reason for privileged actions, a materiality line (N mismatched rows · N field diffs · N open issues) for run outcomes, an expandable before/after field diff, and correlation ids. Run-related entries deep-link straight to the run's evidence.

Placing and releasing a legal hold, every retention-purge execution (including a dry run and a paused no-op), and every period bundle anchored to a write-once store are all recorded here.

Acknowledging or resolving an anomaly is recorded too (anomaly.acknowledged / anomaly.resolved), with who, from which IP, the note or reason given, and the before and after of the acknowledgement and resolution fields.

Tamper evidence¶

Entries are sealed into a hash chain. The header strip shows the signer, the sealing mode, checkpoint count, last-sealed time, and how many entries are pending seal.

Verify integrity recomputes the chain. Success tells you how many sealed entries are intact; failure names the exact entry where the chain breaks, and distinguishes a broken chain from a signed-anchor mismatch — the latter catching a wholesale re-seal that a plain chain check would miss.

Security Posture¶

Administration > Security Posture (/administration/posture) lists every unsafe or unverified setting of the deployment - the same checks the platform runs at startup in production (#2134) - evaluated on any environment. Each finding shows a severity, the consequence, and the Helm value or environment variable that fixes it:

Severity	Meaning
Fatal	In production the platform refuses to start with it. Elsewhere it is reported, not enforced.
Warning	Weakens tamper evidence or transport security; the platform keeps running.
Unverified	The deployment did not declare the fact (for example a non-Helm install). Unverified is not the same as on.

A development deployment always has findings (no external anchor, undeclared network policy) and the page says so in a banner. An empty report reads "No findings across N checks" and lists the checks; a check that could not be measured in this environment (the development-signer check depends on a production-only helper) is listed as not evaluated, never as passed. The Audit Log header strip carries the same consequence text next to "Sealing: Manual" and next to the anchors indicator, so the strip never shows a bare label. Read-only; the settings themselves are deployment-managed.

Export All walks every page and exports the complete matching set with proper RFC-4180 escaping and an Excel-friendly byte-order mark.

Filters¶

Entity type, entity name, entity id, action, actor, category (access, config, reconciliation, evidence, ops), severity (info, notice, warning, critical), source (ui, api, system, scheduler, migration), actor IP for access attribution, and a date range that defaults to today.

Entity types and actions are loaded live from the server, so the filter list can never drift from what the platform actually records.

Notifications and alerting — channel setup¶

Navigate: Notifications (/notifications) → Channels → Add channel (/notifications/channels/new).

Paridity sends outbound alerts — a run failed a proof, schema drift was detected, a terminal status was reached — to the tools your team already watches. A channel is a destination (a Slack workspace, a PagerDuty service, a Jira project); a rule decides which events go to which channel. This page is the per-channel setup reference: what credential each channel needs, the configuration fields it reads, how to create the credential in the vendor tool, and how to verify delivery before you rely on it.

Notifications are outbound only. Paridity never reads back from these tools except where a channel returns a ticket URL it just created (ServiceNow and Jira).

How credentials are stored¶

You never paste a raw secret into a channel. Every channel stores a credential_ref — a URI pointer into your secrets manager, exactly like a system connection (aws-secrets://…, azure-keyvault://…, gcp-secretmanager://…, infisical://…, or vault://…). The secret itself is resolved through the SecretsClient at the moment a notification is sent and is never written to the platform database. The non-secret routing information (a Slack channel name, a Jira project key, a ServiceNow instance URL) lives in the channel's channel_config and is not sensitive.

So for every channel below there are two things to provide:

The credential — created in the vendor tool, stored in your secrets manager, referenced by credential_ref.
The channel_config — the non-secret fields, entered in the channel form.

Verify a channel before you trust it¶

Every channel supports a test send. Use it immediately after creating a channel — a wrong token or a blocked URL surfaces here, not during your first real incident.

In the UI: open the channel and use its Send test action. It fires a synthetic "Paridity test notification" straight at the adapter (bypassing rules) and reports sent or failed with the provider's reason, cut to its first line (at most 200 characters). If the channel's credential pointer cannot be resolved, the reason says so without the secrets backend's own text; that is in the server log under the request's reference (X-Request-ID).
By API: POST /api/v1/notification-channels/{channel_id}/test. Requires notification_rule.create (roles platform_admin, tenant_admin, or operator).
ServiceNow and Jira only — POST /api/v1/notification-channels/{channel_id}/test-ticket opens a real ticket titled [TEST] and returns its URL, so you can confirm the ITSM integration end to end (and then close the test ticket).

A failed test returns the real vendor error (an invalid Slack token, an AADSTS7000215 bad client secret, an SSRF-blocked URL), which is almost always enough to fix the setup on the spot.

Who may set a channel's credential and destination¶

A channel pairs a secret (the credential_ref) with the place it is sent (a URL or host). Whoever sets both decides where a secret goes, so:

Setting or changing credential_ref, or a destination field, needs platform_admin. The destination fields are base_url (Jira), instance_url (ServiceNow), url (webhook), and smtp_host, smtp_port and tenant_id (email). Slack and Teams incoming webhooks carry their destination inside the secret, so for them the credential_ref is the destination. Anyone else gets HTTP 403 CHANNEL_DESTINATION_REQUIRES_ADMIN.
An operator can still rename a channel, turn it on or off, edit its other settings (recipients, project key, issue type) and run Send test. Resending the stored value unchanged is not a change.
A channel's credential_ref must be its own. A ref that names a system's connection_ref, an AI provider or identity-provider secret, a decrypt key, a signing or encryption-key ref, a replication tool credential or a platform signing setting is refused with HTTP 422 CHANNEL_CREDENTIAL_CROSS_USE, and a stored channel that names one is refused at send time.
Every change of credential_ref or a destination writes a notification_channel.destination_changed audit entry with the actor, the old and new destination hosts, and a SHA-256 fingerprint of the ref (never the ref itself or the secret).

All seven channels below were live-delivery verified end to end on 2026-08-24 (#1712 / PR #1961) — each one both delivering a real message and surfacing a deliberately bad credential as a failure.

Slack¶

Credential: either a Slack Incoming Webhook URL or a Bot OAuth token (xoxb-…). The adapter auto-detects which you gave it: a value starting with xoxb- is treated as a bot token and posted via chat.postMessage; anything else is treated as the Incoming Webhook URL and posted directly.

channel_config:

Field	Required	Default	Notes
`channel`	Only with a bot token	`#general`	The channel to post into. Ignored on the Incoming Webhook path — a webhook URL already encodes its channel.

Set up an Incoming Webhook (simplest):

Go to api.slack.com/apps → Create New App → From scratch (Slack's current app-creation dialog; the old "From an app manifest" split is gone — pick From scratch, name the app, choose the workspace).
Open Incoming Webhooks, toggle Activate Incoming Webhooks on.
Add New Webhook to Workspace, choose the target channel, and copy the generated https://hooks.slack.com/services/… URL.
Store that URL in your secrets manager and reference it as the channel's credential_ref. Leave channel blank.

Or use a Bot token (needed if one app must post to several channels): add the chat:write OAuth scope, install the app to the workspace, copy the Bot User OAuth Token (xoxb-…), store it, and set channel to the target (e.g. #recon-alerts). Invite the bot to any private channel it must post to.

Note: the Slack Web API returns HTTP 200 even when a post fails ({"ok": false}); Paridity parses ok and reports those as failures, so a bad bot token shows up correctly in the test send.

Microsoft Teams¶

Credential: a Teams Workflows (Power Automate) webhook URL — the trigger URL of a "Post to a channel when a webhook request is received" flow.

Microsoft has retired the classic Office 365 "Incoming Webhook" connector (deprecation began August 2024, completing through end-2025). Do not use a classic Incoming Webhook — it will stop working. Paridity posts the Workflows-compatible Adaptive Card envelope ({"type":"message","attachments":[{ …adaptive card… }]}), which is what the Power Automate flow expects.

channel_config: none. (The adapter's docstring shows a card_color example, but the code does not read it — card accent colour is fixed per severity. Setting card_color has no effect; leave channel_config empty.)

Set up the Workflows webhook:

In the target Teams channel, ⋯ → Workflows (or start from Power Automate) and pick the template "Post to a channel when a webhook request is received."
Complete the flow (choose the team and channel it posts to) and copy the generated HTTP POST URL.
Store that URL in your secrets manager and reference it as credential_ref.

The card shows the severity, trigger, mapping, and run ID as a fact list.

PagerDuty¶

Credential: a PagerDuty Events API v2 Integration Key (also called the routing key). Events are sent to https://events.pagerduty.com/v2/enqueue.

channel_config:

Field	Required	Default	Notes
`severity_map`	No	`{}`	Maps Paridity severity (`info`/`warning`/`error`/`critical`) to a PagerDuty severity, which must be one of `critical`/`error`/`warning`/`info`. Unmapped values pass through unchanged.

Set up the integration:

In PagerDuty, open the target service → Integrations → Add integration, choose Events API v2.
Copy the Integration Key and store it in your secrets manager as the channel's credential_ref.

Deduplication: when an event carries a run_id, Paridity sets dedup_key = recon-<run_id>, so multiple events for the same run collapse into a single incident rather than paging repeatedly (verified live: 2 events → 1 incident).

Recommendation: keep PagerDuty Intelligent Alert Grouping ON for the service. (Our internal test rig runs it off to make each event individually visible; in production, grouping reduces noise and complements the per-run dedup_key.)

ServiceNow¶

Credential: either a Base64-encoded user:password string (sent as Authorization: Basic …) or a value beginning with Bearer (sent verbatim as a bearer token). Records are created via {instance_url}/api/now/table/{table}.

channel_config:

Field	Required	Default	Notes
`instance_url`	Yes	—	e.g. `https://yourinstance.service-now.com`. Missing → the channel fails with "instance_url is required."
`table`	No	`incident`	Target table.
`impact_map`	No	`{critical:1, error:2, warning:3, info:3}`	Paridity severity → ServiceNow impact (1–3).
`urgency_map`	No	`{critical:1, error:2, warning:3, info:3}`	Paridity severity → ServiceNow urgency (1–3).
`assignment_group`	No	—	Added to the record only if set.
`category`	No	—	Added to the record only if set.

Set up:

Create (or reuse) a ServiceNow integration user with permission to insert into your target table, or mint an OAuth bearer token.
For Basic auth, Base64-encode user:password and store that string in your secrets manager; for a token, store Bearer <token>. Reference it as credential_ref.
Set instance_url and, if you want them, impact_map/urgency_map, assignment_group, and category.

The created record's URL is returned, and you can dry-run the whole path with the test-ticket action, which opens a [TEST] incident.

Jira¶

Credential:

Jira Cloud (*.atlassian.net): [email protected]:api_token (sent as Authorization: Basic, REST API v3, description in Atlassian Document Format).
Jira Data Center / Server: a Personal Access Token (sent as Authorization: Bearer, REST API v2, plain-text description).

The adapter picks Cloud vs DC automatically by whether atlassian.net appears in base_url.

channel_config:

Field	Required	Default	Notes
`base_url`	Yes	—	e.g. `https://acme.atlassian.net`.
`project_key`	Yes	—	e.g. `RECON`. Missing `base_url` or `project_key` → the channel fails.
`issue_type`	No	`Bug`	Issue type name.
`labels`	No	`[]`	Added only if non-empty.
`assignee_account_id`	No	—	Cloud account ID (Cloud) or username (DC).
`priority_map`	No	`{critical:Highest, error:High, warning:Medium, info:Low}`	Unmapped severity → `Medium`.

Set up (Cloud):

Create an API token at id.atlassian.com → Security → API tokens for a user who can create issues in the project.
Store email:api_token in your secrets manager; reference it as credential_ref.
Set base_url and project_key (and issue_type if not Bug).

Set up (Data Center): create a Personal Access Token for the service user, store the raw PAT, and set the same channel_config.

The created issue's URL ({base_url}/browse/{KEY}) is returned; use the test-ticket action to open a [TEST] issue end to end.

Jira channels require platform build ≥ migration 184 (which added jira to the channel-type enum). Earlier builds could run the adapter but not persist a Jira channel.

Email¶

Email has two auth modes, chosen by channel_config.auth_mode. Pick graph for Microsoft 365 / Exchange Online (which has disabled basic SMTP AUTH tenant-wide), and smtp_basic for any other SMTP server.

Mode `smtp_basic` (default)¶

Credential: the SMTP password (or app password).

Field	Required	Default	Notes
`auth_mode`	No	`smtp_basic`	Omit for SMTP.
`smtp_host`	Yes	—	e.g. `smtp.example.com`.
`smtp_port`	No	`587`	—
`smtp_user`	No	`""`	The login user; also the default `from_addr`.
`from_addr`	No	= `smtp_user`	Envelope sender.
`to`	Yes	`[]`	Recipient list. Empty `to` → the channel fails.
`cc`	No	`[]`	—
`use_tls`	No	`true`	`true` → STARTTLS on `smtp_port`; `false` → implicit TLS (SMTP over SSL).

Mode `graph` (Microsoft 365 / Exchange Online)¶

Credential: the Azure app registration's client secret (never in channel_config).

Field	Required	Default	Notes
`auth_mode`	Yes	—	Set to `graph`.
`tenant_id`	Yes	—	Azure directory (tenant) ID.
`client_id`	Yes	—	App registration (client) ID.
`from_addr`	Yes	—	The sending mailbox UPN.
`to`	Yes	—	Recipient list.
`cc`	No	`[]`	—

Set up Graph:

In Entra ID → App registrations → New registration, create an app.
Under API permissions, add Microsoft Graph → Application permissions → Mail.Send, then Grant admin consent (this is an application permission, not delegated).
Under Certificates & secrets, create a client secret; store its value in your secrets manager and reference it as credential_ref.
Set auth_mode: graph, tenant_id, client_id, from_addr (the mailbox to send as), and to.

Production tenants — scope the app. Mail.Send as an application permission lets the app send as any mailbox in the tenant. Restrict it with an Exchange application access policy so the app may only send as the specific from_addr mailbox(es) you intend. Do this before enabling the channel in production.

Paridity uses the OAuth2 client-credentials flow and calls Graph sendMail (expecting 202 Accepted); it does not read the mailbox back. Auth failures surface the Azure error (e.g. AADSTS7000215 for a bad client secret) in the test send.

Webhook (generic)¶

Credential: a bearer token / API key, sent as Authorization: Bearer <credential> (your own static headers can override).

channel_config:

Field	Required	Default	Notes
`url`	Yes	—	Must be `https://`. Missing → the channel fails.
`method`	No	`POST`	Upper-cased.
`headers`	No	`{}`	Static headers merged into every request.

The adapter posts a fixed JSON envelope:

{
  "trigger": "schema_drift",
  "severity": "critical",
  "title": "...",
  "body": "...",
  "run_id": "...",
  "mapping_id": "...",
  "mapping_name": "...",
  "tenant_id": "...",
  "details": { }
}

SSRF protection (important for self-hosted receivers). Every channel adapter that sends a credential - webhook, Jira, ServiceNow, Slack, Teams, PagerDuty and Microsoft Graph email - is deliberately restrictive so it cannot be turned into a request forgery primitive (#3365 extended the webhook policy to all of them; SMTP email gets the address check below, without the https rule):

The URL scheme must be https — plain HTTP is rejected.
The hostname is DNS-resolved and rejected if it resolves to a private, loopback, link-local, or CGNAT address (10/8, 172.16/12, 192.168/16, 127/8, 169.254/16, 100.64/10, and the IPv6 equivalents). Point the webhook at a publicly resolvable HTTPS endpoint. This applies to a self-hosted Jira Data Center, an on-premises ServiceNow or an internal SMTP relay too: each is refused unless a platform admin has added it to the private destination allowlist (next section).
HTTP redirects are not followed, to prevent redirect-based bypass.
After connecting, the address actually connected to is checked again before any response is read (for SMTP: before the password is sent), so a DNS answer that changes between the check and the connection cannot reach a blocked address.

There is a deployment-level dev/CI escape hatch (NOTIFY_WEBHOOK_ALLOW_PRIVATE=1 in the worker environment) that disables the SSRF check so a docker-network sink can receive deliveries. It is environment-only — it can never be enabled through channel_config or the API — and must never be set outside a local dev or CI stack.

Since #2142 this policy is shared with the SSO paths (OIDC discovery and the JWKS fetch), and the escape hatch has a policy-wide name: OUTBOUND_URL_ALLOW_PRIVATE=1 relaxes it for every outbound surface in the process, which is what a dev stack running Keycloak on plain HTTP over a private docker network needs. NOTIFY_WEBHOOK_ALLOW_PRIVATE continues to work unchanged; either variable being set relaxes the check. Both are environment-only, both are dev/CI only, and the Helm chart sets neither — a production deployment has no way to turn the policy off.

Private destinations (internal systems)¶

A customer-hosted installation often sends alerts to systems on its own network: an internal SMTP relay, Jira Data Center, an on-premises ServiceNow, an internal webhook receiver. A platform admin can permit such a destination for the tenant under Notifications -> Private destinations (API: POST /api/v1/notification-private-destinations). Everyone who can read notification channels (notification_rule.read) sees the list; only a platform admin can add or remove entries (notification_channel.configure_destination, the same permission that already decides where a channel secret is sent).

Each entry is exactly one host and one port:

Host form	Example	Permits
Hostname	`jira.corp.example`	A channel URL or SMTP host that is exactly this name (case and a trailing dot ignored). No wildcards; `x.jira.corp.example` needs its own entry.
IP address or CIDR	`10.20.0.5`, `10.20.0.0/24`, `fd12:3456::/64`	A channel URL or SMTP host written as an IP address inside it. A CIDR never permits a hostname that happens to resolve into it.

The port is the URL's port (443 when https:// names none) or the email channel's smtp_port. A different port is refused.

What an entry permits. A destination whose host and port match an entry may resolve to an address in 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 or IPv6 fc00::/7. Nothing else about the policy changes:

HTTP channels still require https, and redirects are still not followed.
The connection must land on one of the addresses the name resolved to when it was checked. If DNS answers with a different private address at connect time, the send is refused (DNS rebinding cannot move an allowlisted name to another internal host).
The entry applies only to that tenant's notification channels. It does not apply to sign-in (OIDC / JWKS) or any other outbound path.

What an entry can never permit. Loopback (127.0.0.0/8, ::1), link-local (169.254.0.0/16, fe80::/10, which includes the AWS, Azure, GCP and OCI metadata address 169.254.169.254), the AWS IPv6 metadata address fd00:ec2::254, the Alibaba Cloud metadata address 100.100.100.200, unspecified (0.0.0.0/8, ::) and multicast addresses, and the names localhost / *.localhost / metadata.google.internal. Such an entry is refused when it is added (HTTP 422 PRIVATE_DESTINATION_NEVER_PERMITTED; a CIDR that overlaps one of these ranges is refused the same way), and such an address is refused at send time whatever the list says. A public address needs no entry and is refused as one (PRIVATE_DESTINATION_NOT_PRIVATE).

A tenant holds at most 200 entries. Each addition and removal writes a notification_private_destination.added / .removed audit entry (notice severity) with the actor and the host, port and note. Removing an entry takes effect on the next send.

An entry does not relax certificate checks. An internal endpoint whose certificate is issued by a private CA needs that CA trusted - see the next section.

Internal CA certificates¶

An HTTPS channel validates the server certificate against the public CA bundle shipped with the platform (certifi), and SMTP STARTTLS / SSL validates against the container's system trust store. An internal Jira, ServiceNow, webhook receiver or SMTP relay whose certificate comes from your own private CA fails that check until the CA is trusted.

Set TLS_CA_BUNDLE_PATH to a PEM file holding your internal CA certificate(s) (the root, plus any intermediates the server does not send). The platform adds these CAs to the default trust store; it does not replace it, so public destinations such as Slack, PagerDuty and Microsoft Graph keep working. It applies to every notification channel (HTTPS and SMTP) and to the other outbound fetches that share the same client, including OIDC discovery and JWKS for sign-in.

Certificate verification is never turned off. The server certificate must still chain to a trusted CA and match the host name; a certificate from any CA that is in neither the default store nor your bundle is still refused.

With the Helm chart, put the bundle in a ConfigMap or Secret and enable tls.caBundle. The chart mounts it read-only into the API and every worker and sets TLS_CA_BUNDLE_PATH for you:

kubectl create configmap corp-root-ca --from-file=ca-bundle.pem=./corp-root-ca.pem

tls:
  caBundle:
    enabled: true
    configMapName: corp-root-ca    # or secretName: <a Secret> - exactly one
    key: ca-bundle.pem             # the key that holds the PEM
    mountPath: /etc/paridity/tls   # TLS_CA_BUNDLE_PATH = <mountPath>/<key>

Naming both secretName and configMapName, or neither, fails the helm install / helm upgrade with a message saying so. With Docker Compose, mount the file into the api and worker-notification services and set TLS_CA_BUNDLE_PATH on both (the shipped docker-compose.yml has a commented example).

If the file is missing or is not a PEM certificate bundle, the API and the notification worker log an ERROR at start naming the path, and every outbound HTTPS / SMTP send refuses with the same reason until it is fixed. Nothing falls back to an unverified connection. A changed bundle is picked up when the pods restart.

Delivery history — was anyone told?¶

The Delivery history tab on the Notifications page is the platform's own answer to the question every operator asks after an incident: was anyone told, and if not, why not? It reads the append-only delivery log — every decision the platform made about a notification, newest first — and needs no database access.

Each row shows when the decision was made, the trigger and severity, the mapping, the channel, the status, and the run it belongs to (the run link opens the run's evidence). The status is the point of the page:

Status	What it means	What to look at
Sent	The provider accepted the message. `Sent at` is the acceptance time.	Nothing — unless the recipient still did not see it, in which case the problem is downstream of Paridity (mailbox rules, a muted channel).
Failed	The provider refused the message. The provider's error is shown in full in the row's details — for example Mailtrap's `550 5.7.0 Too many emails per second. Please upgrade your plan`.	The error text. It names the fix: a rate limit, a revoked webhook, an unreachable host.
Suppressed	Paridity decided not to send. This is not a failure — the row records the reason so a deliberate silence is never mistaken for a lost message.	The reason (below).
Pending	Recorded but not yet attempted. A row that stays pending is a worker that has not run.	Worker health.

Suppression reasons, in plain words:

cooldown — a notification for this rule, trigger and mapping was already sent inside the rule's cooldown window (set per rule; 300 s by default).
flag_off — automatic ticketing is switched off for this mapping.
no_channel — automatic ticketing is on for this mapping, but no auto-ticket channel is configured.
channel_disabled — the configured channel is disabled or no longer exists.
no_adapter — the configured channel's type cannot open tickets.
not_applicable — the mapping was not visible to the tenant when the ticket was evaluated.
resend_exhausted — a lost auto-ticket was re-sent the maximum number of times and the platform stopped retrying.

A quiet period is different from all four statuses: a rule that never matched leaves no row. If you expected a notification and see nothing, the rule did not fire — check its triggers, minimum severity and mapping scope.

Filter by status, trigger and channel, or open the tab from a run's Notifications card to see only that run. The history is read-only: it is evidence, and nothing on this page edits or deletes it. Visible to every role that can read runs, including analysts and auditors.

Access, roles and audit — who may create channels and rules, and how notification actions are audited.
Runs and Evidence — the events that trigger notifications.
Secrets are stored the same way as system connections — see your deployment's secrets-manager setup for creating a credential_ref.

Report a problem¶

Navigate: the Report a Problem action on an error message, on the error screen, on a run's page, or in the help drawer. Support is by email to [email protected].

When something goes wrong, the fastest way to get it fixed is to send us a Diagnostic Bundle with your email. It is one .json file that describes the problem without carrying your data, so we can diagnose it without asking you a round of follow-up questions.

When to use it¶

A run ended in a state you did not expect, or failed and you cannot see why.
An error message appeared in the application. Each one shows an error reference - include it; the bundle matches it to what the platform recorded.
A run or a screen seems stuck, slow, or wrong.
You suspect a false pass: a run reported clean and the data differs. Say so in the first line of your email. We treat it as the highest severity whatever the row count.

Who can create one¶

operator, tenant_admin and platform_admin. An analyst or auditor sees the error reference and is asked to pass it to an operator, who creates the bundle. See Access, roles and audit.

Every bundle that is created is recorded in the Audit Log: who created it, for which run or mapping, and whether names were included.

What the bundle contains¶

The preview in the application lists each section before you download. In plain words:

Section	What it carries
Header	The bundle format version, your Paridity version and build, your tenant's opaque id, the role of the person who created it, and the reference of the request that made it.
Error references	For each error reference you include: when it happened and what kind of action it was (time, action, entity type, category, severity). If your access is limited to some business domains, only records about the bundle's run or mapping are matched, and none when you name neither.
Run	The run's status, its phases with start and end times and durations, readiness outcomes, a schema-drift summary (counts only), lag-retry counts, whether a comparison window was found and how long it spanned, and the run narrative and triage with their wording replaced by tokens.
Mapping	The mapping's opaque id, the two engine types, the replication tool, the watermark strategy, the CDC mode, and the non-name configuration settings with per-field counts.
Anomalies	How many anomalies are open, by analyzer and severity. Counts only.
Workers	Each worker's id, queue, version, build and last heartbeat.
Health	Whether the database and the workflow engine answered.
Your browser	The page you were on (without any search text), your browser's user agent, and the time, status and code of recent errors the page showed.

The bundle's own redaction manifest says, section by section, what was kept and what was removed. You can read it in the file before you send it.

What it never contains¶

Under any setting, the bundle never contains:

data values - no field values, key values, watermark or window values, and no field-level differences;
hash totals or row hashes;
credentials, secrets or connection references;
SQL text or decrypt expressions;
column names;
raw logs;
the name or email of the person who created it.

Free text that could quote a name or a value - error messages, reasons, the narrative's prose - is replaced as a whole by a token such as <OBJECT:3f9a1c2e>. Tokens are consistent within one bundle, so we can see that two sections refer to the same thing, and they are salted for that one export, so they cannot be turned back into the names.

Including mapping and table names¶

By default every mapping, table and tenant name is a token. Tick Include mapping and table names when it would help us to know which table is affected. That keeps exactly:

the mapping name;
the source and target table names (schema.table). For a file or object-store path only the scheme is kept (for example abfss://); the bucket, account and path stay a token;
your tenant name.

Column names are never included, with or without this option. The choice is made for each bundle and is recorded in the Audit Log.

Sending it¶

Click Report a Problem, read the preview, choose whether to include names, and download the file.
Your email client opens a message to [email protected] with the error reference and the bundle's file name filled in.
Attach the downloaded file and send. Nothing is sent automatically - the bundle leaves your environment only as the attachment you send.

Paridity keeps a copy of the bundle for up to 24 hours in its own database, so a retried download returns the same file. It is not stored anywhere else and is never sent anywhere by the platform.

If you also have an evidence export for the run, you can attach it as well; see Evidence.

Troubleshooting¶

This page starts from what you see and works back to what to check. Most of what looks like a problem is the product refusing to give a verdict it cannot stand behind: a held run, a refused configuration, a schema it could not verify. Those are the platform working. The sections below say how to tell that apart from a real fault, and what to put in a report when it is one.

Every error the API returns carries a short error reference (the request id). It appears on the error message in the application and in the X-Request-ID response header. Quote it in any report: it is how support finds the server-side log line without you sending logs. See Report a problem for the Diagnostic Bundle, which collects the references, the run's decisions and the fleet state for you.

Start here - symptom to section¶

What you see	Most likely	Go to
A run reads Readiness hold · reason	Replication or the load pipeline had not finished; the run was deferred, not failed.	Readiness holds
A run reads Config refusal · family	The mapping cannot be compared as configured; a retry gives the same answer.	Config refusals
Schema drift, Schema unverified, Schema not verifiable	The live schema moved, or one side could not be introspected.	Schema drift and unverified schemas
Issue detected right after Running · Rechecking in-flight rows	Replication was still catching up when the rechecks ran out.	Rechecks ran out
The Engine Decisions log says the fingerprint ran on the reference path	A column type, transform or engine pairing cannot be hashed inside the database.	Reference path instead of pushdown
Diagnostics shows a worker stale, a version mismatch, or no worker serving a queue	A worker is down, mid-deploy, or running an older build.	Workers and versions
Runs sit at Running · Queued, · Throttled or · Checking readiness and never move	No worker is polling the queue, or the system's concurrency cap is reached.	Workers and versions
An API call answers 409 or 422 mentioning the Idempotency-Key	The same key was reused, or the first request is still being processed.	Idempotency conflicts
You land on the login page with a session expired notice	The access token expired or no longer validates.	Session expiry
Test Connection fails	Credentials pointer, network, or transport encryption.	Connection tests
A schedule is enabled but no run appears	The scheduler behind it is not confirmed running, or the schedule is outside its active window.	Schedules that do not fire
Evidence shows [masked] instead of values	A masking policy, by design.	Masked values
Something went wrong page, or a System problem · System error run	A platform or connector fault, not a statement about your data.	Platform faults

Readiness holds¶

Symptom. A run ends as Readiness hold with a reason: Replication latency, Replication stalled, Replication not running, Iceberg metadata lag, Load incomplete, Source unavailable, Replication status unknown or Configuration issue. Older runs may show the generic Source not ready.

Cause. Readiness checked the replication tool and the watermarks on both sides and could not find a window that both sides had finished delivering. The run was not compared. Comparing would have produced false mismatches.

What to check. Open the run. The Readiness card says what it was waiting on, what happens next, and what to look at, above the full evaluation trail. The reason names the direction:

Replication latency and Replication stalled: the replication job. The second one means the target did not move between checks at all.
Replication not running: the tool reported its task paused, stopped or failed. See Replication profiles.
Load incomplete: the load pipeline has not written its completion marker, or the target was still changing within the quiescence window.
Source unavailable: the platform could not read how far the data had loaded. Start with Test Connection on both systems.
Replication status unknown: strict CDC mode and the tool could not be reached. Either restore the tool's endpoint or relax the integration mode on the mapping if advisory behaviour is acceptable.
Configuration issue: the readiness settings themselves are invalid. Retrying will not help until they are corrected.

The two settings most often behind a recurring hold are max_lag_seconds (how far behind replication may be) and quiescence_seconds (how long the watermark must be stable). Both are in the configuration reference.

When to report. A hold that names Source unavailable while Test Connection succeeds on both systems, or a hold whose evaluation trail shows no evaluations at all. A hold with a clear reason and a replication job that is genuinely behind is not a platform issue. The full status vocabulary is in Runs.

Config refusals¶

Symptom. A run ends as Config refusal with a family: Object or connection config, Watermark config, Filter SQL, Duplicate keys, Partition strategy, Snapshot pin unavailable, Schema unavailable, Engine requirement, Data encoding, or Operator action required when no family applies yet.

Cause. After readiness passed, an activity hit a condition that no retry can change: a table or column that no longer exists, a revoked grant, a watermark that cannot be executed on that side, a filter that fails the SQL safety allowlist, more than one row sharing a key on one side, a partition strategy that overlaps or leaves rows uncovered, a time-travel pin outside the side's retention, an engine below the supported floor, or text that is not valid UTF-8. The platform refused rather than compare against a guess.

What to check. The run detail names the family and the refused setting (the message is VPC-local and never leaves your deployment). Fix what it names, then trigger a new run. Retrying without a change produces the same refusal. A Config refusal notification rule tells you when one happens; it is a warning, never a page.

When to report. A refusal whose family is Operator action required and whose message you cannot act on, or a refusal that recurs after you corrected exactly what it named.

Schema drift and unverified schemas¶

Symptom. One of: Schema drift · Not compared, Schema drift · Available data passed / failed, Pass · Schema unverified, Pass · Schema not verifiable, Schema unverified · Not compared, or the separate Schema review pending badge.

Cause. Every run introspects both sides and compares the live columns with the schema the mapping accepted. An added column never halts (it opens a pending decision). A lost or narrowed reconciled column follows the mapping's drift policy. A lost key column always halts. A side whose schema could not be read at all is recorded as unverified, and such a run can never present as an unqualified clean pass, whatever the policy.

What to check.

Not compared: the Schema Drift card lists what changed. If the change was intended, Refresh Schema on the mapping re-introspects and accepts the live tables as the new baseline.
Available data passed / failed: a reconciled column was excluded this run. Decide whether to accept the new schema or restore the column.
Schema unverified: introspection failed unexpectedly on a side. Almost always catalog permissions. The run detail records which side and why; the amber clears on the next run that verifies.
Schema not verifiable: the side is on an engine that structurally cannot introspect (raw object storage, for example). This is a standing property, not an alarm.

The policy knobs and their meaning are in Mappings.

When to report. Schema unverified on an engine that can introspect, after you have confirmed the service account can read the catalog. A drift finding on a table nobody changed.

Rechecks ran out (lag retries exhausted)¶

Symptom. The run spent time at Running · Rechecking in-flight rows and still ended as Issue detected. In the Engine Decisions log, up to three rows read lag retry attempted, each with how many keys were rechecked, how many resolved and how many remained.

Cause. When the proof fails on a mapping that declares a replication tool, the platform does not trust the first answer: it waits and rechecks the mismatched rows, three times, after 30, 60 and 120 seconds. If every mismatch resolves, the run passes as Pass · Resolved on recheck. If any remain after the third recheck, the run reports the issue. The recheck count and waits are fixed; they are not configurable per mapping.

A mapping with no replication tool gets no recheck at all unless a side was read at a pinned instant; its Process Steps card shows the In-Flight Recheck step as Skipped, with no replication tool as the reason. That is deliberate: without a tool there is no lag to wait out.

What to check.

Read the three recheck rows in order. A remaining count that keeps shrinking means replication was slower than about three and a half minutes of total wait: the rows are arriving, just late. Issues carry forward and are rechecked at the start of the next run, where they normally resolve. Watch the mapping's open-issue count fall.
A remaining count that does not move across all three rechecks, and again on the next run, is a real break. Open the evidence.
A recheck row with the reason code too many changed rows means the recheck was skipped because the changed-row set was larger than the recheck is allowed to re-read; the run's verdict stands on the proof.
If rechecks are routinely needed, replication is running close to your max_lag_seconds. Raising the readiness tolerance holds the run a little longer and avoids the recheck phase entirely.

When to report. Rows that the evidence shows present on the source and absent on the target for longer than your replication tool's own lag figure says they should be. That is a replication problem to raise with the tool's owner, with the run id and the mismatch keys export attached. A run that rechecks but writes no recheck rows at all is a platform issue.

Reference path instead of pushdown¶

Symptom. The Engine Decisions row for the fingerprint comparison says the hash did not run inside the database (pushdown no, fallback used yes), the Key Extraction step in Process Steps says row hashes were computed on the platform's reference path, the run took noticeably longer than usual, or an anomaly reports key-extraction throughput below its floor.

Cause. By default the platform computes each side's fingerprint inside the engine and moves only the aggregate. When a side cannot do that faithfully it declares so, and the platform computes that side on its own reference implementation instead. The result is still exact; it is slower and moves more data. When one side falls back, the other side is recomputed the same way so the two fingerprints are comparable. Common reasons a side cannot push down:

The two engines do not share a hash method, so neither side can be compared natively.
A transform in the field's chain is one the engines do not fold identically: uppercase, lowercase, collation fold, date truncate or numeric quantize. The chain editor says valid but forcing the reference implementation when you add one. See Field configuration.
A binary or large-object column compared content-exact on Oracle, DB2 or Redshift, which cannot hex a large object in SQL.
A JSON or array column, which is compared on canonical JSON rather than the engine's own text.
A hash plan for binary data over a column the engine declares as text.
An engine below the native floor: DB2 before 11.1, SQL Server before 2019.

What to check. The field grid for transforms and for BLOB compare modes (the editor marks a chain that forces the reference path). The two systems' engine versions. Whether the slowdown coincides with a configuration change.

When to report. A fallback on a mapping with none of the reasons above, or a run that became much slower with no configuration change on either side. The run narrative and the Diagnostic Bundle carry the decision rows; attach the run id.

Workers and versions¶

Symptom. The Diagnostics page (Monitor, operator and up) shows a worker as stale, a worker whose version or build does not match the API, no worker serving this queue, or runs that sit at Running · Queued or Running · Throttled or Running · Checking readiness and never advance.

Cause. Each worker writes a heartbeat every fifteen seconds. A row older than three intervals is stale: the process may be up, but it is not doing work. A worker that was replaced (a restart, a new pod) leaves its previous row behind until the platform retires rows older than a day, so a stale row whose queue also has a live worker is history, not an outage. The version column compares each worker's build to the API's: mismatch means an older or newer build is still serving; unverified means a build commit is missing on one of them (a development build), which is not the same as matching.

What to check.

Stale, and the queue has no live worker: the worker for that queue is down or its event loop is wedged. Restart it. Work sent to the queue waits until a worker polls it.
Mismatch during a deployment: expected while pods roll. Wait one heartbeat cycle after the rollout completes.
Mismatch after the rollout: a worker is running an image that was not rebuilt or restarted with the upgrade. Restart that worker's deployment.
Queued or Throttled and not moving, workers healthy: the mapping's systems have reached their concurrency cap. The run starts when a slot frees. Cancel if you need the capacity elsewhere.

When to report. A worker that stays stale after a restart, or a queue with a live worker whose runs still never leave Queued. Generate a Diagnostic Bundle; its fleet section and queue statistics are what support asks for first.

Idempotency conflicts¶

Symptom. An API client receives one of:

Status	Code	Meaning
400	`MISSING_IDEMPOTENCY_KEY`	Every state-changing request needs an `Idempotency-Key` header.
409	`IDEMPOTENT_REQUEST_IN_PROGRESS`	The first request with this key is still running. Retry after the `Retry-After` interval (five seconds) to receive its result.
422	`IDEMPOTENCY_KEY_BODY_MISMATCH`	This key was already used with a different request body. Use a new key for a different operation.
409	`DUPLICATE_IDEMPOTENCY_KEY`	On triggering a run: the `idempotency_key` in the request body was already used for an execution request.

Cause. The key is claimed before the handler runs, so of several identical requests exactly one executes; the rest receive the cached response for twenty-four hours, or the in-progress answer while the first is still running. The run trigger additionally keeps its own key on the execution request, which is unique per mapping and window.

What to check. The application mints a fresh key for every click, so these errors come from scripts and integrations. Reuse a key only to retry the very same request; mint a new one for a new operation.

When to report. An in-progress answer that persists for minutes on a request you know has finished, or a 503 from a notification-channel or ticket test (the key store is unavailable, and for endpoints with an external side effect the platform refuses rather than risk sending twice).

Session expiry¶

Symptom. The application returns you to the login page with a session expired notice, sometimes in the middle of a page.

Cause. An API call answered 401 while you held a token. The token expired, or the identity provider's signing key rotated so the token no longer validates. The application clears the session and sends you to sign in rather than render an empty page that looks like missing data.

What to check. Sign in again. If you are signed out again within seconds of signing in, the problem is between the platform and the identity provider: check Authentication settings (platform or tenant admin) and the identity provider's clock and key rotation. The error reference on the notice goes in the report.

When to report. Repeated immediate sign-outs after a successful login, with the error reference and the time.

Connection tests¶

Symptom. Test Connection on a system fails.

Cause. The pointer to the secret does not resolve, the network path is closed, the credential is wrong, or the server cannot offer encrypted transport and the platform refused to connect unencrypted.

What to check. The failure text is the first line of the driver's message, at most 200 characters; a timeout reads timeout. The full message is in the server log under the error reference. A PostgreSQL or MySQL failure mentioning TLS or SSL is the fail-closed default at work; see Testing a connection for the choice it asks you to make.

When to report. A test that fails with no reason text at all, or a credential that works from another client on the same network and fails here.

Schedules that do not fire¶

Symptom. A schedule shows enabled, but no run appears at the expected time.

Cause. Saving a schedule also asks the workflow engine to start the durable scheduler behind it. If that second step failed, the schedule looks enabled with nothing behind it. Otherwise the schedule is simply outside its active window: days of week, daily start and end, effective dates, or a close-mode toggle that disabled its tag.

What to check. The Next Run column. A red Scheduler not running badge or an amber Syncing badge that persists is the first case; the platform retries every maintenance tick and the badge clears when the scheduler worker and Temporal are reachable. No badge means the scheduler is confirmed running and the schedule's own window explains the gap. See Is the scheduler actually running it?.

When to report. No badge, a window that includes now, and still no run after one full interval.

Masked values¶

Symptom. Evidence shows [masked] where you expected a value.

Cause. One of three policies: the mapping never stores raw diff values, a field is masked in the UI, or your role is not permitted to see that data classification. The first cannot be undone by anyone; the values were never recorded. See Masking.

When to report. Never on its own. If you believe the policy is wrong, that is a configuration conversation with the mapping's owner.

Platform faults¶

Symptom. A Something went wrong page in the application, an error message with a reference and no actionable text, or a run that ends as System problem · System error.

Cause. A platform or connector fault: a worker lost its database, a driver raised something unexpected, an evidence write failed after its retries. None of these is a statement about your data.

What to check. A system-error run has no retry button; trigger a new run. If it recurs, it is a platform issue. The Diagnostics page's recent API errors and run errors list the references.

When to report. Any recurrence. Use Report a Problem from the error or from Diagnostics and attach the Diagnostic Bundle to the support email: it carries the references, the run's engine decisions, the fleet state and the queue statistics, with names tokenised by default and no evidence values. See Report a problem.

What a good report contains¶

The error reference (request id) from the message or the response header, and the time.
The run id when a run is involved.
The Diagnostic Bundle, generated by an operator from Report a Problem.
What you expected, what you saw, and whether it recurs.

Support will not ask you for logs, credentials or data values. Nothing the bundle contains leaves your deployment except by the email you send.

Error codes¶

Every error the Paridity API returns carries a stable, machine-readable code. This page lists every code the platform can return, with the HTTP status it comes with, the message, and what to do about it.

The error body¶

Every error response has the same shape:

{
  "error": {
    "code": "SYSTEM_NOT_FOUND",
    "message": "System 'sys_abc' not found",
    "detail": null,
    "request_id": "3f6c1e0a-..."
  },
  "detail": { "code": "SYSTEM_NOT_FOUND", "message": "System 'sys_abc' not found", "request_id": "3f6c1e0a-..." }
}

error.code - the code from the table below. Branch on this, never on the message text.
error.message - a human-readable sentence. When the raising site sets no message, it is the HTTP status phrase (for example Not Found).
error.detail - the code's extra fields (the table's Detail keys column, for example field or side); for a request that failed validation, the list of invalid fields; otherwise null.
error.request_id - the request's reference. The same value is in the X-Request-ID response header of every error response, including an unexpected 500. Quote it when you report a problem: it locates the full error in the server log.
detail - the pre-error shape, kept unchanged for existing clients. New clients should read error.

The MCP endpoint (/mcp) is the one exception: its transport-level refusals carry error.code and error.message only.

An error the platform did not anticipate is answered 500 with the code INTERNAL_ERROR and a fixed message; the details stay in the server log under the request id.

Reading the catalogue¶

HTTP status lists every status a code is returned with, when sites differ.
Message is the message text when every site that raises the code sends the same text. varies means the sites differ or the text is built at run time (it names the object, field or value involved) - read error.message.
Raised from names the platform module that raises the code and, in brackets, the shared validation class when the code comes from one. It helps support locate the source; you do not need it to act.
What to do is written only where the remedy follows from how the code is raised. Where it says No guidance yet, report the code and the request id (see Report a problem).

The catalogue is generated from the platform's code by scripts/gen_error_catalogue.py, and CI fails if the code and this page disagree, so a code cannot be added without appearing here.

Catalogue¶

Generated from the code under src/ by scripts/gen_error_catalogue.py. Do not hand-edit; CI fails if this table and the code disagree.

Code	HTTP status	Message	Detail keys	Raised from	What to do
`ADAPTER_NOT_FOUND`	503	Service Unavailable	-	`routers.notification_rules`	No guidance yet - quote the code and the request id in your report.
`AI_DISABLED`	404	AI features are not enabled for this deployment or tenant. Contact your platform administrator if this is unexpected.	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`AI_INVALID_RESPONSE`	502	varies	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_ALREADY_APPROVED`	409	Provider is already approved.	-	`routers.ai_providers`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_ALREADY_DISABLED`	409	Provider is already disabled.	-	`routers.ai_providers`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_CONFLICT`	409	varies	-	`routers.ai_providers`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_ENABLED`	409	Disable the provider before deleting it.	-	`routers.ai_providers`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_ERROR`	502	The AI provider rejected the request.	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_NOT_APPROVED`	409	Provider must be approved before enabling.	-	`routers.ai_providers`	No guidance yet - quote the code and the request id in your report.
`AI_PROVIDER_NOT_FOUND`	404	AI provider not found	-	`routers.ai_providers`	The object the request names was not found. Check the id in the request.
`AI_RATE_LIMITED`	429	The AI provider is rate limiting requests; retry shortly.	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`AI_UNAVAILABLE`	503	The AI provider is unavailable; retry later.	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`API_KEY_NOT_FOUND`	404	Not Found	-	`routers.governance`	The object the request names was not found. Check the id in the request.
`API_KEY_PREFIX_EXHAUSTED`	503	Could not allocate a unique API key prefix. Retry the request.	-	`routers.governance`	No guidance yet - quote the code and the request id in your report.
`APPLY_FAILED`	422	varies	-	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`APPROVAL_ALREADY_DECIDED`	409	varies	-	`routers.approvals (ApprovalConflictError)`	No guidance yet - quote the code and the request id in your report.
`APPROVAL_EXPIRED`	409	varies	-	`routers.approvals (ApprovalConflictError)`	No guidance yet - quote the code and the request id in your report.
`APPROVAL_NOT_FOUND`	404	varies	-	`routers.approvals`, `routers.approvals (ApprovalNotFoundError)`	The object the request names was not found. Check the id in the request.
`APPROVAL_NOT_REQUESTER`	403	varies	-	`routers.approvals (ApprovalForbiddenError)`	No guidance yet - quote the code and the request id in your report.
`APPROVAL_REQUIRED`	403	varies	-	`dependencies.auth`, `routers.runs`	No guidance yet - quote the code and the request id in your report.
`APPROVAL_TARGET_INVALID`	409	varies	-	`routers.approvals (ApprovalConflictError)`	No guidance yet - quote the code and the request id in your report.
`APPROVAL_UNSUPPORTED`	409	varies	-	`routers.approvals (ApprovalConflictError)`	No guidance yet - quote the code and the request id in your report.
`ASYMMETRIC_TRANSFORM_REQUIRES_ADMIN`	403	varies	`field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`BREAK_GLASS_CAP`	409	varies	-	`routers.principals`	No guidance yet - quote the code and the request id in your report.
`BULK_WIPE_NOT_ENABLED`	403	Bulk deletion of all runs is disabled. Set ALLOW_BULK_RUN_WIPE=true in the environment to enable it.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_BYTES_MISSING`	409	varies	`bytes_status`, `persist_limit_bytes`, `size_bytes`	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_BYTES_NOT_RETAINED`	409	varies	`bytes_status`, `persist_limit_bytes`, `size_bytes`	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_GENERATION_FAILED`	503	The evidence bundle could not be generated.	-	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_INTEGRITY_MISMATCH`	409	varies	-	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_NOT_FOUND`	404	varies	-	`mcp.tools.verify_evidence_bundle`, `routers.evidence_bundles`	The object the request names was not found. Check the id in the request.
`BUNDLE_NOT_READY`	409	varies	`bytes_status`, `persist_limit_bytes`, `size_bytes`	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_STORE_OBJECT_MISSING`	409	The container's object is no longer present in the object store; the signed metadata is on record.	`bytes_status`, `store_uri`	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_STORE_UNAVAILABLE`	503	The object store holding this container could not be consulted; bytes not released.	-	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUNDLE_TOO_LARGE`	409	varies	`bytes_status`, `persist_limit_bytes`, `size_bytes`	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`BUSINESS_DOMAIN_NOT_FOUND`	404	Target business domain not found.	-	`routers.replication_profiles`	The object the request names was not found. Check the id in the request.
`BUSINESS_DOMAIN_REQUIRED`	422	varies	-	`routers.mappings`, `routers.systems`	No guidance yet - quote the code and the request id in your report.
`CANCEL_FAILED`	503	Temporal workflow cancellation failed; DB unchanged. Retry.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`CANONICAL_TYPE_OVERRIDE_NOT_APPLICABLE`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`CDC_TABLE_SEMANTICS_INVALID`	422	varies	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`CDC_TABLE_SEMANTICS_SYSTEM_MISMATCH`	422	varies	-	`routers.mappings`	`replacing_merge_tree_final` and `clickpipes_peerdb` describe how a ClickHouse table is read, so they need a mapping with a ClickHouse side. Map a ClickHouse system, or choose another `cdc_table_semantics`.
`CHANNEL_CREDENTIAL_CROSS_USE`	422	varies	`field`	`routers.notification_rules`	The `credential_ref` names a secret another part of the platform owns (the message says which kind). Store a separate secret for the channel and reference that one.
`CHANNEL_DESTINATION_REQUIRES_ADMIN`	403	Setting or changing a notification channel's credential_ref or destination (URL / host) requires platform_admin. Other channel fields can be edited without it.	`fields`	`routers.notification_rules`	A notification channel's `credential_ref` and destination (the fields listed in `fields`) can only be set or changed by a platform admin. Ask one to make that change, or resend the request with those fields unchanged.
`CHANNEL_DISABLED`	409	Enable the channel before sending a test ticket.	-	`routers.notification_rules`	No guidance yet - quote the code and the request id in your report.
`CHANNEL_NAME_EXISTS`	409	A notification channel with this name already exists.	-	`routers.notification_rules`	This tenant already has a notification channel with that name. Choose another name, or update the existing channel.
`CHANNEL_NOT_FOUND`	404	Not Found	-	`routers.notification_rules`	The object the request names was not found. Check the id in the request.
`CONFIRMATION_REQUIRED`	400	scope=all requires confirm=true to prevent accidental deletion of all runs.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`CONFLICT`	409	varies	-	`routers.principals`	No guidance yet - quote the code and the request id in your report.
`CONNECTION_FAILED`	422	Could not establish a connection to the system.	-	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`CONNECTION_REF_OVERRIDE_FORBIDDEN`	403	Testing a connection reference other than the stored one requires permission to change it (system.update on this system) or to create systems (system.create).	-	`routers.systems`	Testing an entered (unsaved) connection reference needs the right to change it (system.update on the system, or system.create) - #3384. Save the reference first with a role that may, or test the stored reference without an override.
`CONNECTOR_UNAVAILABLE`	422	The connector for this system is not available.	-	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`DATA_RESIDENCY_VIOLATION`	403	varies	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`DB_ERROR`	500	The subscription could not be saved.	-	`routers.subscriptions`	No guidance yet - quote the code and the request id in your report.
`DECIMAL_IN_FLOAT_ON_NONFLOAT`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`DECRYPT_CONFIG_REQUIRES_ADMIN`	403	varies	`field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`DECRYPT_EXPRESSION_INVALID`	422	Unprocessable Entity	`errors`, `field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`DECRYPT_EXPRESSION_WITH_ALGO`	422	This field uses Python-side decrypt (decrypt_algo). A decrypt_expression cannot be combined with it — remove the Python-decrypt config first, or drop the expression.	`field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`DECRYPT_KEY_REF_REQUIRED_FOR_ALGO`	422	This field uses Python-side decrypt (decrypt_algo) and requires decrypt_key_ref. To remove the decrypt config, omit the field from the body (platform_admin).	`field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`DECRYPT_KEY_REF_WITHOUT_EXPRESSION`	422	decrypt_key_ref requires a decrypt_expression on the field (or a Python-decrypt algo). Set the expression in the same save, or omit the key reference.	`field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`DECRYPT_ON_KEY_COLUMN`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`DESIGN_PARTNER_PENDING`	501	Adapter is design-partner pending; profile is configured but discovery/latency are unavailable.	`tool`	`routers.replication_profiles`	No guidance yet - quote the code and the request id in your report.
`DIAGNOSTIC_MAPPING_NOT_FOUND`	404	No mapping with that id is visible to you.	-	`routers.diagnostics`	The object the request names was not found. Check the id in the request.
`DIAGNOSTIC_RUN_NOT_FOUND`	404	No run or execution request with that id is visible to you.	-	`routers.diagnostics`	The object the request names was not found. Check the id in the request.
`DIAGNOSTIC_TARGET_MISMATCH`	422	mapping_id does not match the run's mapping; send one or the other.	-	`routers.diagnostics`	No guidance yet - quote the code and the request id in your report.
`DISCOVERY_NOT_SUPPORTED`	501	varies	`tool`	`routers.replication_profiles`	No guidance yet - quote the code and the request id in your report.
`DISPATCH_FAILED`	503	Workflow dispatch failed; request marked system_error.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`DOMAIN_ALREADY_CLAIMED`	409	varies	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`DOMAIN_DELETE_FAILED`	422	Domain could not be deleted.	-	`routers.business_domains`	No guidance yet - quote the code and the request id in your report.
`DOMAIN_INVALID`	422	Invalid domain specification.	-	`routers.business_domains`	No guidance yet - quote the code and the request id in your report.
`DOMAIN_NOT_FOUND`	404	Domain not found.	-	`routers.business_domains`	The object the request names was not found. Check the id in the request.
`DOMAIN_OUT_OF_SCOPE`	403	You do not have access to the target business domain.	-	`routers.replication_profiles`	No guidance yet - quote the code and the request id in your report.
`DOMAIN_PATH_EXISTS`	409	varies	-	`routers.business_domains`	No guidance yet - quote the code and the request id in your report.
`DUPLICATE_IDEMPOTENCY_KEY`	409	Idempotency key already used.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`DUPLICATE_MAPPING_GROUP`	409	varies	-	`routers.mapping_groups`	No guidance yet - quote the code and the request id in your report.
`ELEVATION_NOT_FOUND_OR_NOT_APPROVED`	404	Not Found	-	`routers.governance`	No guidance yet - quote the code and the request id in your report.
`ELEVATION_NOT_FOUND_OR_NOT_PENDING`	404	Not Found	-	`routers.governance`	No guidance yet - quote the code and the request id in your report.
`EMPTY_UPDATE`	422	No fields to update.	-	`routers.ai_providers`, `routers.identity_providers`	The update named no fields to change. Include at least one field.
`EVIDENCE_AUDIT_WRITE_FAILED`	503	varies	-	`routers.evidence_bundles`	No guidance yet - quote the code and the request id in your report.
`EXTERNAL_API_ERROR`	502, 503	varies	-	`routers.notification_rules`	No guidance yet - quote the code and the request id in your report.
`FIELD_NOT_PATCHABLE`	422	varies	`fields`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`FORBIDDEN`	403	varies	-	`dependencies.auth`, `mcp.server`, `routers.ai_providers`, `routers.discovery`, `routers.evidence`, `routers.evidence_bundles`, `routers.runs`, `routers.tenant`	Your role or grants do not allow this request. The message says what is missing - usually a permission or a role; ask a platform or tenant admin to grant it.
`GRANT_ROLE_ABOVE_CALLER`	403	varies	-	`routers.grants`	You can grant or revoke only roles at or below your own highest role; `platform_admin` can be granted or revoked only by a platform admin. Ask a principal who holds the role (or a higher one) to make the change.
`GRANT_SELF_ESCALATION`	403	varies	-	`routers.grants`	You cannot grant yourself, or a group you belong to, a role you do not already hold. Ask another administrator to grant it.
`HOLD_ALREADY_ACTIVE`	409	varies	-	`routers.legal_holds`	No guidance yet - quote the code and the request id in your report.
`HOLD_NOT_FOUND`	404	varies	-	`routers.legal_holds`	The object the request names was not found. Check the id in the request.
`HOLD_TARGET_NOT_FOUND`	404	varies	-	`routers.legal_holds`	The object the request names was not found. Check the id in the request.
`IDEMPOTENCY_KEY_BODY_MISMATCH`	422	This Idempotency-Key was previously used with a different request body. Use a new key for a different operation.	-	`middleware.idempotency`	This `Idempotency-Key` was already used with a different request body. Use a new key for a different request.
`IDEMPOTENCY_KEY_TOO_LONG`	400	varies	-	`middleware.idempotency`	Shorten the `Idempotency-Key` to the length the message states.
`IDEMPOTENCY_STORE_UNAVAILABLE`	503	The idempotency store is unavailable and this operation has external side effects. Retry later.	-	`middleware.idempotency`	The idempotency store could not be reached and this operation has external side effects, so it was refused rather than risk running twice. Retry after the `Retry-After` interval.
`IDEMPOTENT_REQUEST_IN_PROGRESS`	409	A request with this Idempotency-Key is currently being processed. Retry to receive its result.	-	`middleware.idempotency`	A request with the same `Idempotency-Key` is still being processed. Retry after the `Retry-After` interval, with the same key and body, to receive its result.
`IDENTITY_LINK_CONFLICT`	403	This login's identity provider does not match the identity already bound to this account. An administrator must reset the account's identity binding (POST /principals/{id}/reset-identity) before this provider can be linked.	-	`dependencies.auth`	No guidance yet - quote the code and the request id in your report.
`IDP_CREDENTIAL_CROSS_USE`	422	varies	`field`	`routers.identity_providers`	The `client_secret_ref` names a secret another part of the platform owns (the message says which kind). Store the provider's client secret as its own entry in the secrets backend and reference that one.
`IDP_INVALID`	422	SAML idp_entity_id must not be empty.	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`IDP_ISSUER_CONFLICT`	409	An identity provider with this issuer/entityID is already registered.	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`IDP_ISSUER_RESERVED`	422	This issuer is the deployment's own OIDC issuer or platform issuer and cannot be registered as a tenant identity provider.	-	`routers.identity_providers`	The issuer (or SAML entityID) is this deployment's own OIDC issuer or its platform issuer, which tokens issued to the whole deployment carry, so no tenant identity provider may claim it. Register the identity provider under its own issuer.
`IDP_NOT_FOUND`	404	varies	-	`routers.identity_providers`, `routers.saml`	The object the request names was not found. Check the id in the request.
`IDP_SECRET_REQUIRES_ADMIN`	403	Setting or changing an identity provider's issuer, JWKS base URL or client_secret_ref requires platform_admin. Other provider fields can be edited without it.	`fields`	`routers.identity_providers`	An identity provider's issuer, JWKS base URL and `client_secret_ref` (the fields listed in `fields`) can only be set or changed by a platform admin. Ask one to make that change, or resend the request with those fields unchanged.
`INSTALLATION_ID_MISSING`	503	This installation has no id (cfg.installation is empty): apply migration 255_installation_and_licence_state.sql.	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`INTERNAL_ERROR`	500	An unexpected error occurred.	-	`main`	An unexpected server error. Report it with the request id (the `X-Request-ID` response header, also `error.request_id`) - it locates the full error in the server log.
`INVALID_CURSOR`	422	varies	-	`routers.evidence_bundles`, `routers.metrics`, `routers.principals`, `routers.runs`	The `cursor` is not one this listing returned. Restart the listing without a cursor, then pass back each page's `next_cursor` unchanged.
`INVALID_HOLD_SCOPE`	400	varies	-	`routers.legal_holds`	No guidance yet - quote the code and the request id in your report.
`INVALID_INPUT_ENCODING`	422	A request value contains a character the database cannot store (for example a NUL byte).	-	`main`	A path, query or body value carries a character the database cannot store - almost always a NUL byte (`%00` / `\u0000`). Remove it and resend (#3364).
`INVALID_PARAM`	422	varies	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`INVALID_SEVERITY`	422	varies	-	`routers.subscriptions`	No guidance yet - quote the code and the request id in your report.
`INVALID_STATE`	409	varies	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`INVALID_WATERMARK`	422	varies	`side`	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`INVALID_WATERMARK_CONFIG`	422	varies	`fields`, `side`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`KEY_COLUMN_BLOB_SKIPPED`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`KEY_COLUMN_EXCLUDED_FROM_DIFF`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`KEY_COLUMN_EXCLUDED_FROM_HASH`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`KEY_IDENTITY_IS_STRICT`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`KEY_IDENTITY_PROFILE_UNAVAILABLE`	422	varies	-	`routers.mappings`	The platform cannot run this key identity profile yet. Choose a profile the message lists as available.
`LICENCE_ACTIVATION_REFUSED`	422	varies	`details`, `errorCode`, `requestId`	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_DEACTIVATION_REFUSED`	422	varies	`errorCode`, `requestId`	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_KEY_IN_USE`	409	This licence key already licenses another tenant of this installation.	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_KEY_UNRESOLVABLE`	422	varies	`reason`	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_NOT_ACTIVATED`	409	This tenant has no activation to deactivate.	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_NOT_REFRESHABLE`	409	This tenant has no online activation to refresh: activate with a licence key first (an offline activation is renewed with a new code).	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_REQUIRED`	402	varies	`last_verdict_code`, `licence_reason`, `licence_state`, `licence_verdict`	`services.entitlement_service`	No guidance yet - quote the code and the request id in your report.
`LICENCE_SERVER_UNAVAILABLE`	503	varies	`class`, `http_status`, `retry_after`	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LICENCE_TRANSFER_COOLDOWN`	429	varies	`nextTransferAvailable`, `requestId`	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`LIVE_REVEAL_DISPATCH_FAILED`	503	The live re-fetch could not be dispatched. Try again shortly.	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`LIVE_REVEAL_FAILED`	502	The live re-fetch failed against the source or target system. The connection may be down; as-of-run reveal still works.	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`LIVE_REVEAL_TIMEOUT`	504	varies	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`LOCAL_AUTH_ONLY`	403	This tenant accepts local login only; external SSO login is not permitted.	-	`dependencies.auth`	No guidance yet - quote the code and the request id in your report.
`MAPPING_GROUP_NOT_FOUND`	404	varies	-	`routers.mapping_groups`	The object the request names was not found. Check the id in the request.
`MAPPING_HAS_HISTORY`	409	varies	`references`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`MAPPING_HAS_NO_CONFIG_VERSION`	400	Mapping has no config version. Run schema-refresh (POST /mappings/{id}/schema-refresh) or update the mapping to create one before submitting a run.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`MAPPING_HAS_RUN_HISTORY`	409	varies	`run_count`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`MAPPING_IN_GROUPS`	409	Mapping belongs to one or more groups. Remove it first or use ?force=true.	`groups`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`MAPPING_LIMIT_REACHED`	402	varies	`active_count`, `limit`, `requested`	`services.entitlement_service`	No guidance yet - quote the code and the request id in your report.
`MAPPING_NAME_CONFLICT`	409	varies	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`MAPPING_NOT_FOUND`	404	varies	-	`mcp.tools.check_key_status`, `mcp.tools.get_config_change_timeline`, `mcp.tools.get_data_issues`, `mcp.tools.get_issue_impact`, `mcp.tools.get_mapping_config`, `mcp.tools.get_mapping_health_history`, `mcp.tools.get_measured_usage`, `mcp.tools.get_readiness_status`, `mcp.tools.get_replication_health`, `mcp.tools.get_schema_drift`, `mcp.tools.get_verification_statement`, `mcp.tools.get_watermark_history`, `mcp.tools.list_anomalies`, `routers.ai`, `routers.evidence_bundles`, `routers.mapping_groups (MappingGroupReferenceNotFound)`, `routers.mappings`, `routers.metrics`, `routers.runs`, `routers.schedules`, `routers.subscriptions`	The object the request names was not found. Check the id in the request.
`MAPPING_NOT_HIGH_RISK`	409	varies	-	`routers.ai`	No guidance yet - quote the code and the request id in your report.
`MCP_DISABLED`	403	The MCP surface is disabled on this platform (CFG_AI_ENABLED).	-	`mcp.server`	No guidance yet - quote the code and the request id in your report.
`MCP_DISABLED_TENANT`	403	MCP is not enabled for this tenant (cfg.tenant_defaults.mcp_enabled).	-	`mcp.server`	No guidance yet - quote the code and the request id in your report.
`MISSING_IDEMPOTENCY_KEY`	400	All state-mutating requests require an Idempotency-Key header. Use a UUID or a deterministic key for your request.	-	`middleware.idempotency`	Send an `Idempotency-Key` header (for example a UUID) on every POST, PUT, PATCH and DELETE request.
`NOT_EDITABLE`	422	varies	`keys`	`routers.tenant`	No guidance yet - quote the code and the request id in your report.
`NOT_ELEVATION_OWNER`	403	varies	-	`routers.governance`	No guidance yet - quote the code and the request id in your report.
`NOT_FOUND`	404	varies	-	`routers.grants`, `routers.principals`, `routers.runs`	No guidance yet - quote the code and the request id in your report.
`NOT_REFRESHABLE`	403	Only SAML SSO session tokens can be refreshed.	-	`routers.me`	No guidance yet - quote the code and the request id in your report.
`NOT_SAML_SESSION`	403	Only SAML SSO sessions can be signed out at the identity provider.	-	`routers.me`	No guidance yet - quote the code and the request id in your report.
`NO_ACTIVE_AI_PROVIDER`	404	No approved + enabled AI provider is configured for this tenant. Configure one via /ai-providers before invoking the Onboarding Copilot.	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`NO_EMAIL`	409	This user has no email; a 'direct' placeholder binding cannot be keyed. Set an email first.	-	`routers.principals`	No guidance yet - quote the code and the request id in your report.
`NO_ITEMS`	400	items must not be empty.	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`NO_PROPOSALS_CONFIRMED`	400	confirmed_proposal_ids must not be empty	-	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`NO_SSO`	404	varies	-	`routers.saml`	No guidance yet - quote the code and the request id in your report.
`NO_TAGS_PROVIDED`	400	tags must not be empty	-	`routers.schedules`	No guidance yet - quote the code and the request id in your report.
`NO_VALUES_PROVIDED`	400	Provide at least one setting to update.	-	`routers.tenant`	No guidance yet - quote the code and the request id in your report.
`NO_WATERMARK_SUPPLIED`	400	Supply source_watermark and/or target_watermark.	-	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`NULL_NOT_ALLOWED`	422	varies	`fields`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`NUMERIC_SCALE_ON_DECRYPT_FIELD`	422	varies	`field`	`routers.mappings (DecryptCanonRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`NUMERIC_SCALE_ZERO`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`OFFLINE_CODE_INVALID`	422	varies	`reason`	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`OFFLINE_CODE_OTHER_MACHINE`	409	This activation code was issued for another installation. Send this installation's challenge to EGMW support for a new code.	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`OIDC_CONFIG_ON_SAML`	422	client_secret_ref / oidc_config apply to OIDC providers only.	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`PARTITION_NOT_FOUND`	404	Not Found	-	`routers.runs`	The object the request names was not found. Check the id in the request.
`PARTITION_STRATEGY_EMPTY`	422	partition_strategy would discover zero partitions — a partitioned run over zero partitions compares nothing. Declare non-empty discovery values/ranges (e.g. discovery: {mode: enumerated, values: [...]}) or remove partition_strategy. (#1173)	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`PARTITION_STRATEGY_INVALID`	422	varies	`detail`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`PARTITION_STRATEGY_NOT_DISJOINT`	422	varies	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`PLATFORM_DEFAULTS_NOT_FOUND`	404	Platform defaults row not found.	-	`routers.tenant`	No guidance yet - quote the code and the request id in your report.
`PLATFORM_SCHEMA_AI_NOT_CONSENTED`	403	No AI provider is configured for schema-bearing operations. Enable 'allow_platform_schema_ai' in tenant AI settings, or configure a dedicated AI provider for the 'ai_onboarding' scope.	-	`services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`PLATFORM_SIGNER_NOT_CONFIGURED`	503	varies	-	`routers.well_known`	No guidance yet - quote the code and the request id in your report.
`PLATFORM_SIGNING_KEY_REQUIRED`	422	SAML sign-in requires a shared platform signing key in production (PLATFORM_SIGNING_KEY_REF / Helm app.saml.signingKeyRef). Without it, each API replica would sign session tokens with its own ephemeral key and logins would fail intermittently. Configure the key, then enable this identity provider.	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`PREVIEW_NOT_FOUND`	404	varies	-	`routers.discovery`	The auto-config preview is unknown or has expired (previews expire after 1 hour). Run `POST /auto-config/preview` again and use the new preview id.
`PRINCIPAL_DISABLED`	403	This account is disabled. Contact your administrator to re-activate it.	-	`dependencies.auth`	An administrator disabled this account (#3332). Signing in again cannot help: ask a platform or tenant admin to set the principal back to active (PATCH /principals/{id}); then sign in again.
`PRIVATE_DESTINATION_EXISTS`	409	This host and port are already on the private destination allowlist.	-	`routers.notification_destinations`	The tenant already allows this exact host and port. Nothing to add.
`PRIVATE_DESTINATION_INVALID`	422	varies	`field`	`routers.notification_destinations (InvalidPrivateDestination)`	A private destination entry is one exact hostname, or an IP address / CIDR for an IP-literal host, plus one port 1-65535. Remove any scheme, port suffix, path or wildcard from `host`, and zero the host bits of a CIDR.
`PRIVATE_DESTINATION_LIMIT_REACHED`	409	varies	-	`routers.notification_destinations`	A tenant holds at most 200 private destination entries. Remove entries that are no longer used, or use one CIDR entry for IP-literal hosts in the same subnet.
`PRIVATE_DESTINATION_NEVER_PERMITTED`	422	varies	`field`	`routers.notification_destinations (InvalidPrivateDestination)`	Loopback, link-local (including every cloud metadata address), unspecified and multicast addresses, and `localhost` / metadata-service names, can never be allowlisted. Point the channel at the internal system's own address instead.
`PRIVATE_DESTINATION_NOT_FOUND`	404	No private destination entry with this id for this tenant.	-	`routers.notification_destinations`	The object the request names was not found. Check the id in the request.
`PRIVATE_DESTINATION_NOT_PRIVATE`	422	varies	`field`	`routers.notification_destinations (InvalidPrivateDestination)`	The address is not wholly inside a private range, so it needs no entry: public destinations are already permitted over HTTPS. Narrow the CIDR to the private part.
`PROFILE_ALREADY_EXISTS`	409	A profile for this source/target/tool combination already exists	-	`routers.replication_profiles`	No guidance yet - quote the code and the request id in your report.
`PROFILE_DETACH_CONFLICT`	422	varies	`fields`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`PROFILE_IN_USE`	409	varies	`mapping_count`	`routers.replication_profiles`	No guidance yet - quote the code and the request id in your report.
`PROFILE_NOT_FOUND`	404	varies	-	`routers.discovery`, `routers.mapping_groups`, `routers.replication_profiles`	The object the request names was not found. Check the id in the request.
`PROFILE_SYSTEM_MISMATCH`	422	varies	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`PROPOSAL_NOT_FOUND`	404	varies	-	`routers.discovery`	The object the request names was not found. Check the id in the request.
`PROPOSAL_REQUIRES_REVIEW`	422	varies	`proposal_id`	`routers.discovery`	No guidance yet - quote the code and the request id in your report.
`PROVIDER_ADAPTER_NOT_IMPLEMENTED`	501	varies	-	`routers.ai_providers`, `services.ai.invocation`	No guidance yet - quote the code and the request id in your report.
`QUERY_ID_NOT_FOUND`	404	No Paridity run in this tenant issued that query ID.	-	`routers.evidence`	The object the request names was not found. Check the id in the request.
`READ_PIN_VIEWS_SYSTEM_MISMATCH`	422	read_pin_views = true pins a Snowflake VIEW at the readiness instant, but neither side of this mapping is a Snowflake system.	-	`routers.mappings`	`read_pin_views` pins a Snowflake view, so it needs a mapping with a Snowflake side. Turn it off, or map a Snowflake system.
`REASON_REQUIRED`	422	varies	-	`routers.principals`, `routers.runs`	No guidance yet - quote the code and the request id in your report.
`RECOMMENDATION_NOT_FOUND`	404	Recommendation not found or already decided.	-	`routers.recommendations`	No guidance yet - quote the code and the request id in your report.
`REPLICATION_PROFILE_NOT_FOUND`	404	varies	-	`routers.discovery`, `routers.mapping_groups (MappingGroupReferenceNotFound)`, `routers.mappings`	The object the request names was not found. Check the id in the request.
`REVEAL_KEY_FAILURE`	409	varies	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`REVEAL_UNAVAILABLE`	409	varies	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`RISK_NOT_COMPUTED`	409	No pre-run risk score has been computed for this mapping yet.	-	`routers.ai`	No guidance yet - quote the code and the request id in your report.
`ROUNDING_MODE_ON_DECRYPT_FIELD`	422	varies	`field`	`routers.mappings (DecryptCanonRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`ROUNDING_MODE_ON_FLOAT`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`RUN_CONSUMPTION_FAILED`	500	The run's consumption could not be read. Retry; if it persists, report the request id.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`RUN_NOT_FOUND`	404	varies	-	`mcp.tools.compare_run_to_last_passing`, `mcp.tools.explain_failed_run`, `mcp.tools.get_run_summary`, `mcp.tools.get_run_timeline`, `mcp.tools.triage_run`, `routers.ai`, `routers.evidence`, `routers.evidence_bundles`, `routers.mappings`, `routers.runs`	The object the request names was not found. Check the id in the request.
`RUN_NOT_IN_FAILURE_STATE`	409	varies	-	`routers.ai`	No guidance yet - quote the code and the request id in your report.
`RUN_NOT_TERMINAL`	409	varies	-	`routers.evidence_bundles`, `routers.runs`	No guidance yet - quote the code and the request id in your report.
`SAML_ASSERTION_INVALID`	401	The SAML assertion could not be validated.	-	`routers.saml`	No guidance yet - quote the code and the request id in your report.
`SAML_CONFIG_ON_OIDC`	422	saml_config applies to SAML providers only.	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`SAML_LOGIN_FAILED`	502	Could not build the SAML request.	-	`routers.saml`	No guidance yet - quote the code and the request id in your report.
`SAML_SP_SIGNING_KEY_INVALID`	503	The SAML SP signing key is configured but unusable; see the API log.	-	`routers.saml`	No guidance yet - quote the code and the request id in your report.
`SAML_SP_SIGNING_KEY_NEXT_INVALID`	503	The staged next SAML SP signing key (SAML_SP_SIGNING_KEY_NEXT_REF) is configured but unusable; see the API log.	-	`routers.saml`	No guidance yet - quote the code and the request id in your report.
`SAML_SP_SIGNING_KEY_REQUIRED`	422	Signed SAML requests need a shared SP signing key in production (SAML_SP_SIGNING_KEY_REF / Helm app.saml.spSigningKeyRef): a PEM bundle with the private key and certificate in your secret backend. Configure the key, then enable request signing on this identity provider.	-	`routers.identity_providers`	No guidance yet - quote the code and the request id in your report.
`SAML_SP_SIGNING_KEY_UNAVAILABLE`	503	The SAML SP signing key could not be loaded from the secret backend; see the API log.	-	`routers.saml`	No guidance yet - quote the code and the request id in your report.
`SCD_SCOPE_INVALID`	422	varies	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`SCHEDULE_AUDIT_WRITE_FAILED`	503	Failed to record bulk schedule toggle in audit log. Toggle not applied.	-	`routers.schedules`	No guidance yet - quote the code and the request id in your report.
`SCHEDULE_LAG_EXCEEDS_MAPPING`	400	varies	-	`routers.schedules`	No guidance yet - quote the code and the request id in your report.
`SCHEDULE_NAME_CONFLICT`	409	A schedule with this name already exists for this mapping	-	`routers.schedules`	No guidance yet - quote the code and the request id in your report.
`SCHEDULE_NOT_FOUND`	404	varies	-	`routers.schedules`	The object the request names was not found. Check the id in the request.
`SCHEDULING_NOT_ENTITLED`	402	Scheduling is not available on your current plan. Upgrade to enable scheduled reconciliation.	-	`routers.schedules`	No guidance yet - quote the code and the request id in your report.
`SCHEMA_DECISION_COLUMN_ONE_SIDED`	409	varies	`column`, `missing_side`, `side`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`SCHEMA_DECISION_NOT_FOUND`	404	Schema decision not found.	-	`routers.mappings`	The object the request names was not found. Check the id in the request.
`SCHEMA_DECISION_NOT_OPEN`	409	Decision already dispositioned.	-	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`SCHEMA_DECISION_PHYSICAL_NAME_UNKNOWN`	409	The true-case physical name for this column is not yet known (the decision predates its capture and the column is not in the schema baseline). Run POST /mappings/{id}/schema-refresh, or wait for the next run to re-observe the column, then retry.	`column`, `side`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`SCHEMA_DRIFT_PENDING`	400	Mapping has unresolved schema drift. Use schema-refresh to accept the new schema.	-	`routers.runs`	No guidance yet - quote the code and the request id in your report.
`SCHEMA_REFRESH_UNVERIFIED`	409	varies	`outcome`, `reason`, `side`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`SELF_APPROVAL_FORBIDDEN`	403	varies	-	`routers.approvals (ApprovalForbiddenError)`, `routers.governance`	No guidance yet - quote the code and the request id in your report.
`SERVICE_ACCOUNT_NOT_FOUND`	404	Not Found	-	`routers.governance`	The object the request names was not found. Check the id in the request.
`SESSION_MAX_LIFETIME`	401	Session reached its maximum lifetime; please sign in again.	-	`routers.me`	No guidance yet - quote the code and the request id in your report.
`SESSION_REVOKED`	401	Session was ended by the identity provider; please sign in again.	-	`routers.me`	No guidance yet - quote the code and the request id in your report.
`SSO_REQUIRED`	403	This tenant requires SSO login. Local/platform-issuer login is not permitted.	-	`dependencies.auth`	No guidance yet - quote the code and the request id in your report.
`STALE_WRITE`	409	varies	`current_version`	`optimistic_lock`	No guidance yet - quote the code and the request id in your report.
`SUBSCRIPTION_NOT_ASSIGNED`	402	No subscription tier is assigned to this tenant. Creating or enabling mappings and schedules, bulk import and auto-configuration apply are refused until a platform operator assigns one. Existing configuration stays viewable and editable; reads, evidence and runs are unaffected.	-	`services.entitlement_service`	No guidance yet - quote the code and the request id in your report.
`SUBSCRIPTION_NOT_FOUND`	404	Subscription not found or not owned by caller.	-	`routers.subscriptions`	The object the request names was not found. Check the id in the request.
`SUBSCRIPTION_SUSPENDED`	402	varies	-	`routers.runs`, `services.entitlement_service`	No guidance yet - quote the code and the request id in your report.
`SUGGESTION_ALREADY_APPLIED`	422	varies	-	`routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`SUGGESTION_FIELD_MISSING`	422	varies	-	`routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`SUGGESTION_NOT_APPLICABLE`	422	varies	-	`routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`SYSTEM_IN_USE`	409	varies	-	`routers.systems`	No guidance yet - quote the code and the request id in your report.
`SYSTEM_NAME_CONFLICT`	409	varies	-	`routers.systems`	No guidance yet - quote the code and the request id in your report.
`SYSTEM_NOT_FOUND`	404	varies	-	`mcp.tools._discovery_helpers`, `routers.discovery`, `routers.mappings`, `routers.metrics`, `routers.replication_profiles`, `routers.systems`	The object the request names was not found. Check the id in the request.
`TENANT_DEFAULTS_NOT_FOUND`	404	Tenant defaults row not found.	-	`routers.tenant`	No guidance yet - quote the code and the request id in your report.
`TIMESTAMP_ROUNDING_ON_DECRYPT_FIELD`	422	varies	`field`	`routers.mappings (DecryptCanonRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`TIMESTAMP_ROUNDING_ON_NON_TIMESTAMP`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (DecryptCanonRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`TIMESTAMP_ROUNDING_WITH_FIELD_PRECISION`	422	varies	`field`	`models.api.requests (FieldConfigRejection)`, `routers.mappings (FieldConfigRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`TOOL_CONFIG_INCOMPLETE`	409	varies	`tool`	`routers.replication_profiles`	No guidance yet - quote the code and the request id in your report.
`TOO_MANY_ITEMS`	422	At most 50 diffs may be revealed per call.	-	`routers.evidence`	No guidance yet - quote the code and the request id in your report.
`TRANSFORM_CHAIN_INVALID`	422	Unprocessable Entity	`errors`, `field`	`routers.mappings`	No guidance yet - quote the code and the request id in your report.
`TRIAGE_CACHE_MISS`	404	No cached triage exists for this run with the active provider config.	-	`routers.ai`	No guidance yet - quote the code and the request id in your report.
`TRIAL_ALREADY_USED`	409	This tenant has already consumed its trial. A trial is granted once per tenant; an extension is an operator database change with its own audit entry.	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`TSTZ_DECRYPT_PLAINTEXT`	422	varies	`field`	`routers.mappings (DecryptCanonRejection)`, `routers.suggestions (SuggestionNotApplicableError)`	No guidance yet - quote the code and the request id in your report.
`UNAUTHENTICATED`	401	varies	-	`dependencies.auth`, `middleware.auth`, `middleware.idempotency`, `routers.me`	Send a valid bearer token, or sign in again if it has expired. If the message says authentication is temporarily unavailable, the sign-on provider could not be reached - retry, and report it with the request id if it persists.
`UNKNOWN_TENANT`	401	varies	-	`dependencies.auth`	No guidance yet - quote the code and the request id in your report.
`UNKNOWN_TIER`	422	varies	-	`routers.subscription_status`	No guidance yet - quote the code and the request id in your report.
`VALIDATION_ERROR`	400, 404, 409, 422	varies	`key`	`main`, `routers.anomaly_policies`, `routers.grants`, `routers.mappings`, `routers.notification_rules`, `routers.principals`, `routers.recommendations`, `routers.runs`, `routers.schedules`, `routers.systems`, `routers.tenant`	The request was rejected as invalid. `error.message` says what was rejected; when the body or a parameter failed validation, `error.detail` lists each invalid field. Correct the request and resend.
`HTTP_<status>`	any	The HTTP status phrase (for example `Not Found`), or the detail text when the detail is a plain string	-	any route whose error carries no code (a plain-string, list or absent detail)	No guidance yet - quote the code and the request id in your report.

241 codes from 444 raising sites, plus the HTTP_<status> family.

Known limitations¶

This page is an index. Each limitation below is documented on the guide page it belongs to, and the link takes you to that section. The one-line summaries use the words of the linked section; when you need the detail or the workaround, follow the link. If an area is not listed, its guide page records no limitation.

Connectors and engines¶

Area	Limitation	Where it is documented
ClickHouse	On ClickHouse 25.8, `Date32` and `DateTime64` store only 1900-01-01 through 2299-12-31; a source value outside the column's range is not held by the replica, and the row is reported as a real difference.	ClickHouse - Known limits
ClickHouse	`Array`, `Map`, `Tuple`, `Nested`, `JSON` / `Object`, `Variant`, `Dynamic`, `IPv4` / `IPv6` and the aggregate-function types have no native rendering and take the slower reference path; no cross-engine verification has been recorded for these types.	ClickHouse - Known limits
ClickHouse	A container (`Array`, `Map` or `Tuple`) whose element is a `Date` or `DateTime` type is refused on the reference path as well; exclude such a column.	ClickHouse - Known limits
ClickHouse	Rounding modes other than half-up, numeric tolerance, float compare mode, canonical type override, timestamp rounding, a non-UTC timezone for naive timestamps, and transform chains other than `null_to_empty`, `empty_to_null` and `decimal_separator_normalize` are computed by the reference engine, not in ClickHouse.	ClickHouse - Known limits
ClickHouse	An in-SQL `decrypt_expression` is not supported; use a Python-side decryption algorithm (`decrypt_algo`) instead.	ClickHouse - Known limits
ClickHouse	`Distributed` tables under `FINAL` are not supported.	ClickHouse - Known limits
ClickHouse	TLS is verified with a private CA, not yet against ClickHouse Cloud; mutual TLS (client certificates) is not supported.	ClickHouse - Known limits
ClickHouse	ClickHouse Cloud has not been verified live yet.	ClickHouse - Known limits
ClickHouse	Hierarchical narrowing (segment filters) is not available on ClickHouse in this version.	ClickHouse - Known limits
ClickHouse	A `String` column holding bytes that are not valid UTF-8 stops the run with an error naming the column.	ClickHouse - Known limits
Delta Lake via Trino	Trino's `delta` catalog reads a Delta `TIMESTAMP` at millisecond precision; a passing run that reads such a column shows Pass - Reduced timestamp fidelity and never presents as a clean pass.	Delta Lake - Trino reads a Delta TIMESTAMP at millisecond precision
Delta Lake via Snowflake	Snowflake Delta Direct has refresh-driven visibility, no Row Tracking metadata, no change reads and a 1,000-commits-per-refresh vendor limit; `commit_snapshot` stays refused on Snowflake.	Delta Lake - Snowflake Delta Direct
Delta Lake via File Dataset	`delta` mode reads S3, GCS, ADLS and local paths only, under a reader-feature allowlist; a VACUUMed pin is refused, never read as latest.	Delta Lake - File dataset `delta` mode
Delta Lake mirrored as Iceberg	A Databricks UniForm table publishes its Iceberg metadata asynchronously after the Delta commit, so an Iceberg reader can briefly see an older commit.	Delta Lake - UniForm lag, Iceberg - UniForm metadata lag
Iceberg	With no SQL engine (DuckDB), equality deletes read only on unpartitioned format v2 tables with NULL-free equality columns of the proven types, and format v3 reads only unpartitioned tables without nanosecond timestamps, column defaults or deletion vectors (#3328); other shapes are refused by name.	Iceberg - What is not there yet
Systems	SAP Application and Kafka CDC Stream are shown as Future Release: visible but disabled, and cannot be configured.	Systems - Connectors

Comparison settings¶

Area	Limitation	Where it is documented
Key identity	V1 offers no case-insensitive identity profile and no accent-insensitive one.	Key identity - What is not offered in V1
FLOAT / DOUBLE columns	An exact FLOAT comparison runs on the DuckDB reference for Oracle, SQL Server, Snowflake, BigQuery, DB2 LUW and Redshift: still exact, but the raw values leave the warehouse instead of being hashed in place.	Float column handling - Current behavior
BLOB / binary columns	Metadata only is content-blind: a change that preserves length is invisible to it.	Field configuration - BLOB / binary columns
XML columns	The comparison is whitespace-significant: if the two sides pretty-print differently, every row will differ.	Field configuration - BLOB / binary columns
Encrypted columns	Some per-field overrides are disabled on encrypted columns for engine pairs where the decrypt cast is not yet byte-verified.	Field configuration - Encrypted columns
Field configuration editors	The three editing surfaces do not expose identical fields.	Field configuration - Things worth knowing

Replication tools¶

Area	Limitation	Where it is documented
Replication profiles	Most tools in the picker show Future Release and are disabled, including SAP SLT, SAP BODS and Qlik Replicate; with Custom / Other comparison stays fully accurate but tool-aware readiness is lost.	Replication profiles - Which tools are usable today

Evidence and consumption¶

Area	Limitation	Where it is documented
Evidence coverage	Evidence may not cover everything: sensitive columns may carry no values, what is stored is the comparison rendering rather than the original value, caps can truncate or skip the diff set, and evidence is a point in time.	Repair posture - Coverage limits
Reconciliation Consumption	Consumption is not priced, budgeted, reconciled to a provider invoice, or projected.	Reconciliation Consumption - What Paridity does not do
Reconciliation Consumption	Self-hosted engines have no provider meter, so their systems report usage only (for example bytes scanned) and are hidden from the Consumption page unless you turn on Show systems without a provider meter. Redshift node-hours and RPU-hours are not captured; Redshift reports bytes scanned only.	Reconciliation Consumption - Metered vs usage

Configuration Reference¶

Every setting Paridity resolves at run time, where it lives, how conflicting values are combined, and what it changes about a reconciliation.

How to read this document. Paridity does not have one settings screen. A value can be declared at up to seven levels, and the platform combines them by an explicit rule per setting — usually "the most specific wins", but sometimes "the tightest wins" or "all must agree". If you only read one section, read The seven layers and the Reachability legend.

Source of truth. The tables below are rendered from src/services/config_resolver.py and the migration chain in db/migrations/, not from a design spec. Where a spec and the code disagreed, the code won and the spec was corrected (2026-08-01).

The "Default (fresh install)" cell of every L1 row is generated by scripts/gen_config_reference_defaults.py and gated in CI: it is the value the cfg.platform_defaults row holds on a freshly migrated database (#3106). Your deployment can differ - an operator may have changed the row since (the development stack's demo data, for example, raises egress_row_limit) - and Settings → Platform & Tenant Defaults always shows the live value.

The seven layers¶

Configuration resolves from seven layers. For most settings the most specific layer that defines a value wins; the exceptions are listed in Resolution rules and flagged on every affected row.

Layer	Where it lives	Scope	Who sets it
L1 Platform defaults	`cfg.platform_defaults` (one row)	The whole installation	Your Paridity operator, at deployment. Visible read-only in Settings → Platform & Tenant Defaults
L2 Tenant defaults	`cfg.tenant_defaults`	One tenant	Platform admin, in Settings → Platform & Tenant Defaults. Canonicalization defaults are visible there but operator-managed
L3 System profile	`cfg.systems`	One source or target system	Platform admin, in Systems
L4 Mapping	`cfg.mappings`	One mapping	Operator, in Mappings → Advanced
L5 Field override	`cfg.mapping_fields`	One column of one mapping	Operator, in Field Config
L6 Schedule	`cfg.schedules`	One schedule of one mapping	Operator, in Schedules
L7 Run override	`execution_requests.run_overrides`	One run	`platform_admin`, at run submission

A run is pinned to the configuration that existed when it started. Resolution produces an immutable snapshot stored in cfg.config_versions, and the run references that snapshot by id. Editing a mapping mid-run never changes the run in flight, and the snapshot's config_checksum is what proves to an auditor which policy was in force.

Reachability legend¶

Every stored value is visible in the product. What varies is where you can change it. Each option below carries one of these markers.

Marker	Meaning
UI	Settable in the web application.
API	Settable through the REST API (and therefore automatable), but with no dedicated screen.
Ops	Visible in Settings → Platform & Tenant Defaults, but changed by your Paridity operator with direct SQL rather than from a form. (The platform row and its defaults come from the database migrations; `db/seeds/platform_defaults.sql` sets no L1 value.)
Derived	Not set directly — computed from other settings or introspected from the engine.

Why "Ops" exists — and why it no longer means "hidden". Every L1 and L2 value is listed in Settings → Platform & Tenant Defaults, grouped, with the layer it comes from. An auditor or evaluator can therefore read the platform's entire comparison policy — including exactly what it treats as equal — without database access.

A subset stays operator-managed rather than editable in a form. That is deliberate, and there are two reasons:

Canonicalization defaults (null_token, numeric_scale, timestamp_precision, timezone, trim_strings, null_equals_empty, hash_spec_version) define what "equal" means. Changing one mid-stream silently alters every future comparison and makes prior evidence no longer directly comparable — there is no re-baseline step that would make that safe as a settings toggle. The screen shows each one with that explanation.

Signing key references point into your secrets backend and belong to the deployment, not the application. (The evidence key pointer, evidence_kek_ref, is the exception since #2395: it is editable in-app by a platform admin, because a tenant that cannot set it has masked-evidence reveal permanently off.)

Everything else at L2 — capacity, readiness, diff-safety limits, retention floors, evidence privacy, and the comparison-methodology flags — is editable on that screen by a platform admin, and every change is written to the audit log. L1 is read-only throughout: it is the floor a tenant inherits when it sets nothing.

Resolution rules¶

Most settings are "lowest layer wins". These are the exceptions — each is a deliberate safety property, not an inconsistency.

Setting(s)	Rule	Why
`max_concurrency`	MIN across L1, L2, L3, L4, L6	A system's own limit is a hard ceiling — a mapping can never ask a database for more parallelism than that database was declared to tolerate.
`max_partition_concurrency`	MIN across L4, L1 default, and both systems' L3 `max_concurrency`	Same ceiling logic, applied to parallel partitions.
`egress_row_limit`, `per_column_diff_limit`, `global_diff_cell_limit`	MIN across L1, L2, L4, L6 — and an L7 run override may only tighten	Cost and blast-radius caps must not be raisable by a single run.
`store_raw_diff_values`	AND across L2, L4, L5 — any `FALSE` suppresses	A privacy suppression at any level must be final. Values are stored as `NULL` with `values_masked = TRUE`.
`retain_*_days`	L4 wins, else the L1 default; cannot go below the L2 minimum	A mapping can retain evidence longer than the tenant floor, never shorter.
`priority`	MAX across L1 and L6	A platform floor can be raised for an urgent schedule and never dropped below.
`cdc_integration_mode`	L4 only	Changing how CDC signals gate a comparison is a policy change; a schedule or a single run must not be able to weaken it.
`cdc_table_semantics`	L4 only	Declares how the table's commit point relates to the one the comparison reads (#2853); a schedule or a single run must not be able to switch the hold off.
`read_pin_views`	L4 only	Declares that the mapping's Snowflake view(s) may be pinned at the readiness instant (#2933); what a run reads must not vary by schedule or run.
`fail_fast_threshold_pct`	Lowest wins; an L7 override may lower it	—
`diff_truncation_method`	L1 → L2 → L4, L4 wins; L6/L7 may select only `first_n` or `random_sample`	`stratified` has resource implications, so it stays a mapping-level decision.
`mask_in_ui`	L5 (field) only	A tenant-wide "mask everything" policy is not implemented — do not assume one exists.

Comparison semantics (canonicalization)¶

These decide what "equal" means. They are the highest-consequence settings in the product: changing one changes which rows are reported as differences.

Option	Layers	Default (fresh install)	Values	What it changes
`null_token`	L1, L2 — Ops	`<NULL>`	any string	The literal a NULL becomes before hashing. Both sides must agree or every NULL-bearing row mismatches.
`trim_strings`	L1, L2, L4 — Ops/API	`true`	bool	Strips the six ASCII whitespace characters (space, tab, newline, vertical tab, form feed, carriage return) — not Python's 29 Unicode whitespace codepoints. Non-breaking space and U+3000 are deliberately kept.
`null_equals_empty`	L1, L2, L4 — Ops/API	`false`	bool	When true, an empty string (after trimming) is treated as NULL. Turn this on only when one side genuinely stores `''` for absent data.
`numeric_scale`	L1, L2 — Ops	`6`	1–38	Decimal places every numeric is quantized to before hashing. Too low silently equates genuinely different values; too high can surface float noise as differences. Per-field overrides are rejected — this is global by design.
`timestamp_precision`	L1, L2 — Ops	`millis`	`seconds`, `millis`, `micros`	Truncation applied to every timestamp. Set this to the coarsest precision your two systems agree on.
`timezone`	L1, L2 — Ops (DB-only: the config-defaults API lists it read-only)	`UTC`	IANA name	How naive timestamps are interpreted. Non-UTC values change which rows fall inside an incremental window, not just how they render — and, for a `time` partition strategy, which rows each partition holds: a partition day is a day in this timezone, rendered per side by the column's comparison frame (naive columns compare wall-clock digits, timezone-aware columns compare instants). There is no per-mapping timezone; a change takes effect on the next config version each mapping mints.
`numeric_tolerance`	L4 only — API	none	decimal	Absolute difference below which two numerics are considered equal. Use for known float drift; it weakens exactness, so prefer fixing the source.
`hash_spec_version`	L1 — Ops	`1.0`	—	The canonicalization contract version. Changing it changes every hash.

Both sides always use the same rules. Canonicalization is resolved once per mapping and applied identically to source and target. That is what makes a cross-engine comparison meaningful.

SCD scope (SCD2 / bitemporal tables)¶

Scopes the comparison to the current rows of a side that legitimately keeps closed history rows. Mapping-level only, always operator-declared — the platform never infers SCD2-ness. See Mappings — declared scope.

Option	Layers	Default	Reach	What it changes
`scd_scope`	L4 only	none	UI	The whole declaration: scoped `side`, `mode` (`current_only` is the only v1 mode), a typed `current_predicate` (`flag_equals`, `end_ts_sentinel`, or `end_ts_null` — columns and values, never SQL), optional `interval_columns`, and `assertions`. Stored in the checksummed config snapshot, so every run records the exact predicate it was scoped by.
`assertions.unique_current_per_key`	—	`true`	UI (fixed)	The I1 invariant — always checked when a scope is declared; not disableable.
`assertions.no_interval_overlap`	—	`false`	UI	The I2 invariant — opt-in (extra self-join per run), requires `interval_columns`.

Rules the reference should not let you discover the hard way: the scoped side's engine must be live-verified for scope rendering (currently postgres, snowflake — anything else is rejected, fail-closed); scope columns may not also be key or mapped comparison columns; key_columns must be declared; and the scope composes with (never replaces) the side's filter_sql. Watermark queries are never scoped by scd_scope — see Watermarks.

Per-field overrides (L5)¶

Set in Field Config. All are UI-reachable unless noted.

Option	Values	What it changes
`exclude_from_hash`	bool	Column is omitted from the row fingerprint — it stops affecting pass/fail. Rejected on a key column (422 `KEY_COLUMN_EXCLUDED_FROM_HASH`): the proof phase compares an order-insensitive aggregate of the row hashes, so with the key outside the hash two datasets whose payloads are swapped between keys compare equal, and a proof-only mapping never runs the diff phase that would catch it.
`exclude_from_diff`	bool	Column still proves equality but is not reported cell-by-cell. Rejected on a key column (422 `KEY_COLUMN_EXCLUDED_FROM_DIFF`): the field diff finds each row by its key. A key column stored excluded before this rule still runs correctly, because the field diff always fetches the key.
`hash_column_order`	int	Position in the row serialization. Must match on both sides; normally leave to the platform.
`blob_compare_mode`	`hash`, `metadata_only`, `skip`, `full`	How a BLOB/CLOB is compared. Defaults to `hash` (content-exact) when unset. `metadata_only` compares only length and existence - a same-length content change becomes invisible, so it is an explicit opt-in, never a default. The length unit is the category's: bytes for BLOB/binary (`BLOB:<n>`), code points for CLOB/text (`CLOB:<n>`) and for JSON / XML (`SEMI:<n>`, one render on the proof, the recheck and the field diff; JSON / XML metadata_only always runs on the reference implementation, never native - #2996). `skip` is rejected on a key column (422 `KEY_COLUMN_BLOB_SKIPPED`): it removes the column from the row hash, so payloads swapped between keys compare equal.
`sensitivity_level`, `mask_in_ui`	— / bool	Masks the value in evidence views. Field-level only.
`store_raw_diff_values`	bool	The third term of the AND-chain. Any `FALSE` above suppresses storage.
`per_field_diff_limit`	int	Overrides the per-column cap for this column only.
`transform_chain`, `transform_chain_target`	ordered list	Per-side normalizations applied before comparison (e.g. case folding). Five transforms deliberately force the reference implementation rather than engine SQL — see the transform matrix in `CLAUDE.md`.
`canonical_type_override`	`date`, `timestamp`, `string`	Forces how a column is canonicalized when the engine's declared type is misleading. `date` / `timestamp` remap a date/timestamp column (inert on any other type). `string` declares that a ClickHouse `FixedString(N)` holds NUL-padded text, compared as text with the trailing NULs dropped; it is rejected on any other column type and with `blob_compare_mode` `metadata_only` / `skip` (422 `CANONICAL_TYPE_OVERRIDE_NOT_APPLICABLE`). A mapping with an effective override runs on the reference implementation on both sides unless that engine pair is verified for native rendering (none is for `string`).
`rounding_mode`, `timestamp_rounding`	mode	Rounding semantics for numerics/timestamps. Engines disagree on `.5` boundaries, so this is explicit.
`dual_compare_mode`, `float_compare_mode`	mode	Qlik dual values and float comparison strategy.
`decrypt_algo`, `decrypt_params`, `decrypt_key_ref`, `decrypt_expression`	—	Encrypted-column handling. `decrypt_algo`/`decrypt_params` are API-read-only today (set them with the field-config import path). Keys are always resolved through the secrets backend and never stored.
`declared_type`	—	Derived — introspected from the engine, refreshed by schema-refresh.

Compute and diff safety¶

Option	Layers	Default (fresh install)	Reach	What it changes
`compute_mode`	L1, L4, L6, L7	`hybrid`	UI	Where comparison work happens (pushdown vs. platform-side).
`proof_only_mode`	L1, L4, L6, L7	`false`	UI	Stop after the proof phase. A failure becomes terminal `fail_hash` with no field-level diff — cheap, but no evidence of what differed.
`max_concurrency`	L1, L2, L3, L4, L6 (MIN)	`5`	UI	Parallel queries against a system.
`max_partition_concurrency`	L4, L1, both L3 (MIN)	`4`	UI	Parallel partitions in a partitioned run.
`query_timeout_seconds`	L1, L3, L4	`300`	UI	Per-query timeout. Size it for the proof query at full volume.
`max_diff_rows`	L1, L4	`500000`	UI	Cap on rows handed to the diff engine. (No schedule-level override exists.)
`diff_sampling_mode` / `diff_sample_size`	mode: L1, L4 · size: L4 only	`all`	UI	Diff every mismatched row (`all`), a deterministic hash-selected N (`sample_n`), or percentage (`sample_pct`) of them. Selection is stable per run+key, not first-N. Sampled diffs are flagged `is_sampled = TRUE` in evidence and the total mismatch count is always recorded, so a sampled run still tells you the true scale of the difference. A `sample_*` mode with no `diff_sample_size` set on the mapping diffs 100% (the safe direction).
`egress_row_limit`	L1, L2, L4, L6 (MIN), L7 tighten-only	`1000000`	UI	Max mismatched rows pulled back to the worker — the main egress-cost control.
`per_column_diff_limit`	same	`10000`	UI	Max diff cells per column.
`global_diff_cell_limit`	same	`5000000`	UI	Max diff cells for the whole run.
`fail_fast_threshold_pct`	L1, L2, L4, L7	`50.00`	UI	If this share of rows mismatch, skip the diff phase and fail immediately — the run is already conclusive. It ends `fail_count` if the row counts differ, otherwise `fail_hash` (never `fail_field`: no column was compared).
`diff_truncation_method`	L1 → L2 → L4 (L4 wins); L6/L7 may select only `first_n` / `random_sample`	`first_n`	UI	Which rows survive truncation once a limit binds.

Readiness (when a comparison may begin)¶

Readiness is what makes Paridity CDC-aware: it refuses to compare a window that replication has not finished delivering, which is the difference between a real failure and a false alarm.

Option	Layers	Default (fresh install)	Reach	What it changes
`readiness_mode`	L1, L4, L6	`hybrid`	API	How strictly CDC/watermark signals gate the run.
`max_lag_seconds`	L1/L4, MIN with L6	`900`	UI	Maximum tolerated replication latency before the run defers.
`quiescence_seconds`	L1, L4, L6	`60`	UI	How long the watermark must be stable before proof starts.
`retry_max_attempts`	L1, L4, L6	`10`	API	Readiness re-check attempts before the run is skipped.
`retry_backoff_seconds`	L1	`60,120,300,600,900`	Ops	Backoff ladder between re-checks.
`cdc_integration_mode`	L4 only	`advisory`	UI	`strict` halts on CDC degradation; `advisory` warns and proceeds - except that a snapshot mapping is held (not compared) while a reachable replication tool reports a connector paused, stopped or failed; `disabled` skips CDC checks. Forced to `disabled` when no replication profile is attached.
`cdc_table_semantics`	L4 only	`none`	UI	`uniform_iceberg_mirror`: the Iceberg side is a Databricks UniForm mirror of a Delta table; each run is held (`not_ready`, cause `uniform_metadata_lag`) until the Iceberg table property `delta-version` has reached the Delta version (the commit-instant comparison is the fallback when the property is unreadable; unreadable both ways holds). Needs an Iceberg `commit_snapshot` watermark on one side and a Delta one on the other (422 otherwise). `replacing_merge_tree_final` / `clickpipes_peerdb` (#2900, ClickHouse): the ClickHouse side holds unmerged row versions (ReplacingMergeTree and the other merge-dependent engines); every read of it goes through `FINAL`, and `clickpipes_peerdb` also excludes the ClickPipes soft-delete rows (`_peerdb_is_deleted = 0`). Needs a ClickHouse system on one side (422 `CDC_TABLE_SEMANTICS_SYSTEM_MISMATCH` otherwise); the run refuses with the reason if the table's engine or columns do not fit. `none` on a merge-dependent ClickHouse table is allowed but flagged (an advisory, not a block): unmerged versions are counted until they merge. `none` changes nothing.
`read_pin_views`	L4 only	`false`	UI	Snowflake only (#2933). `true`: readiness also pins the mapping's Snowflake view side(s) at the readiness instant (`AT(TIMESTAMP)`), so every phase of a run reads the same state; the pinned read is verified first and a base table whose `DATA_RETENTION_TIME_IN_DAYS` is 0 refuses the run (a configuration hold naming that table). Needs a Snowflake system on one side (422 `READ_PIN_VIEWS_SYSTEM_MISMATCH` otherwise). Native tables need no flag: they are pinned when their retention is 1 day or more and read latest per phase at retention 0. `false` changes nothing.

Evidence and retention¶

Option	Layers	Default (fresh install)	Reach	What it changes
`evidence_kek_ref`	L2 only	(unset)	UI / API — platform admin only	Secrets-backend pointer (`aws-secrets://`, `azure-keyvault://`, `gcp-secretmanager://`, `infisical://`, `vault://`; `env://` in development only) to the tenant secret that protects masked evidence values — the KEK that wraps each run's data key and the HMAC key behind the fingerprints are both derived from it. A pointer, never the key: the platform resolves it at run time and never stores or returns the material. While unset, masked diffs are written fingerprint-only and "Reveal masked values" is unavailable for every run of the tenant (the diff worker degrades to NULL rather than lose evidence, so nothing fails — the UI states the reason instead). Runs recorded before it was set stay fingerprint-only; a change takes effect on each mapping's next run. Set on Settings → Platform & Tenant Defaults → Evidence key (secrets pointer), which also shows Masked-value reveal: available / not configured; every change is audit-logged with the caller's IP. Added #2395 — until then this was DB-only (no API, no UI, no seed).
`store_raw_diff_values`	L2 AND L4 AND L5	`true`	UI (L4/L5)	Whether actual differing values are stored. Any `FALSE` in the chain stores `NULL` and sets `values_masked`. (The L1 column exists but is not read — it cannot suppress storage platform-wide.)
`retain_summary_days`	L4, else L1; floored by the L2 minimum	`540`	UI	Retention for run summaries.
`retain_mismatch_keys_days`	L4, else L1; floored by the L2 minimum	`365`	UI	Retention for mismatched-row keys.
`retain_field_diffs_days`	L4, else L1; floored by the L2 minimum	`180`	UI	Retention for cell-level differences — usually the largest and shortest-lived evidence.
`bundle_retention_days`	L2, floored by L1	`2555`	UI	Retention for audit evidence bundles and their stored period containers. A bundle ages from its own generation time. The tenant value cannot go below the platform value. Enforced by the bundle leg of the same retention purge; `GET /evidence-bundles/retention` reports the resolved value, what awaits the next purge run, and the last purge on record.

Evidence tables are append-only. Deletion happens only through the retention purge, which writes its own audit record of what it removed.

Legal holds override every value above. A legal hold (Settings → Legal Holds) freezes a tenant, a mapping, or a single bundle so the purge skips it — a held record is never deleted no matter how far past its retention window it is. Placing and releasing a hold is approval-gated (evidence.legal_hold, a second person approves) and audited; the purge records each skip. A hold is an action, not a resolved-config field — it does not appear in the seven-layer resolution — but it is the control that lets you preserve evidence for litigation or an audit without changing anyone's retention numbers. See docs/runbooks/evidence_retention.md.

The retention purge runs on a built-in schedule (#2025). These four settings define how long evidence is eligible to be kept; the platform's built-in scheduler (RetentionPurgeWorkflow, a durable Temporal singleton on the scheduler worker) enforces them automatically. It is on by default but dry-run-first: for the first retention_purge_dry_run_days after install (default 7) it computes and attests what would be deleted without deleting anything, so a fresh deployment never deletes on day one. Every fire writes one cfg.audit_log attestation row. An ad-hoc manual purge stays available and approval-gated (evidence.retention_purge).

Deployment settings (env / Helm values, not per-tenant resolved config): retention_purge_enabled (default on), retention_purge_interval_seconds (86400 = daily), retention_purge_hour_utc (4), retention_purge_dry_run_days (7), retention_purge_force_dry_run (off — keep dry-run indefinitely).

Pausing the schedule is an approval-gated, audited operator act (evidence.retention_purge elevation → retention_purge.paused/.resumed), distinct from disabling it in deployment config. A paused schedule still fires but deletes nothing — the no-op is attested — so a paused schedule is a compliance fact GET /evidence-bundles/retention.enforcement reports as retention_purge_paused. Pause and resume it in Settings → Retention Schedule. See docs/runbooks/evidence_retention.md for the full procedure.

Legal holds (above) still override every value here: automatic deletion always skips a held scope. This is why the schedule ships default-on safely — evidence under a preservation order can always be frozen.

Windows, watermarks and partitions¶

Option	Layers	Reach	What it changes
`windowing_mode`	L4	API	`fixed_interval` (a rolling time window), `watermark_range` (bounded by observed watermarks), or `partitioned_snapshot` (per-partition frozen snapshots).
`watermark_domain`	L4	UI	`time_utc` or `sequence`. Never mix domains within a mapping — an SCN and a timestamp are not comparable.
Source/target watermark config	L4	UI	Strategy and field per side. Prefer, in order: `control_marker`, `delta_version`, `ingest_ts`, `partition_max`, `timestamp`, `custom_sql`, then snapshot modes.
`window_interval_seconds`, `min_advance_seconds`	L4	API	Window size and the minimum advance required for a new run to be worth executing.
`partition_strategy`	L4	UI	`time` (with a grain), `key_range`, `list`, or `composite` (two components, no nesting). Per-partition SQL scoping is decided per (engine, kind, column-type family) cell and per side — see the matrix below: on an unverified cell the partition is proven over the whole window on that side (correct, no speed benefit). The mapping page shows the per-side prediction; the run page shows what each run recorded.
`max_partitions_per_run`	L1 only	UI	Safety cap on partitions per run (default `256`).
`window_mode`	L6	UI	`auto` (incremental) or `full` (forced full-table comparison).

Per-partition SQL scoping — the live-verified cells (generated from src/services/partition_coverage.py by scripts/gen_partition_scoping_matrix.py; do not hand-edit). Scoping is decided per side of a mapping and per (engine, partition kind, column-type family) cell: a partition's proof and extract SQL are scoped to that partition on a side only when the side's engine has a live-verified cell for the partition's kind and the partition column's type family — on any other cell the partition's proof runs over the whole window on that side (results stay correct: the coverage check still holds; the per-partition speed benefit does not apply). A composite partition is scoped only when every component is. The mapping page shows the pre-run prediction per component and side; the run page shows what each run actually recorded.

Engine	`time`	`is_null`	`key_range`	`list`
`bigquery`	`date`, `timestamp`	`null`	`numeric`, `date`	`numeric`, `date`, `string`
`databricks`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`db2_luw`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`mysql`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`oracle`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`postgres`	`time`, `date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`redshift`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`sap_hana`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`snowflake`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`sqlserver`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`
`trino`	`date`, `timestamp`	`null`	`numeric`, `date`, `timestamp`	`numeric`, `date`, `timestamp`, `string`

Family names: time = a family-less legacy time predicate (PostgreSQL's type-agnostic cell); null = the NULL bucket; numeric / date / timestamp / string = the partition column's source-introspected type family (predicate_type_family). A column whose type the schema snapshot does not know has no family and is never scoped (refresh the schema).

time partitions under a non-UTC global timezone additionally need the side's comparison frame (classified when the run starts, from the column's type) to be live-verified for the engine — so the mapping page reports such a component as classified at run time until a run records it. PostgreSQL keeps its own per-side cast path and needs no frame cell.

Engine	verified frames (tz ≠ UTC)
`bigquery`	`utc_frame`, `wall_frame`
`databricks`	`utc_frame`, `wall_frame`
`db2_luw`	`wall_frame`
`mysql`	`utc_frame`, `wall_frame`
`oracle`	`utc_frame`, `wall_frame`
`postgres`	own per-side cast path (any classified frame)
`redshift`	`utc_frame`, `wall_frame`
`sap_hana`	`wall_frame`
`snowflake`	`utc_frame`, `wall_frame`
`sqlserver`	`utc_frame`, `wall_frame`
`trino`	`utc_frame`, `wall_frame`

key_range / list partitions on a timestamp column under a non-UTC global timezone (#2329) are rendered per side frame too — a naive authored bound means local wall-clock time in the tenant timezone, an offset-bearing bound the instant as written — but the >= / < / = bound shapes are verified SEPARATELY from time partitions, per (engine, kind, frame); PostgreSQL has no exemption here. An engine with no cell for the kind keeps such a partition an honest whole-window proof on that side (reason unverified_bound_frame); an engine with a cell reports the component as classified at run time until a run records the frame.

Engine	`key_range` (tz ≠ UTC, timestamp column)	`list` (tz ≠ UTC, timestamp column)
`bigquery`	—	—
`databricks`	—	—
`db2_luw`	`wall_frame`	`wall_frame`
`mysql`	`utc_frame`, `wall_frame`	`utc_frame`, `wall_frame`
`oracle`	`utc_frame`, `wall_frame`	`utc_frame`, `wall_frame`
`postgres`	`utc_frame`, `wall_frame`	`utc_frame`, `wall_frame`
`redshift`	—	—
`sap_hana`	—	—
`snowflake`	—	—
`sqlserver`	`utc_frame`, `wall_frame`	`utc_frame`, `wall_frame`
`trino`	`utc_frame`, `wall_frame`	`utc_frame`, `wall_frame`

Reason codes (the reason / explanation fields on GET /runs/{id} → partition_scoping, GET /runs/{id}/partitions → scoping_detail and GET /mappings/{id} → partition_scoping; a composite reports composite_component:<index>:<reason> for its first failing component):

Reason	What it means
`composite_shape`	The composite partition is malformed (a nested composite, a non-object component, or no components); the partition's proof ran over the whole window.
`frame_classified_at_run`	Partition scoping for a partition kind partition on a timestamp column under a non-UTC global timezone depends on the affected partition column's comparison frame, which is classified when the run starts; until a run records it the mapping reports scoping as not applied.
`no_side_column`	The partition predicate names no affected-side column (declare source_column and target_column on the strategy); the partition's proof ran over the whole window.
`no_typed_spelling`	The the engine engine has no exact literal spelling for this column-type partition kind bound (a bound that does not parse in the column's type, a missing bound, or a string list without a known trim setting); the partition's proof ran over the whole window.
`quantize_unverified`	This column-type partition kind partition compares a column whose canonical value is rounded (or truncated) before hashing, and quantized membership on the engine has not been live-verified (only PostgreSQL half-up rounding is); a raw-column bound could place a boundary row in different partitions on the two sides, so the partition's proof ran over the whole window.
`timestamp_bound_non_utc_tz`	This run was recorded before per-frame rendering of timestamp partition kind bounds existed (#2320, superseded by #2329): under a non-UTC global timezone the bounds could not be rendered per comparison frame (wall-clock vs instant), so the partition was not scoped on any engine; re-run the mapping to get per-frame scoping where the engine is verified; the partition's proof ran over the whole window.
`unclassified_frame`	Under a non-UTC global timezone the affected partition column's comparison frame could not be classified (wall-clock vs instant), so a time bound cannot be rendered safely; the partition's proof ran over the whole window.
`unknown_family`	The partition column's type is not known to the mapping's schema snapshot (refresh the schema, or partition on a mapped column), so no typed boundary literal can be rendered; the partition's proof ran over the whole window.
`unsupported_kind`	This partition predicate is not one of the scopable kinds (time, is_null, key_range, list, or a composite of those); the partition's proof ran over the whole window.
`unverified_bound_frame`	The the engine engine has not been live-verified to render a timestamp partition kind partition against a comparison-frame column under a non-UTC global timezone (the >= / < / = bound shapes are verified separately from time partitions); the partition's proof ran over the whole window.
`unverified_cell`	The the engine engine has not been live-verified to place every boundary row of a column-type partition kind partition exactly; the partition's proof ran over the whole window.
`unverified_frame`	The the engine engine has not been live-verified to render a time partition against a comparison-frame column under a non-UTC global timezone; the partition's proof ran over the whole window.

Schedules¶

Schedules are interval-based, not cron. Set in Schedules; all UI.

Option	Values	What it changes
`interval_value` + `interval_unit`	int + `minutes`/`hours`/`days`/`weeks`	Recurrence.
`days_of_week`	ISO 1–7, or empty	Restrict to specific days.
`active_start_time`, `active_end_time`	time	Daily active window.
`effective_from`, `effective_until`	date	Calendar bounds — how a quarter-end or year-end schedule is expressed.
`tags`	list	Grouping used by close mode.
`priority`	`low`, `normal`, `high`, `critical`	Ordering when the platform is congested. Resolved as MAX with the platform floor.
`window_mode`	`auto`, `full`	Forces a full comparison for this schedule (e.g. a quarter-end sweep).
`enabled`	bool	Master toggle.

Close mode is a bulk convenience toggle over tags, not a separate scheduling engine: it enables/disables by tag, may step priority by −3..+3 enum levels, and may apply the two tightening overrides (max_lag_seconds, quiescence_seconds). Overrides looser than a schedule's mapping are skipped per schedule and reported back. Available at Configure → Schedules → Close Mode, and as POST /schedules/bulk-toggle. See Schedules and close mode.

Deployment settings¶

Set as environment variables by your operator; not part of run configuration. The complete list lives in src/config/settings.py. The ones a deployer must understand:

Variable	Purpose
`DATABASE_URL`	Control-plane Postgres. Required, no default.
`APP_ENV`	`development` / `test` / `local` / `production`. Must be set explicitly — the dev auth bypass is refused unless this names a non-production environment.
`TEMPORAL_HOST`, `TEMPORAL_NAMESPACE`	Workflow engine.
`SECRETS_BACKEND`	`aws`, `azure`, `gcp`, `infisical`, or `vault`.
`OIDC_ISSUER`, `OIDC_AUDIENCE`, `OIDC_JWKS_BASE_URL`	Identity provider.
`SCIM_AUDIENCE`, `MCP_AUDIENCE`	Dedicated audiences for the provisioning and agent surfaces. A token minted for one surface is rejected on the others.
`PUBLIC_BASE_URL`	Used in SCIM resource URLs and OAuth metadata.
`CFG_AI_ENABLED`, `CFG_AI_REQUIRE_TENANT_OPT_IN`, `MCP_RATE_LIMIT_PER_MINUTE`	AI/agent surface controls. Production forces tenant opt-in.
`AUDIT_SIGNING_KEY_REF`	Signing key for the audit hash-chain anchor.
`AUDIT_SIGNING_PREVIOUS_KEY_REFS`	Retired signing keys, verify-only (comma-separated). Move the outgoing ref here when you rotate: a checkpoint is verified under the key that signed it, and only declared keys are trusted. Keys left out are reported as unverified on `GET /audit/verify` — not as tampering, but no longer provable either.

Glossary — Mapping, Replication Profile, watermark, hash_total vs row_hash.
Float column handling — the one canonicalization topic with its own guide.
docs/specs/spec_canonical_hashing.md — the full canonicalization contract.
docs/specs/spec_watermark_config.md — per-strategy watermark configuration.

Proof & credibility

How Paridity guarantees accuracy - and how to verify it yourself¶

The claim¶

For every engine pair marked Verified in the table below, Paridity detects every byte-level difference between a source and target dataset - with zero known false passes - using a deterministic cross-engine canonical hash comparison, not sampling and not inference. A false pass (reporting a dataset clean when it is not) is treated as a compliance failure, never an acceptable trade-off.

This is a narrow, falsifiable claim. It does not claim every supported engine is at the same verification depth (see Limitations), and it does not claim Paridity infers or judges correctness - correctness is always computed, never guessed. It is also scoped to a stated threat model - the proof defends against replication faults and corruption, not against a privileged insider deliberately tampering with the databases being compared (see Threat model); stating that scope precisely is part of the claim.

How it works (deterministic)¶

Aligned time window. Before any comparison, Paridity aligns the source and target to the same point in time using watermark state (CDC offset, control marker, or snapshot) - this is what makes it CDC-aware: comparing datasets mid-replication produces false positives, so Paridity waits for a stable, comparable window rather than comparing whatever happens to be in each table right now. Where the storage engine itself can change the picture between the readiness instant and the read, the read is pinned or merge-resolved rather than taken at "latest": a Snowflake native table with Time Travel retention is read AT(TIMESTAMP) of the readiness instant (views opt in per mapping with read_pin_views; a table with retention 0 is read unpinned and logged as such, never silently), and a ClickHouse ReplacingMergeTree side is read through FINAL when the mapping declares cdc_table_semantics (replacing_merge_tree_final or clickpipes_peerdb) - an operator declaration, never an inference.
Proof phase - hash_total. Paridity computes a hash_total - an aggregate fingerprint of every row in the window - independently on the source and the target, then compares the two values. This is an O(1) pass/fail signal: it proves an entire dataset identical (or not) without pulling the rows across the network. hash_total is not a checksum in the informal sense; it is two independent commutative aggregates (an XOR and a SUM of per-row hash integers) so that accidental cancellation between differing rows is statistically negligible - quantified, and scoped precisely, in Threat model below.
Canonicalization - the part that makes cross-engine comparison possible. Every column value is normalized to an identical string form before it is hashed, regardless of which database produced it:
NULL → a literal null token (<NULL> by default), and empty strings are folded into it only if configured to.
Numeric values → fixed-point decimal strings at a configured scale (e.g. 1250.5 at scale 4 → "1250.5000") - this is what makes an Oracle NUMBER and a Snowflake NUMBER hash identically for the same value.
Timestamps → normalized to UTC and rendered as ISO 8601 (2024-03-15T14:30:00.123Z) at a configured precision, never a database-local format.
Row fields are joined with the ASCII Unit Separator (\x1F) as the delimiter - chosen specifically because it cannot appear in normal text, and any literal occurrence is escaped by doubling it.
Column order is fixed per mapping (hash_column_order) and stored in the resolved config snapshot, so it is identical on both sides for the life of the run.

Full canonicalization rules are available to reviewers on request through your Paridity account contact. 4. row_hash - key extraction. If hash_total does not match, Paridity computes a SHA-256 row_hash per row on both sides and correlates them to identify exactly which rows differ - without re-comparing rows that already match. 5. field_diff - the deliverable. For each mismatched row, Paridity compares column-by-column and records the specific field(s) that differ, with the source and target values (subject to masking policy). This is the evidence an operator or auditor actually reads.

Note: AI never determines correctness in Paridity. Correctness is always established deterministically - by aligned time windows, hash_total aggregate fingerprints, row_hash per-row fingerprints, and field_diff column comparisons. AI assists triage and explanation of already-computed results only; it never decides whether data matches and never auto-confirms an operator action.

Threat model¶

The proof phase is an integrity check for replication correctness, and the accuracy claim above is scoped to that threat model. Paridity states this explicitly rather than leaving it implied.

What the proof defends against. Non-adversarial divergence between source and target: replication defects (missed, duplicated, or partial applies), corruption, canonicalization and schema drift, in-flight replication latency. hash_total is a homomorphic multiset hash - two commutative aggregates (XOR and SUM) over per-row SHA-256 fingerprints. Faults do not choose their bytes, so their contribution to the aggregate is effectively random, and the probability that a genuinely different dataset produces a matching aggregate is ≈10⁻¹⁹ per comparison (quantified below; smaller still on engines where the XOR component is real). Row counts are compared alongside the aggregate, so any fault that adds or removes rows is caught by count alone.

What the proof does not defend against. A malicious insider who can write the datasets being compared and knows the construction. Any 64-bit SQL-computable aggregate - however strong the per-row hash - can in principle be defeated by an adversary who modifies a row and then crafts other rows in the same window whose fingerprint deltas cancel in the aggregate (a generalized-birthday search, computationally feasible at this width). This is a property of every SQL-pushable aggregate construction, not an implementation defect. Adversarially secure alternatives (LtHash-class multiset hashes) cannot be pushed down into your engines as SQL - adopting one would trade away the property that the proof is computed inside your own engines, where you can verify it independently. Paridity scopes the claim honestly instead of making that trade silently.

What covers the insider case. Deliberate dataset tampering by a privileged insider is an access-control and audit problem, addressed by controls rather than by the aggregate:

Your own database access controls and change auditing on source and target - writing production data is already a privileged, audited act under your existing SOX/ICFR controls. Paridity's proof complements those controls; it does not replace them.
Paridity's own records are adversarially hardened even though the proof gate is not: evidence is append-only; the audit log is hash-chained, written through a single enrichment chokepoint, and sealed with signed checkpoints; evidence bundles are signed, hash-chained, and verifiable offline (see the audit & compliance proof). An insider could aim to fool a future proof; they cannot silently rewrite what past runs recorded.
RBAC separation (four roles, API-enforced) with actor and IP attribution on every administrative action inside Paridity.

If your threat model requires adversarial-grade tamper evidence over the databases themselves, that calls for database-side controls (WORM storage, native database auditing) - Paridity sits alongside them as the replication- correctness control, and its own evidence trail is built to the adversarial standard those controls expect.

The evidence¶

Every row below is backed by a passing, reproducible test vector or a live end-to-end run - not just a code review.

Claim component	Evidence source	Result
Canonicalization produces byte-identical concat strings across engines	The cross-engine byte-verified allowlist is the complete 66-pair C(12,2) matrix (12 engines since ClickHouse, 2026-10-08); `tests/hashing/test_vectors.json` tv011-tv022	Verified at the canonicalization (concat-string) level for all 12 pushdown engines - PostgreSQL, MySQL, Snowflake, Databricks, SQL Server, Oracle, Trino, BigQuery, Redshift, SAP HANA, DB2 LUW, ClickHouse - via live per-engine byte-identity test classes. Programmatically counted: 66 entries, 12 engines, no missing or extra pair.
Every one of the 55 engine pairs of the matrix as it stood at the 2026-08-23 run (the 11 ClickHouse pairs came later and are not part of that run) is verified directly, end-to-end - not only proven transitively via the reference implementation	Artifact `docs/live_gates/pairing_matrix_latest.{json,txt}` (run `2026-08-23T13:33:47Z`, `all_55_direct: true`)	Verified - the pairing-matrix lane ran all 55 pair configs through the real pipeline (API → Temporal → readiness → SQL → diff → evidence) in one recorded run: 55/55 native `pass_clean` with `fallback_used=False` (`hash_method=xor_sum_sha256`), each fail-case leg reaching `fail_count` (except `bigquery_to_redshift`, PASS-only - neither engine is harness-injectable). No pair false-failed on identical seeds.
`row_hash` computed via each engine's native SQL is byte-identical to the Python `HashService` reference, not just string-identical	The live byte-identity suites `tests/integration/connectors/test_cross_engine_byte_identity.py` (one `Test<Engine>ByteIdentity` class per engine; its own 33-case vector list - 29 ids drawn from `tv001`-`tv035`, not every id in that range, plus the four astral `tv048`-`tv051` - includes a NULL decimal, a zero-fraction timestamp and non-ASCII text) and `test_clickhouse_byte_identity_2827.py`	Verified live for all 12 pushdown engines - each class executes the engine's own `row_hash` SQL (the exact expression `extract_keys` emits) and asserts the hex equals the reference SHA-256 of the canonical row string. DB2 LUW, once run by hand outside the suite because of a host driver limitation, is now a gated class in the same suite, run from the worker container where the driver is installed. The full required-mode live-gate run of 2026-10-28 (`docs/live_gates/run_20261028T151921Z.json`, also `latest.json`; verdict GREEN, 1,614 passed, 0 failed, 0 errors across 12 engines) records DB2 on its own under `per_engine.db2_luw`: 88 passed, 0 failed, 0 errors, 2 skipped, across four classes - `TestDb2ByteIdentity` (32 passed, 1 skipped), `TestDb2DecryptByteIdentity` (40 passed, 1 skipped), `TestExtractRowsForHashByteIdentity` (8 passed) and `TestPreNullChainLiveDb2` (8 passed). Both skips are the same by-design case, named in `skipped_tests` with the reason: `tv017` (32 integer digits) exceeds DB2's 31-digit DECIMAL and is not representable. The per-test junit for that run is `run_20261028T151921Z.junit.xml`; the earlier hand-run per-vector record `docs/live_gates/db2_byte_identity_20261027T1333Z.txt` (DB2 11.5.9.0, 32 vectors passed at concat and hash level) agrees with it. Concat-string parity alone does not imply hash-level parity, so the suites assert at the hash level for every engine.
`hash_total` aggregate arithmetic (XOR + SUM, including the SUM-only path) is correct at defined edge cases	`tests/hashing/test_vectors.json` `ht001`-`ht010`	Verified - covers single-row, empty dataset, XOR-cancellation, row-count mismatch, SUM-modulo-boundary, the `xor_is_real=False` SUM-only comparison path (`ht009`), and the aggregate over the four astral-text row hashes (`ht010`).
Where an engine cannot render a value byte-faithfully in SQL, it refuses the native path rather than hashing different bytes	Connector refusal sites, each pinned by unit tests and recorded in the cross-engine hash verification record (Issues #68, #69, #70, fixed 2026-10-10/11): Trino `to_hex` and BigQuery `TO_HEX` over a text column under a binary plan (`NotSupportedError` after one cached schema read); SAP HANA `BINTOHEX` over an NVARCHAR column (the same refusal - HANA would otherwise accept it silently and render CESU-8); the SAP HANA row concat hashed as `STRTOBIN(..., 'UTF-8')`, never `TO_VARBINARY` (CESU-8 false-failed every astral character against every peer); the PostgreSQL row hash as `sha256(convert_to(concat, 'UTF8'))`, never `(concat)::bytea` (which parsed bytea escape input - a false pass of `\101` against `A`); PostgreSQL arrays and MySQL `json`, and `metadata_only` on any semi-structured column, always compared on the reference	Verified - every refusal lands on the DuckDB reference path with the reason logged, and the run is never a silent pass under a different byte stream. The HANA defect was found by astral (non-BMP) text, and the astral vectors `tv048`-`tv051` were added to the standing vector set so every engine is gated on that class; the PostgreSQL defect was found while disproving an unrelated array-render hypothesis - a reminder that the record is kept by testing claims, not by assuming them.
Every connector can extract raw rows for the DuckDB reference (fallback) path and produces the same `hash_total` as every other connector on a shared fixture	Live byte-identity across all 12 connectors on a shared fixture	Verified - all 12 connectors produce the identical `hash_total` (`4a136562c19cd220:4e2c14af69caec14`) on the shared 6-row fixture, live byte-identity-verified against `HashService` for every type-fidelity-bearing engine.
Cloud warehouse pairs are hash-level verified live, not only in CI fixtures	Live cloud-warehouse PASS + FAIL runs	Verified - PostgreSQL↔Snowflake, ↔Databricks, ↔BigQuery, ↔Redshift each ran a live PASS (`pass_clean`, native pushdown) and a live FAIL (`fail_count`, mismatch=1) against a hardened dataset containing CJK text, a NULL decimal, a foreign `timestamptz` source type, and uppercase target columns.
Accuracy is verified at production data volume, not only in small test fixtures	Live cross-engine runs to 100,000,000 rows	Verified - three flagship pairs (PostgreSQL→Snowflake, PostgreSQL→Databricks, Db2 LUW→BigQuery) are each verified by live reconciliations from 1M through 100,000,000 rows, and Oracle→Snowflake through 10M; every clean run was preceded by a fail-first proof in which a planted 0.01% field tamper and planted deletions were recovered exactly (timings and conditions in the companion performance proof document). Volume is also where subtle defects surface: an earlier 1,000,000-row MySQL↔Snowflake run caught Snowflake hex aggregate components that were not fully zero-padded - a genuine at-volume false-failure risk, since fixed - and in the runs cited here Oracle ran full-strength SHA-256 `DBMS_CRYPTO` pushdown with matching cross-engine `hash_method` - the weak 32-bit `ora_hash_sum` fallback, which engages when the `DBMS_CRYPTO` grant is missing (see Limitations), was never engaged in these runs. A 2026-08-31 audit of PostgreSQL ±infinity dates and timestamps found and fixed a third such class: the driver's own codec collapsed `infinity` onto the `9999-12-31` SCD2 sentinel in both directions, and `TO_CHAR` rendered an infinite timestamp as the null token - so a stored `infinity` could compare equal to a stored literal. ±infinity are now distinct values canonicalized as PostgreSQL's own text render, with a live pin.
SQL Server, Oracle, and Trino are hash-level verified by live end-to-end runs, not only by byte-identity fixtures	Live PASS + FAIL runs (SQL Server UTF-8 hashing, Oracle target-side field resolution + UTC session TZ, Trino NULL-decimal rendering)	Verified live (`pass_clean` + `fail_count`) for each. Because concat-string parity does not imply hash parity, hash-level assertion is mandatory for every engine, not just the canonicalization string.
SUM-only comparison (engines without a native XOR aggregate) is statistically sound, not just "good enough"	Collision-probability analysis	Verified - collision probability quantified at ≈1.08×10⁻¹⁹ per comparison for the SHA-256 SUM path; the weak 32-bit `ora_hash_sum` / `checksum_agg_sqlserver` fallback paths (≈2.3×10⁻¹⁰ per comparison) are flagged as weak in the proof evidence and treated as advisory, not audit-grade.

Verify it yourself¶

These steps reproduce the claim in your own environment. Steps 1-4 can be run self-serve by a technical evaluator with operator access; step 5 is independent of the running application and can be run by anyone with the repository.

Create a Mapping between two engines from the Verified list below (e.g. PostgreSQL → Snowflake), pointed at a dataset you control.
Run it once cleanly. Confirm the run reaches pass_clean and that the Run detail view shows a native-pushdown proof (not a DuckDB fallback, unless one of the two engines requires it - see Limitations).
Force a known, single-cell mismatch on the target - change one value in one row directly in the target system.
Run the Mapping again. Confirm:
The proof phase's hash_total values now differ between source and target (visible on the Run detail page).
The run's status reaches Issue detected - terminal state fail_field, because you changed a value (row counts still match, so the proof hash_total fails, then the diff phase isolates the field). Had you instead added or removed a row, it would reach fail_count. Either way - never a silent pass.
The Evidence tab's field_diff record identifies the exact row and column you changed, with the source and target values shown side by side (subject to your masking configuration).
Run the shipped test vectors independently of the running application. tests/hashing/test_vectors.json contains the canonical row-hash vectors (tv001-tv051) and hash_total vectors (ht001-ht010) that gate every engine implementation before it can be declared production-ready. These are plain input/expected-output pairs - you do not need to trust Paridity's own test runner to check them; you can independently compute SHA-256 over the documented canonical string for any vector and confirm it matches expected_row_hash.

Tip: Steps 1-4 require a live source and target you're willing to write a test value into - most evaluators do this with an SE in a sandboxed environment rather than a production system. Step 5 requires no Paridity environment at all - it can be done offline against the JSON file alone.

Limitations¶

Accuracy verification depth is not identical across every engine. This table states, honestly, what "Verified" means for each scope - over-stating this would be exactly the kind of over-claim this document exists to prevent.

Scope	Status	Notes
PostgreSQL, MySQL, Snowflake, Databricks, Trino, BigQuery, ClickHouse (native XOR aggregate available)	Verified	`xor_is_real=True` - real XOR and SUM components, native SHA-256 pushdown, hash-level byte-identity vs the `HashService` reference, including cloud pairs. ClickHouse (added 2026-10-08) is byte-verified on 26.8 and on the 25.8 LTS floor; a server below 25.8 is refused at connect time, never run on a fallback.
Runs whose schema could not be verified, or whose read was at lower fidelity than stored	Verified - never an unqualified clean pass	Schema introspection is tri-state per side (`verified` / `unavailable` / `failed`); a run with an unverified side records it in the run's drift detail, displays as "Pass · Schema unverified", and is excluded from pass counts (filterable via `display_status=schema_unverified`). Likewise a column the connector reads at lower fidelity than stored - today Trino `delta_lake` reads a Delta `TIMESTAMP` rounded to the millisecond, and Trino `hive` does the same at its default `hive.timestamp-precision` - is recorded on the run, displayed as "Pass · Reduced timestamp fidelity" and excluded from pass counts (`display_status=reduced_fidelity`). Neither label is configurable away.
Oracle, SQL Server, Redshift, SAP HANA, DB2 LUW (v11.1+)	Verified - SUM-only by design	`xor_is_real=False`: no native `BIT_XOR` aggregate on these engines, so `hash_total` comparison uses the SUM component alone; the XOR component is a fixed `0000000000000000` and is never compared. This is an architectural limitation, not an unverified gap - the SUM-only collision probability is quantified and documented (≈1.08×10⁻¹⁹ per comparison). Native pushdown is byte-verified live for all five engines.
DB2 LUW v10.5 / v11.0	Verified - via DuckDB reference fallback	These versions lack `HASH(expr, 2)`; `hash_total()` and `extract_keys()` raise `NotSupportedError` and Paridity automatically falls back to the DuckDB reference implementation (ground truth) for both proof and diff phases. Slower than native pushdown, not less accurate.
Object Store, File Dataset connectors	Verified	No SQL engine to push down to - the DuckDB reference path is always used. File Dataset: byte-identity plus live pairings. Object Store: inventory-level byte-identity vs an independent SDK listing live against S3-compatible storage, orchestrator legs, and a live end-to-end manifest-vs-bucket run (`pass_clean` + a same-length content tamper caught as `fail_field`), with per-flavor content-fingerprint constraints pinned by tests (multipart/ADLS/GCS etag semantics).
Salesforce	Verified	API-only source; always uses the DuckDB reference path. Live end-to-end verified against four target warehouses (PostgreSQL, Snowflake, Databricks, BigQuery) - `pass_clean` and `fail_field` on injected drift, with full datetime-string canonicalization. Verified via permanent pairing harness legs: Campaign and Account faithful replicas, clean pass plus injected-drift catch, DATE/DATETIME/CURRENCY/INT/PICKLIST coverage.
Oracle `ora_hash_sum` fallback / SQL Server `checksum_agg_sqlserver` fallback	Supported - advisory only, not audit-grade	Only engaged when SHA-256 pushdown is unavailable. 32-bit hash, collision probability ≈2.3×10⁻¹⁰ per comparison - non-negligible at volume for a compliance platform. Flagged as weak in the proof evidence; Paridity does not present these results as equivalent to the SHA-256 path.
Adversarial tampering of the compared datasets by a construction-aware insider	Out of scope of the proof aggregate - by design, stated explicitly	See Threat model. The `hash_total` proof targets replication faults and corruption; a privileged insider deliberately writing compensating rows into the compared datasets is covered by compensating controls (your database access controls and change auditing, plus Paridity's append-only evidence, hash-chained audit log, and signed bundles), not by the aggregate.
Cross-engine pairs where source and target `hash_method` values differ	Verified (by automatic fallback)	Paridity detects the mismatch and forces both sides to the DuckDB reference path rather than attempting a native comparison it cannot guarantee - this is the platform failing safe, not a gap. In the current engine set every proof connector emits `xor_sum_sha256`, so this path is a safety net rather than the common case.
Mappings with a declared SCD2 scope	Verified - scoped; PostgreSQL and Snowflake only	A mapping may declare a current-rows-only scope for an SCD2/bitemporal side. The proof then covers that side's current rows, not the whole table, and the run is badged "Scoped: SCD current rows" so a scoped pass is never displayed as a whole-table pass. The scope is always operator-declared and versioned inside the checksummed config snapshot - never inferred, because inferring that one side is a superset by design is a false-pass machine - and is rejected outright on any engine whose scope rendering is not live byte-verified (today `postgres` and `snowflake`; fail-closed at both validation layers). Every scoped run first checks the scope's own premise: exactly one current row per business key (always), and optionally no overlapping validity intervals. A violation reclassifies the run SCD invariant violated and excludes it from pass counts.

This document reflects the audit state as of last_verified above. New engine pairs and deeper verification depth are added continuously - the live status of every open item is available to reviewers on request through your Paridity account contact.

How Paridity stays efficient at scale - and what we have actually measured¶

The claim¶

Paridity establishes whether two datasets match without transferring either of them, and the volume of data it moves afterwards is proportional to the size of the difference, not the size of the dataset. A clean 100-million-row reconciliation moves a row count and a short fingerprint string from each side - and nothing else.

Every performance figure presented here as a result is a measurement taken on named hardware, at a named row count, by a named harness, at a named commit. No result in this document is extrapolated, modelled, or projected, and none is a contractual commitment. One extrapolation does appear, in Limitations, shown specifically to demonstrate why we do not publish them. Where we have not measured something, this document says so rather than estimating it.

That second paragraph is part of the claim, not a disclaimer around it. A reconciliation platform that overstates its own throughput has the same defect as one that overstates a match: it produces a number that gets acted on and is not true.

How it works (deterministic)¶

Efficiency in Paridity is not the result of tuning. It comes from doing the cheap test first and only paying for the expensive one when the cheap test fails.

The proof gate runs inside your engine. Paridity asks the source and target databases to each compute a hash_total - a single aggregate fingerprint over every row in the window - using their own native SQL. What crosses the network is a row count and a fingerprint pair per side, on the order of tens of bytes. For a dataset that matches, this is the entire cost of the reconciliation: no rows are read out of either system, and there is no egress charge to pay.
The comparison window is aligned before anything is computed. Paridity uses watermark state - a CDC offset, a control marker, or an explicit snapshot - to pick a window in which both sides are comparable. This is an accuracy mechanism first, but it is also an efficiency one: comparing mid-replication produces a false difference, and a false difference costs a full diff cycle to disprove.
Only the rows that could differ are ever extracted. When the proof gate fails, Paridity does not pull both tables. The optional stratified proof splits the aggregate into 256 independent buckets, so a failing run can tell which buckets disagree and extract only those. A changed row dirties at most two buckets, so on a sparse difference the extraction is a small fraction of the table.
Only the rows that actually differ are compared column by column. Row fingerprints (row_hash) identify the differing rows; the field_diff phase then runs per-cell comparison on those rows alone.
Pushdown is the default, and every fallback is logged with its reason. The hash and key-extraction work runs inside the customer's engine wherever the engine can express it. Where it cannot - an engine without a native SHA-256 aggregate, or a transform whose semantics differ across engines - Paridity falls back to its own reference implementation and records why. The fallback is slower; it is never less accurate.

The consequence worth internalising: the expensive phases only run when something is wrong. A healthy estate is cheap to prove continuously.

Note: AI never determines correctness in Paridity. Correctness is always established deterministically - by aligned windows, hash_total aggregate fingerprints, row_hash per-row fingerprints, and field_diff column comparison. AI assists triage and explanation of already-computed results only; it never decides whether data matches, and it plays no part in any figure on this page.

The reference hardware¶

Every figure published as a result below was taken on one machine, referred to throughout as the reference box - with three exceptions, each labelled where it appears and each explained at the end of this section. Numbers were captured with the machine otherwise idle, and with the three CI runners confirmed not busy first. Where a figure was taken with the worker process deliberately capped at 4 CPUs on that box (the ClickHouse key-extraction readings), the cap is stated beside it.


CPU	AMD Ryzen AI 9 HX 470 - 12 cores / 24 threads
Memory	64 GB
Storage	Samsung SSD 990 PRO 2 TB NVMe

This is our hardware, not a model of yours. A 4 vCPU worker pod - the baseline our internal engineering floors are written against - will produce lower absolute figures, and a DuckDB path that uses 24 threads here will not find 24 threads there. Where a figure depends on thread count, this document says so.

No throughput or duration figure measured on a developer laptop or on the Docker development stack is published as a result in this document. Such measurements exist internally, and on the laptop they run at roughly half the box across the board, which is why they are not a basis for anything told to a customer.

There are exactly three exceptions, each marked at the point it appears:

The narrowing egress cut, in the evidence table below. It is a ratio of rows extracted to rows extracted, which is hardware-independent by construction - how many rows a query returns does not depend on how fast the machine running it is.
The readiness short-circuit timing, in the floors table further down. It is a genuine development-stack duration and it is labelled as one. It appears because the margin against its target is four orders of magnitude, which no change of hardware closes; it is presented as evidence that a floor is met, never as a performance figure to quote.
The Databricks partition-reuse comparison, in its own subsection below. Its durations were recorded on the development stack against a cloud serverless warehouse and are labelled as such. It appears because the point it establishes is a comparison between two code paths on the same run shape, and the per-row gap between those paths (a Python reference hash at tens of microseconds per row against a native pushdown at tens of nanoseconds) is three orders of magnitude, which no change of worker hardware closes. The durations are not performance figures to quote.

The evidence¶

Each row states the row count it was taken at and the commit at which it was measured. Every one of these harnesses drives the real production code path - the actual Temporal activity or the actual connector method - not a reimplementation of it. The internal records behind each figure are available to reviewers on request through your Paridity account contact.

Proof phase - establishing a match without moving data¶

What was measured	Result	Scale and conditions
PostgreSQL `hash_total` native pushdown	434,204 rows/sec (230.31 s)	100,000,000 rows, reference box idle, 2 alternating repeats, commit `60057634`
The same path, independently re-run at a different commit	433,657 rows/sec (230.60 s)	100,000,000 rows, same rig, same day, commit `610f86a4`-era
PostgreSQL `hash_total` at smaller scale	922,680 rows/sec at 1M; 945,781 rows/sec at 5M	1,000,000 and 5,000,000 rows, reference box, commit `7fde8a9b`
Engine-less proof path, vectorized	29,909 → 345,646 rows/sec (11.6×) at 1M; 29,470 → 400,185 rows/sec (13.6×) at 5M	1,000,000 and 5,000,000 rows, 12 columns, 8% nulls, reference box, commit `58940a72`
Stratified proof - the cost of localizing a failure	+32.9% over the plain proof (305.98 s vs 230.31 s)	100,000,000 rows, reference box, commit `60057634`

The 100M-row figure is the one to hold on to: it is the largest-scale proof measurement we hold, and the two independent runs agree to within 0.13%. Note that it is materially lower than the 1M and 5M figures above it - see the note on extrapolation in Limitations, which is the single most important caveat on this page.

The vectorized row matters for a different reason: it is the path used when neither side is a SQL engine - files, object stores, Qlik - where there is nothing to push down to. That path runs at 1.7× our internal proof floor. It is enabled by default for the file_dataset and qlik_cloud connectors and nowhere else, because those are the two surfaces where it has been byte-identity verified at volume.

Key extraction - the first phase that reads rows¶

Key extraction runs only when the proof gate fails. Each side streams its row fingerprints into the platform's staging store; every figure below is the shipped code path, and where the span includes that staging write it says so.

What was measured	Result	Scale and conditions
PostgreSQL key extraction (`stream_keys`)	507,375 rows/sec at 1M; 531,727 rows/sec at 5M	1,000,000 and 5,000,000 rows, reference box, commit `7fde8a9b`
Keyset-paginated extraction, completeness asserted	PostgreSQL 637,265 rows/sec; MySQL 575,420 rows/sec	200,000 rows each, reference box, commit `7fde8a9b`; every key returned exactly once
Databricks target-side extraction, full activity including the staging write	p50 55,990 rows/sec (57,624 / 55,990 / 55,769; spread 3.3%)	10,000,000 rows x 20 columns, reference box worker against a 2X-Small serverless warehouse, 3 fail-path runs (`ef9cba2e` / `eb4cdf57` / `003661ab`), box commit `ec5d516a`
Snowflake target-side extraction, including the staging write	123.9K rows/sec (PostgreSQL source side 95.2K rows/sec on the same run)	100,000,000 rows x 20 columns, reference box worker against an X-Small warehouse, one fail-path run (`d0fc9aae`), commit `a8d6eba7`
ClickHouse extraction through the worker's staging write (`_stage_side_copy`)	145,642-152,167 rows/sec at 50,025 rows; 157,217-163,567 rows/sec at 2,000,000 rows	50,025 rows (7 columns) and 2,000,000 rows (3 columns), reference box with the bench process capped at 4 CPUs, five uncontended runs across ClickHouse 25.8.33.6 and 26.8.20.9, each a median of 5 after a warm-up, commits `70910123` (runs 1-2 per server) and `04f14672` (run 3)

The Databricks and Snowflake rows are the target-side half of fail-path runs from the flagship campaigns further down, isolated per side from the activity history rather than from a micro-benchmark, and the span is the whole extraction activity - stream, canonicalise, write to staging. The Databricks figure clears its internal floor by 1.12x on three runs. The Snowflake figure is a single run and is recorded here as such, not certified. The ClickHouse rows carry a can-fail control: the same bench with its process throttled to 0.1 CPU read 17,299 rows/sec, reported BELOW FLOOR and exited non-zero.

Difference phase - cost proportional to the difference¶

What was measured	Result	Scale and conditions
Field diff, as shipped today	503,712 cell-pairs/sec (25,186 keys/sec)	1,000,000 keys × 20 columns, 1% of cells differing, reference box idle, commit `c6b54e3a`
Field diff, cumulative improvement across three changes	222.77 s → 37.34 s = 5.97×	1,000,000 keys × 20 columns, reference box, `6a61110f` → `5b9884a6`
Connections opened by a 1M-key diff	4,000 → 2	1,000,000 keys, reference box
Egress avoided by narrowing on a sparse difference	12.7× fewer rows extracted (20 of 256 buckets)	100,000 rows / 10 planted differences, commit `84d5bdbd` - see note below

On the field-diff unit. A cell-pair is one source cell compared against one target cell, and the denominator counts cells processed, including the ones that turn out to be equal. Any rows/sec restatement must carry the column width, because the same run has a different rows/sec at every width. The measurement above restates as 25,186 rows/sec at 20 columns; our internal floor for this phase, discussed below, restates as ≈7,500 rows/sec at the same width. Those two figures are often confused - one is what we measured, the other is the minimum we will accept. We do not quote a width-free rows/sec number for this phase, and neither should anyone quoting us.

On the narrowing row. This is the one measurement here not taken on the reference box - it was taken on the Docker development stack. It is included because it is a ratio of rows extracted to rows extracted, and how many rows a query returns does not depend on how fast the machine running it is. The cut is also scale-independent by construction: it is a function of how many of the 256 buckets contain a difference, which depends on the density of the difference and not on the row count. The measurement bears that out - 100,000 rows and 300,000 rows with 10 planted differences gave 12.7× and 12.8×. At high difference density the benefit disappears and the run degrades gracefully to full extraction, measured at 1.0× with no penalty.

Concurrency and platform overhead - the cost is your engine, not the platform¶

What was measured	Result	Scale and conditions
Paridity's own CPU cost while a reconciliation runs	0.8% of CPU (engine ~1202%, the whole platform stack ~9% on its isolated partition)	Reference box, PG→PG, 2M × 20-col slices, engine pinned to 16 vCPU, concurrency N = 1 → 30, cpuset hard-partition isolating platform CPU from engine CPU, 2026-08-23
Aggregate throughput vs engine vCPU	~9.5-10K rows/sec per engine vCPU, ~linear (8 vCPU → 80.8K rows/s; 16 vCPU → 152.5K rows/s)	Same rig, cpuset-partitioned
Enterprise-pattern concurrency, error-free	20 fully-overlapping reconciliations - 0 errors, 0 throttles, 0 lost evidence across 78 runs, every run `pass_clean` on native pushdown; submission→start queue ≤ 0.24 s, ~2 s non-proof overhead per run even at N = 20	Reference box, single worker-container stack, PG→PG 2M × 20-col, 2026-08-17

What this establishes, and what it does not. Paridity's own orchestration cost stays flat and sub-second as concurrency rises - the platform is not the bottleneck, and its overhead is a fraction of one percent of CPU. The per-run slowdown visible at high concurrency in these tests is the test engine saturating, not Paridity - both sides of all N mappings deliberately shared one stock-configured 128 MB Postgres container, a fraction of any enterprise database. Aggregate reconciliation throughput is bounded by the source/target engine capacity you size, and scales roughly linearly with the vCPU you give it - a property you provision, not one this platform caps. A single large reconciliation saturates about 8 engine vCPU and then flattens (one proof query parallelizes to ~4 workers), so the way to use more capacity is to run reconciliations concurrently. The figures in this section are PostgreSQL-to-PostgreSQL. Four flagship cross-engine pairs have been measured end-to-end on the same reference configuration, under shipped defaults, each preceded by the fail-first protocol (a planted 0.01% field tamper and planted deletions recovered exactly before any clean number was accepted):

Flagship pair	Proof p50, 1M rows	Proof p50, 10M rows	Proof p50, 100M rows
PostgreSQL → Snowflake	13.1 s	79.9 s	12.5 min
PostgreSQL → Databricks	12.7 s	79.5 s	12.6 min
Oracle → Snowflake	12.5 s	108.8 s	not measured - the Oracle source used for these runs is the free edition, capped at 2 CPU threads / 12 GB (disclosed deliberately-unflattering hardware); 100M is out of scope on that cap
Db2 LUW → BigQuery	52.5 s	8.5 min	85.9 min

Rows are 20 columns wide throughout - a rows-per-second figure is meaningless without the width, so it is stated. Every cell is the p50 of three cache-busted clean runs with run ids recorded in the internal benchmark register; where a source engine is the bottleneck (Oracle's 2-thread cap, Db2's in-database SHA-256 rate) that is disclosed rather than averaged away.

Flagship cross-engine pair - Db2 LUW → BigQuery, measured to 100M rows¶

This is the most heterogeneous of the four measured pairs - a Db2 LUW source reconciled against a BigQuery target, from one million to one hundred million rows, on the reference box (one worker container, no CPU cap recorded, against BigQuery on-demand) - and the pair documented in most depth here. Every clean number below is preceded by a fail-first proof: a 0.01% field tamper and a batch of planted deletions were each recovered exactly before any clean timing was accepted.

Rows (20 columns)	Proof phase (p50)	End-to-end, clean	BigQuery data scanned per proof
1,000,000	52.5 s	~55 s	~297 MB
10,000,000	8.5 min	~8.7 min	~3.1 GB
100,000,000	85.9 min	~87 min	~29 GiB

Each tier is three cache-busted clean runs with a tight spread (0.2%-1.0%), plus corroborating fail-path runs. The long pole is the source engine, not the platform: Db2's native SHA-256 hash, computed in-database at its measured optimal parallel degree, runs at roughly 19K rows/sec on this 20-column shape and dominates every tier; the BigQuery side and Paridity's own orchestration are a small fraction of the wall clock. Proof duration and BigQuery bytes scanned both scale close to linearly with row count. When a proof fails and the platform extracts and diffs the mismatched rows, the full end-to-end on a 0.01% mismatch is key-extraction bound - roughly 2 minutes at 1M, 24 minutes at 10M, and 2.6 hours at 100M - and every planted corruption at every tier was recovered exactly.

100M rows is the largest measurement held; 1B and 10B are deliberately unmeasured and are established on a customer-engagement basis, not projected from this range.

Full-run partition reuse on Databricks - measured at 10M rows, and slower than the plain proof below a break-even¶

Paridity has an opt-in optimisation for full-table runs on partitioned lakehouse tables (fullrun_reuse_enabled, default off): keep the per-partition proof of the previous run and re-hash only the partitions the table's change feed says moved. It was measured before any default was considered, and the measurement went against it. This is the third labelled exception: the durations are development-stack runs against Databricks' serverless warehouse, single runs each, and they are published as a comparison, not as throughput.

Two identical 10,000,000-row Delta tables (4 columns, 12 partitions, change data feed on), one row appended to one partition per side between the first and second run, every run pass_clean, proof durations read from the platform's own phase record:

Run shape	Rows hashed	Before (#3192, recorded at commit `062d9117`, run-tag `20261023131118`)	After (#3238, measured at commit `3efc0d27`, run-tag `20261024155805`)
Plain native pushdown, reuse off (control)	20,000,002	10,250 ms	9,225 ms
Reuse, first run - no baseline, every partition re-hashed	20,000,000	779,431 ms (76x the control)	62,675 ms (6.8x)
Reuse, 1 of 12 partitions changed per side	1,666,668 re-hashed, 18,333,334 reused	79,670 ms (7.8x)	13,809 ms (1.5x)

The "before" column found the cause: a changed partition was re-hashed through the Python reference implementation at about 41 microseconds per row, while the plain path it replaced is the engine's native hash at about 45 nanoseconds per row above a fixed cost of roughly 9 seconds. The "after" column is the fix - a changed partition is now re-hashed through the engine's own scoped native hash, so the combined result stays byte-identical to a full scan - and it is still slower than the plain proof at this shape, because listing the changed partitions has a fixed cost of several seconds that a 10M-row native scan does not pay. The harness's fit puts the break-even at roughly 14M skipped rows per side on this warehouse class; that break-even is a two-point projection and is labelled indicative, not measured. The consequence is already in the product: the setting stays off by default, and its help text says it is slower than a plain full run below that break-even.

Why these numbers can be trusted¶

This section is the actual differentiator, and it is worth more than any individual figure above.

The benchmarks that carry the load have been proven able to fail. A green benchmark that has never been seen to fail is not evidence. Two of the harnesses behind the figures above ship a --prove-can-fail mode that plants a specific defect and requires the run to go red: the field-diff harness (five legs) and the vectorized-hash harness (two planted defects plus an untampered control). Two more carry a can-fail control of a different kind: the ClickHouse key-extraction bench was throttled to 0.1 CPU and seen to report BELOW FLOOR and exit non-zero, and the frontend first-paint gate was seen to fail on the build it was introduced against (425.5 KB) before it passed on the fixed one. The others - the PostgreSQL floor, evidence-write, PostgreSQL and MySQL key-extraction, stratified and narrowing benchmarks - do not have a documented can-fail proof; several were established before we adopted that requirement. That is a real gap in our evidence and we are not going to describe it as anything else.
Every performance run is simultaneously an accuracy run. The field-diff benchmark asserts that the set of (key, column) cells written to evidence is exactly the planted set - count, membership, and no duplicates - and the 1M-key certification run above cleared that on 200,126 planted cells. Every stratified proof run is gated on the recombined hash being byte-identical to the plain hash, on both repeats. The vectorized proof harness required byte-identical results on 6 of 6 lanes at both 1M and 5M rows, and separately demonstrated that all four of its fallback paths return the reference answer rather than a different one. A fast wrong answer fails these benchmarks, which is the only arrangement compatible with our accuracy posture.
Scaling behaviour is measured wherever we could measure it. The PostgreSQL proof, vectorized proof, PostgreSQL and ClickHouse key-extraction and field-diff results were each taken at two or more scales, which is what distinguishes "fast" from "fast until it isn't" - a single-point benchmark cannot. Four results are single-point and should be read as such: the stratified overhead (100M only), the field-diff certification run (1M only), the Databricks key-extraction floor (10M only) and the Snowflake key-extraction reading (100M only, one run).
Corrections are recorded rather than quietly fixed. Where a benchmark figure has been superseded - a measurement re-run after contamination was found, or a predicted scaling regression that the benchmark later refuted - the correction stands in the record rather than being silently overwritten.

Our internal throughput floors - design gates, not commitments¶

Paridity engineering maintains throughput floors as merge gates: a change that drops a path below its floor is treated as a regression and must be fixed before it lands. They are written against a 4 vCPU worker pod.

These are internal engineering rules. They are not service levels, they are not contractual, and they are not a prediction of what you will see on your hardware against your data. They are published here for one reason: to be honest about which of them we have actually demonstrated and which we have not.

Path	Floor (4 vCPU baseline)	Demonstrated?
Proof `hash_total` pushdown	≥ 200,000 rows/sec	PostgreSQL only. Cleared at 100M rows on the reference box (2.2×). Proof throughput has been measured for the other eleven pushdown engines, but only at 50,000-60,000 rows and only on development or cloud rigs - never on the reference box. At that size each figure is dominated by something other than per-row throughput (per-row engine cost on Oracle and DB2 LUW, network transport on the cloud engines, fixed query latency on ClickHouse), so none is a valid floor reading in either direction and none is published here.
Key extraction - PostgreSQL / MySQL	≥ 25,000 rows/sec	Yes, both. PostgreSQL 507,375 rows/sec at 1M and 531,727 at 5M on the box (commit `7fde8a9b`); keyset pagination at 200K rows measured PostgreSQL 637,265 and MySQL 575,420 rows/sec, each asserting every key was returned exactly once.
Key extraction - Snowflake / Databricks	≥ 50,000 rows/sec	Databricks: yes, p50 55,990 rows/sec at 10M rows × 20 columns over the full extraction activity including the staging write, three runs, a 1.12× margin (box commit `ec5d516a`). Snowflake: one run, 123.9K rows/sec at 100M rows × 20 columns on the same scope (run `d0fc9aae`, commit `a8d6eba7`) - recorded, not certified, because a single run is not a floor demonstration.
Key extraction - ClickHouse	≥ 50,000 rows/sec	Yes. 145,642-163,567 rows/sec across five uncontended runs at 50,025 and 2,000,000 rows on the box with the bench process capped at 4 CPUs, on both CI-pinned server versions (25.8, the oldest the platform accepts, and 26.8), about 3× the floor (commits `70910123` / `04f14672`). The verdict was seen to go red under a 0.1-CPU throttle.
Field diff	≥ 150,000 mismatched cell-pairs/sec	Yes. 503,712 cell-pairs/sec at 1M keys × 20 columns on the box, a 3.4× margin. The floor is set from this measurement.
Evidence writes	≥ 50,000 rows/sec	Yes. 101,764-126,787 rows/sec across four write paths at 200,000 rows on the box, a 2.0-2.5× margin (commit `7fde8a9b`).
Readiness, when no replication profile is configured	< 5 seconds	Cleared by a wide margin, but on the development stack - p50 3 ms, p95 146 ms over 540 recorded attempts. Not re-measured on the reference box, so the figures are not published as a result above. The margin is four orders of magnitude, so the conclusion is not in doubt even if the numbers are rig-bound.
CDC probe	< 5 seconds	No figure. When this document was first written the probe recorded no duration at all (2,121 attempts with no timing). The platform now records a timed `cdc_probe` phase row on every run where the probe actually executes, so the telemetry exists - but no measurement against the budget has been taken on the reference box, so we still cannot state a figure.
State machine transitions	< 50 ms	No. Never measured.
Frontend first-paint JavaScript	≤ 350 KB gzipped	Yes - 313.3 KB (320,823 bytes), and this is the only performance budget checked automatically on every code change. A correction is owed here. The previous edition of this document reported 347 KB against a gate set at 400 KB; that figure counted only the entry chunk, while the browser also preloads every chunk the landing page imports statically. Counted properly, first paint had been 425.5 KB - over budget - since before the figure was published. The gate now counts the entry chunk plus every preloaded chunk, is set at the 350 KB budget itself, and was seen to fail on the 425.5 KB build (commit `03ff55d2`) before the dashboard's chart library was moved off the critical path (commit `31b78329`). Margin today: 36.7 KB.
Frontend time-to-interactive, chart re-render	≤ 2.5 s / < 100 ms	No. Never measured.

Three rows of that table still say we have not measured something, and the proof row says it for eleven of twelve engines. Publishing a table where every row says "met" would have been easy and would have been false.

What the platform checks at runtime, and what it does not. The floors above are merge gates. Since the throughput_floor analyzer landed, the key-extraction floors are also evaluated on every real run that extracts keys: the platform computes the run's key-extraction rate - the same number the run narrative prints - and compares it with the floor for the engine pair (PostgreSQL and MySQL 25,000; Snowflake, Databricks and ClickHouse 50,000 rows/sec; the slower side's floor governs). A run materially below floor - under half of it, by platform default - opens one anomaly on the mapping, and no second one while the first stands unresolved. A run whose rate cannot be computed (a phase shorter than ten seconds, or a side whose rows were not counted) is reported as unmeasured, never as a pass, and never as zero. An engine pair with no documented floor is reported as having none. The proof and field-diff floors are not evaluated at runtime; they remain benchmark and pre-merge gates only, and nothing in the product checks them against a real run.

Verify it yourself¶

Efficiency is the easiest of Paridity's three proof claims for you to test, because unlike accuracy it needs no adversarial setup - you simply run the product against your own data and read the timings it records about itself.

Run a clean reconciliation and watch what does not happen. Point a Mapping at a dataset of at least 50,000 rows that you expect to match. When it reaches pass_clean, open the run and read the Proof Phase card: it shows the two hash totals and whether they matched. A clean run has no Compare Phase card at all - because the diff phase never ran. That absence is the claim in section 1 of How it works, and it is the whole efficiency argument in one screen.
Compute the throughput yourself, from the platform's own record. The Proof Phase card reports Proof DB time (both sides, run in parallel). Divide the row count shown on the Data Window card by that duration. That figure is yours - your hardware, your engines, your network - and it is the only throughput number that should inform your sizing. Note the card sums both engines, so it can legitimately exceed the run's wall-clock time.
Make the difference sparse, then make it dense. Introduce ten differences into a large target table and run it; then introduce thousands and run it again. Compare the Data Volume card between the two runs, and the volume and egress tracking at /administration/cost-usage. This is the proportional-cost claim, and it is the one most worth confirming yourself, because it is what sets your steady-state running cost.
Confirm the failure path is not silently cheap. An efficiency claim only means something if the platform is doing the work. Check that the run which found ten differences reported exactly ten, with the right rows and columns named in the evidence. Any reconciliation platform can be made faster by checking less, so verify these two properties together or neither is worth much.
Read the platform's own floor verdict. On a run that extracted keys, the run narrative's Key Extraction step prints the throughput it computed across both sides, or says why it could not ("not measured" when a side's rows or the phase timing were not recorded). A run materially below the floor for your engine pair opens a throughput_floor anomaly on the mapping. The absence of that anomaly on a run whose throughput the narrative prints is the platform's statement that the floor was met on your hardware; the absence of a printed throughput is not.

Tip: Steps 1-3 need nothing from us - they read timings the platform already records about itself. For sizing a production deployment against your row volumes and engine pair, ask your Paridity contact for an SE-led benchmark against a dataset shaped like yours; the engine pair and the shape of your data matter more than any figure on this page.

Limitations¶

Stated with the same discipline as the evidence. Over-claiming here would undo the point of the document.

Scope	Status	Notes
PostgreSQL proof, key extraction, evidence writes, field diff	Verified	Measured on the reference box at 200K-100M rows against the real production code paths. Verification depth is not uniform: the field-diff result carries both an exact-set accuracy gate and a can-fail proof, the stratified proof result carries a byte-identity gate on every run, and the key-extraction result asserts completeness (every key exactly once). The plain PostgreSQL floor and evidence-write results carry neither an accuracy gate nor a can-fail proof - they are throughput measurements only.
Engine-less proof path (`file_dataset`, `qlik_cloud`)	Verified	Vectorized path byte-identity verified at 1M and 5M rows and enabled by default for these two connectors only. `qlik_cloud` carries no volume measurement of its own - its engine read is page-decode bound and the throughput gain there is small; it is on the verified surface because it is verified, not because it is fast.
Cross-engine sizing matrix at production scale	Verified	Measured - four pairs. Four flagship pairs are measured on the reference box (see the flagship table above): PostgreSQL → Snowflake, PostgreSQL → Databricks, and Db2 LUW → BigQuery at 1M / 10M / 100M rows; Oracle → Snowflake at 1M / 10M (100M is out of scope on the disclosed free-edition Oracle cap of 2 CPU threads / 12 GB - a paid-tier Oracle source would be a separate measurement). 1B and 10B rows are deliberately unmeasured for every pair (customer-engagement basis).
Proof floor on engines other than PostgreSQL	Roadmap	Eleven of the twelve SQL pushdown engines have no valid measurement against the 200,000 rows/sec floor. Proof figures do exist for all eleven - MySQL, Oracle, DB2 LUW, SQL Server, Trino and ClickHouse on development rigs, Snowflake, Databricks, BigQuery, Redshift and SAP HANA on live cloud rigs - but every one was taken at 50,000-60,000 rows, and our own record warns against reading a proof figure at scales of this order as a floor result. Each is dominated by something other than per-row throughput: a per-row `DBMS_CRYPTO` call on Oracle, hex-to-integer arithmetic on DB2 LUW, network transport on the cloud engines, fixed query latency on ClickHouse. This cuts both ways - Redshift measured ~958K rows/sec at 60,000 rows (Redshift Serverless, 2026-07-12, result cache disabled; no commit recorded for that reading) and ClickHouse ~985K rows/sec at 50,025 rows (commit `6c24087d`, development rig), five times the floor, and neither is any more publishable than a sub-floor reading, because neither is measuring what the floor measures.
Snowflake / Databricks / ClickHouse key extraction	Supported	Measured unevenly. Databricks is demonstrated at 10M rows on three runs; ClickHouse at 50K and 2M rows on five runs in a 4-CPU-capped process; Snowflake on one run at 100M rows. None of the three has been measured at a second scale on the reference box except ClickHouse, and ClickHouse has not been measured above 2M rows. The floor is also evaluated at runtime on every real run for these engines (see the floors section), which is a check on each deployment, not a substitute for the benchmark.
Full-run partition reuse (Databricks Delta, Trino Iceberg)	Supported	Measured - slower than the plain proof below a break-even. Measured at 10M rows per side on Databricks, on the development stack, before and after the re-hash path was moved to the engine's native hash. Reuse remains off by default and is 1.5× slower than the plain proof at that shape; the break-even (about 14M skipped rows per side) is a two-point projection, not a measurement. The Trino Iceberg path takes the same code route and passes its correctness drill, but carries no timing at any scale - unmeasured.
Runtime evaluation of the proof and field-diff floors	Not built	Only the key-extraction floors are evaluated on real runs. A deployment whose proof or field-diff phase runs below its internal floor produces no signal from the platform; those two floors are benchmark gates only.
Any duration figure at 1B or 10B rows	Roadmap	We have never run a reconciliation at those volumes. The 100M-row measurement above is the largest we hold.
Extrapolating our figures to a larger row count	Not supported	Our own record shows why. PostgreSQL proof throughput is flat from 1M to 5M rows (922,680 → 945,781 rows/sec), and extrapolating that flatness to 100M predicts roughly 106 seconds. The measured 100M result is 230.31 seconds - about half the rate. We have not established the cause (a plausible one is that 100M rows no longer fit in cache the way 5M does, but that is a hypothesis we have not tested), and the cause does not change the guidance: flatness across one order of magnitude does not license a projection across two. Use the 100M measurement; do not project from the 5M one, and do not project past 100M at all.
Field-diff I/O concurrency gains	Supported	Measured on PostgreSQL only. The reduction from 4,000 connections to 2 applies to every engine and is the larger win. The concurrent-fetch overlap was measured on PostgreSQL. When it was measured, all but two of the SQL connectors called their blocking driver directly inside the coroutine, so a prefetch could not start until the compare yielded; every SQL connector's keyed fetch has since been moved onto a dedicated driver thread per connection (commit `0a2bb532`), which removes that structural reason - but the overlap has not been re-measured on any other engine. Do not apply the fetch-concurrency figures to a cloud warehouse.
Cross-region and cloud-warehouse latency figures	Not published	Internal artifacts contain a latency model built by injecting round-trip delay. It is a model and it is labelled as one; no modelled figure appears in this document.
API endpoint response times	Not published	Measured across sixteen endpoints and comfortably inside every internal target, but on the Docker development stack rather than the reference box, so they are excluded under this document's own rule.
Contractual performance commitments	None	Nothing in this document is a service level. Our support terms state our actual contractual commitments, which cover support response times and do not include throughput.

This document reflects the measurement record as of last_verified above. The authoritative internal record - including every figure deliberately excluded here - is available to reviewers on request through your Paridity account contact.

How Paridity produces audit-grade, independently-verifiable evidence¶

The claim¶

Every reconciliation run Paridity executes produces a complete, immutable audit trail - config state, phase timings, evidence, and terminal outcome - that an external auditor can verify independently, without trusting the platform's own dashboards. Evidence in the ev schema is append-only, the run's compliance timestamp (compare_started_at) never changes once set, and the trail can be exported as a cryptographically signed, hash-chained bundle that is verifiable offline.

This is a narrow, falsifiable claim about trail completeness and tamper-evidence, not about data-matching accuracy - see the companion accuracy proof doc for the hash_total / row_hash / field_diff correctness claim. The two are independent: this document is about whether you can trust the record of what happened, not whether the reconciliation itself was correct.

How it works (deterministic)¶

Paridity's audit trail is built from deterministic writes at defined points in the run lifecycle - never inferred or reconstructed after the fact.

Config audit before comparison. Before any data comparison begins, the workflow resolves the run's policy through the 7-layer config resolution chain and writes an ev.ev_config_audit row recording the config_checksum - the SHA-256 of the canonicalized, resolved config JSON (sorted keys, sorted arrays, metadata fields stripped). This proves the policy was fixed and known before the run touched any data, and whether it changed since the previous run of the same mapping. config_checksum is a policy fingerprint - it says nothing about the data itself, and must never be confused with hash_total.
Append-only evidence. All ev.* tables (proof results, mismatch keys, field diffs, run metrics, in-flight rechecks, evidence bundles, and the stored period-bundle containers) are INSERT-only. No application code path issues UPDATE or DELETE against them except two documented, narrow lifecycle exceptions (below) and the retention purge script, which itself writes an audit record before deleting anything. Derived figures - dashboard summaries, hourly rollups, anomaly baselines, per-run diff limits, consumption ledgers - are not evidence and do not live in ev at all. Since #1724 they sit in their own proj schema, where being rewritten in place is their purpose: migration db/migrations/177_projection_schema.sql moved the four such tables that had sat in ev (and were being upserted every few minutes) into proj, and later projections such as proj.run_consumption (260_proj_run_consumption.sql) are created there directly. An auditor reads ev.*; proj.* is recomputable from it at any time and is never an audit source. The split is what makes "ev.* is append-only" a rule with no remembered exceptions rather than an approximation.
An immutable audit anchor. compare_started_at is set once, when the run's CreateRunActivity executes, and is never written to again - not during in-flight rechecks, not on reruns. It is the timestamp an auditor anchors the whole trail to.
Per-phase timing. Readiness, SCD-invariant (scd_invariants, only for a mapping with a declared SCD scope), proof, key extraction, field diff, and in-flight recheck (lag_retry) phases each write a run.run_attempts row with phase, started_at, completed_at, outcome, and worker_id (the Temporal activity/workflow identity that ran it) - so an auditor can see exactly how long each phase took and who/what executed it.
Append-only watermark history. run.run_watermarks uses a composite (run_id, recorded_at) key, so every readiness evaluation - including retries - adds a new row rather than overwriting the last observed watermark. Nothing about what the platform saw at each check is lost.
Terminal-status and admin-action logging. Every terminal run status (pass_clean, pass_resolved_lag, fail_hash, fail_count, fail_field, schema_drift_error, system_error, readiness_error, config_error, cancelled, skipped, no_new_data, no_new_data_open_issues - the RUN_TERMINAL_STATES set) and every admin action (cancel, bypass_readiness, retention purge) is written to cfg.audit_log with actor, entity, action, and timestamp - through a single enrichment chokepoint, so every call site gets the same guarantees rather than each populating them individually. The action is run.<status>, so a terminal outcome is filterable by name. config_error (#3034) is the platform declining to compare as configured after readiness passed - an unresolvable decryption key reference, a filter that does not validate, duplicate rows per key, a partition set that does not cover the table - where a retry cannot change the verdict and an operator must change the configuration. It is kept apart from system_error (a platform fault) so an auditor can tell a refusal from an outage: it is a distinct value of run.run_status_t (migration 245_run_status_config_error.sql), has its own audit action run.config_error (246_audit_action_config_error.sql), raises a WARNING-level config_refusal notification rather than the CRITICAL system_error page (247_notification_trigger_config_refusal.sql), and is reachable only from the post-readiness states - the allowed edges are declared in RUN_VALID_TRANSITIONS (src/models/internal/enums.py) and enforced by src/services/state_machine.py, which rejects any other transition. The runs list shows it as Config refusal.
Network attribution. actor_ip is extracted from the request (X-Forwarded-For or direct connection, in the API authentication layer) and stamped onto API-originated audit entries.
Tamper-evidence on the audit log itself. The audit log is hash-chained: each sealed row carries entry_hash = SHA-256(prev_hash + canonical(row)), so altering, deleting, or reordering any row breaks every entry_hash after it. Chains are periodically sealed with a signature over the chain head, so a full-database rewrite that recomputes every hash still fails verification against the last signed checkpoint.
Signed, hash-chained evidence bundles. An auditor or platform_admin can generate a single exportable archive (ev.ev_evidence_bundles) per run containing the resolved config snapshot, config audit, readiness, proof, mismatch keys, field diffs, in-flight rechecks, run metrics, and the relevant audit-log entries - whose manifest commits to every file's SHA-256 and is sealed with a detached signature over exactly the bytes that ship - ECDSA-P256 from a KMS-backed key (AWS KMS, Azure Key Vault, GCP KMS) or a local ECDSA key, so an outside auditor verifies with the public key alone, or HMAC-SHA256 with a customer-held key - linked to the mapping's previous bundle via previous_bundle_hash, forming a tamper-evident sequence over time. Period exports are a signed container over individually signed per-run bundles.
Retention deletion is itself audited. The one process that is allowed to delete ev.* rows (the retention-purge process) writes a cfg.audit_log row - table, mapping, deleted-row count, cutoff, retention policy applied - before each DELETE, so a purge never happens silently. Evidence bundles are covered by the same process under bundle_retention_days (tenant value floored at the platform value): the bundle's audit entry additionally records the stored-container rows and bytes removed by the cascade, the bundle and run ids, and the hash-chain boundary - so a later chain verification can prove a missing predecessor was purged (CHAIN_HEAD_PURGED) rather than tampered with.
Signed single sign-on requests, with key rotation that never leaves an unsigned gap. Access to the platform is the first ITGC an auditor asks about. Where a customer's identity provider is configured to require signed requests (saml_config.sign_requests, per provider), the sign-in, sign-out and sign-out-response messages Paridity sends are signed RSA-SHA256 with one deployment-wide key held as a PEM bundle in the customer's secrets backend (SAML_SP_SIGNING_KEY_REF, Helm app.saml.spSigningKeyRef) - the key is a pointer, never a column, and production refuses to enable such a provider or to start without it (SAML_SP_SIGNING_KEY_REQUIRED) rather than sending an unsigned request in its place. The certificate the IdP verifies against is published with its SHA-256 fingerprint at GET /saml/{idp_id}/metadata and GET /saml/{idp_id}/sp-info and on the Authentication settings screen. Rotation (#3281) is two-phase: a next bundle is staged (SAML_SP_SIGNING_KEY_NEXT_REF, Helm app.saml.spSigningKeyNextRef) and both certificates are published side by side while every IdP picks up the new one; only the current key ever signs, the staged private key is parsed to prove it matches its certificate and then discarded; cut-over is a pointer move. A staged bundle that is unusable, identical to the current one, or unreachable at boot stops the API from starting or returns 503 SAML_SP_SIGNING_KEY_NEXT_INVALID on the publish surfaces - the platform never publishes one certificate while the operator believes two are published. Code: src/services/saml_sp_signer.py, src/services/saml_sp_service.py, src/api/routers/saml.py, src/api/routers/identity_providers.py; the IdP onboarding runbook (section 4b) is available to reviewers on request through your Paridity account contact.

Note: AI never determines correctness or completeness of the audit trail. Every fact above is a deterministic database write triggered by a defined workflow/activity step or a SQL script - not an inference. Where Paridity's AI features touch runs or evidence at all, they assist triage and explanation only; they never decide whether a run passed, never auto-confirm an admin action, and never write to ev.* or cfg.audit_log.

The two approved append-only exceptions¶

Evidence is append-only with exactly two documented lifecycle exceptions, both narrow and both logged:

ev.ev_anomalies - the lifecycle columns acknowledged_by, acknowledged_at, acknowledged_note, resolved_at, resolved_reason may be updated as an operator works an anomaly. Every other column on the row is immutable and is never touched by that update.
ev.ev_evidence_bundles - a two-phase write: the INSERT creates a pending row (metadata only); one subsequent UPDATE fills file_count, total_size_bytes, content_hash, previous_bundle_hash, manifest, and - since the period container is signed over exactly the bytes that ship (#2014) - manifest_sha256, signature, and signing_key_id, once generation completes. No further mutation is permitted after that. The statement is the one in src/workers/evidence/generate_bundle.py (GenerateEvidenceBundleActivity, the period container); a per-run bundle is written complete in a single INSERT (src/api/routers/evidence_bundles.py) and never updated. The three signature columns are not new columns: manifest_sha256 and signing_key_id date from migration 023_evidence_bundles.sql and signature from 062_evidence_bundles_enhancement.sql; #2014 added them to the existing UPDATE rather than introducing a second one.

Nothing else in ev is rewritten. The mutation sites the code tree may contain are declared once, in src/services/sox_audit_assertions.py: APPROVED_EV_MUTATION_SITES holds exactly the two sites above, APPROVED_EV_UPDATE_PATHS names the only columns each may touch (every other column is immutable by complement, so a column added later is immutable by default), and DERIVED_EV_PROJECTIONS - the list that would widen the rule - has been deliberately empty since the proj split. The static scan and the live xmin check (below) are both asserted against those declarations.

The evidence¶

Claim component	How it's enforced	Result
Config audit written before proof	The config-audit write executes in the reconciliation workflow before the proof phase begins; recorded in `ev.ev_config_audit`.	Verified in code - the activity executes before the proof phase begins on every run path.
`config_checksum` canonicalization is order/whitespace-insensitive	Canonicalization and checksum computation in the config-resolution layer.	Verified in code - sorted keys, sorted arrays, metadata stripped, `json.dumps(..., sort_keys=True, separators=(",", ":"))`, SHA-256 over UTF-8.
`compare_started_at` is never updated after being set	The run state machine (`src/services/state_machine.py`) rejects any status transition whose extra fields name `compare_started_at` (`ValueError: compare_started_at is immutable`), and every status write goes through it; the column is set once by `CreateRunActivity`.	Verified in code and pinned by unit tests: one asserts the rejection, one asserts no evidence activity can update the column, and two assert the in-flight recheck and config re-pin UPDATEs never name it (the test pins are available to reviewers on request through your Paridity account contact). There is still no database-level trigger: `db/migrations/001_base_schema.sql` documents the column as immutable but no `CREATE TRIGGER` on `run.runs` exists (searched the migration set for triggers and for functions referencing the column). Immutability therefore rests on the application chokepoint plus the tests, not on the database engine.
`ev.*` evidence is append-only except two documented exceptions	The SOX audit-trail verifier's check G4, a static code scan, and the single sanctioned deletion path (the retention purge).	Verified and actively enforced, not left to convention. Check G4 snapshots every `ev.` row's Postgres `xmin` across a real run and fails if any row's `xmin` moves outside the two approved lifecycle updates - a content comparison would miss a no-op rewrite, whereas `xmin` moves on UPDATE and nothing else. A separate static scan rejects both* `UPDATE ev.…` and `INSERT … ON CONFLICT DO UPDATE` anywhere in the code tree - checking one pattern alone would miss the other. Retention purge is the only sanctioned `DELETE FROM ev.`; the `ev_anomalies`/`ev_evidence_bundles` exceptions match the documented lifecycle columns (`APPROVED_EV_UPDATE_PATHS` in `src/services/sox_audit_assertions.py`). The scan is a unit test in the platform's suite (#1713, available to reviewers on request through your Paridity account contact); G4 is in `scripts/verify_sox_audit_trail.py`, which snapshots every base table in the `ev` schema by catalogue lookup (so a table added later is covered without editing the check). The `proj` split (#1724) is what made this row true as written:* the live `xmin` check found four derived tables in `ev` being rewritten by upsert behind the "two exceptions" claim; migration `177_projection_schema.sql` moved them to `proj` instead of adding them as exceptions, `DERIVED_EV_PROJECTIONS` is deliberately empty, and `proj.*` is by design not scanned - rewriting a projection is its purpose, and no projection is an audit source.
Watermark history is append-only	A composite primary key `(run_id, recorded_at)` on the watermark-history table.	Verified - the table records one row per observation, preserving full watermark history for SOX rather than overwriting the last-observed value.
Per-phase timestamps + `worker_id` in `run.run_attempts`	The `run.run_attempts` table plus per-phase attempt-writing activities in the reconciliation workflow for `readiness`, `proof`, and other phases.	Verified - readiness and proof phases both confirmed writing attempt rows with `worker_id` in the workflow.
Terminal statuses + admin actions logged to `cfg.audit_log`	A single audit-log write chokepoint in the services layer.	Verified - single chokepoint function referenced from workflow/worker/API call sites.
A configuration refusal is a named terminal, not an outage (`config_error`, #3034)	The `config_error` value on `run.run_status_t` (migration 245), its audit action `run.config_error` (migration 246), its WARNING notification trigger `config_refusal` (migration 247), the allowed edges in `RUN_VALID_TRANSITIONS` (`src/models/internal/enums.py`) enforced by `src/services/state_machine.py`, and the terminal action map in `src/workflows/reconcile_mapping.py`.	Verified in code - `config_error` is in `RUN_TERMINAL_STATES`, is reachable only from `proof_running`, `proof_passed`, `diff_queued`, `diff_running`, `lag_retry_pending` and `transient_error` (never from readiness, whose refusals stay `readiness_error`), has no outgoing edges, and writes one `run.config_error` audit row through the same chokepoint as every other terminal. All three enum values landed together so the audit write cannot fail for a missing action value (the defect that left `readiness_error` without audit rows from 2026-06-04 to 2026-08-19, about eleven weeks, repaired by migration 181). The SOX audit-trail harness (`scripts/verify_sox_audit_trail.py`) drives a dedicated leg - a mapping whose decryption key reference cannot be resolved - and asserts the run terminates `config_error` with its audit row, never `system_error`.
`actor_ip` populated from request	The API authentication layer.	Verified - the auth layer extracts `X-Forwarded-For` and feeds it to `actor_ip`.
Hash-chained, tamper-evident audit log	The audit-log hash-chain columns (`entry_hash`, `prev_hash`, `sealed_at`) and the chain-verification logic in the services layer.	Verified in code - chain fields exist, verification recomputes the chain from genesis and checks stored hashes.
Signed checkpoints anchor the chain against a full re-seal forgery	Checkpoint records in `cfg.audit_chain_checkpoints`, exposed via the `/audit/seal` and `/audit/verify` endpoints.	Supported - HMAC signing is the default (shipped with a development-insecure key that must be replaced before production); an ECDSA-P256 asymmetric signer is shipped and selectable via configuration. Each checkpoint is verified under the key that signed it, so rotating a signing key does not strand history - the outgoing key is declared among the deployment's recognized previous keys. Verification is tri-state: a checkpoint whose key this deployment no longer declares reads as unverified, never as tampering.
KMS-backed signing (private key never enters the app)	The AWS KMS, Azure Key Vault, and GCP KMS signing clients in the services layer.	Supported - the AWS KMS, Azure Key Vault, and GCP KMS signers are each live-verified against a real key in that cloud. The private key never leaves the vault/KMS, so a full application or database compromise still cannot forge a checkpoint.
Automatic chain sealing	The automatic chain-sealing workflow, gated by a configuration flag.	Supported - on by default since #2729 (set `audit_chain_sealing_enabled=false` / `app.audit.sealingEnabled=false` to opt out); manual "Seal now" / "Verify" via `/audit/seal` and `/audit/verify` are always available regardless of the flag.
External anchor / WORM export of the sealed chain	Five external anchor clients in the services layer - S3 Object Lock, Azure Blob immutability, GCS retention lock, an RFC 3161 timestamp authority, and SIEM forwarding - wired into the seal/verify workflow and the `/audit/seal` endpoint.	Verified - each signed checkpoint can be anchored to a write-once external store (S3 Object Lock, Azure Blob immutability, or GCS retention lock), timestamped by an RFC 3161 authority, and forwarded to a SIEM. Live-verified against real cloud stores, all five anchors together on one deployment; on verify, the four write-once anchors read back `valid` and the SIEM forward reads `delivery_attested`. A deliberate delete or overwrite is refused by the store itself (e.g. S3 `AccessDenied`, Azure `BlobImmutableDueToPolicy`), not by application code.
Signed, hash-chained evidence bundles	The evidence-bundle schema (`content_hash`, `previous_bundle_hash`, `signature`, `masking_state`), the bundle generation service, the bundle signer resolver, and the bundle API endpoints.	Verified - table, hash-chain columns, RBAC-gated generate/list/get/verify endpoints, and audit logging on every action all present. The bundle is built once and signed over exactly the bytes that ship: a detached signature over the manifest, which commits to every file's SHA-256 (reworked 2026-08 - the earlier format signed a pre-rebuild archive hash that never shipped and was not offline-verifiable). Which key signs is the tenant's `bundle_signing_key_ref` (settable through the API and the Settings screen, tenant-level with a platform fallback): a KMS key (AWS KMS / Azure Key Vault / GCP KMS) or a local ECDSA key signs asymmetrically - ECDSA-P256, the same signer adapters as the audit-chain checkpoints - and the bundle embeds the public key plus its SHA-256 fingerprint; a secrets-URI HMAC key keeps symmetric signing. Period exports are signed containers (#2014): the container manifest lists every nested per-run bundle by SHA-256 and carries its own detached signature; the signature over the container is recorded on the bundle row as `manifest_sha256`, `signature` and `signing_key_id` by the same single pending-to-generated UPDATE that completes the row (`src/workers/evidence/generate_bundle.py`), and `APPROVED_EV_UPDATE_PATHS` names exactly those columns, so a recorded signature can never be rewritten after generation. The container bytes are persisted in the platform database (append-only, inside the customer VPC) and downloadable through the product; the download serves exactly the signed bytes - the writer refuses to store bytes that do not hash to the recorded `content_hash`, the endpoint re-checks the hash as it streams, and every download is audit-logged before release. Live-verified 2026-08-28: a 327-run period container requested, generated and downloaded through the product verified credential-less with the KMS public key (#2018 transcript, available to reviewers on request through your Paridity account contact). Object-store tier (#2026): a tenant (or the platform) may name an S3 / Azure Blob / GCS bucket - the same destinations and cloud identity as the audit-chain anchors - and period containers are then written there (create-once, SHA-256 in the object metadata) with the database keeping only the location, hash and the store's receipt; the download streams the exact recorded object through the same hash check, and a container whose object is missing or whose store cannot be reached is refused with a named reason. Live-verified 2026-08-29 against a real S3 Object Lock bucket and LocalStack (#2026 transcript, on request as above), and 2026-08-30 on a real Azure Blob container under an immutability policy and a real GCS bucket under a retention policy - the same 29 legs each as S3, with the S3 runs repeated green first (#2038 transcript, on request as above).
Standalone offline verification tool	The standalone offline bundle-verification tool shipped with the platform (also embedded inside every bundle).	Verified - and not merely "ships": automated writer↔verifier agreement tests generate real bundles through the production writer for every signer kind (HMAC, local ECDSA, AWS KMS, Azure Key Vault, GCP KMS, and the period container) and run the shipped verifier against the produced bytes; every negative (tampered member, stripped signature, wrong key, wrong fingerprint, attacker-embedded key, HMAC key against an asymmetric bundle, public key against an HMAC bundle, tampered nested bundle) was proven to fail against a deliberately broken verifier before being trusted. Public-key mode is stdlib-only (ECDSA-P256 implemented in the single verifier file, no packages to install): `--public-key <pem>` for a key obtained out of band, or `--public-key-fingerprint <sha256>` to pin the copy embedded in the bundle. The embedded key is never trusted unpinned. The verifier is fail-closed: unsigned, unverifiable, wrong-key, wrong-kind-of-key, and fingerprint-mismatch outcomes exit non-zero with a named reason; hash-level-only checking requires an explicit, loudly-labelled flag. Live-verified 2026-08-28 against a real AWS KMS key: a bundle and a period container signed via KMS were verified in a throwaway container with no cloud credentials, no packages, and only the public key read directly from KMS; six can-fail legs exited 1 with the named reason (#2014 transcript, available to reviewers on request through your Paridity account contact). Azure Key Vault and GCP KMS bundle signing are covered by the same adapters that were live-verified for audit checkpoints (#1729) and by fake-client agreement tests, not by a live bundle run.
Bundles anchored to write-once storage (SEC 17a-4 / FINRA 4511 posture)	The bundle-anchor service, the `bundle.anchored` audit action (migration 192), the anchor block on the bundle metadata endpoints, the anchor read-back in `POST /evidence-bundles/{id}/verify`, and the `anchored_copy_retained_until` facts on the retention purge's audit entry.	Verified - when a tenant's object store is a write-once store (S3 Object Lock, Azure Blob immutability, GCS retention lock - the same destinations as the audit-chain anchors), the signed period container written there is the write-once copy, and the platform attests it rather than making a second one: in the same transaction as `bundle.generate` it writes a `bundle.anchored` audit entry naming the store, the exact object (bucket, key, version), the store's own protection statement and retain-until date, and the container's SHA-256 - only when the store confirmed immutability protection on that object; an unprotected bucket is shown as unprotected and never attested as an anchor. Bundle metadata and the Period audit bundles card show the anchor posture. Verification reads the anchored object back in full from the store and compares its SHA-256 with the platform record (`anchor_valid`): a write-once copy that disagrees with the record, or has vanished, fails the verification even when the caller's copy agrees with the record; an unreachable store is reported as unverifiable, never as a pass. The retention purge asks the store to delete the object with the record; a locked store refuses, and the purge's audit entry records how many anchored copies the store kept and the latest date the store said its protection lapses (`anchored_copy_retained_until`) - the copy outlives the platform record under the store's own lock, findable by its recorded location. Live-verified 2026-08-29 against a real S3 Object Lock bucket through the product and against a LocalStack COMPLIANCE-locked bucket (where the lock alone - no IAM - refused the delete): attestation, full read-back verification, tampered-copy / altered-record / absent-copy can-fails, overlay resistance, and the purge record (#2027 transcript, available to reviewers on request through your Paridity account contact). Live-verified 2026-08-30 on a real Azure Blob container under an immutability policy and a real GCS bucket under a retention policy - the same 29 legs each as S3, with the S3 runs repeated green first so the store-neutral refactor could not have changed what the S3 proof asserts. Each store's behaviour is asserted as that store's own: Azure refuses both the delete and an overlay write with `BlobImmutableDueToPolicy`, GCS with its retention-policy 403, and where Azure's container-level policy exposes no per-object retain-until date the record carries none rather than an invented one - pinned both ways (#2038 transcript, on request as above).
Masking is enforced end-to-end (`mask_in_ui` as a field-level setting - there is no tenant-wide mask policy today - and the `store_raw_diff_values` AND-chain across tenant, mapping and field)	The `mask_in_ui` / `store_raw_diff_values` resolution rules and the bundle `masking_state` (`full`/`masked`/`metadata_only`).	Supported - masking policy computation is verified in the config resolver (see the companion accuracy proof doc); bundle-level enforcement of the same policy at export time is reflected in the `masking_state` column. Independent confirmation that bundle generation re-applies `mask_in_ui`/`store_raw_diff_values` per field at export time (rather than only at live-UI query time) is a recommended verification step.
Retention purge is itself audited	The retention-purge process.	Verified - writes one `cfg.audit_log` row per (table, mapping) pair before each `DELETE`, including disabled mappings.
Evidence-bundle retention (`bundle_retention_days`) is enforced	The bundle leg of the retention-purge process, the shared retention resolver, and `GET /evidence-bundles/retention`.	Verified - the same purge process deletes bundle records generated longer ago than the resolved `bundle_retention_days` (tenant value, floored at the platform value - a tenant cannot shorten bundle retention below the platform minimum). The stored period container is removed by the database cascade, which the purge verifies before committing (an orphaned container aborts the run). A container kept in the tenant's object store (#2026) is deleted from the store by the same purge, and the store's answer is written per object on the same audit entry - deleted, retained by store (a bucket under Object Lock / immutability / retention, or an identity without delete rights, refused - recorded with the lock facts, never reported as freed) or already absent; a store that cannot be consulted aborts that group before any database change. Live-verified 2026-08-29: S3 refused the delete of a locked object and the audit entry recorded `retained_by_store` with the lock; on an unlocked store the object was deleted and confirmed absent. The audit entry written before the delete records rows removed from both tables, bytes freed, bundle and run ids, cutoff, policy, and the hash-chain boundary. Live-verified 2026-08-28 on the development database: a bundle past the cutoff and its container removed, a bundle inside the window untouched, the audit entry present, and a cutoff that qualified nothing deleted nothing (#2021 transcript, available to reviewers on request through your Paridity account contact). The nightly SOX audit-trail gate (check G5) seeds an aged bundle with a stored container and asserts the same.
Hash chain stays honest across retention	The server-side chain verification (`verify_evidence_bundle`) and the purge's audit record.	Verified - the first bundle surviving a purge names a predecessor that no longer exists; verification resolves that hash against the purge's audit entry and reports `CHAIN_HEAD_PURGED` (with purge date and policy) as a valid link. A missing predecessor that no purge entry names is `BROKEN`; a predecessor that still exists but disagrees is `BROKEN` regardless of any purge entry - retention can explain an absence, never a mismatch.
Legal holds freeze evidence against the purge, and the skip is audited	The legal-hold entity (`cfg.legal_holds`), the hold exclusion on every purge leg, and the place/release API (`POST /legal-holds`, `POST /legal-holds/{id}/release`, `GET /legal-holds`).	Verified - a legal hold on a tenant, a mapping, or a single bundle makes the retention purge skip the held scope: a held record past its cutoff is never deleted. Placing and releasing a hold is gated by an approved elevation (`evidence.legal_hold`, the same separation-of-duties gate as an ad-hoc purge) and both acts are written to `cfg.audit_log`. When the purge skips a held record it writes a `legal_hold.skipped` audit row naming the hold(s) and the count per table, so an auditor sees the hold working, not merely an absence of deletions. Releasing the hold makes the scope eligible again; an expired hold (past its optional `expires_at`) no longer protects. Live-verified 2026-08-29 on the development database: a held bundle past the cutoff survived `--execute` (its `xmin` unchanged) while an unheld aged bundle was purged in the same run, the skip was on the audit record naming the hold, releasing the hold made the bundle eligible and the next run deleted it, an expired hold protected nothing, and the API refused an ungated place with `APPROVAL_REQUIRED` and accepted it with an approved elevation - 24/24 legs (#2024 transcript, available to reviewers on request through your Paridity account contact). The live proof exercised the bundle, mapping, and tenant scopes against the bundle leg; the run-evidence and metrics legs apply the same hold exclusion (the identical `covers_mapping` predicate and active-hold SQL, which ran live with zero-row results in the same harness) and their non-zero skip paths are proven by unit tests rather than a live seed.
Retention purge runs on a built-in schedule	A durable Temporal singleton (`RetentionPurgeWorkflow`) on the scheduler worker that runs the same `run_purge` chokepoint on a cadence, plus the deployment settings (`retention_purge_*`) and the schedule state on `cfg.retention_purge_schedule`.	Verified - the platform ships a built-in scheduler that runs the retention purge automatically (daily at 04:00 UTC by default), so a retention policy is executed rather than merely intended. It is on by default but dry-run-first: fires within the first `retention_purge_dry_run_days` after install (default 7) compute and attest what would be deleted without deleting anything, so a fresh deployment never deletes on day one. It ships default-on safely because legal holds (#2024) land first - automatic deletion always skips a scope under a preservation order. Each fire writes one `cfg.audit_log` row (`retention_purge.executed`) so an auditor can list every scheduled run. Live-verified 2026-08-29 on the development database: the boot singleton fired dry-run-first on its own; a scheduled dry-run fire deleted nothing; a scheduled execute fire purged an aged bundle while a legal hold made it skip a held bundle in the same fire (the held bundle's `xmin` unchanged, a `legal_hold.skipped` row naming the hold); every fire wrote exactly one attestation row (#2025 transcript, available to reviewers on request through your Paridity account contact).
Pausing the scheduled purge is an approval-gated, audited compliance fact	The pause/resume endpoints (`POST /evidence-bundles/retention/pause`\|`/resume`), the `evidence.retention_purge` approval gate, and the `retention_purge.paused`/`.resumed` audit actions.	Verified - an operator can pause the built-in schedule; a paused schedule still fires but deletes nothing (the no-op is attested with mode `skipped_paused`), which is itself a visible compliance fact - `GET /evidence-bundles/retention` reports `enforcement=retention_purge_paused`. Pausing and resuming require an approved `evidence.retention_purge` elevation (the same separation-of-duties gate as an ad-hoc purge; 403 `APPROVAL_REQUIRED` without one) and both acts are written to `cfg.audit_log`. Live-verified 2026-08-29: a paused schedule left a past-cutoff bundle intact, resume deleted it, the API refused an ungated pause and accepted an elevated one, and both acts were audited.
SAML sign-in requests are signed, and the signing key rotates without an unsigned gap (#250, #3281)	The per-provider `sign_requests` opt-in (`src/models/identity_provider.py`), the SP signer (`src/services/saml_sp_signer.py`), the SP settings builder that pins RSA-SHA256 / SHA-256 (`src/services/saml_sp_service.py`), the metadata and `sp-info` publish surfaces (`src/api/routers/saml.py`), the IdP-enable guard (`src/api/routers/identity_providers.py`), and the Helm values `app.saml.spSigningKeyRef` / `app.saml.spSigningKeyNextRef` rendered to `SAML_SP_SIGNING_KEY_REF` / `SAML_SP_SIGNING_KEY_NEXT_REF` (`deploy/helm/paridity/values.yaml`, `templates/configmap.yaml`).	Verified in code (PR #3288, merged 2026-10-26). One RSA key per deployment, loaded from a secrets-URI PEM bundle (PKCS#8 key + X.509 certificate; a non-RSA key is refused because RSA-SHA256 is the pinned algorithm); the PEM never lands in the database, settings or logs. Production with `sign_requests` on and no key is a refusal at IdP-enable (422 `SAML_SP_SIGNING_KEY_REQUIRED`) and at API start, never an unsigned send. The certificate and its SHA-256 fingerprint are public by design (`/saml/{idp_id}/metadata` as a signing `KeyDescriptor`, `/saml/{idp_id}/sp-info`, and the Authentication settings screen). Rotation: with a next ref staged the metadata carries both certificates (current first) and `sp-info` reports `sp_signing_rotation_staged: true` with the next certificate, its fingerprint and expiry; only the current key signs - the signing path reads the next certificate from cache and never loads the next private key, which is parsed once to prove it matches its certificate and then discarded. A staged bundle that is unusable, whose certificate equals the current one, or that is set while production has no current key fails closed: the API refuses to start naming which ref failed, and the publish surfaces answer 503 `SAML_SP_SIGNING_KEY_NEXT_INVALID`. Cut-over is a pointer move (`spSigningKeyRef` to the next secret, clear `spSigningKeyNextRef`); rollback within the window is the reverse move. Runbooks: the IdP onboarding runbook (section 4b) and the security checklist, available to reviewers on request through your Paridity account contact. Scope label: the key refs are a Helm / environment setting and the two-phase procedure is operator-run; the UI shows the certificates and the staged state but does not perform the rotation. This control is not live-verified against a customer IdP in this document's evidence; it is verified by code reading and the unit suite that merged with PR #3288.

Verify it yourself¶

Confirm append-only evidence. As an auditor or platform_admin, attempt an UPDATE or DELETE against any ev.* table other than the two documented exceptions, from an ordinary application code path (not a database console) - there is no API surface that permits it. Ask your Paridity contact to walk through the retention-purge process if you want to see the one exception in code.
Recompute a config_checksum. Pull a run's resolved config snapshot (cfg.config_versions.resolved_config_snapshot, linked via run.runs.config_version_id), apply the documented canonicalization rules (strip spec_version/generated_at/ generated_by, sort all object keys and the documented arrays, serialize with no whitespace, UTF-8 encode), take SHA-256, and confirm it matches the config_checksum stored on ev.ev_config_audit for that run.
Verify an evidence bundle offline. Generate a bundle for a run (POST /runs/{execution_request_id}/evidence-bundle), download it, and run the shipped verifier. With a KMS / ECDSA signing key configured, anyone can verify with the public key alone - obtain it out of band from the data owner's KMS (e.g. aws kms get-public-key, az keyvault key download, gcloud kms keys versions get-public-key) and run python tools/verify_bundle.py --bundle <file> --public-key <pem>, or pin the embedded copy with --public-key-fingerprint <sha256>. With an HMAC key, run --key-from-env HMAC_KEY with the tenant's key from its own secrets backend. The verifier is fail-closed: exit 0 means every file hash, the hash chain, and the detached signature all verified; an unsigned, tampered, wrong-key, or wrong-kind-of-key bundle never reports as verified. The platform's own POST /evidence-bundles/{bundle_id}/verify round-trip endpoint is the server-side complement; the offline tool is the one that does not require trusting the platform, and the out-of-band public key is the one that does not require trusting the bundle.
Walk the bundle hash chain. Generate two sequential bundles for the same mapping and confirm the second bundle's previous_bundle_hash equals the first bundle's content_hash.
Verify the audit-log hash chain. Call GET /audit/verify (or the equivalent chain-verification endpoint) and confirm it reports the chain intact; if automatic sealing is enabled, confirm the signature on the latest checkpoint validates against the configured signing key/public key.
Trace a run's phase timings. Pull run.run_attempts for a run_id and confirm there is a row for each phase the run went through (readiness, proof_count and proof_hash, and - on a failed proof - key_extraction, field_diff, lag_retry; plus scd_invariants where the mapping declares an SCD scope), each with started_at, completed_at, and worker_id populated.
Trace a terminal status to its audit entry. Pick a run that reached a terminal status and confirm cfg.audit_log has a matching entry with actor, action, entity, and timestamp - and, for an admin-triggered cancellation or a bypass_readiness run, that a reason is present.
Verify the write-once copy, not just your copy. For a period bundle written to a write-once store, call POST /evidence-bundles/{bundle_id}/verify and read anchor_valid / anchor.state: the server reads the anchored object back from the store in full and compares its SHA-256 with the platform record. Ask your Paridity contact to alter the platform record, tamper the anchored object, or remove it in a sandbox - verification must report the disagreement or the absence, never a pass, and an unreachable store must report unverifiable rather than either.
Watch a legal hold stop a deletion. Place a hold (POST /legal-holds) on a mapping whose evidence is already past its retention cutoff - the API must refuse an ungated request with APPROVAL_REQUIRED until an evidence.legal_hold elevation is approved - then run the retention purge. The held records survive, cfg.audit_log carries a legal_hold.skipped row naming the hold and the per-table count, and releasing the hold makes the next purge delete them.
Confirm no projection lives in ev. List the base tables in the ev schema (information_schema.tables WHERE table_schema = 'ev') and confirm none of run_summary, metric_rollup_hourly, run_metric_baselines, diff_run_limits or run_consumption (with or without an ev_ prefix) is among them; they are in proj, whose schema comment states it is never an audit source. Then run the static scan (the #1713 unit test in the platform's suite, available to reviewers on request through your Paridity account contact) and confirm it reports exactly the two declared mutation sites.
Trace a configuration refusal. Pick a run whose status is config_error (shown as Config refusal) and confirm cfg.audit_log holds a run.config_error row for it with actor, entity and timestamp, that the run's error_code names the configuration defect, and that no run.system_error row exists for the same run. If the deployment uses notifications, confirm the alert fired under the config_refusal trigger at WARNING, not under system_error.
Check the certificate your IdP verifies against. Call GET /saml/{idp_id}/sp-info (or open the Authentication settings screen) and compare sp_signing_certificate_fingerprint_sha256 with the certificate uploaded to your identity provider. During a staged rotation, confirm sp_signing_rotation_staged is true, that GET /saml/{idp_id}/metadata lists both certificates as signing keys, and that the signature on a captured AuthnRequest still verifies against the current certificate; after cut-over confirm the fingerprint has moved and sp_signing_rotation_staged is false.

Tip: An SE-led session can walk through steps 2-6 live against a sandbox mapping, including a deliberately tampered bundle so you can see verification fail as expected.

Limitations¶

Scope	Status	Notes
Append-only `ev.*` evidence with two documented lifecycle exceptions	Verified	`ev_anomalies` lifecycle columns and the `ev_evidence_bundles` pending→generated update are the only sanctioned mutations; core evidence columns are never touched. Enforced on every run, not by convention alone: the SOX audit-trail check G4 snapshots each row's `xmin` across a real run, and a static scan rejects both `UPDATE ev.…` and `INSERT … ON CONFLICT DO UPDATE` in the tree.
Derived projections live outside `ev` (`proj` schema, #1724)	Verified	Migration 177 moved the four rewritten-in-place tables out of `ev`; migration 260 created a later projection directly in `proj`. `proj.` is recomputable from `ev.`, is never an audit source, and is deliberately not covered by the append-only checks. It is data-plane: derived from evidence values, so it stays inside the customer VPC like `ev`.
Config audit before proof (`config_checksum`)	Verified	Confirmed in the workflow code path for every run.
`compare_started_at` immutability	Supported	Enforced by the state-machine chokepoint (a write naming the column is rejected) and pinned by unit tests; not by a database-level trigger or constraint - see the evidence table.
Configuration refusals are a distinct audited terminal (`config_error`)	Verified	Migrations 245/246/247; edges in `RUN_VALID_TRANSITIONS`, enforced by the state machine; one `run.config_error` audit row per run; WARNING `config_refusal` notification. Exercised by a dedicated leg of the SOX audit-trail harness. Readiness-phase refusals remain `readiness_error`; partition sub-rows have no `config_error` value of their own and record `system_error` at the partition level while the run terminates `config_error`.
Signed SAML requests and two-phase SP key rotation	Supported	Verified in code and by the unit suite merged with PR #3288 (2026-10-26); not live-verified against a customer identity provider in this document's evidence. The key refs are Helm / environment settings; the rotation is operator-run from the runbook, with the UI showing the certificates and the staged state. Only providers configured with `sign_requests` receive signed requests; an IdP that does not verify request signatures is unaffected either way.
Per-phase timing + `worker_id` in `run.run_attempts`	Verified	Confirmed for readiness and proof; the same attempt-writing activity is called for each phase.
Append-only watermark history	Verified	Migration-level enforcement (`(run_id, recorded_at)` composite PK).
Audit-log hash chain (tamper-evidence)	Supported	Chain + verification code shipped; sealing is on by default (#2729) - a deployment that opts out has a hash-chain mechanism present but not actively advancing until sealing is re-enabled.
HMAC checkpoint signing	Supported	Default signer; symmetric - only as strong as the secret's storage (an inherent property of HMAC signing, not a Paridity-specific weakness). Ships with an insecure default key that must be changed before production use.
ECDSA-P256 asymmetric signing	Supported	Shipped; lets an external auditor verify with only the public key.
Asymmetric (KMS / ECDSA) evidence-bundle signing - third-party verification with the public key alone	Verified	Live-verified against AWS KMS: bundle + period container verified credential-less with the public key read from KMS, no shared secret; six can-fail legs. Azure Key Vault / GCP KMS bundle signing use the live-verified checkpoint adapters and are covered by agreement tests, not a live bundle run.
Period evidence bundles downloadable through the product, bytes exactly as signed	Verified	Live-verified 2026-08-28 (#2018): period container requested via the API, persisted, downloaded, and verified credential-less with the KMS public key; sha256(download) == server `content_hash`; named refusals for pending / over-cap / not-retained; download audit-logged; serving mutates no evidence row.
Period containers can live in the customer's own object store, with no size cap and no bytes in the platform database	Verified	Live-verified 2026-08-29 (#2026): with `evidence_bundle_store_ref` set to a real S3 Object Lock bucket the container was written by the product (create-once, SHA-256 in the object metadata), the database row recorded location + hash + the store's receipt and no bytes, the download streamed the exact object byte-identical and verified offline, S3 itself refused the retention delete (`retained_by_store` on the audit entry with the lock facts), and on LocalStack a tampered object was refused mid-stream and an unlocked object was deleted and confirmed absent. Live-verified 2026-08-30 (#2038) on a real Azure Blob container under an immutability policy and a real GCS bucket under a retention policy - the same 29 legs each, with the S3 runs repeated green first so the store-neutral refactor could not have changed what the S3 proof asserts. Each store's behaviour is asserted as that store's own: Azure refuses the delete and an overlay write with `BlobImmutableDueToPolicy`, GCS with its retention-policy 403, and where Azure's container-level policy exposes no per-object retain-until date the record carries none rather than an invented one - pinned both ways.
Period bundles anchored to write-once storage, attested and verifiable by read-back (SEC 17a-4 / FINRA 4511 posture)	Verified	Live-verified 2026-08-29 (#2027): the container the product wrote to a real S3 Object Lock bucket was attested (`bundle.anchored`, same transaction as generation, with the lock mode, retain-until, object version and SHA-256), read back in full by the verify endpoint and matched; a tampered local copy, a platform record altered to disagree with the anchored object, and an absent anchored copy each failed verification with a named reason; S3 refused the delete of the anchored version and an overlay write left the anchored version verifiable; the retention purge recorded the anchored copy the store kept and until when. On LocalStack a COMPLIANCE lock alone (no IAM) refused the delete. The store's lock is what makes the copy write-once - the platform attests and verifies it; an unlocked bucket is reported unprotected, never anchored. Live-verified 2026-08-30 (#2038) on a real Azure Blob container under an immutability policy and a real GCS bucket under a retention policy - the same 29 legs each, with the S3 runs repeated green first so the store-neutral refactor could not have changed what the S3 proof asserts. Azure refuses both the delete and an overlay write with `BlobImmutableDueToPolicy` and GCS with its retention-policy 403 - each store's own refusal, not application code - and where Azure's container-level policy exposes no per-object retain-until date the record carries none rather than an invented one, pinned both ways.
AWS KMS signing (private key never leaves KMS)	Supported	Live-verified against a real AWS KMS key.
Azure Key Vault / GCP KMS signing	Supported	Both signers live-verified against a real key in that cloud, alongside AWS KMS.
External WORM anchor (S3 Object Lock / Azure Blob immutability / GCS retention lock / RFC 3161 TSA / SIEM forward)	Verified	All five anchors live-verified together against real cloud stores; a deliberate delete or overwrite is refused by the store itself, not by application code.
Automatic continuous chain sealing	Supported	On by default since #2729; manual "Seal now" remains available. A deployment that opts out and never clicks Seal has a verifiable-on-demand chain, not a continuously-advancing one.
Signed, hash-chained evidence bundles	Verified	Schema, generation service, and RBAC-gated API confirmed in code.
Bundle masking enforcement at export time	Supported	Masking policy computation itself is verified; independent confirmation that bundle generation re-applies `mask_in_ui`/`store_raw_diff_values` per field at export time is a recommended verification step - flagged in the evidence table.
Retention purge is audited before deletion	Verified	Confirmed in the retention-purge process.
Evidence-bundle retention is enforced, cascade included	Verified	Bundle records and their stored containers age out under `bundle_retention_days` through the same audited purge; the chain boundary is recorded so verification reports a purged predecessor as `CHAIN_HEAD_PURGED`, never as tampering. Live-verified 2026-08-28.
Retention purge runs unattended on a built-in schedule	Verified	A durable Temporal singleton runs the purge automatically (daily by default), on by default and dry-run-first for the first days after install; legal holds make automatic deletion always skip preserved evidence. Live-verified 2026-08-29 (#2025): the boot singleton fired dry-run-first, a scheduled execute fire purged an aged bundle while honouring a hold, and every fire wrote one attestation row.
Pausing the schedule is approval-gated and audited	Verified	Pause/resume requires an approved `evidence.retention_purge` elevation and is written to `cfg.audit_log`; a paused schedule fires but deletes nothing (attested), and the retention status reports the paused state. Live-verified 2026-08-29.
Deployment model	Verified	The entire platform - including all evidence, the audit log, and any signing keys - runs inside your VPC. Nothing described in this document sends evidence or audit data to Paridity-operated infrastructure.

Paridity Architecture Technical Brief¶

Verified data. Trusted outcomes.

CDC-Aware Reconciliation Platform

Prepared for: Customer, partner and procurement evaluators (Customer Confidential) Version: v0.2 - Date: 2026-10-27

Executive summary¶

Paridity's architecture makes three commitments that most reconciliation tools cannot: the entire platform runs inside your own cloud account/VPC, so credentials, object names, and data values never leave your environment; a single vendor-neutral connector contract with cross-engine canonical hashing lets it compare heterogeneous source/target pairs (e.g. Oracle against Snowflake) natively, with a DuckDB reference engine as ground truth; and the platform is engineered for corporate-scale datasets - 100M-10B rows per Mapping - with bulk-only I/O and hash pushdown enforced as design rules, not aspirations. This brief describes how the architecture delivers each commitment and is explicit about what is roadmap rather than shipped. It is the architecture companion to the Paridity Security Whitepaper; security controls are covered there and are linked, not restated, here.

Problem¶

An enterprise architect evaluating a reconciliation platform is really evaluating three risks at once. First, data-residency risk: conventional SaaS reconciliation tools require exporting database credentials, schema/object metadata, and often the data values themselves to a vendor-hosted environment. Second, lock-in and coverage risk: tools built around one warehouse or one replication vendor cannot prove correctness across a heterogeneous estate (SAP ECC on Oracle or DB2 replicating to Snowflake, Databricks, or ADLS). Third, scale risk: an architecture that compares tables by pulling rows to a central service collapses at billions of rows. Paridity's architecture is designed around all three: customer-hosted deployment, a vendor-neutral engine model, and pushdown-first comparison that moves fingerprints, not rows.

Approach / architecture¶

Deployment model - customer-hosted, in your own cloud account¶

Today's shipping architecture. The full Paridity stack deploys inside the customer's own cloud account/VPC: the FastAPI control-plane API, the React/Vite UI, the Temporal orchestrator, all six Temporal workers (scheduler, readiness, SQL, diff, evidence, notification), and a single PostgreSQL instance holding all four schemas. Deployment is via a shipped Helm chart and Terraform reference modules for AWS, Azure, and GCP that provision the cluster, a managed Postgres instance with backups/PITR, the customer's own secret backend, and an egress-locked network - node egress deny-by-default, reaching cloud services over private paths (AWS PrivateLink endpoints, GCP Private Google Access, Azure NSG service tags) rather than the internet, with only the operator's declared source/target endpoints allowed outbound. Verified live on all three clouds. Paridity operates no part of this infrastructure - there is no Paridity-hosted control plane, no Paridity-visible copy of customer data, and no Paridity credential store in this model. This is the architecture, delivered and CI-gated - the entire platform runs inside the customer's own cloud account.

Four schemas, one separation of concerns¶

The platform's own PostgreSQL database is partitioned into four schemas whose concerns are never mixed:

Configuration schema - configuration, versioning, and audit. Holds the 7-layer config model, frozen resolved-config snapshots, and the audit log. The config_checksum (SHA-256 of the resolved config JSON) lives here - it fingerprints policy, never data.
Execution schema - execution lifecycle: runs, execution requests, watermarks, CDC state, schema-drift detection, per-phase attempt records with timestamps.
Evidence schema - append-only evidence: hash_total proof results, row_hash mismatch keys, field_diff output, run metrics, in-flight recheck records, and hash-chained evidence bundles. Evidence tables are never updated or deleted outside the audited retention purge - which now runs on a built-in schedule and skips any scope under a legal hold - and two narrowly-scoped lifecycle exceptions.
Projection schema - derived, recomputable projections (run summaries, hourly metric rollups, baselines) that dashboards read instead of raw evidence. Anything rewritable lives here by construction, so the evidence schema's append-only rule holds with no exceptions to remember - and it is actively enforced by a live row-version check plus a static code scan, not left to convention.

The separation is enforced by classification: every table and column is classified as control-plane (governance/metadata) or data-plane (sensitive - credentials, object/schema/column names, resolved snapshots, evidence values), and the two are never mixed in one table.

Orchestration - six workers on dedicated task queues¶

All execution flows through Temporal (self-hosted, MIT-licensed) durable workflows. Work is divided across dedicated task queues - scheduler, readiness, SQL, diff, evidence, and notification - so that pushdown SQL, in-memory diffing, and evidence writes scale independently. Workers execute single activities and contain no orchestration logic; connectors execute SQL and contain no reconciliation logic; comparison logic lives in workers and services. Data moves between workers and the DuckDB diff engine as Apache Arrow - a columnar interchange format, not row-at-a-time serialization.

Controls / details¶

Vendor-neutral engine model¶

Every source and target system is accessed through one contract: the BaseConnector abstract base class. Connectors are deliberately dumb adapters - they execute SQL and return data; all comparison logic lives above them. This is what keeps Paridity vendor-neutral: adding an engine means implementing one interface, not re-architecting the comparison pipeline.

Native hash pushdown with a reference fallback. For the proof phase, Paridity asks each engine to compute the hash_total - the aggregate dataset fingerprint - inside the engine itself, using that engine's native SHA-256 facility (e.g. sha256() + BIT_XOR on PostgreSQL, SHA2 on Snowflake and Databricks, DBMS_CRYPTO on Oracle, HASHBYTES on SQL Server, HASH(expr, 2) on DB2 LUW 11.1+, HASH_SHA256 on SAP HANA, SHA256 + groupBitXor on ClickHouse). Only fingerprints cross the network, never rows. Where an engine cannot compute a compatible hash, the connector raises NotSupportedError and the orchestrator falls back to the DuckDB reference implementation - the ground-truth engine every native implementation is byte-verified against - rather than silently computing something different.

Cross-engine canonical hashing is the enabling mechanism. Heterogeneous comparison (Oracle source, Snowflake target) is possible because every column value is normalized to an identical canonical string form - null token, fixed-point numerics, UTC ISO-8601 timestamps, \x1F-delimited row serialization - before hashing, so identical data hashes identically regardless of which engine produced it. Source and target must use the same hash_method; if they differ, both sides fall back to the DuckDB reference. The per-engine verification record - which pairs are byte-verified live, which are SUM-only, and every open accuracy issue - is maintained in the platform's cross-engine hash verification record and distilled for evaluators in How Paridity guarantees accuracy; this brief points to that record rather than restating it.

Engine coverage today: native xor_sum_sha256 hash pushdown on twelve engines - Oracle, DB2 LUW (11.1+), SQL Server (2019+), Snowflake, Databricks, BigQuery, Redshift, SAP HANA, MySQL/Aurora MySQL, PostgreSQL, Trino (covering ADLS/Hive external tables), and ClickHouse (25.8 LTS and later; an older server is refused at connect, never run on a fallback). Object-store and file datasets, and Salesforce, are compared via the DuckDB reference path (no SQL engine to push down to); Kafka CDC participates in readiness evaluation, not dataset proof. Replication-tool integration is likewise adapter-based: Debezium, Airbyte, Fivetran, Oracle GoldenGate and Qlik Talend Cloud are the Tier-1 (production) adapters; the design-partner-pending stubs are exactly SAP SLT, SAP BODS, Qlik Replicate (with its legacy attunity alias) and Informatica CDC - Roadmap per the status vocabulary below, not presented as shipped. Any other tool in the catalogue has no adapter and is Evaluating.

CDC-aware windowing - comparing at the same point in time¶

Most reconciliation platforms compare tables. Paridity compares datasets within a deterministically aligned time window, using CDC tool state to avoid false positives from in-flight replication. A readiness evaluation - CDC probe, watermark fetch, latency evaluation, stability check, advancement check - runs before every comparison and follows an explicit state machine. Watermark strategies are selected in a defined preference order (control_marker first, snapshot last), and Paridity never infers freshness from business dates such as posting date - it uses replication metadata, CDC state, or explicit snapshot mode. Two storage-engine behaviours that can move the picture after readiness are handled as declared mapping settings, never guessed: a Snowflake native table with Time Travel retention is read AT(TIMESTAMP) of the readiness instant (views opt in with read_pin_views; a no-retention table is read unpinned and logged as such), and a mapping may declare cdc_table_semantics - uniform_iceberg_mirror holds readiness until the Iceberg side has published the compared Delta commit; replacing_merge_tree_final and clickpipes_peerdb read a ClickHouse ReplacingMergeTree side through FINAL (the latter also filtering the PeerDB soft-delete marker) so unmerged row versions cannot false-fail a run.

Systems without a native change timestamp - SAP and similar¶

Many SAP application tables carry no column that advances on every insert, update, and delete. Paridity treats that as a supported configuration rather than an exception. A watermark strategy is chosen per mapping from a defined preference order: control_marker (a completion record your pipeline writes when a load finishes - preferred), delta_version / kafka_offset / oracle_scn / lsn / db2_commit_ts (a CDC offset or commit sequence), ingest_ts (a load-completion timestamp), partition_max, timestamp (a fallback - it infers rather than observes), custom (an operator-reviewed SQL expression), and finally snapshot or partitioned_snapshot.

Two rules are absolute. Paridity never infers freshness from a business date: a posting date or document date records when something happened in the business, not when the row arrived, so a backdated or corrected document would fall outside every future window and never be re-compared. And auto-configuration never selects a column as a change watermark unless its name identifies it as a tracking timestamp; any other candidate is surfaced for operator review and is never applied silently.

Where no trustworthy signal can define a complete window, the mapping runs as an explicit snapshot. snapshot compares the whole current dataset on every run; partitioned_snapshot scopes each run to partitions declared on the mapping (by key range, value list, time grain, or a composite of two - for SAP typically client, company code, or fiscal year), and a schedule with window_mode = full provides the periodic full sweep. In both modes the readiness phase has nothing to fetch - no watermark, latency, or stability check is evaluated - and the run proceeds straight to the same count, hash-total, mismatch-key, and field-diff proof as every other run, with the comparison mode recorded in the run's evidence.

Declared scope for history-bearing tables¶

An SCD2 or bitemporal side legitimately retains closed history rows the other side no longer shows, so comparing the two whole fails forever on data that is correct. Paridity resolves this with an operator-declared scope, never an inferred one: a typed current-row predicate (a flag column, a far-future end-date sentinel, or a NULL end date - columns and values, never operator SQL) on one named side, versioned inside the checksummed resolved-config snapshot. Inferring that one side is a superset by design would be a false-pass machine, so the platform refuses to infer it, and refuses the declaration outright on any engine whose scope rendering is not live byte-verified. Every scoped run first checks the scope's own premise - exactly one current row per business key, and optionally no overlapping validity intervals - before the proof. A broken premise never silently passes: the run keeps its data verdict, is reclassified SCD invariant violated, and is excluded from pass counts. Watermark queries are deliberately never scoped: the current-row boundary is mobile, so scoping it would hide a pure logical delete behind a "no new data" verdict.

Scale & performance - engineering commitments, not marketing numbers¶

Paridity is engineered to remain viable at corporate scale - 100M-10B rows per Mapping - and treats throughput as the gate immediately after accuracy. The following are binding engineering rules enforced in code review, and are presented here as design commitments distinct from benchmarked results. Measured results now exist at production scale and live in the Paridity performance proof document: four flagship cross-engine pairs measured end-to-end on a published reference configuration - three of them (PostgreSQL→Snowflake, PostgreSQL→Databricks, Db2 LUW→BigQuery) through 100M rows - each preceded by a fail-first corruption-recovery proof. This brief cites those measurements rather than restating them.

No row-by-row I/O on the hot path. Reads use server-side cursors with bulk prefetch (asyncpg cursors with prefetch ≥ 10,000, Oracle array fetch, Snowflake Arrow-batch streaming, Databricks Arrow fetch); staging writes use bulk APIs (PostgreSQL COPY FROM STDIN via copy_records_to_table). Row-at-a-time INSERT loops are forbidden on extract/stage paths, with a per-engine bulk pattern defined for every SQL engine (ClickHouse streams keys as Arrow batches).
Pushdown is the default, fallback is the exception. Hash-total and key-extraction work runs inside the source/target engine wherever the engine supports it; every fallback to the DuckDB reference path is logged with its reason.
Throughput floors (per worker pod, 4 vCPU baseline) are defined as merge gates: ≥ 200,000 rows/sec for proof (hash-total) pushdown on Tier-1 engines; ≥ 25,000 rows/sec for PostgreSQL/MySQL key extraction; ≥ 50,000 rows/sec for Snowflake/Databricks key extraction. A change that drops below a floor is treated as a regression and must be fixed before merge. Measured reference points now sit well above these floors - PostgreSQL key extraction has been measured at ~507,000 rows/sec at 1M rows on the reference configuration (see the Paridity performance proof document); the floors remain the merge gate, not the marketing number.
The platform's own database is protected the same way. Hot-path queries must be index-backed; dashboards read pre-aggregated summary tables, never raw evidence rows in-request; evidence tables are always queried with a run_id predicate and explicit column projection.

Reliability¶

Durable workflows. Each run executes as one Temporal workflow instance per execution request - never two for the same request. Workflow state survives worker restarts; scheduling uses Temporal's durable timers, not an external cron daemon.
Explicit state machines. Execution requests, runs, and readiness evaluations each follow a defined state machine; every status transition goes through the defined paths and terminal transitions are written to the audit log - no ad-hoc status updates anywhere in the codebase. A deterministic configuration refusal discovered after readiness (a connector rejecting the mapping's declared types, a partition plan that cannot be executed) terminates in its own config_error state - operator work, raised as a warning - and is never conflated with system_error, the fault terminal that pages on-call.
Idempotency at two layers. All state-mutating API endpoints require an Idempotency-Key header with a 24-hour server-side response cache, and duplicate execution requests for the same mapping+window are rejected by a database UNIQUE constraint on the idempotency key - not silently ignored.
Evidence writes cannot be quietly lost. Evidence-write activities run under a dedicated retry policy (10 attempts, no non-retryable error types); long-running activities are heartbeat-monitored.
Audit anchors are immutable. The run's compare-start audit timestamp is set once when a run starts and never changes - not during in-flight rechecks, not on reruns. In-flight rechecks happen inside the same workflow and each iteration is recorded as recheck evidence.
Retention is executed, not merely configured. A durable Temporal singleton on the scheduler worker runs the retention purge on a cadence (daily at 04:00 UTC by default), writing one audit attestation per fire. It is on by default but dry-run-first for the first seven days after install, so a fresh deployment never deletes on day one, and legal holds are evaluated on every leg - automatic deletion always skips a scope under a preservation order.
Backpressure, not overload. Per-system concurrency is enforced through a database-backed counter with max_concurrency resolved as the minimum across config layers - each system's profile, source and target, is a hard ceiling no schedule or run can exceed.

Security controls (summary - see the Security Whitepaper)¶

Authentication (OIDC/JWT), RBAC, tenant isolation, secrets handling (connection_ref is always a URI pointer resolved via SecretsClient against the customer's own secret backend - Paridity never stores a credential), evidence masking, and the append-only hash-chained evidence trail are covered in depth, with the same Verified/Supported/Roadmap grading, in the Paridity Security Whitepaper. One caveat is repeated here because it is architectural: PostgreSQL row-level security is live as a defense-in-depth backstop - the application-layer tenant predicate on every query is the primary isolation mechanism today, because the application's database role currently has BYPASSRLS and FORCE ROW LEVEL SECURITY is intentionally deferred pending a call-site audit (see the Security Whitepaper, "Tenant isolation").

Proof / evidence¶

Paridity's correctness claim is deterministic end to end: aligned time windows, an aggregate hash_total as the fast pass/fail proof gate, per-row row_hash fingerprints to isolate exactly which rows differ, and field_diff output as the reconciliation evidence. Every native hash implementation is byte-verified against the DuckDB reference engine, and the per-engine verification status - including open issues, which are never closed without live test vectors - is tracked in the platform's cross-engine hash verification record and presented for evaluators, with a verify-it-yourself procedure, in How Paridity guarantees accuracy. Every run records its config_checksum to the append-only audit evidence before the proof phase begins, so an auditor can independently confirm which policy was in force.

Note: AI in Paridity assists triage and explanation only - it never determines whether data matches. Correctness is always established by the deterministic hash_total / row_hash / field_diff pipeline above, never by a model's judgment.

Limitations & roadmap honesty¶

Capability	Status	Notes
Customer-hosted deployment - full stack in customer VPC	Verified	Helm + Terraform, packaging delivered and CI-gated.
Live `helm install` / `terraform apply` validation on a real cluster/cloud account	Verified	From-empty EKS, AKS, and GKE installs with fail-first signature verification, plus Azure and GCP `terraform apply`, live-attested against the published release artifacts. CI gates `helm lint`/`template` and `terraform fmt`/`validate` for all three clouds, plus a compose smoke of the production images, on every change to those surfaces; `terraform plan`/`apply` are live-attested in the committed install transcripts, not run in CI.
Vendor-neutral connector contract (`BaseConnector`) across all listed engines	Verified	Single ABC; per-engine verification depth tracked in the cross-engine hash verification record.
Native cross-engine hash pushdown	Verified per pair	Every native-pushdown engine pair in the 66-pair cross-engine matrix is byte-verified, and the full matrix has additionally been run end-to-end to native clean passes - see the cross-engine hash verification record and the accuracy proof doc for per-pair depth and the SUM-only engine caveats.
Replication adapters - Debezium, Airbyte, Fivetran, Oracle GoldenGate, Qlik Talend Cloud	Verified	Tier-1 production adapters. Fivetran live-verified against a real account: adapter read-path plus reconciliation end-to-end. GoldenGate live-verified on a real GG Free 26ai replicating pair: three-verdict profiled e2e (pass_clean / no_new_data / fail_count), MA REST and classic_heartbeat modes. Qlik Talend Cloud live-verified on a live trial tenant (RDS PostgreSQL → Snowflake): the same three-verdict profiled e2e, CDC probe consulted on every run.
Replication adapters - SLT, BODS, Qlik Replicate, Informatica CDC	Roadmap	Design-partner-pending stubs; contract defined, not production-validated.
Throughput floors (≥200K rows/sec proof pushdown; ≥25-50K rows/sec key extraction)	Supported	Binding design commitments enforced as merge gates; presented as engineering rules, not published benchmark results.
Published benchmark sizing matrix (1M-10B rows, per engine pair)	Partially measured	Four flagship pairs are measured on the reference box (see the performance proof doc): PostgreSQL → Snowflake, PostgreSQL → Databricks, and Db2 LUW → BigQuery at 1M / 10M / 100M rows, and Oracle → Snowflake at 1M / 10M (100M is out of scope on the disclosed free-edition Oracle cap). All 1B and 10B cells remain deliberately unmeasured for every pair.
Declared SCD2 scope for history-bearing tables	Verified - PostgreSQL and Snowflake	Scope rendering is live byte-verified on those two engines; every other engine rejects the declaration rather than applying it on faith.
Tenant isolation - PostgreSQL RLS as sole enforcement	Roadmap	RLS live as defense-in-depth; `FORCE ROW LEVEL SECURITY` deferred pending call-site audit (see Security Whitepaper).
Security certifications (SOC 2 / ISO 27001), independent penetration test	Roadmap	Covered honestly in the Security Whitepaper - no certification or third-party pentest exists today.
Cost & Usage - the cloud cost a comparison incurs	Scan volume measured on eleven engines; stated as Not measured, never estimated, elsewhere	See the statement below.

What Paridity can and cannot tell you about cost¶

Paridity reports the additional cloud cost you incur because it ran a comparison - never a total cost of operation, and never a guess. What it can say, and the limits it states on the page:

Scan volume is measured, not modelled, on eleven engines - Snowflake, Databricks, BigQuery, file datasets, Qlik Cloud, Trino/Hive and ClickHouse, whose drivers or servers return bytes scanned per statement (ClickHouse on the proof statements; a streamed key extraction there is recorded as unmeasured, never as zero), plus Oracle, SQL Server and Redshift (#2456) and Db2 LUW (#2457), where Paridity reads the figure from the engine's own catalogue immediately after each proof statement. Those four bill by instance/RPU/cluster hours or per core, so their bytes are a calibration denominator rather than a vendor unit. On every other engine the page says Not measured and states how many mappings its figures cover; it never renders an unmeasured estate as $0.00 or ranks one it cannot see. PostgreSQL is a deliberate Not measured: every per-statement source it offers is either aggregated across connections, instance-wide, or costs a second execution of the proof query - and no PostgreSQL estate bills per byte (Aurora bills I/O requests, which are not bytes).
Every dollar is your rate × recorded bytes, frozen at run time. A charge appears only for a system you gave a rate to (scheduled change → system → tenant default → otherwise $0). Each run's cost is written as append-only evidence at the rates in effect when it started; changing a rate never rewrites history, and the page footnotes the runs that are not frozen.
Egress is billed per side - source bytes at the source system's rate, target bytes at the target's. The suggested egress rate is 0 - configure one only if your provider bills you for result traffic, that is, your topology crosses regions or clouds. On the reference Snowflake and Databricks estates neither vendor recorded a transfer charge for in-region query results.
A $/TB rate is only true for the cadence it was measured at. Warehouse-billed engines charge for uptime; at the reference rigs' short-run cadence the measured effective rate was tens of times the list ballpark. Paridity ships a calibration procedure that derives your rate from your own bill, and does not publish a portable rate. Every rate is stored with its calibration provenance - the date, cadence, rows per run, bill source and attributable share of the window it was measured in - shown beside the number, and is flagged stale when the observed cadence has drifted more than 2× from the calibrated one.
"Last 30 days" is the cost incurred in the trailing 30 days - a measured total, not a forecast. Paridity does not project spend.

Appendix¶

Paridity Security Whitepaper - the security companion: access control, secrets, data protection, supply chain, DR.
How Paridity guarantees accuracy - the accuracy claim, verification method, and per-engine status.
Paridity glossary - canonical terminology (hash_total, row_hash, field_diff, config_checksum, Mapping, Replication Profile, watermark).
Supporting internal references - the deployment split-readiness model and data-classification manifest, the three state-machine references, the BaseConnector connector contract, the cross-engine canonicalization and hashing algorithm, the per-engine hash verification record and open accuracy issues, the hardware baseline and reproducible benchmark harness, and the full dependency/license inventory - are available to reviewers on request through your Paridity account contact.
Follow-up questions: route through your Paridity account contact or the Trust Center (in progress).

How Paridity compares to the alternatives¶

How to read this document. Every statement about Paridity is cited to one of the proof documents (accuracy, architecture brief, audit and compliance, performance, verify it yourself) or to the public connector compatibility matrix - each of which names the code, migration, test, or live transcript behind it. Every statement about another vendor is limited to that vendor's own public documentation, re-verified against current official sources on 2026-10-27 (no hands-on testing), and "not publicly evidenced" means the reviewed documentation did not establish a capability, not that the capability is absent. Anything Paridity does not do today is labelled Roadmap.

The honest starting point¶

Paridity is not the only product that compares a source dataset to a target dataset. Replication vendors bundle validation with their pipelines, observability platforms offer reconciliation policies, diff tools compare tables across databases, and every data team has written a COUNT(*) check at some point. Several of these products are mature and genuinely good at what they do - this document says so where it is true, because an evaluation built on strawmen produces a decision you cannot defend later.

What this document does claim is narrower and more useful: the alternatives belong to different categories, each built to answer a different question, and the right way to evaluate any of them - Paridity included - is to decide which question your organization actually needs answered, then test the candidates against it. Paridity's category is replication-correctness assurance: proving, on a recurring basis and with retained evidence, that a copied dataset matches its source inside a defensible comparison window, independently of whichever tool moved the data (architecture brief, Problem and Approach).

Paridity runs inside your own cloud account. Your data, credentials, and evidence stay inside your VPC - nothing described in this document sends them to Paridity-operated infrastructure, and Paridity operates no part of the deployment (architecture brief, Deployment model). The one outbound path to Paridity is the licence service, used at licence activation and - once the licence release is in force - for a daily licence refresh; it carries the licence key, a machine id derived from the installation, the product name, and the platform and architecture, never a credential, an object or column name, or an evidence value; the licence service records the request's source address (licensing guide, deployment models).

Four structural properties define the category, and they are the recurring test throughout this document:

CDC-aware window alignment - the comparison window is derived from replication state (watermarks, CDC offsets, control markers, or explicit snapshots), not from a clock or an operator's guess about replication latency, so in-flight changes are not reported as differences (architecture brief, CDC-aware windowing).
A deterministic proof-then-diff pipeline - a whole-dataset aggregate fingerprint (hash_total) decides whether the datasets match before any rows move; per-row fingerprints (row_hash) isolate which rows differ; column-by-column comparison (field_diff) records what differs. The expensive phases only run when something is wrong (accuracy proof, How it works).
Append-only audit evidence - configuration snapshots, phase timings, proof results, and mismatch evidence are retained immutably and can be exported as signed, hash-chained bundles verifiable offline, without trusting the platform's own dashboards (audit and compliance proof).
Vendor neutrality - the same comparison logic, canonicalization rules, and evidence model apply across every source/target pair and every replication tool, so one control covers the whole estate (architecture brief, Vendor-neutral engine model; cross-engine byte identity is verified on all 66 pairs of the 12 SQL engines, accuracy proof, The evidence).

The categories of alternatives¶

Replication-vendor validation add-ons¶

What they are good at. Several replication and migration platforms ship a validation capability alongside data movement. The strongest of these are mature products in their own right: they compare source and target, account for data still in flight by rechecking provisional mismatches, produce row-level difference detail, and - a capability Paridity deliberately does not offer - can repair the target. If your estate runs entirely on one replication product and your control owner accepts validation from the same vendor that moves the data, these tools can be a practical answer, and they may already be in your contract.

Where the category structurally differs. Two questions decide whether a bundled validator fits your requirement. First, coverage: a validator built into a replication platform validates what that platform moves. Most enterprises run more than one movement mechanism - a CDC product here, a SaaS ELT tool there, custom jobs elsewhere - and every pipeline outside the platform is outside the validator. Second, independence: with a bundled validator, the vendor that moves the data also attests that the movement was correct. Many audit and separation-of-duties frameworks prefer the checker to be independent of the thing being checked. Neither point is a criticism of the tools; both are properties of the architecture, and both are things your control owner - not the vendor - should rule on.

Data-quality and observability monitors¶

What they are good at. Observability platforms watch pipelines and datasets for anomalies - freshness, volume, distribution shifts, rule violations - across a broad estate with relatively little setup. Some also offer reconciliation as a policy or metric within the platform: row counts, key and column checks, configurable match thresholds, scheduling, and APIs. For teams that already run one of these platforms, adding a reconciliation policy is a low-friction expansion, and for many monitoring purposes it is enough.

Where the category structurally differs. Observability answers "does this data look healthy?" - a statistical question, usually gated by a configurable pass threshold (for example, a rule that passes once a configurable score or match percentage is reached). Replication-correctness assurance answers "does this copy match its source?" - a deterministic question where the interesting answer is precisely the rows inside the remaining 1%. Related structural differences to probe: whether the comparison window is derived from replication state or from a timestamp or increment column - and if so, how a row modified without advancing that column enters a comparison window; and whether the result is retained as immutable, exportable evidence or as an operational alert that ages out. Ask each vendor how their reconciliation handles these - the answers vary by product and by edition, and they are checkable in each vendor's documentation.

Point diff tools¶

What they are good at. Cross-database diff tools built for engineering workflows are excellent at what they were designed for: showing a developer exactly what changed between two tables or between two versions of a transformation, integrated into CI so a code change cannot silently alter data. The best of them have strong developer ergonomics, value-level diffs across engines, column remapping, sampling controls, and APIs. For migration cutover validation and CI-style regression testing, they are often the most natural tool in this document.

Where the category structurally differs. A diff tool answers "what is different between these two tables right now?" - typically on demand, for an engineer, with engineer-oriented output. A recurring production control needs three things a point diff is not built around: a comparison window aligned to replication state so that live CDC traffic does not produce false differences; a governed run lifecycle (schedules, readiness checks, in-flight rechecks, configuration versioning) rather than an invocation; and evidence designed for a control owner or auditor to review months later, not a diff for an engineer to act on today. If your requirement is a one-time migration or CI validation, a diff tool may fit well; if it is a recurring attestation, ask how the diff becomes a control record.

ETL-testing and data-testing suites¶

What they are good at. Enterprise data-testing platforms are mature, broadly connected, and strong at what QA organizations need: authoring and reusing test cases, scheduling suites, integrating with CI/CD, testing BI reports, and producing test reports. Several deploy in the customer's environment and publish their licensing basis. For a testing organization validating transformations and warehouse loads, they are a proven choice.

Where the category structurally differs. A test suite is authored: someone writes the query pairs, maintains them as schemas change, and decides what "pass" means per test. A reconciliation control is configured once per mapping and then governed: canonicalization across engines is the platform's job, not the test author's; schema drift is detected rather than discovered as a broken test; and the comparison window is aligned to replication state rather than to the test schedule. The practical questions are maintenance cost per mapping at estate scale, how much data must move to the testing server for each comparison, and how a test result becomes evidence a non-QA control owner can rely on for a prior period.

Hand-rolled SQL, dbt checks, and internal scripts¶

What they are good at. For one table, one migration, or one team, a hand-written comparison is fast to build, free of license cost, and perfectly transparent - you can read every line. Most organizations start here, and for a bounded, one-time question it is often the right call.

Where the category structurally differs. The build decision changes when the requirement becomes hundreds of mappings, heterogeneous engine pairs, active CDC, and audit evidence. The hard problems are exactly the invisible ones: making an Oracle NUMBER and a Snowflake NUMBER hash identically (canonicalization across null handling, numeric scale, timestamp precision, time zones, string trimming, and binary values); avoiding false mismatches while replication is in flight; reproducing precisely which script and which configuration ran for a period twelve months ago; and keeping all of it working as schemas drift and the original author moves on. Honest questions for a build option: who owns the scripts, what is the annual engineering and database-compute cost, and can the team hand an auditor evidence rather than a query?

Category comparison¶

Directional, category-level comparison - individual products within a category vary, and the honest evaluation method is to verify each cell against the specific product and edition you are quoted. "Varies" means exactly that: some products in the category have it, and the vendor's public documentation will tell you whether yours does. "Not assessed" means our review of public vendor documentation did not examine that capability for the category at all - ask the vendor, and do not read it as a gap. The Paridity column is proven row by row in the list that follows the table.

Capability	Paridity	Replication-vendor validation	Observability monitors	Point diff tools	Data-testing suites	Hand-rolled SQL
Independent of the tool that moves the data	Yes	Typically no - the validator is the movement vendor's product; some in the named list run without that vendor's replication component (see Named alternatives)	Yes	Yes	Yes	Yes
One control across multiple replication tools	Yes - vendor-neutral by design; native replication-state adapters for five tools today, generic watermarks for the rest, further native adapters Roadmap	Typically limited to the vendor's pipelines; verify per product	Varies	Yes	Yes, with per-pair authoring	Yes, with per-pair effort
Comparison window derived from replication state	Yes - watermarks, CDC probes, control markers, snapshots	Typically an in-channel recheck after a latency period, or the channel's own transaction state - verify the mechanism	Typically timestamp / increment-column based	Typically point-in-time or filter-based	Not publicly evidenced as replication-state aware in the products reviewed	Manual
Whole-dataset proof before any row-level extraction	Yes - `hash_total` proof gate; rows move only when it fails	Some do (a hash or checksum phase before row detail) - verify the mechanism	Varies - often threshold-gated policy checks	Varies	Typically query-result comparison	Typically counts / ad-hoc checksums
Documented cross-engine canonicalization (nulls, numerics, timestamps, binary)	Yes - published rules, byte-verified across engine pairs	Varies - verify against the pairs you need	Varies	Varies	Author's responsibility per test	Author's responsibility
Field-level difference evidence on mismatched rows	Yes - `field_diff` records, subject to masking policy	Common	Varies	Yes	Yes	Author-built
Append-only evidence, exportable and verifiable offline	Yes - signed, hash-chained bundles with a stdlib-only offline verifier; with KMS/ECDSA signing a third party verifies with the public key alone, holding no shared secret	Reports and run history; cryptographic export not publicly evidenced in the products reviewed	Not assessed - alerts and dashboards are the usual surface	Engineering-oriented run history and reports	QA reports and run history	Rarely
Evidence retained on write-once storage, under an enforced retention schedule, with legal hold	Yes - period bundles anchored to S3 Object Lock / Azure Blob immutability / GCS retention lock and verified by full read-back; a built-in, audited purge schedule; approval-gated legal holds	Not assessed	Not assessed	Not assessed	Not assessed	Author-built
Slowly-changing / bitemporal history tables compared without silently excluding rows	Yes - an operator-declared, versioned current-row scope (PostgreSQL and Snowflake today; other engines reject the declaration rather than apply it on faith), never inferred; the run is badged as scoped so a scoped pass never reads as a whole-table pass, and the scope's own premise is re-checked every run	Not assessed	Not assessed	Not assessed	Not assessed	Author's responsibility
Schema drift as a first-class control	Yes - detected, classified, and a run with reduced coverage is never presented as a clean pass	Not assessed	Not assessed	Not assessed	Not assessed	Manual
Target repair / write-back	No - deliberately read-only (see below)	Common	Not documented in the products reviewed	Not documented in the products reviewed	Not documented in the products reviewed	Author-built

On the write-once row, one qualification belongs in the open: it is the store's lock that makes the copy write-once, not Paridity's code. What Paridity does is attest it only when the store confirms protection, and verify it by reading the object back - an unprotected bucket is reported unprotected, never as an anchor.

Where each Paridity row is proven. Each entry names the proof document and section; those documents in turn cite the code path, migration, test, or live transcript.

Independent of the tool; one control across tools - the connector contract and engine model are in the architecture brief, Vendor-neutral engine model. The connector compatibility matrix, Replication tools lists the five replication-tool adapters in production today: Debezium, Airbyte, Fivetran, Oracle GoldenGate, Qlik Talend Cloud. The same table marks the four Roadmap adapters, which are not yet operational: a profile can declare them, but every adapter call answers 501 DESIGN_PARTNER_PENDING, so they give no replication-state readiness today - SAP SLT, SAP BODS, Qlik Replicate, Informatica PowerExchange CDC. A pipeline with no tool, or with a tool not yet natively integrated, is reconciled on generic watermarks or an explicit snapshot window, which is not equivalent to reading that tool's native checkpoint state.
Comparison window from replication state - readiness state machine, watermark preference order, Snowflake read pin, and cdc_table_semantics in the architecture brief, CDC-aware windowing.
Whole-dataset proof before extraction; canonicalization; field-level evidence - accuracy proof, How it works and The evidence: the hash_total / row_hash / field_diff pipeline, the published canonicalization rules, and the 66-pair cross-engine byte-identity verification across 12 SQL engines. field_diff values are shown subject to the masking policy, as that document notes.
Append-only evidence, offline-verifiable bundles - audit and compliance proof, The evidence: the append-only rule and how it is enforced, the detached-signature bundle format, the stdlib-only public-key verifier, and the live credential-less verification against a KMS key.
Write-once storage, purge schedule, legal hold - the same document's evidence table rows for bundles anchored to write-once storage, the built-in retention-purge schedule, and legal holds, each live-verified against real S3, Azure Blob, and GCS stores.
Slowly-changing / bitemporal scope - accuracy proof, Limitations (row "Mappings with a declared SCD2 scope") and architecture brief, Declared scope for history-bearing tables.
Schema drift - detection is part of the evidence model (architecture brief, Four schemas); a run whose schema could not be verified never reads as an unqualified pass (accuracy proof, Limitations). The reduced-coverage rule is a code claim: src/models/api/display_status.py reclassifies a pass_clean run that dropped a reconciled column to Schema drift, and the policy that decides halt versus continue is resolved in src/services/config_resolver.py::_resolve_schema_drift (docs/specs/spec_schema_drift.md).
Read-only - connectors execute reads and return data; no connector writes to a source or target system, and no repair feature exists (architecture brief, Vendor-neutral engine model).

Two rows deserve emphasis because they cut against Paridity, and leaving them out would make the table advertising rather than evaluation:

Repair. Several alternatives can fix the target after finding a difference - generating or executing corrective SQL. Paridity detects and pinpoints divergence but does not write to your source or target systems. This is a deliberate separation-of-duties posture - the checker does not modify what it checks - but if automatic repair is a mandatory requirement, a replication-vendor validator may fit your workflow better, and you should weigh that openly.
Fit. If your entire estate runs through a single replication product whose bundled validation your control owner accepts, or if your requirement is CI-style testing rather than a recurring production control, Paridity is not automatically the best answer. It is strongest in heterogeneous estates - mixed engines, multiple movement tools, regulated controls - where one independent, evidence-producing control layer is the requirement.

Named alternatives¶

The vendors below are the ones evaluators ask about most often. Each paragraph is limited to what the vendor's own public documentation supported when we re-verified it against current official sources on 2026-10-27 (no hands-on testing); treat it as orientation, and verify current capabilities against the vendor directly.

Oracle GoldenGate Veridata is a mature, capable database-comparison product with a long enterprise track record. It uses an initial hash-based comparison phase followed by a recheck after a configurable latency period to distinguish in-flight rows from persistent differences, supports heterogeneous comparison and delta processing, offers REST operations, and can repair the target. Oracle's documentation positions it for any replication scenario and does not state a dependency on GoldenGate replication - confirm with Oracle for a non-GoldenGate estate. Its documented targets now include Snowflake, Databricks, MongoDB and SingleStore alongside relational databases. Questions worth asking in an evaluation: how the in-flight delay is determined (an operator-configured setting versus a value derived from replication state), which hash algorithm is used and how it is documented, and in what format difference evidence is retained and reviewed over time.

Fivetran HVR Compare is among the closest technical alternatives to Paridity inside an HVR estate: checksums calculated inside the locations with compact results transferred, row-by-row comparison that identifies inserts, updates, and deletes, an online compare that accounts for changing data, and target refresh/repair. If every relevant pipeline runs through HVR, it is a serious option. The evaluation questions are the category ones: how pipelines that HVR does not move are certified under the same control, and whether your control framework accepts the replication vendor as the assessor of its own replication. Note that HVR is a distinct product surface from the Fivetran SaaS ELT service, priced on its own consumption basis.

Striim Validata is a standalone reconciliation product that works with Striim, other replication tools, and custom pipelines - the closest positioning match to Paridity in this document. It is offered self-managed (Validata Platform) and as a managed SaaS (Validata Cloud, generally available April 2026). It offers several validation methods, rechecks provisional mismatches after a configurable wait, retains reports and run history, and generates target-side repair SQL. To its credit, Striim publishes its datatype comparison rules - how values are normalized before comparison, including how differing numeric scales, float precision, time zone offsets, and null or duplicate key columns are handled. Read those published rules as part of your evaluation and decide whether each normalization trade-off is acceptable for your control: the central question for any reconciliation product is which genuinely different values its normalization will treat as equal, and which rows are excluded from the comparison rather than reported.

AWS DMS Data Validation is built into AWS Database Migration Service: it compares source and target rows, continues validating as CDC changes are applied, reports mismatch state, and offers data resynchronization (documented for Oracle and SQL Server sources to PostgreSQL / Aurora PostgreSQL targets, DMS engine 3.6.1 and later). For supported DMS migration paths it is a practical, included capability. Its scope is DMS endpoints and tasks; evaluation questions are how pipelines outside DMS are covered, how its documented endpoint-specific rules and limitations map to your tables, and how its control tables, metrics, and logs become retained audit evidence.

Datafold provides cross-database data diffing with strong developer workflow integration - CI, APIs, column remapping, filtering and sampling - and is particularly strong in migration validation and developer-led teams. If the requirement is engineering-led migration or code-change validation, it is a natural candidate. If the requirement is a recurring production control, the questions are the point-diff category questions: window alignment under active CDC (its public migration guidance is primarily timestamp or filter based), run governance, and evidence for control owners.

QuerySurge is a mature enterprise data-testing platform: broad connectivity, query-based and visual (no-SQL) test authoring, scheduling, CI/CD integration, BI testing, and customer-managed deployment with transparent user-based licensing; QuerySurge 15 (October 2026) adds Mapping Studio and Validation Lineage. For QA organizations it is well proven. The evaluation questions are the testing-suite category questions: per-mapping authoring and maintenance effort at estate scale, data movement to the comparison server, behavior under active replication, and turning test results into a control record.

Questions to ask any vendor - including us¶

These encode the structural differences above as neutral evaluation criteria. They are fair questions for Paridity too, and the proof documents linked in the next section are our answers.

Window alignment. How does the product establish that source and target are comparable at the moment of comparison? Is the mechanism derived from replication state, or is it a configured wait? What happens to rows that change mid-comparison?
Every-row accounting. How are rows with null comparison keys handled? Duplicate keys? Are such rows reported, or excluded from the comparison - and if excluded, where does the report say so?
Normalization trade-offs. When source and target represent the same value differently (numeric scale, timestamp precision, time zones, trailing spaces, character encodings, binary), what are the documented comparison rules - and which genuinely different values would those rules treat as equal?
Proof cost on a clean run. When the datasets match, how much data crossed the network to establish that? Is the comparison computed inside your engines, or shipped to the vendor's comparison server?
Evidence. What artifact survives after the run - and after the operational data changes? Can an auditor verify it without trusting the product's own UI? Is it tamper-evident? Can you export it, and can you verify the export offline without a secret the vendor also holds? Is the retained copy on storage that refuses deletion - and can the product prove that by reading it back, rather than asserting it? Can you freeze a period against retention deletion while a matter is open, and is that freeze itself audited?
Configuration proof. For a run executed nine months ago, can the product show exactly which configuration governed it - and prove that configuration was recorded before the comparison began?
Schema change. If a column is dropped, renamed, or retyped between runs, does the product detect it - and can a comparison that silently lost a column still report a clean pass?
Repair authority. If the product can write to the target, who authorizes it, how is it audited, and is that compatible with your separation-of-duties policy? If it cannot, is read-only acceptable for your workflow?
Determinism. Is the match decision a deterministic computation, or does any part of it involve thresholds, sampling defaults, or inference? (In Paridity, AI never determines correctness - it assists triage and explanation of already-computed results only; accuracy proof, How it works.)
Cost basis. What is the licensing meter, what does each run consume in source load, target compute, and egress, and are development, test, and DR environments licensed separately?

Paridity's own answers: questions 1 and 7 are in the architecture brief (CDC-aware windowing; Reliability) and the accuracy proof; questions 2, 3 and 9 in the accuracy proof (How it works; Threat model; Limitations); question 4 in the performance proof (egress avoided by narrowing, and the volume and egress tracking it points to); questions 5 and 6 in the audit and compliance proof; question 8 is answered above (read-only, no repair); the licensing meter in question 10 is in the licensing guide - ask your Paridity contact about non-production and DR environments, which that guide does not address.

How to verify, not take our word¶

This document has made claims about Paridity while urging you to verify claims about everyone. The same standard applies to us - and the verification is designed to be run by you, in your environment, without trusting our dashboards:

Accuracy proof - the deterministic hash_total / row_hash / field_diff claim, the per-engine verification status, the stated threat model, and its limitations. Includes forcing a known single-cell mismatch and watching it caught, plus offline recomputation of the shipped hash test vectors.
Performance proof - what has actually been measured, on named hardware at named row counts (four flagship cross-engine pairs measured end-to-end, three of them to 100 million rows), what has not been measured, and why we do not publish extrapolations.
Audit & compliance proof - the append-only evidence model, hash-chained audit log, signed evidence bundles, and the offline verification tool, each labeled Verified or Supported.
Verify it yourself - the consolidated proof-of-value walkthrough: accuracy, audit integrity, speed and scale, and provenance and supply chain, each independently reproducible.
Connector compatibility matrix - which engines and replication tools are supported today, at what verification depth, with roadmap items labeled as roadmap.

The most useful competitive exercise is not reading this document - it is a proof of concept that runs the same tests against every candidate: active writes during comparison, null and duplicate keys, decimal scale and time zone edge cases, a schema change between two runs, a deliberately tampered evidence export, and a large-table run with the transferred bytes measured. Any product in this document should welcome that test. We do.

Verify Paridity yourself - a proof-of-value guide¶

The claim¶

You don't have to take our word for any of it. Paridity's core claims - accuracy (every byte-level difference detected, no false passes on Verified engine pairs, scoped to the threat model the accuracy proof states explicitly: replication faults and corruption, not a privileged insider deliberately tampering with both compared datasets), audit integrity (a complete, tamper-evident trail you can check without trusting our dashboards), speed (per-phase timings recorded for every run), and provenance (the images you run are exactly the ones we built and signed) - are each independently reproducible by you, in your own environment. This guide consolidates the concrete verification steps from the companion proof documents into one proof-of-value walkthrough.

This is Paridity's own thesis - prove it, don't assure it - applied to the evaluation process itself. The detailed claims, evidence citations, and limitations live in the companion documents; this guide tells you how to test them:

Accuracy proof - the hash_total / row_hash / field_diff correctness claim, per-engine verification status.
Audit & compliance proof - the append-only, hash-chained, offline-verifiable audit trail claim.

How it works (deterministic)¶

Everything you will verify below is deterministic computation, not inference - which is precisely why it is independently reproducible:

Proof phase - hash_total. Paridity computes an aggregate dataset fingerprint (hash_total) independently on source and target within a deterministically aligned time window, and compares the two values. Identical data produces identical fingerprints; a single changed byte does not.
Diff phase - row_hash, then field_diff. If the hash_total values differ, Paridity computes a per-row SHA-256 row_hash on both sides to isolate exactly which rows differ, then compares those rows column-by-column to produce the field_diff evidence - the exact row, column, and source/target values (subject to masking policy).
Audit trail - deterministic writes at defined points. The config_checksum (SHA-256 of the canonicalized resolved config) is written before any comparison begins; every phase writes timestamps and a worker_id to the run's per-phase attempt records; evidence is append-only; and the trail can be exported as a hash-chained, signed evidence bundle verifiable offline.

Because every one of these is a defined algorithm over defined inputs - never a judgment call - you can recompute the same values yourself and check that they match what Paridity recorded.

Note: AI never determines correctness in Paridity. Correctness is always established deterministically - by aligned time windows, hash_total aggregate fingerprints, row_hash per-row fingerprints, and field_diff column comparisons. AI assists triage and explanation of already-computed results only; it never decides whether data matches and never auto-confirms an operator action.

What you can verify¶

Claim	How you verify it	What proves it
Accuracy - a single changed byte is always detected, never silently passed	Force a known one-cell mismatch on the target and re-run the Mapping (Track A, steps 1-4); independently recompute the shipped hash test vectors offline (Track A, step 5)	The run reaches `fail_field` and the `field_diff` evidence identifies the exact row and column you changed; your own SHA-256 computation over the documented canonical string matches the shipped `expected_row_hash`
Audit integrity - the trail is complete and tamper-evident, verifiable without trusting the platform	Recompute a `config_checksum` from the resolved config snapshot; verify an evidence bundle offline with `tools/verify_bundle.py`; walk the bundle hash chain; verify the audit-log hash chain; trace the per-phase attempt records (Track B)	Your independently computed SHA-256 matches the stored `config_checksum`; the bundle's `content_hash` and signature verify offline; each bundle's `previous_bundle_hash` equals the prior bundle's `content_hash`; the chain-verification endpoint reports the audit log intact; every phase has timestamps and a `worker_id`; and with asymmetric (KMS/ECDSA) signing the signature verifies against a public key you obtained from the data owner's KMS out of band - no shared secret, and nothing of ours to trust
Speed - run performance is measured and recorded per phase, not asserted	Run a Mapping over a dataset of ≥50,000 rows you control and read the proof and key-extraction phase durations from the per-phase attempt records (Track C)	Recorded `started_at`/`completed_at` per phase are the platform's own source of truth for run latency - you compute the throughput yourself from your own row counts and the recorded timings
Provenance - the images you run are exactly the ones Paridity built and signed	`cosign verify` each released image and the Helm chart by digest against its keyless Sigstore identity; pull and scan the per-image SPDX SBOM; read the three committed cloud install transcripts (Track D)	The signature validates against the public Rekor transparency log for the exact build-workflow identity - no shared key to trust; the SBOM enumerates the shipped artifact, not a description of it; each install transcript reproduces a from-empty, signature-verified install with a live pass and a live fail

Verify it yourself¶

The four tracks below are independent - run any or all of them. Tracks A (steps 1-4) and C require a live source and target you're willing to write a test value into; most evaluators do this with an SE in a sandboxed environment rather than a production system. Track A step 5, Track B step 3's offline verification, and Track D's signature and SBOM checks require no running Paridity application at all - only the released artifacts and a read-only registry pull token.

Track A - Accuracy¶

Reproduces the accuracy claim: a forced mismatch is always detected and pinpointed. (Source: accuracy proof, "Verify it yourself".)

Create a Mapping between two engines from the accuracy proof's Verified list (e.g. PostgreSQL → Snowflake), pointed at a dataset you control.
Run it once cleanly. Confirm the run reaches pass_clean and that the Run detail view shows a native-pushdown proof (not a DuckDB fallback, unless one of the two engines requires it - see the accuracy proof's Limitations table).
Force a known, single-cell mismatch on the target - change one value in one row directly in the target system.
Run the Mapping again. Confirm:
The proof phase's hash_total values now differ between source and target (visible on the Run detail page).
The run's status reaches Issue detected - terminal state fail_field, because you changed a value (row counts still match, so the proof hash_total fails, then the diff phase isolates the field). Had you instead added or removed a row, it would reach fail_count. Either way - never a silent pass.
The Evidence tab's field_diff record identifies the exact row and column you changed, with the source and target values shown side by side (subject to your masking configuration).
Run the shipped test vectors independently of the running application. tests/hashing/test_vectors.json contains the canonical row-hash vectors (tv001-tv051) and hash_total vectors (ht001-ht010) that gate every engine implementation before it can be declared production-ready. These are plain input/expected-output pairs - you do not need to trust Paridity's own test runner to check them; you can independently compute SHA-256 over the documented canonical string for any vector and confirm it matches expected_row_hash. This step can be done offline against the JSON file alone.

Track B - Audit integrity¶

Reproduces the audit-trail claim: complete, append-only, tamper-evident, and verifiable without the platform. (Source: audit & compliance proof, "Verify it yourself".)

Recompute a config_checksum. Pull a run's resolved config snapshot (linked from the run's config-version reference), apply the documented canonicalization rules (strip spec_version/generated_at/ generated_by, sort all object keys and the documented arrays, serialize with no whitespace, UTF-8 encode), take SHA-256, and confirm it matches the config_checksum stored on the run's config-audit evidence record. This proves the policy in force was recorded before the run touched any data - and that you can check it without trusting Paridity's serialization.
Confirm append-only evidence. As an auditor or platform_admin, attempt an UPDATE or DELETE against any evidence table other than the two documented lifecycle exceptions, from an ordinary application code path (not a database console) - there is no API surface that permits it. Ask your Paridity contact to walk through the retention-purge routine if you want to see the one sanctioned deletion path in code.
Verify an evidence bundle offline. Generate a bundle for a run (POST /runs/{execution_request_id}/evidence-bundle - the run is addressed by its execution request id), download it, and run the shipped verifier. With a KMS or ECDSA signing key configured, anyone can verify with the public key alone - obtain it out of band from the data owner's KMS (aws kms get-public-key, az keyvault key download, gcloud kms keys versions get-public-key) and run python tools/verify_bundle.py --bundle <file> --public-key <pem>, or pin the copy embedded in the bundle with --public-key-fingerprint <sha256>; the verifier never trusts an embedded key unpinned. Public-key mode is stdlib-only - no packages to install. With an HMAC key, run --key-from-env HMAC_KEY with the tenant's key from its own secrets backend. The platform's POST /evidence-bundles/{bundle_id}/verify endpoint is the server-side complement; the offline tool is the one that does not require trusting the platform, and the out-of-band public key is the one that does not require trusting the bundle.
Walk the bundle hash chain. Generate two sequential bundles for the same Mapping and confirm the second bundle's previous_bundle_hash equals the first bundle's content_hash.
Verify the audit-log hash chain. Call GET /audit/verify (or the equivalent chain-verification endpoint) and confirm it reports the chain intact; if automatic sealing is enabled, confirm the signature on the latest checkpoint validates against the configured signing key/public key.
Trace a run's phase timings. Pull the per-phase attempt records for a run_id and confirm there is a row for each phase the run went through (readiness, proof_count and proof_hash, and - on a failed proof - key_extraction, field_diff, lag_retry), each with started_at, completed_at, and worker_id populated.
Trace a terminal status to its audit entry. Pick a run that reached a terminal status and confirm the audit log has a matching entry with actor, action, entity, and timestamp - and, for an admin-triggered cancellation or a bypass_readiness run, that a reason is present.

Track C - Speed and scale¶

Paridity records how long every phase of every run took - so the speed evidence is your own environment's timings, not a vendor benchmark slide.

Run a Mapping over a dataset of at least 50,000 rows you control, on engines from the accuracy proof's Verified list, and let it complete (pass_clean, or a fail_* state if you combine this with Track A).
Read the phase durations from the per-phase attempt records for that run_id (the same query as Track B, step 6): the proof_count and proof_hash rows show how long the count and hash_total proof took, and - on a failed proof - the key_extraction and field_diff rows show the diff-phase timings. started_at and completed_at are written by the platform for every phase of every run; these recorded timings, not ad-hoc stopwatch numbers, are the source of truth.
Compute throughput yourself from your own row count and the recorded phase durations. On Verified engine pairs the proof phase is pushed down into the engines as SQL aggregation - the rows are fingerprinted where they live rather than pulled across the network - so proof-phase duration is dominated by your engines' aggregation speed, not by data egress.
Interpret the numbers honestly. Paridity engineering holds internal per-worker throughput floors as design targets, and this document deliberately does not quote a guaranteed rows/sec figure: observed throughput depends on your engines, warehouse sizing, network, and whether the run used native pushdown or the DuckDB reference fallback (visible on the Run detail page, and slower by design - see the accuracy proof's Limitations). What you can verify self-serve is that every phase's timing is recorded and attributable in your environment - and you can re-run the same Mapping on the same dataset to check the timings for yourself rather than relying on a one-off number. For sizing a production deployment against your row volumes, ask your Paridity contact for an SE-led benchmark against a dataset shaped like yours.

Track D - Provenance and supply chain¶

Reproduces the supply-chain claim: the images you run are exactly the ones Paridity built and signed, and you can prove it without trusting Paridity's build system. (Source: the digests-vX.Y.Z.{md,json} assets on each GitHub Release; further supporting material available to reviewers on request through your Paridity account contact.)

Verify the release signatures yourself - keyless, against a public transparency log. Paridity signs every released container image and the Helm chart with cosign keyless signing (Sigstore): each signature is bound to the GitHub Actions workflow identity that built the artifact and recorded in the public Rekor transparency log - there is no shared signing key to trust or to leak. After Paridity issues you a read-only registry pull token (the images are in a private registry - see the note below), verify each digest. The identity string, the image names and the exact digests are printed in the release's own digests-vX.Y.Z.md - copy them from there rather than from this page. Every release published before the repository rename on 2026-10-27 - v1.3.1 and earlier, including this v1.2.0 example - was built under the product's former name: the identity names EGMW2/DataAlign and the artifacts are ghcr.io/egmw2/dataalign-{api,ui,docs} and the chart ghcr.io/egmw2/charts/dataalign. Those signatures are permanent and verify exactly as published. The release workflow now builds under the new name, so a release published after the rename names EGMW2/Paridity in its identity and paridity-{api,ui,docs} / charts/paridity for its artifacts (at the time of this verification no such release had been published yet).
```
cosign verify \
  --certificate-identity 'https://github.com/EGMW2/DataAlign/.github/workflows/release-images.yml@refs/tags/v1.2.0' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  ghcr.io/egmw2/dataalign-api@sha256:<digest>   # repeat for ui / docs / chart
```
The digest for each artifact is published in that release's own digests-v1.2.0.{md,json} asset. A verify that passes proves the image was built by that exact workflow at that exact tag; change a byte and it fails.
Pull and read the SBOM. Each release ships a machine-generated SPDX-JSON SBOM per image (sbom-dataalign-{api,ui,docs}-v1.2.0.spdx.json for releases through v1.3.1; sbom-paridity-{api,ui,docs}-vX.Y.Z.spdx.json from the first post-rename release), generated by syft from the published digests - not a local rebuild - so it lists exactly what is inside the artifact you pull. Take it as a GitHub Release asset, or verify it as a cosign in-toto attestation bound to the same identity:
```
cosign verify-attestation --type spdxjson \
  --certificate-identity 'https://github.com/EGMW2/DataAlign/.github/workflows/release-images.yml@refs/tags/v1.2.0' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  ghcr.io/egmw2/dataalign-api@sha256:<digest>
```
Feed the SBOM into your own vulnerability scanner and license policy - you are auditing the shipped artifact, not a description of it.
Read the reproducible install proofs. Paridity publishes a committed, fail-first install transcript for each of the three managed Kubernetes clouds - EKS, AKS, and GKE. Each records the same sequence you would run: cosign-verify the signatures with the tamper and wrong-identity legs proven to fail first, mirror the images digest-preserved into your own cloud registry, terraform apply from empty, helm install the signed chart by digest, and a live reconciliation reaching both pass_clean and fail_count on native pushdown. Ask your Paridity contact for the transcripts and walk them against your own cluster.
Inspect the verification record behind these documents. Every proof document in this pack carries a last_verified date and cites the exact artifact behind each claim - the committed live-gate results (docs/live_gates/, including the 55/55 direct pairing-matrix run) and the dated benchmark artifacts. Paridity maintains a living verification ledger on a documented edition cadence with an explicit fail-first bar: a claim is not recorded as passing until it has been seen to go red against a planted defect. Its verified results are what these customer-facing documents render from. The provenance you can hold in your hand is the citation trail in these documents plus the committed artifacts they point to.

Important - the registry is private by design. Paridity's release images and chart are published to a private registry, so an anonymous cosign verify returns a 401. This is deliberate: customers mirror the signed images into their own registry and run from there. You verify with a read-only pull token Paridity issues you - and once you have read access the signature check is fully independent of Paridity, because it validates against Sigstore's public transparency log, not against anything Paridity operates. Note also that crane copy-style mirroring copies the image only; signatures and attestations do not travel with it, so verify against the source registry before you mirror.

Tip: An SE-led session can walk through all four tracks live against a sandbox Mapping - including a deliberately tampered evidence bundle so you can see offline verification fail as expected, which is the most convincing five minutes of the exercise.

Limitations¶

Be clear-eyed about what each track does and doesn't establish, and what you can run entirely on your own versus with an SE.

Scope	Status	Notes
Offline hash test-vector verification (Track A, step 5)	Verified	Fully self-serve, no Paridity environment needed - plain input/expected-output pairs in `tests/hashing/test_vectors.json`, checkable with any SHA-256 implementation.
Offline evidence-bundle verification (Track B, step 3)	Verified	`tools/verify_bundle.py` runs independently of the platform. Requires a bundle generated by a live deployment first.
Third-party bundle verification with the public key alone (Track B, step 3)	Verified	With KMS/ECDSA signing, verification needs only the public key - no Paridity credential, no shared secret, and no packages to install. Live-verified against a real AWS KMS key in a throwaway container holding no cloud credentials, with six can-fail legs each exiting non-zero for a named reason.
Write-once anchor read-back (Track B)	Verified	Where the tenant's bundle object store is a write-once store, the verify endpoint reads the anchored object back in full and compares its hash with the platform record. A disagreement, an absence, and an unreachable store are each reported as exactly that - never as a pass. Live-verified on S3 Object Lock, and on Azure Blob immutability and GCS retention lock.
Forced-mismatch accuracy test (Track A, steps 1-4)	Verified	Requires a live source and target you can write a test value into - typically an SE-led sandbox, not production. Applies to engine pairs marked Verified in the accuracy proof; see that document's Limitations table for per-engine verification depth (including SUM-only engines and DuckDB-fallback paths). What this track demonstrates is the accuracy proof's stated threat model - an accidental or fault-induced change is detected; it is not a claim about a construction-aware insider crafting compensating changes (accuracy proof, Threat model section). If the Mapping under test declares an SCD scope, this track proves that declared current-rows scope rather than the whole table - the run carries a "Scoped: SCD current rows" badge to say so.
`config_checksum` recomputation (Track B, step 1)	Verified	Requires read access to the platform database (or an SE pulling the snapshot for you) plus the published canonicalization rules.
Audit-log hash-chain verification (Track B, step 5)	Supported	The chain and verification endpoint are shipped; automatic sealing is on by default (#2729) - a deployment that opts out has a verifiable-on-demand chain, not a continuously advancing one. See the audit proof's Limitations for signing-key options (HMAC default, ECDSA-P256, and KMS-backed signing on AWS KMS, Azure Key Vault and GCP KMS - each live-verified against a real key in that cloud) and their status.
Phase-timing trace and speed measurement (Track C)	Supported	Per-phase attempt-record timings are recorded for every run. This guide quotes no guaranteed throughput figure - your observed numbers depend on your engines and infrastructure; internal throughput floors are engineering design targets, not contractual SLAs. SE-led benchmarking is available for production sizing.
Release signature verification (Track D, step 1)	Verified	cosign keyless signatures on every released image and the Helm chart, live-attested since v1.1.1 and re-exercised by the release workflow's own fail-first verify job (correct identity passes, a tampered image and a wrong workflow identity are required to fail) on every release through v1.3.1, the latest at this verification. Independent of Paridity once you hold a read-only pull token - validated against Sigstore's public Rekor transparency log. The registry is private by design, so anonymous verification is not the model; you verify the images you mirror.
SBOM inspection (Track D, step 2)	Verified	SPDX-JSON per image, generated by syft from the published digests and shipped as both a GitHub Release asset and a cosign in-toto attestation. No CycloneDX output today.
Cloud install transcripts (Track D, step 3)	Verified	EKS / AKS / GKE from-empty installs with fail-first signature verification, live-attested against the published v1.1.1 artifacts; the v1.2.0 release was additionally install-verified on AKS. These are committed evaluation materials, not a live sidecar page - request them from your Paridity contact.
SLSA build-provenance predicate	Roadmap	Provenance today is cosign keyless signing plus an SPDX SBOM in-toto attestation; a formal SLSA provenance predicate is not yet generated. Do not read a SLSA level into these steps.
Roadmap replication adapters and engines	Roadmap	Tracks A and C apply to the engine pairs listed as Verified/Supported in the accuracy proof. Design-partner-pending replication adapters (e.g. SAP SLT, Qlik Replicate, Informatica CDC) and any engine not in that table are Roadmap - do not extend these verification results to them. Debezium, Airbyte, Fivetran, Oracle GoldenGate and Qlik Talend Cloud are the Tier-1 (production) replication adapters today. Note that Fivetran HVR is a separate product with no adapter - Evaluating, not Roadmap.
Deployment model	Verified	Everything in this guide runs inside your VPC - evidence, audit log, and signing keys never leave your environment.

This guide consolidates the verification steps from the two companion proof documents as of last_verified above. Where this guide and a companion document differ, the companion document - which carries the underlying evidence citations - wins.

Deployment

Paridity Customer Deployment Guide¶

This documentation pack explains how Paridity is deployed after a customer buys the product and wants to run it in their own cloud account.

It is written for sales engineers, customer platform teams, security reviewers, DBAs, and implementation partners. It avoids assuming deep Kubernetes or Temporal expertise, while still naming the concrete responsibilities and technical requirements.

Document Map¶

Document	Purpose
01_deployment_overview.md	Plain-language explanation of what gets deployed and how it fits into the customer's environment
02_requirements.md	Infrastructure, identity, network, database, and operational prerequisites
03_responsibility_matrix.md	Who owns each part of the deployment and ongoing operation
04_deployment_process.md	Step-by-step process from sale through production handoff
05_security_and_networking.md	Security architecture, network flows, secrets, SSO, and evidence handling
06_cost_and_license_considerations.md	Cloud cost drivers, sizing, open-source license posture, and customer-provided licenses
07_customer_intake_checklist.md	Questions to collect before implementation begins
08_temporal_operations.md	What the customer must operate for Temporal and what Paridity owns

These are internal engineering references, not part of the customer documentation set. They are named rather than linked, because they are not published to the customer documentation site - ask your Paridity contact if you need one of them during an implementation.

Reference	Repository path
Deployment runbook	`docs/runbooks/deployment.md`
Security checklist	`docs/runbooks/security_checklist.md`
Reference deployments	`docs/architecture/reference_deployments.md`
Disaster recovery	`docs/architecture/disaster_recovery.md`
Deployment split-readiness	`docs/architecture/deployment_split_readiness.md`
Helm chart README	`deploy/helm/paridity/README.md`
Terraform README	`deploy/terraform/README.md`

Customer Deployment Overview¶

Short Version¶

Paridity is deployed as a customer-owned application inside the customer's cloud or private platform. The customer keeps their existing data systems, identity provider, secret manager, and replication tools. Paridity adds the control plane and execution workers that prove replicated datasets are complete and correct.

In the Phase 1 deployment model, Paridity does not operate customer production infrastructure. Credentials, object names, schema names, run history, and evidence stay inside the customer's environment.

What the Customer Already Has¶

Most customers already have these systems before Paridity is introduced:

Area	Customer-owned examples
Source systems	SAP ECC/S4, Oracle, DB2 LUW, SQL Server, PostgreSQL, MySQL, SAP HANA
Target systems	Snowflake, Databricks, BigQuery, Redshift, Trino/Hive, cloud object stores
Replication tools	SAP SLT, Qlik Replicate, GoldenGate, Debezium, Airbyte, Fivetran, Informatica CDC
Identity provider	Okta, Microsoft Entra ID, Keycloak, Cognito, generic OIDC/SAML
Secret manager	AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, Vault, Infisical
Cloud platform	AWS, Azure, GCP, OpenShift, private Kubernetes, or another container platform

Paridity integrates with those systems. It does not replace them.

What Paridity Deploys¶

Paridity's core application stack contains:

Component	Purpose
React UI	Browser interface for operators, analysts, admins, and auditors
FastAPI API	Authenticated control plane API for configuration, runs, evidence, and audit
PostgreSQL	Paridity metadata, run lifecycle, audit, and append-only evidence store
Temporal	Durable workflow engine for long-running reconciliation jobs
Scheduler worker	Starts scheduled reconciliation work
Readiness worker	Checks CDC/watermark readiness before a run starts
SQL worker	Connects to source and target systems and performs proof/key extraction work
Diff worker	Runs targeted row and field-level diffs using DuckDB/Arrow
Evidence worker	Writes immutable proof, diff, audit, lag retry, and metric records
Notification worker	Sends operational and audit notifications
Connectors	Runtime adapters for source and target data systems
Replication adapters	Runtime adapters for declared replication tools and CDC state

Temporal is an internal Paridity runtime dependency. Customers do not need to write Temporal workflows or expose Temporal to business users, but in the customer-deployed model their platform/SRE team must operate the Temporal service unless a separate managed-service agreement says otherwise.

Logical Architecture¶

Customer users
    |
    v
Customer IdP  ---->  Paridity UI
                         |
                         v
                    Paridity API
                         |
          +--------------+--------------+
          |                             |
          v                             v
  Paridity PostgreSQL              Temporal
   cfg / run / ev / proj               |
                                       v
                          Paridity worker pods
                            |       |       |
                            v       v       v
                       Sources  Targets  Replication tools
                            |
                            v
                    Customer secret manager

Deployment Packaging¶

The standard production packaging is:

Artifact	Purpose
Backend container image	Runs the API and all workers; the container command selects the role
Frontend container image	Serves the React UI
Helm chart	Installs the full Paridity stack into Kubernetes
Terraform references	Provision cloud infrastructure such as Kubernetes, managed Postgres, networking, and secret references
Migration job	Applies Paridity database migrations on install or upgrade
Preflight job	Verifies the customer's secret backend configuration before rollout

Kubernetes is the recommended production deployment target because it gives a portable operating model across AWS, Azure, GCP, OpenShift, and private cloud. Other deployment models are possible, but they require more customer-specific packaging and runbooks.

What Paridity Stores¶

Paridity's own PostgreSQL has four logical schemas:

Schema	Contents
`cfg`	Configuration, identity records, grants, schedules, policies, audit
`run`	Execution requests, run lifecycle, watermarks, attempts, CDC state
`ev`	Append-only proof results, mismatch keys, field diffs, metrics, evidence bundles
`proj`	Derived projections (run summaries, metric rollups, baselines) - recomputable from `ev.*`, rewritten in place, never an audit source

Paridity stores secret references, not credentials. A connection_ref points to the customer's secret manager, where the actual credential lives.

What Does Not Leave the Customer Environment¶

In the customer-deployed model, these stay inside the customer boundary:

Source and target credentials
Object, schema, table, and column names
Resolved configuration snapshots
Watermarks and CDC state
Proof results and row-level mismatch evidence
Field-level diff evidence
Audit history

Typical Production Shape¶

For a medium enterprise deployment:

Kubernetes cluster: EKS, AKS, GKE, OpenShift, or private Kubernetes
Managed PostgreSQL with backups and point-in-time recovery
External or dedicated Temporal persistence
Multiple API replicas
Scaled SQL and diff worker pools
Customer IdP integration
Customer secret manager integration through workload identity
Ingress with TLS
Network egress restricted to approved customer systems
Logs and audit events integrated with the customer's monitoring/SIEM tooling

Customer Deployment Requirements¶

This document lists what must exist before Paridity can be deployed into a customer environment.

1. Cloud and Runtime Requirements¶

Requirement	Production expectation
Cloud account or subscription	Customer-owned AWS, Azure, GCP, or private cloud account
Container platform	Kubernetes preferred: EKS, AKS, GKE, OpenShift, or private Kubernetes
Container registry access	A customer-owned registry (ECR/ACR/GAR) to mirror the Paridity images into, plus one-time-per-release read access to Paridity's private GHCR via an issued pull token (`crane copy`, digest-preserving - see `docs/runbooks/deployment.md` §2.0). Installs pin by digest from the release manifest
Helm	Used to install and upgrade Paridity
Terraform	Optional reference path for infrastructure provisioning
DNS	Customer-owned DNS name for the UI/API, such as `paridity.customer.example`
TLS certificate	Customer-issued or managed TLS certificate for ingress

Kubernetes is recommended, not conceptually mandatory. If the customer requires ECS, Azure Container Apps, Cloud Run, or VM-based deployment, that becomes a customer-specific packaging project.

2. Database Requirements¶

Paridity requires PostgreSQL for its own metadata, run lifecycle, and evidence.

Requirement	Production expectation
PostgreSQL version	Managed PostgreSQL compatible with the application's migrations
Network access	Private network access from Paridity pods to PostgreSQL
Encryption	Storage encryption enabled
TLS	TLS required for application connections where supported
Backups	Automated backups and point-in-time recovery
Extensions	`uuid-ossp`, `pgcrypto`, and `ltree` available or pre-created
Database user	Privileges to run migrations or a DBA-supported migration process

For production, use RDS, Azure Database for PostgreSQL Flexible Server, Cloud SQL for PostgreSQL, or an approved customer-managed PostgreSQL service.

3. Temporal Requirements¶

Temporal is required for durable reconciliation workflows. Paridity is currently built on Temporal as its orchestration engine; alternative customer schedulers such as Airflow, Control-M, Step Functions, or Argo can trigger Paridity through the API, but they do not replace Paridity's internal Temporal runtime.

Requirement	Production expectation
Temporal server	Deployed in the customer environment
Persistence	PostgreSQL-backed persistence with backup coverage
Network access	API and worker pods can reach Temporal on the configured host/port
Namespace	default or customer-approved namespace
Operations owner	Customer platform/SRE owner assigned for Temporal monitoring, restart, patch, upgrade, backup, and restore
Version control	Temporal version stays within the Paridity-supported compatibility range

Small pilots may use embedded Temporal. Production should use a persistent, operationally supported Temporal deployment.

Customer users do not need direct Temporal access. Customer operations teams do need runbooks and permissions to restart Temporal, review logs, monitor health, apply Paridity-approved upgrades, and restore Temporal persistence during DR.

4. Identity and Access Requirements¶

Paridity authenticates users through the customer's identity provider.

Requirement	Production expectation
IdP	OIDC or SAML-capable provider
Issuer/entity ID	Stable issuer URL or SAML entity ID
Audience/client ID	API audience and UI client configuration
User claims	Email/subject, tenant identifier if applicable, role/group claims
Role mapping	Mapping from IdP groups to Paridity roles
Break-glass policy	Customer-approved emergency admin access procedure

Main application roles:

Role	Typical use
`platform_admin`	Deployment-wide administration
`tenant_admin`	Tenant-scoped administration
`operator`	Configure mappings, schedules, and runs
`analyst`	Investigate reconciliation results
`auditor`	Read audit and evidence records

5. Secret Manager Requirements¶

Paridity must be able to resolve customer-owned connection references at runtime.

Requirement	Production expectation
Secret backend	AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, Vault, or Infisical
Workload identity	IRSA, Azure Workload Identity, GKE Workload Identity, or equivalent
No static cloud keys	Static keys should not be stored in Kubernetes secrets
Connection references	URI-style references stored in Paridity, credentials stored only in the secret backend
Canary secret	A test `connection_ref` for install-time preflight validation

Example connection reference:

aws-secrets://arn:aws:secretsmanager:us-east-1:123456789012:secret:paridity/oracle-prod

6. Network Requirements¶

Paridity workers need controlled outbound access to the systems they reconcile.

Destination	Required from
Source databases	SQL/readiness workers
Target databases/warehouses	SQL/diff workers
Replication metadata APIs or databases	Readiness workers
Customer secret backend	API and worker pods
Paridity PostgreSQL	API, workers, migration job
Temporal	API and workers
OIDC/JWKS or SAML IdP endpoints	API and UI/browser flow
OpenTelemetry/SIEM/log endpoints	Optional, if customer requires telemetry export

Production deployments should use default-deny egress with explicit allow-lists.

7. Source and Target Access Requirements¶

Paridity needs least-privilege access to compare data.

Typical permissions:

Read metadata for schemas, tables, columns, and keys
Read selected source and target tables/views
Execute hash/proof SQL functions where supported
Read replication state or CDC metadata where configured
Create no objects in source/target systems unless a specific connector path requires customer-approved staging

Credentials should be service accounts created and owned by the customer.

8. Operational Requirements¶

Area	Requirement
Monitoring	Pod health, API health, worker queues, Temporal health, PostgreSQL metrics
Logging	Application logs retained according to customer policy
Audit	`cfg.audit_log` retained and optionally exported to SIEM
Backup	PostgreSQL and Temporal persistence backups enabled and tested
Restore testing	Restore validation before production go-live
Upgrade window	Customer-approved maintenance window for Helm upgrades and migrations
Evidence retention	Customer-approved retention and purge policy
Temporal operations	Customer-owned operational procedure for Temporal health, restart, patch, upgrade, backup, and restore

9. Sizing Inputs¶

Before sizing the deployment, collect:

Number of mappings
Largest table sizes
Expected schedule frequency
Required concurrency
Source and target systems involved
Whether proof-only mode is acceptable for some mappings
Whether field-level diff evidence is required for every mismatch
Evidence retention duration
Required RPO/RTO
Cloud region and data residency constraints

Customer Deployment Responsibility Matrix¶

This matrix clarifies who owns each part of a customer deployment. Adjust it for each contract, but do not leave ownership implicit.

Roles¶

Role	Description
Paridity implementation team	Vendor or partner team that installs/configures Paridity
Customer platform team	Owns cloud, Kubernetes, networking, DNS, ingress, and runtime standards
Customer IAM/SSO team	Owns IdP, groups, claims, SAML/OIDC apps, and access reviews
Customer DBA/data platform team	Owns source/target database access, service accounts, warehouse controls
Customer replication team	Owns SLT/Qlik/GoldenGate/Debezium/Airbyte/Fivetran configuration and state access
Customer security/compliance team	Owns security review, evidence retention, audit, SIEM, and policy approval
Customer SRE/operations team	Owns monitoring, incident response, backups, upgrades, and day-2 operations

RACI Matrix¶

Legend:

R = Responsible for doing the work
A = Accountable for approval/outcome
C = Consulted
I = Informed

Activity	Paridity team	Platform	IAM/SSO	DBA/data	Replication	Security	SRE/Ops
Confirm deployment architecture	R	A	C	C	C	C	C
Select sizing tier	R	A	I	C	C	C	C
Provision Kubernetes	C	R/A	I	I	I	C	C
Provision managed PostgreSQL	C	R/A	I	C	I	C	C
Deploy/provision Temporal	R	R/A	I	I	I	C	R
Operate Temporal day to day	C	C	I	I	I	C	R/A
Configure backups/PITR	C	R	I	C	I	A	R
Configure DNS/TLS/ingress	C	R/A	C	I	I	C	R
Configure OIDC/SAML app	C	C	R/A	I	I	C	I
Define role/group mapping	R	I	R	I	I	A	I
Configure secret backend access	R	R	I	C	C	A	C
Create Paridity service accounts in source/target systems	C	I	I	R/A	C	C	I
Grant replication metadata access	C	I	I	C	R/A	C	I
Configure network allow-lists	R	R/A	I	C	C	C	R
Install Helm release	R	C	I	I	I	I	C
Run preflight and migrations	R	C	I	C	I	I	C
Configure systems/mappings/schedules	R	I	I	C	C	C	C
Validate first reconciliation run	R	I	I	C	C	C	C
Approve production go-live	C	C	C	C	C	A	A
Monitor production health	I	C	I	I	I	C	R/A
Perform upgrades	R	C	I	I	I	C	A
Patch/upgrade Temporal within Paridity support window	R	C	I	I	I	C	A
Respond to application incidents	R	C	I	C	C	C	A
Respond to cloud/network incidents	C	R	I	I	I	C	A
Respond to source/target access incidents	C	I	I	R	R	C	A
Evidence retention and purge policy	C	I	I	I	I	A	R

Ownership Boundaries¶

Paridity Owns¶

Product container images (published to private GHCR per release with a digest manifest on the GitHub Release; customer pull tokens issued read-only per docs/runbooks/release_publishing.md)
Helm chart and reference Terraform modules
Product configuration guidance
Connector behavior and reconciliation correctness
Application-level troubleshooting
Supported Temporal version range and chart configuration
Temporal workflow compatibility testing
Upgrade instructions and compatibility notes

Customer Owns¶

Cloud account/subscription
Kubernetes or container runtime operation
Temporal runtime operation when deployed in the customer's environment
Network routes, firewalls, private endpoints, DNS, and TLS
Identity provider, users, groups, and access reviews
Source/target database accounts and permissions
Replication tools and replication health
Secret manager contents and access policies
Backups, restore tests, monitoring, log retention, and SIEM
Temporal monitoring, restart, patching, upgrade execution, backup, and restore
Compliance approval for evidence retention and purge policy

Shared Responsibilities¶

Sizing and performance tuning
Security review
Go-live validation
Incident triage across product, cloud, database, and replication layers
Upgrade planning

Decision Record Template¶

Capture these decisions before production go-live:

Decision	Selected value	Owner	Approval date
Cloud provider/region
Kubernetes platform
Sizing tier	Small / Medium / Large
PostgreSQL service
Temporal mode	Embedded / external / dedicated
Secret backend	AWS / Azure / GCP / Vault / Infisical
Identity provider
Network egress policy
Evidence retention
Backup/PITR target
RPO/RTO
SIEM/log export

Customer Deployment Process¶

This is the recommended process from closed sale through production handoff.

Phase 0: Commercial and Technical Kickoff¶

Goal: align scope, stakeholders, deployment model, and success criteria.

Tasks:

Confirm the customer will run Paridity in their own environment.
Identify cloud provider, region, and platform team.
Identify source systems, target systems, and replication tools in scope.
Confirm whether this is a pilot, production deployment, or regulated/SOX deployment.
Assign owners using the responsibility matrix.
Complete the customer intake checklist.
Define success criteria for the first production mapping.

Exit criteria:

Stakeholders identified
Deployment tier selected provisionally
Data systems and replication tools listed
Security review path known

Phase 1: Architecture and Security Review¶

Goal: agree how Paridity will fit into the customer's cloud and control model.

Tasks:

Review the customer deployment overview.
Confirm runtime target: EKS, AKS, GKE, OpenShift, or equivalent.
Confirm PostgreSQL strategy: managed Postgres preferred.
Confirm Temporal strategy: embedded only for pilot; persistent external/dedicated for production.
Assign a customer platform/SRE owner for Temporal operations.
Confirm IdP protocol and role/group mappings.
Confirm secret backend and workload identity pattern.
Confirm network egress destinations and CIDR allow-lists.
Confirm evidence retention, backup, and audit requirements.
Review license and cost considerations.

Exit criteria:

Security team approves deployment pattern
Network flow list approved
Identity and secret manager designs approved
Sizing tier selected

Phase 2: Infrastructure Provisioning¶

Goal: create the customer-owned infrastructure Paridity needs.

Typical Terraform-backed path:

cd deploy/terraform/aws    # or azure / gcp
terraform init
terraform apply \
  -var region=<region> \
  -var sizing_tier=<small|medium|large> \
  -var 'allowed_egress_cidrs=["<source>","<target>","<secret-backend>"]'

Tasks:

Provision or identify Kubernetes cluster.
Provision or identify managed PostgreSQL.
Provision or identify Temporal and its persistence backend.
Enable backups and point-in-time recovery for PostgreSQL and Temporal persistence.
Create database URL secret for the Helm chart.
Configure workload identity for secret manager access.
Configure ingress controller, DNS, and TLS secret.
Configure network policies/firewall allow-lists.
Mirror the release images from Paridity's private GHCR into the customer registry with the issued read-only pull token, and verify digests survived the mirror (docs/runbooks/deployment.md §2.0). Digests come from the release's digests-vX.Y.Z.md manifest on the GitHub Release.

Exit criteria:

Kubernetes access available
PostgreSQL reachable from cluster
Temporal reachable from API and worker pods
Secret backend access configured
DNS/TLS path ready
Network allow-lists implemented

Phase 3: Identity and Secret Setup¶

Goal: make authentication and runtime credentials work without embedding secrets in Paridity.

Tasks:

Create OIDC or SAML application in the customer IdP.
Configure redirect URLs, issuer/entity ID, audience/client ID. For SAML, list the email domains your users sign in with on the provider (Settings -> Authentication). The login page finds a SAML provider only by email domain, so an enabled provider with no domains is not offered at sign-in; the settings page flags it. A domain can route to only one enabled SAML provider. To move from one IdP to another, add the domains to the new provider while it is still disabled, then disable the old one and enable the new one. The SSO URL, signing certificate and domains can be changed later without re-registering, so rotating an expiring IdP certificate keeps the same ACS URL. Only the entity ID is fixed after creation.
Map customer groups to Paridity roles.
Create source/target/replication service accounts.
Store credentials in the customer secret backend.
Create a preflight canary secret.
Validate workload identity can read the canary secret.

Exit criteria:

Login path defined
Role mapping approved
Secret references created
Preflight canary available

Phase 4: Paridity Installation¶

Goal: install the application stack.

Example Helm install:

helm install recon deploy/helm/paridity \
  -f deploy/helm/paridity/values-medium.yaml \
  --namespace paridity --create-namespace \
  --set image.backend.repository=<customer-registry>/paridity-api \
  --set image.frontend.repository=<customer-registry>/paridity-ui \
  --set image.docs.repository=<customer-registry>/paridity-docs \
  --set image.backend.digest=sha256:<backend-image-digest> \
  --set image.frontend.digest=sha256:<frontend-image-digest> \
  --set image.docs.digest=sha256:<docs-image-digest> \
  --set database.external.existingSecret=recon-db \
  --set temporal.host=<temporal-host:7233> \
  --set oidc.issuer=<customer-idp-issuer> \
  --set oidc.audience=recon-platform-api \
  --set app.publicBaseUrl=https://<paridity-hostname> \
  --set preflight.connectionRef=<customer-secret-ref> \
  --set app.audit.signingKeyRef=<kms-key-ref>            # required in production (#2134) \
  --set app.evidenceBundles.signingKeyRef=<kms-key-ref>  # evidence-bundle signer - see below (#2558) \
  --set networkPolicy.enabled=true \
  --set ingress.enabled=true \
  --set ingress.host=<paridity-hostname> \
  --set ingress.tlsSecret=<tls-secret>

Two signers, two keys¶

Paridity signs two different things, and each has its own setting. Setting one does not set the other.

Setting	What it signs	Who checks it	If you leave it empty
`app.audit.signingKeyRef`	The audit log's hash-chain checkpoints	Paridity's chain verification	A production install refuses to start
`app.evidenceBundles.signingKeyRef`	Exported evidence bundles (`manifest.json.sig`)	Your auditor, offline, with `verify_bundle.py`	Bundles ship unsigned, and the offline verifier refuses to call them verified

Use an asymmetric key for the evidence-bundle signer - a cloud KMS key ref (awskms://, azurekv://, gcpkms://), or a secrets-backend URI whose secret is a private ECDSA P-256 PEM. Your auditor then verifies bundles with the PUBLIC key alone. An HMAC key also signs, but only someone who holds that key can verify it, so it does not give an auditor independent verification.

The value sets the platform-wide default when Paridity first starts, and again after a database rebuild. After that, the product setting wins: an administrator can change the default on the configuration screen, or set a different signer for a tenant, and the Audit page shows which signer is in effect. If you choose to ship unsigned bundles, the API logs a warning at every production start that names this setting.

What happens during install:

Preflight job verifies the customer secret backend.
Migration job bootstraps or upgrades Paridity PostgreSQL.
API, UI, Temporal, and workers roll out.
Ingress exposes the UI/API.

Exit criteria:

All pods ready
API /health returns healthy
UI loads over HTTPS
User can authenticate through customer IdP
Workers connect to Temporal
The Audit page shows the evidence-bundle signer as configured, and one exported evidence bundle verifies offline with verify_bundle.py --public-key <your public key> (skip only if you have chosen unsigned bundles)

Phase 5: Connectivity Validation¶

Goal: confirm Paridity can reach only the approved systems and can perform its minimum required work.

Tasks:

Validate source system connection.
Validate target system connection.
Validate replication metadata access if applicable.
Validate secret references resolve at runtime.
Confirm no plaintext credentials are stored in Paridity tables.
Confirm network policies block unapproved destinations.
Confirm logs do not contain secrets or row payloads.

Exit criteria:

Source, target, and replication connectivity verified
Network/security checks pass
Customer security team accepts the runtime path

Phase 6: First Mapping and Reconciliation Run¶

Goal: prove an end-to-end reconciliation using a representative but controlled dataset.

Tasks:

Configure systems.
Configure replication profile.
Create mapping and fields.
Refresh schema.
Configure schedule or manual run.
Run full mode.
Run auto mode, if incremental reconciliation is in scope.
Review proof results, mismatch evidence, and audit records.

Exit criteria:

First run completes successfully
Evidence is visible to authorized users
Audit log contains expected actions
Customer understands how to interpret pass/fail statuses

Phase 7: Production Readiness Review¶

Goal: verify the deployment is ready for business use.

Checklist:

Security checklist complete
Backups enabled
Restore test completed or scheduled before regulated use
Monitoring dashboards configured
Alerts configured
Runbook owners assigned
Temporal operations owner assigned
Temporal restart, backup/restore, and upgrade procedure reviewed
Evidence retention policy configured
Upgrade procedure reviewed
Incident process documented
Initial mappings and schedules approved

Exit criteria:

Customer production owner signs off
Security/compliance signs off
Paridity implementation team signs off

Phase 8: Handoff and Day-2 Operations¶

Goal: transfer normal operation to the customer operations team.

Handoff package:

Installed version and image digests
Helm values used
Terraform outputs or infrastructure notes
IdP configuration summary
Secret backend configuration summary
Network allow-list summary
Backup/restore details
Temporal operations owner and runbook
Monitoring and alert details
Known limitations and agreed follow-ups
Support escalation path

Normal operations include:

Monitoring API, workers, Temporal, and PostgreSQL
Restarting Temporal when required by the runbook
Reviewing failed or delayed runs
Adding mappings and schedules
Managing user access through IdP groups
Rotating source/target credentials in the customer secret manager
Applying Paridity upgrades through Helm
Applying Paridity-approved Temporal patches/upgrades
Running periodic restore validation

Security and Networking Guide¶

Security Posture¶

Paridity is designed to run inside the customer's trust boundary. The standard deployment keeps credentials, object names, data system metadata, proof results, and field-level evidence inside the customer's cloud or private platform.

Identity and Authentication¶

Paridity uses the customer's identity provider for user authentication.

Supported patterns:

OIDC/JWT
SAML, where configured

Production requirements:

No local development auth bypass
TLS on all browser-facing endpoints
IdP groups mapped to Paridity roles
Break-glass admin access documented and audited
Access reviews owned by the customer

Main roles:

Role	Typical access
`platform_admin`	Full deployment-wide administration
`tenant_admin`	Tenant administration
`operator`	Configure systems, mappings, schedules, and runs
`analyst`	Investigate reconciliation evidence
`auditor`	Read audit records and evidence

Secrets¶

Paridity must not store source, target, or replication credentials in its own database.

Correct pattern:

Paridity cfg.systems.connection_ref
    |
    v
Customer secret manager
    |
    v
Actual credential

The connection_ref is a pointer. The actual credential lives in AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, Vault, or Infisical.

Production requirements:

Use cloud workload identity where possible.
Do not store static cloud keys in Kubernetes secrets.
Use a preflight canary secret to verify access during install.
Rotate credentials in the customer's secret manager, not in Paridity tables.

Network Boundaries¶

Only the UI/API should be reachable by users. Worker pods should not be exposed publicly.

Inbound:

Source	Destination	Purpose
Customer users/browser	Paridity ingress	UI and API access
IdP browser redirects	Paridity ingress	Login callback where applicable
Customer monitoring	Health endpoints	Optional health checks

Outbound:

Source	Destination	Purpose
API	PostgreSQL	Config, run, evidence metadata
API	IdP/JWKS endpoint	Token validation
API/workers	Secret manager	Resolve connection references
API/workers	Temporal	Workflow dispatch and execution
Workers	PostgreSQL	Run state and evidence writes
SQL/readiness workers	Source systems	Proof, key extraction, metadata, readiness
SQL/diff workers	Target systems	Proof, key extraction, metadata, diff
Readiness workers	Replication tools	CDC/watermark/lag state
API/workers	OTel/SIEM endpoints	Optional telemetry export

Egress Control¶

Production deployments should use two layers of egress control:

Cloud network controls: security groups, firewall rules, NSGs, private endpoints, routing tables.
Kubernetes NetworkPolicy: default-deny with explicit allowed destinations.

The approved destination list should include only:

Source database endpoints
Target database/warehouse endpoints
Replication metadata endpoints
Secret manager endpoints
IdP/JWKS endpoints
PostgreSQL and Temporal endpoints
Approved telemetry/SIEM endpoints

Data Handling¶

Paridity handles several classes of sensitive data:

Data class	Stored where	Notes
Credentials	Customer secret manager	Paridity stores only references
Object/schema/column names	Paridity PostgreSQL	Sensitive customer metadata
Resolved config snapshots	Paridity PostgreSQL	Sensitive because they reference systems and fields
Proof results	Paridity PostgreSQL `ev.*`	Append-only evidence
Mismatch keys	Paridity PostgreSQL `ev.*`	Append-only evidence
Field-level diffs	Paridity PostgreSQL `ev.*`	May contain sensitive values; masking policy applies
Audit log	Paridity PostgreSQL `cfg.audit_log`	Administrative and lifecycle evidence

Field-level evidence must respect masking and raw-value storage policy. If raw values are not permitted, evidence records should preserve the fact of mismatch without storing sensitive values.

Audit and Compliance¶

Production requirements:

Administrative actions are written to cfg.audit_log.
Terminal run status transitions are audited.
Evidence records are append-only.
Retention purge actions are logged before deletion.
Audit logs can be exported to the customer's SIEM.
compare_started_at is immutable and serves as an audit anchor for runs.

Backup and Recovery¶

Customer operations owns backup and restore, with Paridity guidance.

Minimum expectations:

Managed PostgreSQL automated backups
Point-in-time recovery for production
Temporal persistence backup
Quarterly restore validation for regulated use
Restore runbook reviewed before go-live

Temporal is internal to Paridity, but it is still production infrastructure. The customer operations team must be able to monitor it, restart it, collect logs, patch it, upgrade it within Paridity's supported version range, and restore its persistence backend during DR.

Production Security Checklist¶

Before go-live, verify:

APP_ENV=production
No development auth bypass enabled
Images pinned by digest
TLS configured
Customer IdP configured
Customer secret backend configured
Workload identity configured
Network egress allow-list applied
PostgreSQL private and encrypted
Backups and PITR enabled
Temporal persistence backup and restore path validated
Temporal operations owner assigned
Logs reviewed for secret leakage
Audit log retention configured
SIEM export configured if required

Cost and License Considerations¶

Cost Buckets¶

Paridity cost has two categories:

Cost to run the Paridity platform.
Cost created in customer data platforms when reconciliation queries run.

The second category can be larger at scale and must be included in customer planning.

Paridity Platform Runtime Cost¶

Runtime cost includes:

Kubernetes nodes or serverless container runtime
API, UI, and worker pods
Managed PostgreSQL
Temporal compute and persistence
Load balancer or ingress
Logs, metrics, traces, and SIEM export
Evidence storage
Backups and point-in-time recovery
NAT gateways or private endpoint charges where applicable

Reference AWS estimates from the architecture docs:

Tier	Typical scope	Directional monthly cost
Small	Pilot, up to 50 mappings	About $110/month
Medium	Enterprise starting point, up to 500 mappings	About $865/month
Large	Large estate, up to 5,000 mappings	About $5,800/month
Multi-region	V1 planning only, large per region	About $17,400/month for three large regions

These are planning numbers, not quotes. Region, discounts, reserved capacity, evidence retention, log volume, and HA requirements can materially change cost.

Data Platform Cost¶

Reconciliation work can create cost in source and target platforms:

Platform area	Cost driver
Snowflake	Warehouse credits consumed by proof and extraction SQL
Databricks	SQL warehouse/serverless/job compute
BigQuery	Bytes scanned and slot usage
Redshift	Cluster/serverless compute
Source databases	CPU, I/O, read replicas, and possible license capacity
Object stores	Read requests, scanned bytes, and transfer
Network	Cross-AZ, cross-region, private endpoint, NAT, or egress charges
Logging/SIEM	Ingested application and audit logs

For large datasets, customer warehouse/database resource monitors should be configured before production schedules are enabled.

Cost Controls¶

Recommended controls:

Start with a controlled pilot mapping.
Use auto incremental mode where appropriate.
Use full mode intentionally for baseline or periodic controls.
Use proof-first execution so field diff work runs only after proof failure.
Set worker concurrency limits per source/target system.
Schedule heavy jobs outside business-critical windows.
Use partitioned reconciliation for very large tables.
Configure evidence retention and purge policy.
Monitor PostgreSQL storage growth.
Monitor warehouse credits/bytes scanned for target systems.
Review cost estimates after the first production week and first month.

License Posture¶

Paridity is designed around commercially safe dependencies. Core components are generally permissive or commercially accepted:

Component	License posture
FastAPI, Pydantic, SQLAlchemy	Permissive ecosystem
React, Vite, TanStack, Zustand, shadcn/ui, lucide	Permissive ecosystem
PostgreSQL	PostgreSQL License
DuckDB	MIT
Apache Arrow/PyArrow	Apache 2.0
Trino	Apache 2.0
OpenTelemetry	Apache 2.0
Kubernetes, Helm	CNCF/open-source ecosystem
Temporal self-hosted / Python SDK	Commercially usable for this packaging model

The product should maintain a third-party notices file and SBOM as part of the release process.

License Cautions¶

Important cautions:

Customer database drivers may have separate licenses or download terms.
Oracle, DB2, SAP, Snowflake, Databricks, and similar systems may require customer-owned client libraries, accounts, or subscriptions.
Customer replication tools remain separately licensed by the customer.
Paridity integrates with replication tools; it does not grant licenses to Qlik, GoldenGate, SLT, Fivetran, Airbyte Cloud, or similar products.
HashiCorp Vault should be optional, not required. Many customers will prefer their cloud-native secret manager.
Avoid making Confluent Platform components required. Apache Kafka core is acceptable; Confluent Platform components may carry license constraints that need legal review.

Commercial Packaging Considerations¶

Before selling production deployments, define:

Paridity commercial license/EULA
Third-party notices and SBOM delivery
Support boundaries between Paridity and customer cloud operations
Supported connector tiers
Supported deployment targets
Upgrade policy and supported version window
Security patch SLA
Whether implementation services are included or separate

Customer Deployment Intake Checklist¶

Use this checklist before implementation begins. It is intended for a sales engineer, solution architect, or implementation lead to complete with the customer.

1. Customer and Project¶

Question	Answer
Customer name
Project sponsor
Business owner
Technical owner
Security owner
Operations owner
Target go-live date
Pilot or production?
Regulated/SOX/ICFR scope?

2. Cloud and Runtime¶

Question	Answer
Cloud provider	AWS / Azure / GCP / private / other
Region(s)
Kubernetes platform	EKS / AKS / GKE / OpenShift / private / other
Existing cluster or new cluster?
Temporal deployment preference	Embedded / dedicated / existing customer-managed
Temporal operations owner
Container registry restrictions
Helm allowed?
Terraform allowed?
Required tagging/labeling standards
Private endpoint requirements

3. Sizing¶

Question	Answer
Expected number of mappings at go-live
Expected number of mappings after 12 months
Largest source table row count
Largest target table row count
Expected run frequency
Required concurrent runs
Required RPO/RTO for Paridity
Evidence retention duration
Preferred sizing tier	Small / Medium / Large

4. Identity and Access¶

Question	Answer
Identity provider	Okta / Entra ID / Keycloak / Cognito / other
Protocol	OIDC / SAML
Issuer URL or entity ID
API audience/client ID
UI client ID
Group claim name
Email/user claim name
Role mapping approved?
Break-glass process

5. Secret Backend¶

Question	Answer
Secret manager	AWS SM / Azure KV / GCP SM / Vault / Infisical
Workload identity mechanism	IRSA / Azure WI / GKE WI / other
Secret naming convention
Secret rotation policy
Canary `connection_ref` available?
Static keys prohibited?	Yes / No

6. Network¶

Question	Answer
Paridity hostname
TLS certificate source
Ingress controller
Source system CIDRs/endpoints
Target system CIDRs/endpoints
Replication tool endpoints
Secret backend endpoints
IdP/JWKS endpoints
SIEM/OTel endpoints
NetworkPolicy enforced by CNI?	Yes / No
NAT/private endpoint constraints

7. Source Systems¶

System	Type	Environment	Owner	Endpoint	Access approved?
		Dev/Test/Prod
		Dev/Test/Prod
		Dev/Test/Prod

For each source system, collect:

Database/platform type and version
Network endpoint
Required TLS settings
Service account name
Secret reference
Schemas/tables in scope
Expected load constraints
Preferred read replica, if applicable

8. Target Systems¶

System	Type	Environment	Owner	Endpoint	Access approved?
		Dev/Test/Prod
		Dev/Test/Prod
		Dev/Test/Prod

For each target system, collect:

Warehouse/database type and version
Compute warehouse or cluster to use
Resource monitor or budget control
Service account name
Secret reference
Schemas/tables in scope
Query timeout or concurrency limits

9. Replication Tools¶

Tool	Source	Target	Owner	Metadata access method	Approved?
				API / DB / files / none
				API / DB / files / none

For each replication tool, collect:

Tool name and version
Task/job names
How lag/watermark state is exposed
Service account or API token secret reference
Whether Paridity should treat replication state as advisory or strict

10. First Mapping Candidates¶

Choose a simple first mapping before a large or sensitive one.

Question	Answer
Source table
Target table
Primary/business key
Approximate row count
Watermark column or strategy
Replication tool/task
Full mode required?	Yes / No
Auto/incremental mode required?	Yes / No
Field-level diff required?	Yes / No
Raw value storage allowed?	Yes / No

11. Operations¶

Question	Answer
Monitoring system
Log retention
SIEM integration required?
Alert channels
Backup owner
Temporal restart owner
Temporal patch/upgrade owner
Temporal persistence backup owner
Restore test date
Upgrade window
Incident bridge process
Support contact list

12. Go-Live Approval¶

Approval	Owner	Date
Architecture approved
Security approved
Network approved
Identity approved
Source/target access approved
Backup/restore approved
First run validated
Production handoff complete

Temporal Operations Responsibility¶

Paridity uses Temporal internally for durable orchestration. In the customer-deployed model, Temporal runs inside the customer's environment as part of the Paridity stack.

Required Positioning¶

Temporal is a bundled internal runtime dependency, not a customer-facing feature. Customer business users do not need to use Temporal directly, and customer teams do not need to write Temporal workflows for Paridity.

However, Temporal is still production infrastructure. Unless the contract explicitly includes a Paridity-managed service, the customer platform/SRE team must operate the Temporal runtime deployed in their environment.

What Temporal Does for Paridity¶

Temporal provides:

Durable reconciliation workflow execution
Long-running proof and diff orchestration
Activity retries and backoff
Worker task queues
Cancellation and recovery behavior
Schedule execution
Recovery after worker restarts

Because Paridity workflows rely on these semantics, Temporal is not currently a drop-in replaceable component.

External enterprise schedulers can still trigger Paridity:

Airflow / Control-M / Step Functions / Argo
        |
        v
Paridity API execution request
        |
        v
Paridity Temporal workflow
        |
        v
Paridity workers

Customer-Owned Temporal Operations¶

In the Phase 1 customer-deployed model, the customer owns:

Area	Customer responsibility
Availability	Keep Temporal services/pods running
Restart	Restart Temporal using the approved runbook when needed
Monitoring	Monitor Temporal health, task queues, persistence, and pod status
Logging	Collect Temporal logs into the customer's logging platform
Backup	Back up Temporal persistence
Restore	Restore Temporal persistence during DR
Patching	Apply security patches through Paridity-approved chart/image updates
Upgrades	Upgrade within Paridity's supported Temporal compatibility range
Network	Ensure API and workers can reach Temporal
Incident response	Participate in incidents involving orchestration, stuck runs, or queue backlog

Paridity-Owned Temporal Responsibilities¶

Paridity owns:

Supported Temporal version range
Helm chart configuration for the supported deployment model
Workflow and worker compatibility testing
Product-level troubleshooting guidance
Upgrade instructions and release notes
Recommendations for monitoring and alerting

Paridity should not support arbitrary customer modifications to Temporal configuration unless they are reviewed and accepted for that deployment.

Deployment Options¶

Option	Who operates it	Recommended use
Embedded Temporal from the Paridity chart	Customer platform/SRE	Pilot or small non-critical deployments
Dedicated Temporal deployed for Paridity	Customer platform/SRE	Recommended production default
Existing customer-managed Temporal cluster	Customer platform/SRE	Only if version, namespace, access, and support boundaries are approved
Paridity-managed Temporal	Paridity	Future or separately contracted managed offering

Go-Live Requirements¶

Before production go-live:

Temporal owner is named.
Temporal health checks are monitored.
API and workers can reach Temporal.
Temporal persistence is backed up.
Restore path is documented.
Upgrade path is documented.
Logs are available to operations.
Incident escalation path is known.

Security & trust

Paridity Trust & Security - secure by design¶

Verified data. Trusted outcomes. Paridity's strongest security property isn't a certificate - it's the architecture. The platform is built so that whole categories of vendor risk simply don't exist in its design.

Why the design is the security answer¶

Paridity runs entirely inside your own cloud account and stores no customer data or credentials of its own. That single design choice answers most of what a security review is built to probe - structurally, not with a policy promise. The one call an installation makes to Paridity is a licence check to EGMW's own licence service, and it carries licence data only (the licence key, a machine id, an activation id, and the product, platform, and architecture), never customer data - see the Data-Flow Disclosure.

Design property	What it means	The risk it removes
No stored customer data	Only hashes, diffs, and metadata are retained - in your Postgres, in your VPC. Source/target row values never persist in a Paridity-controlled store.	No vendor-side data store to breach; no data-residency question; no cross-customer blast radius.
No stored credentials	`connection_ref` is a URI pointer into your own secrets backend (AWS SM / Azure KV / GCP SM / Infisical). The platform resolves it at runtime from your backend; Paridity never holds a secret.	No credential store to leak; no "who has our DB passwords" exposure.
Proper protocols & methods	SSO over SAML 2.0 or OIDC (authorization code + PKCE); bearer-token auth (RS256/ES256, no `alg=none`; `aud`/`iss`/`exp` enforced), TLS in transit, RBAC least-privilege (4 roles), tenant isolation with PostgreSQL RLS as a defense-in-depth backstop, evidence masking (a tenant+mapping+field AND-chain for raw-value storage, plus per-field display masking).	No home-grown auth; least-privilege by construction; sensitive values maskable end-to-end.
Deterministic & auditable	Correctness is computed (`hash_total`/`row_hash`/`field_diff`), never inferred. Evidence is append-only and immutable; the audit log is hash-chained with signed checkpoints that can be anchored to external WORM storage, an RFC 3161 timestamp authority, and your SIEM; evidence bundles are signed and offline-verifiable with your own key. Evidence retention runs on a built-in, dry-run-first schedule that audits every deletion, and an approval-gated legal hold can freeze evidence from deletion entirely. AI never determines correctness.	No black-box "trust us"; the record is tamper-evident and independently checkable.
Customer-operated	You deploy and run it (Helm/Terraform); Paridity operates nothing in your reconciliation path except the licence service, which releases after v1.4.0 check once a day.	No dependency on a vendor's security operations for your reconciliation, and a bounded one on its uptime: without the licence service the installation runs normally for 7 days, then a paid licence runs proof-only until it is reachable - never a halt.

Start your review here. The documents below are the detail behind each property.

Two mechanisms worth a closer look¶

A robust, tamper-evident audit log. Paridity's audit trail is engineered so an external auditor can trust it without trusting Paridity:

Append-only + hash-chained. Each sealed audit row carries entry_hash = SHA-256(prev_hash + canonical(row)), so altering, deleting, or reordering any entry breaks every hash after it. A full-database rewrite that recomputes every hash still fails against the last signed checkpoint.
Complete by construction. Every terminal run status and every admin action (cancel, bypass_readiness, retention purge) is written through a single point of enrichment with actor, entity, action, timestamp, and source IP - no code path can omit the guarantees. Even evidence deletion (retention purge) writes an audit record before it removes anything.
Immutable evidence + signed bundles. Evidence is append-only; a run's trail exports as an evidence bundle whose manifest commits to every file's hash and to a previous-bundle-hash chain, sealed with a detached signature over exactly the bytes that ship - verifiable offline with the shipped fail-closed tool and your own signing key (held in your secrets backend; an unsigned or tampered bundle never reports as verified).

Keys stay locked down. Paridity is built to never become a place your keys live:

No credential custody. connection_ref is a pointer; the platform resolves the real secret from your backend (AWS SM / Azure KV / GCP SM / Infisical) at runtime. Paridity holds a pointer, never a password. Column-decrypt keys (TDE, etc.) resolve the same way and are never persisted.
Signing keys you control. Audit/evidence signing supports HMAC → ECDSA-P256 → cloud KMS (AWS KMS, Azure Key Vault, or GCP KMS), where the private key never leaves the KMS/vault - Paridity asks the KMS to sign; it never sees the key. All three KMS signers are live-verified against a real key in that cloud. The default dev key is insecure by design and must be replaced before production, which the deployment checklist enforces.
Least-privilege everywhere. OIDC/JWT (RS256/ES256, no alg=none), four-role RBAC enforced only at the API layer, tenant isolation with RLS as a database-level backstop.

The documents¶

Document	Answers	Status
Security Whitepaper	The controls in depth: access, secrets, data protection, OSS posture	In-Review
Architecture Technical Brief	Control-plane/data-plane design, vendor-neutral engine model, scale	In-Review
Data-Flow & Sub-Processor Disclosure	Where data lives (your VPC), the one licence-data call to EGMW's licence service, and the sub-processor list: none for customer data; Railway (licence service hosting) and Microsoft 365 (support mailbox) for licence and support data	In-Review
Accuracy Proof	How correctness is guaranteed - and how to verify it yourself	In-Review
Compliance & Audit Proof	Append-only, hash-chained, offline-verifiable audit trail	In-Review
Verify It Yourself	Independently confirm accuracy, audit integrity, and speed	In-Review
Business Continuity & DR Summary	Backup/PITR, RPO/RTO, restore - and who owns what (customer-hosted)	In-Review
Open-Source & Supply-Chain Posture	Permissive-only license posture; pinned lockfiles; CI dependency and image scanning; signed release images with SPDX SBOMs	In-Review
Questionnaire responses (SIG/CAIQ reference)	Pre-answers for an inbound SIG/CAIQ - many answers are "N/A: customer-hosted"	In-Review (reference)
Accessibility Conformance Posture (on request; not on this site)	WCAG 2.1 AA target, what is enforced vs unverified, the VPAT path	In-Review
Security Policy / responsible disclosure (on request; not on this site)	How to report a vulnerability, response SLAs, safe harbor	Active

Formal certifications¶

EGMW holds no SOC 2 report or ISO 27001 certification, and no such audit is planned now or in the immediate future. This is a statement about applicability, not a gap waiting to be closed: SOC 2 and ISO 27001 attest a service organization's controls over the customer data it hosts or processes. Paridity is software you run in your own environment; EGMW does not host it and does not process your data. The controls that matter for your deployment - access, encryption, backup, network, monitoring - are yours, configured in your own cloud account and covered by your own certifications and audits. Open-source and supply-chain hygiene is documented separately in the OSS & Supply-Chain Posture.

Company-level assurances, stated plainly¶

Evil Genius MindWorks LLC (EGMW), the company behind Paridity, is a small, privately held business. These are the answers a vendor-risk review usually asks for, as they stand today:

Assurance	Today
SOC 2 / ISO 27001	None held, none planned (see above).
Third-party penetration test	None performed and none scheduled. What runs on every code change is automated: Semgrep static analysis, dependency vulnerability audits (`pip-audit`, `npm audit`), a full-history Gitleaks secret scan, and a Trivy configuration scan; release images are Trivy-scanned before release and re-scanned weekly. A dynamic scan (DAST) runs weekly in CI: OWASP ZAP against a throwaway stack built from the current code - a passive baseline scan of the production UI image, and an authenticated, active scan of the API driven by the API's own OpenAPI document. The job fails on any finding in a FAIL-level class (injection, cross-site scripting, path traversal, XML external entities, remote code execution, server-file disclosure, open redirect, and the security headers already in place) and when the API scan cannot be shown to have run authenticated. This is automated scanning, not a penetration test: it does not chain findings, test business logic or tenant isolation, or test a real deployment's ingress, TLS or identity provider. Because Paridity runs in your environment, you may test your own deployment with your own tools or testers (Acceptable Use Policy §6.1); testing EGMW-operated systems such as the licence service needs EGMW's written authorization first (§6.2).
Internal security review	An internal, AI-assisted code review of the Paridity source (#3357), carried out by EGMW with an AI model in four passes from 2026-10-30 to 2026-11-01. No outside party took part. Scope: tenant isolation and authorization; authentication (bearer tokens, SAML, OIDC, SCIM, API keys); generated SQL; licensing; secrets and logging. Every item in scope was read line by line and either demonstrated as a finding, ruled out with a recorded counter-check, or pinned by a test. A finding required a test, committed to the repository, that failed against the code before the fix. Five findings: three high (#3365, #3369, #3398), one medium (#3383), one low (#3384); all five were fixed by 2026-11-01. This is a secure-development control, not a penetration test and not a third-party assessment: it reviewed source code and a development stack, did not test a customer deployment's ingress, TLS or identity provider, and did not exercise exploit chains against live credentials. The review record (method, coverage, findings, hypotheses not demonstrated, ruled-out items with their counter-checks, follow-ups) is retained by EGMW and available on request.
Cyber insurance / certificate of insurance	EGMW holds no cyber-insurance policy and cannot provide a certificate of insurance at this time. If a customer contract requires coverage, EGMW will review the requirement.
Company viability	No formal viability statement and no financial statements are published. The mitigant is architectural, and it has a limit: see what happens without EGMW.
Formal security program / incident-response plan	Neither is published. EGMW is a small business and has no formal, documented security program or incident-response plan today. What exists is the engineering practice these documents describe, and the coordinated vulnerability-disclosure policy.

What happens without EGMW¶

Paridity runs entirely in your environment. Its only runtime dependency on EGMW is the licence service. If that service becomes unreachable - an outage, or EGMW ceasing to operate it - an installation keeps running normally for 7 days on its last signed licence token. After that, a paid licence is degraded until a refresh succeeds: runs continue and are never stopped, but they are proof-only (counts and hash totals, no field-level diffs), and bringing new configuration into service (for example, creating or enabling a mapping or schedule) is refused. No data is touched. The honest risk: if the licence service never returns, that degraded state is permanent. There is no source-code escrow, no perpetual-licence fallback, and no offline activation path today.

How to use this with your VRM process¶

Read the five design properties above - most control questions resolve there.
Go deeper via the Security Whitepaper + Architecture Brief + Data-Flow disclosure.
For SOX/ICFR or audit stakeholders, use the Accuracy + Compliance proofs - and actually run the verify-it-yourself steps.
If you must map a SIG/CAIQ, the questionnaire reference is pre-answered; many rows are "not applicable - customer-hosted, no stored data."
Route anything unanswered through your Paridity account contact.

Classification: Customer Confidential - a curated index; each linked document carries its own verification date and status. Do not distribute externally without Paridity approval.

Paridity Security Whitepaper¶

Verified data. Trusted outcomes.

CDC-Aware Reconciliation Platform

Prepared for: General distribution Version: v0.1 · Date: 2026-10-29

Executive summary¶

Most vendor-risk reviews start with the same question: where does our data go, and who can touch it. For Paridity, the answer today is structural, not procedural: it doesn't go anywhere, and no one at Paridity can touch it. In Paridity's current shipping architecture, the entire platform - API, UI, workflow orchestration, all workers, and the PostgreSQL evidence store - is deployed inside the customer's own cloud account/VPC via a Helm chart and Terraform modules the customer controls. Paridity, the vendor, operates no infrastructure that holds or processes customer data, and has no network path to a customer's credentials, schema, object names, or evidence values. This removes an entire category of vendor risk that a hosted SaaS reconciliation tool cannot: there is no Paridity-operated environment holding customer data to breach, and no cross-customer blast radius from a shared multi-tenant service, because none exists in this deployment model. The one Paridity-operated service an installation talks to is the licence service, run by Evil Genius MindWorks LLC (EGMW), the company that makes Paridity, and hosted on Railway (the Data-Flow Disclosure names EGMW's sub-processors). It receives licence data only: the licence key, a machine id derived from a random installation id, the activation id, the product name, the platform and CPU architecture, and the request's source address - never customer data. In releases that enforce the licence (every release after v1.4.0), an installation that cannot reach it runs normally for 7 days, then a paid licence runs proof-only (counts and hash totals, no field-level diffs, and no new configuration brought into service) until it can; runs are never stopped.

On top of that architectural position, Paridity layers conventional defense-in-depth: OIDC/JWT-based authentication with role-based access control, PostgreSQL row-level security scoped to tenant as a database-layer backstop, a secrets-adapter model that never stores a credential in Paridity's own tables, a chained masking model that governs whether sensitive diff values are stored or displayed at all, and an append-only, cryptographically hash-chained evidence trail built for SOX/ICFR-grade audit. This paper documents what is implemented today and is explicit about what remains roadmap - per Paridity's own accuracy and honesty standard, an unverified claim is not stated as fact.

Problem¶

Enterprises evaluating a reconciliation platform to prove that data from their operational systems of record - SAP, Oracle, DB2, and other databases, plus SaaS and file sources - lands correctly in analytical targets (Snowflake, Databricks, BigQuery, Redshift, ADLS) are, by definition, evaluating a tool that will sit next to systems of record and produce evidence that may be relied on for SOX/ICFR controls. Extending that kind of access to a new vendor is itself a risk decision. Conventional SaaS reconciliation and data-observability tools typically require exporting database credentials, schema/object metadata, and often the underlying data values, to a vendor-hosted environment - expanding the customer's external attack surface, creating a new compliance-review burden at every renewal, and making the customer's audit evidence dependent on a third party's own security posture and uptime.

A vendor-risk team's standard questions - where does data live, who can access it, what happens in a vendor breach, is there a certification behind these claims - are exactly the questions this document answers directly, without inflating what is built versus what is planned.

Approach / architecture¶

Deployment & data-residency architecture¶

Today's shipping architecture. The full Paridity stack deploys inside the customer's own cloud account/VPC:

FastAPI control plane, React/Vite UI, Temporal orchestrator, and all six Temporal workers (scheduler, readiness, SQL, diff, evidence, notification)
A single PostgreSQL instance holding the configuration, run-state, and evidence schemas

Deployment is via an in-repo Helm chart covering the full stack (API, UI, Temporal, workers, preflight/migrate jobs, autoscaling, ingress) and Terraform reference modules for AWS, Azure, and GCP provisioning the cluster, a managed Postgres instance with backups/PITR, the customer's own secret backend, and an egress-locked network. Node egress is deny-by-default on all three clouds and reaches cloud services over private paths rather than the internet - AWS via PrivateLink VPC endpoints, GCP via Private Google Access, Azure via NSG service-tag rules - with only the operator's declared endpoints permitted outbound. This egress lockdown is verified live on all three clouds, by a from-empty deployment plus a probe confirming that a non-declared destination is refused outbound. Paridity operates no part of this infrastructure - there is no Paridity-hosted control plane, no Paridity-visible copy of customer data, and no Paridity credential store in this model. This is today's shipping architecture: the packaging is delivered and continuously validated in CI.

The one outbound path to Paridity: the licence check. The API pods and the scheduler worker call the EGMW licence service over HTTPS (port 443; default https://license.evilgenius.io, set by LICENCE_SERVER_URL; http:// is refused outside development and test) to activate and deactivate a licence and, in releases that enforce the licence, to refresh a signed licence token once a day. The request bodies carry only the licence key, the machine id, the activation id, the product name, and the platform and architecture - never an object, schema, or column name, a credential, an evidence value, or a tenant or user identity. The installation verifies the returned token offline against public keys built into the release image; the keys are a release constant, not a runtime setting, file, or database value. Because the licence service has a stable name but not a stable IP address, an egress-locked deployment routes the call through the customer's HTTPS egress proxy (the client honours HTTPS_PROXY and accepts a custom CA for a TLS-inspecting proxy); neither the Helm chart nor the Terraform modules open this path by default. Without it, the installation runs normally for 7 days on the token it holds, then a paid licence runs proof-only until a refresh succeeds - never a halt. An air-gapped installation cannot be activated today: the product reads a staff-issued offline activation code, but the licence service cannot yet issue one for Paridity. The Data-Flow Disclosure §3 gives the full detail.

Notification egress. Alerts and tickets go only to the destinations a platform admin configures on a notification channel (setting or changing a channel's credential or destination needs platform_admin). Every send is checked before the credential leaves the process: HTTP channels must use https, redirects are not followed, the destination's resolved addresses must not be private, loopback, link-local (including the cloud metadata endpoint 169.254.169.254), CGNAT, multicast or unspecified, and the address actually connected to is checked again before any response is read (for SMTP, before the password is sent). For internal systems (an SMTP relay, Jira Data Center, an on-premises ServiceNow) a platform admin can add an exact host and port to the tenant's private destination allowlist. An entry permits only addresses in 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 or fc00::/7 for that host and port, only for that tenant's notification channels; the https, no-redirect and connected-address checks still apply, and the connection must land on an address the name resolved to when it was checked. Loopback, link-local and the AWS / Alibaba metadata addresses can never be allowlisted: such an entry is refused when it is added and such an address is refused at send time. Every allowlist change is audited. The Data-Flow Disclosure §4 describes the providers.

This whitepaper describes the shipping, customer-hosted architecture. A hosted deployment option is on our internal roadmap with no committed date; if it ships, the same hard invariant already governs its design - credentials, object/schema/column names, and evidence field values never leave the customer VPC and never reach Paridity-operated infrastructure - and that invariant is enforced as a review gate on all new work today.

Controls / details¶

Access control¶

Authentication. All user-facing API endpoints require an OIDC/JWT bearer token. Token validation enforces RS256/ES256 signature algorithms only (no alg=none), and verifies audience, issuer, and expiry before a request is authenticated. Audiences are applied per path: SCIM provisioning endpoints require their own dedicated audience and the /mcp agent surface another, so a user token cannot drive SCIM or /mcp and a SCIM or agent token cannot reach user endpoints.
Development bypass is production-guarded. A development-only authentication bypass exists for local use; the platform refuses to honor it outside a development/test environment, and it only short-circuits when no credentials are supplied at all - if a real bearer token or API key is present, it is validated normally, so RBAC still applies.
RBAC. Four roles form a strict hierarchy: platform_admin > operator > analyst > auditor. Role checks are enforced in the API layer only - never inside workers or services. Access is further scoped through grants that bind a principal to specific resources (e.g. a business domain), not just a role.
Single sign-on. Interactive browser SSO is available over SAML 2.0 (Okta and Microsoft Entra ID both live-verified) and, since v1.4.0, over OIDC: a server-side authorization-code flow with PKCE, where the API holds the client secret through the customer's secrets backend and accepts an ID token signed only with RS256, PS256, or ES256. OIDC sign-in is live-verified with Microsoft Entra ID; Okta OIDC sign-in is not yet live-tested. Both sign-in paths end in the same session step. SCIM user/group provisioning automates onboarding/offboarding. Separately, OIDC bearer tokens from a registered per-tenant issuer are validated for API and service-account access. Admin-sensitive actions (e.g. bypassing readiness on run creation) require platform_admin and are recorded in the tamper-evident audit log.
Tenant isolation. Every tenant-scoped table carries a tenant_id column, and every application query filters by it. As a defense-in-depth backstop below the application layer, PostgreSQL row-level security (RLS) is live in the production schema: 30+ tenant-scoped tables each carry a policy restricting visibility to the current tenant, and an unset session context fails closed to zero rows rather than failing open. Stated honestly: the application's database role currently retains a bypass on RLS, so RLS is today a backstop against a missing tenant predicate rather than the sole enforcement mechanism; binding the owner role with FORCE ROW LEVEL SECURITY is intentionally deferred until every application call-site is confirmed to set the tenant context. Primary tenant isolation today is the application-layer tenant_id predicate on every query; RLS is the layered backstop.

Secrets management¶

Paridity never stores customer credentials in its own database. A system's connection reference is always a URI pointer, never a secret, e.g. aws-secrets://..., azure-keyvault://..., gcp-secretmanager://..., infisical://.... A secrets client resolves the pointer to the actual credential at runtime by dispatching to the customer's own configured backend:

AWS Secrets Manager
Azure Key Vault
GCP Secret Manager
Infisical (self-hosted, MIT-licensed)
HashiCorp Vault - supported as an optional adapter for existing Vault customers only, never a default or required component.

In multi-tenant deployment configurations, each backend enforces per-tenant access scoping at the IAM/policy layer (e.g. AWS Secrets Manager resource-tag conditions, Azure Key Vault tenant-prefixed secret names, GCP Secret Manager IAM conditions on label match, Infisical per-tenant projects), so one tenant's application identity cannot resolve another tenant's secret. Because Paridity is single-tenant per deployment, this per-tenant secret scoping is inherent to the deployment model itself - each customer's Paridity instance only ever holds a pointer to that customer's own secret backend, in that customer's own cloud account.

Data protection¶

Evidence masking is an AND/OR-chained policy, not a single toggle. store_raw_diff_values - whether a mismatched field's actual before/after values are persisted at all - is an AND-chain across the tenant default, the mapping config, and the field config: any layer set to FALSE suppresses storage entirely and the row is written with NULL values and a masked flag. mask_in_ui - whether a field is masked on display even when raw values are stored - is a per-field control: set it TRUE on a field and display is masked (a tenant-wide display-mask policy is not currently offered). Evidence retrieval endpoints apply this masking at query time; a masked field never returns raw values regardless of caller role.
Evidence is append-only and tamper-evident. Evidence tables are never updated or deleted outside the retention purge function and two narrowly-scoped lifecycle exceptions (anomaly acknowledgment state; the single pending→generated evidence-bundle fill). Core evidence values are immutable once written. Evidence bundles are built once and sealed with a detached signature over the bundle manifest - which commits to every file's SHA-256 - plus a previous-bundle-hash linking each bundle to the prior one per mapping: a chain an auditor can walk independently, offline, with the shipped fail-closed verifier. The signing key is the customer's choice. A cloud-KMS key (AWS KMS, Azure Key Vault, or GCP KMS) or a local ECDSA key signs asymmetrically, so a third party verifies with the public key alone - no customer secret is disclosed and no platform access is needed; the private key never leaves the KMS. A customer-held HMAC key remains available. The bundle embeds the public key as a convenience and records its SHA-256 fingerprint in the manifest, but the assurance path is a key or fingerprint obtained out of band from the customer's own KMS - an attacker who rebuilds a bundle can embed their own key, so the embedded copy is never treated as proof by itself. A bundle whose signing key cannot be resolved ships unsigned, and the offline verifier refuses to report it verified.
Encryption in transit / at rest. Ingress terminates TLS for the UI/API; the pre-production security checklist requires Postgres connections to use TLS (sslmode=require or stricter) and managed-Postgres storage encryption, with a customer-managed key (CMK) as an available option. Stated honestly: these are deployment-configuration requirements verified at install time via the checklist, not properties Paridity's application code enforces unilaterally - because Paridity deploys into the customer's own cloud account, the customer's own cloud provider (KMS, TLS termination, managed-Postgres encryption) is the actual enforcement point, configured per the Terraform/Helm defaults and the pre-production checklist.
Encrypted source/target columns. Where a source or target column is itself encrypted (e.g. Oracle TDE, Snowflake native encryption), a per-field decryption expression is injected into the connector's generated SQL at query time only; decryption keys are resolved through the customer's secret backend, and Paridity never persists a decrypted value outside the normal evidence-masking rules above.
Object-name masking is per-surface, and we name the difference rather than let a reviewer find it. A tenant setting replaces table, schema, and column names with opaque <OBJECT:sha256[:8]> tokens - but it delivers a different guarantee on each of the two surfaces that can carry a name outward. On AI prompts Paridity itself routes to a model provider, the masking is absolute: masking is deny-by-default, so a string survives unmasked only if it is on a curated safe list, sits in fixed instructional text, or is mechanically provable as a non-name (an existing token, UUID, pure number, ISO-8601 timestamp, or phase/attempt id). A pattern miss over-masks; it cannot leak. On the /mcp agent surface it is best-effort: only values under a defined list of object-name fields are tokenized, and any other field carrying a name is returned in clear text.
Why the MCP behaviour is deliberate, and the residual risk it leaves. /mcp is an authenticated customer API at parity with REST, not a Paridity-routed prompt path: the customer's own agent authenticates with the customer's own credentials against a model the customer chose, so the customer is the disclosing party. Deny-by-default masking would defeat the surface's purpose - triage requires naming which column drifted, not merely that one did. The residual risk we state explicitly: same authorization is not the same disclosure surface. A person reading a table name on a Paridity screen does not thereby place it in an LLM context window, a provider's request logs, or a retention window; an MCP client does, automatically, on every call. The permission check is identical to REST; the practical egress is not. In short: on /mcp it is your agent, your credentials, your disclosure - a single setting deliberately governs both surfaces, because a dedicated MCP toggle would present a best-effort mechanism as a guarantee a reviewer could rely on. Compensating controls, each enforced in code rather than by convention: the surface is off by default (with a 403 when disabled) and, in production, the API refuses to start with AI enabled unless a tenant has explicitly opted in; per-tool RBAC uses the same permission strings as the equivalent REST routes; tenant scope always comes from the authenticated principal, never a tool argument; the catalog is read-only, with arbitrary SQL, shell, secret retrieval, file access, and direct database modification permanently excluded; every dispatched tool call is audited; and a per-principal, per-tool rate limit sheds abuse before dispatch. /mcp is classified data-plane and runs entirely inside the customer's own environment, like the rest of the platform. The control that closes the gap is the customer's: govern which MCP clients and which models may hold Paridity credentials, exactly as you would govern REST API keys.

Supply-chain / OSS posture¶

Every dependency in the technology stack is MIT, Apache 2.0, or a comparably permissive license (e.g. the PostgreSQL License) - no GPL, AGPL, or SSPL component is used anywhere in the platform. Two components are deliberately excluded from default use for licensing reasons rather than adopted and then carved around: HashiCorp Vault (BSL 1.1) is optional-only, never a default or required dependency; Confluent Platform components (Schema Registry, ksqlDB, REST Proxy) are excluded in favor of Apache Kafka core only, because the Confluent Community License bars competing SaaS offerings. Every release additionally publishes a machine-generated SPDX SBOM per container image, generated from the published image digests, and signs every released image and the Helm chart with keyless signing (Sigstore) so a customer can verify provenance before mirroring the images into their own registry (most recently verified on the v1.4.0 release, 2026-10-28). Every code change is gated in CI by dependency-vulnerability audits (Python and production JavaScript), Semgrep static analysis, a full-history secret scan (Gitleaks), and a configuration scan (Trivy); release images are additionally Trivy-scanned for fixable HIGH/CRITICAL vulnerabilities before they are pushed, and the latest release's images are re-scanned weekly. A dedicated supply-chain posture document is available for third-party-risk review.

Proof / evidence¶

Paridity's core correctness claim - the thing a security/audit reviewer should actually trust the product on - is deterministic, not AI-assisted or heuristic. A run establishes correctness through aligned time windows, an aggregate dataset fingerprint (hash_total) as a fast pass/fail proof gate, per-row fingerprints (row_hash) to isolate exactly which rows differ, and column-level output (field_diff) as the actual reconciliation evidence. Hash implementations are verified byte-for-byte against a reference engine on every supported database, and an accuracy issue is never closed until a live test vector confirms the fix. Every run's config is fingerprinted (config_checksum) and written to an append-only config audit before the proof phase begins, so an auditor can confirm the policy in force during a run independent of its outcome.

The proof gate's threat model is scoped explicitly. The hash_total aggregate defends against non-adversarial divergence - replication defects, corruption, drift - with a quantified collision probability (about one in ten quintillion per comparison). It is not designed to resist a construction-aware insider who can write the datasets being compared: a 64-bit SQL-pushable aggregate is adversarially weak by construction, and adversarially secure alternatives cannot be computed inside the customer's own engines, which is the property the proof's verifiability rests on. That insider case is covered by compensating controls, not by the aggregate: the customer's own database access controls and change auditing, Paridity's append-only evidence, the hash-chained and checkpoint-signed audit log, signed offline-verifiable evidence bundles, and RBAC separation. The full threat-model statement is given in Paridity's accuracy proof documentation.

Note: AI in Paridity assists triage and explanation only - it never determines whether data matches. Correctness is always established by the deterministic hash_total / row_hash / field_diff pipeline above, never by a model's judgment.

Limitations & roadmap honesty¶

Capability	Status	Notes
Customer-hosted deployment - customer data never leaves customer VPC	Verified	Helm + Terraform; the full stack deploys in the customer VPC, delivered and CI-gated. The one outbound call to Paridity is the licence check, which carries licence data only (next row).
Licence check to the EGMW licence service	Supported	Activation, deactivation, and (in releases after v1.4.0) a daily token refresh over HTTPS 443; licence data only. 7 days of normal running without it, then proof-only for a paid licence, never a halt. Egress-locked deployments need an HTTPS egress proxy for it; neither the chart nor the Terraform modules open the path by default. Offline (air-gapped) activation is not yet available from the licence service.
OIDC/JWT authentication, RBAC, scoped grants	Verified	Enforced in the API authentication layer.
SSO - Okta (SAML)	Verified	Live-verified.
SSO - Microsoft Entra ID (SAML)	Verified	Live-verified, including SCIM provisioning.
SSO - OIDC sign-in (authorization code + PKCE)	Verified (Entra); Supported (other providers)	Live-verified with Microsoft Entra ID before v1.4.0; Okta OIDC sign-in not yet live-tested.
SCIM user/group provisioning	Verified	Create / update / deprovision live-verified against Microsoft Entra.
Tenant isolation - application-layer `tenant_id` predicate	Verified	Enforced on every query.
Tenant isolation - PostgreSQL RLS (defense-in-depth)	Supported	Live in the production schema; `FORCE ROW LEVEL SECURITY` intentionally deferred pending a call-site audit (the app role currently retains an RLS bypass).
Secrets never stored by Paridity; pluggable backends	Verified	The connection reference is a URI pointer only, resolved at runtime against the customer's own secret backend.
Evidence masking (`store_raw_diff_values` AND-chain; `mask_in_ui` per-field display mask)	Verified	Enforced at config-resolution and query time. A tenant-wide display-mask policy is not currently offered - masking display is a per-field control.
Object-name masking - AI prompt path	Verified (absolute)	Deny-by-default; a pattern miss over-masks and cannot leak.
Object-name masking - `/mcp` agent surface	Best-effort by design - stated, not implied	Allowlisted object-name fields are tokenized; other fields carrying a name are returned in clear text, at REST parity. Deliberate - the customer's agent and model are the disclosing party. Bounded by the controls in "Data protection" above; do not represent it as absolute.
Append-only, hash-chained evidence bundles	Verified	Per-file SHA-256 manifest + detached signature over the shipped bytes + previous-bundle chain; verifiable offline with the shipped fail-closed tool. Asymmetric signing (AWS KMS, Azure Key Vault, GCP KMS, or a local ECDSA key) lets a third party verify with the public key alone - no customer secret; a customer-held HMAC key remains available. An unsigned or tampered bundle never reports as verified.
Evidence-bundle retention with a built-in purge schedule	Supported	`bundle_retention_days` resolves per tenant with a platform floor; a built-in `RetentionPurgeWorkflow` runs on a default-on schedule, dry-run first, and writes an audit row for every deletion - including the object store's own answer per object. Cadence and thresholds are deployment configuration.
Legal hold - freeze evidence from retention purge	Supported	An approval-gated hold placed on a tenant, mapping, or bundle causes the purge to skip and audit the skip. Placing and releasing a hold each require the `evidence.legal_hold` permission and are written to `cfg.audit_log`.
Period bundles in customer-owned, write-once object storage	Verified	Optional per-tenant tier: the signed container is written to the customer's own S3, Azure Blob, or GCS bucket and the platform database keeps no bytes - only the object URI, size, SHA-256, and the store's receipt. Where the store confirms immutability (S3 Object Lock, Azure immutability policy, GCS retention), the bundle is attested as anchored; an unprotected store is never attested as one.
TLS in transit / encryption at rest	Supported	Enforced via deployment configuration (Terraform/Helm defaults + pre-production checklist), not unilaterally by application code - see "Data protection" above.
OSS license posture (no GPL/AGPL/SSPL)	Verified	Permissive-only by binding engineering rule.
Machine-generated SBOM + signed release artifacts	Verified	SPDX SBOM per image + keyless (Sigstore) image and chart signatures, published every release.
SOC 2 / ISO 27001 certification	None held, none planned	No certification exists, and no SOC 2 or ISO 27001 audit is planned now or in the immediate future. These attest a service organization's controls over the customer data it hosts or processes. Paridity is software the customer runs in its own environment; EGMW does not host it and does not process customer data, so the controls that matter for the deployment are the customer's own, under the customer's own certifications and audits.
Proof gate vs. adversarial dataset tampering	Out of scope by design - stated, not implied	The `hash_total` proof targets replication faults/corruption (quantified collision bound); a construction-aware insider writing compensating rows into the compared datasets defeats any SQL-pushable 64-bit aggregate. Covered by compensating controls - see "Proof / evidence" above.
Independent penetration test	Not performed, none scheduled	No third-party penetration test has been performed and none is scheduled. What runs is automated, on every code change: Semgrep static analysis, dependency vulnerability audits, a full-history Gitleaks secret scan, and a Trivy configuration scan; release images are Trivy-scanned before release and re-scanned weekly. A dynamic scan (DAST) runs weekly in CI: OWASP ZAP against a throwaway stack built from the current code - a passive baseline scan of the production UI image, and an authenticated, active scan of the API driven by the API's own OpenAPI document. The job fails on any finding in a FAIL-level class (injection, cross-site scripting, path traversal, XML external entities, remote code execution, server-file disclosure, open redirect, and the security headers already in place) and when the API scan cannot be shown to have run authenticated. This is automated scanning, not a penetration test: it does not chain findings, test business logic or tenant isolation, or test a real deployment's ingress, TLS or identity provider. The customer may test its own deployment, which runs in its own environment (Acceptable Use Policy §6.1).
Internal security review	Performed 2026-10-30 to 2026-11-01 (internal, not third-party)	An internal, AI-assisted code review of the Paridity source (#3357), carried out by EGMW with an AI model in four passes from 2026-10-30 to 2026-11-01. No outside party took part. Scope: tenant isolation and authorization; authentication (bearer tokens, SAML, OIDC, SCIM, API keys); generated SQL; licensing; secrets and logging. Every item in scope was read line by line and either demonstrated as a finding, ruled out with a recorded counter-check, or pinned by a test. A finding required a test, committed to the repository, that failed against the code before the fix. Five findings: three high (#3365, #3369, #3398), one medium (#3383), one low (#3384); all five were fixed by 2026-11-01. This is a secure-development control, not a penetration test and not a third-party assessment: it reviewed source code and a development stack, did not test a customer deployment's ingress, TLS or identity provider, and did not exercise exploit chains against live credentials. The review record (method, coverage, findings, hypotheses not demonstrated, ruled-out items with their counter-checks, follow-ups) is retained by EGMW and available on request.
Formal security program / incident-response plan	Not in place	EGMW is a small business with no formal, documented security program or incident-response plan today. Contractual incident terms are in the agreement.
Disaster recovery (backup/PITR, restore playbook)	Supported	RPO/RTO targets and restore procedures are documented per deployment tier. The restore-validation drill is automated in Paridity's CI on a quarterly schedule against its reference environment; it has executed once to date (2026-08-17, a manual dispatch), and the first scheduled run is 2026-11-01. RTO at realistic evidence volume is not yet measured.

Further information¶

Trust Center - the index to Paridity's security and compliance materials.
Supply-chain & OSS posture - full dependency/license inventory, SBOM, and signing verification steps.
Accuracy proof & verification guide - the deterministic correctness record behind the hash_total / row_hash / field_diff claims, with steps to reproduce them yourself.
Additional architecture and control documentation (deployment model, tenant isolation, disaster recovery, the pre-production security checklist) is available to reviewers on request through your Paridity account contact.

Paridity data-flow & sub-processor disclosure¶

Procurement and data-protection teams ask two questions by name: where does our data flow and who are your sub-processors. Because Paridity's architecture runs entirely inside the customer's own cloud account, the honest answers are unusually short - and that is the point.

1. The one-sentence answer¶

Customer data never leaves the customer's cloud account, and no Paridity sub-processor receives customer source data, schema/object names, or evidence values. Two things can leave, neither of them customer data: a licence check to EGMW's own licence service, which carries licence data only, and a support email with an optional redacted diagnostic bundle, only when an operator chooses to send one (§2, §3). EGMW uses two sub-processors for that data, named in §4: Railway hosts the licence service, and Microsoft 365 hosts EGMW's support mailbox. The optional AI features are bring-your-own-key: when a customer enables them, prompts go to the AI provider the customer configures and contracts with directly (§4) - a vendor relationship the customer owns, not a Paridity sub-processor.

2. Data classes (what data exists, and where it lives)¶

Data class	Examples	Where it lives	Leaves customer VPC?
Source/target row data	The actual values being reconciled	Read from the customer's own systems by in-VPC workers; only hashes/diffs are retained	No
Evidence	`hash_total`, `row_hash`, `field_diff`, bundles	Customer's own PostgreSQL (`ev` schema), in the customer's cloud. Optionally, signed evidence-bundle containers are written instead to the customer's own object store (S3, Azure Blob, or GCS) named by a tenant-configured destination pointer - the customer's bucket, the deployment's own credential chain; the database then retains only the object URI, size, hash, and the store's receipt.	No
Metadata	Object/schema/column names, mapping config, `resolved_config_snapshot`, `known_column_set`	Customer's own PostgreSQL (`cfg`/`run`)	No
Credentials	Source/target DB logins, keys	Never stored by Paridity - `connection_ref` is a URI pointer into the customer's own secrets backend; resolved at runtime	No
Audit trail	`cfg.audit_log`, actor/IP, config audit	Customer's own PostgreSQL	No
Telemetry (optional)	OpenTelemetry traces/metrics	Customer's own observability stack; span attributes never include row values	No (customer-controlled)
Licence	Licence key; a machine id (a SHA-256 of a random installation id that the platform's own database mints - never a host name, network adapter, or hardware id); the activation id the licence service issued; the product name (`paridity`); the operating-system platform and CPU architecture	The key stays in the customer's own secrets backend (the platform holds a pointer to it, plus a hash and a masked form). The values are sent over HTTPS to the Paridity licence service: activation sends the key, machine id, platform, architecture, and product; the daily refresh sends the key, machine id, and product; deactivation sends the activation id and machine id. The licence service records the request's source address.	Yes - licence data only, never customer data
Support diagnostic bundle (operator-initiated)	Run status, timings, readiness and drift history, anomaly counts, worker health, and an audit excerpt for one run or mapping	Generated inside the deployment as a JSON file an operator downloads; nothing is sent automatically. It leaves only if the operator attaches it to a support email. Column names, evidence values, hashes, connection references, secrets, and SQL text are never included; object names are replaced with opaque tokens unless the operator opts in to keep exactly the mapping name, the source/target `schema.table` names, and the tenant name. Requires the `diagnostics.export` permission.	Only if the operator sends it

There is no Paridity-operated database, control plane, or evidence store in this model. The only data Paridity holds is licence data in its licence service (hosted on Railway): the licence and its customer record, the activated machine id, the platform and architecture reported at activation, and the source addresses of licence requests - plus any support email, and any diagnostic bundle attached to it, that a customer chooses to send to EGMW's support mailbox (hosted on Microsoft 365). Paridity holds no customer data.

3. Data flow (customer VPC)¶

   Customer source systems ─┐
   (SAP/Oracle/DB2/…)       │  (read, in-VPC)
                            ▼
              ┌─────────────────────────────┐   Everything inside this box runs in the
              │   Customer cloud account /  │   customer's own account. No customer data
              │   VPC (customer-controlled) │   crosses out to Paridity-operated
              │                             │   infrastructure.
              │  Workers → hash/diff →      │
              │  PostgreSQL (cfg/run/ev)    │
              │  API/UI ← operators (SSO)   │
              │                             │  HTTPS 443, licence data only
              │  API pods + scheduler ──────┼─────────────► EGMW licence service
              │  worker (licence calls)     │  (license.evilgenius.io by default)
              └─────────────────────────────┘
                            ▲
   Customer target systems ─┘
   (Snowflake/Databricks/…)   (read, in-VPC)

Credential resolution (SecretsClient → the customer's own AWS SM / Azure KV / GCP SM / Infisical) and identity (the customer's own OIDC/SAML IdP - Okta or Microsoft Entra ID) are also entirely within the customer's control. Paridity is not in the path.

The licence path. The API pods and the scheduler worker call the licence service over HTTPS (port 443) at the address in the LICENCE_SERVER_URL setting (default https://license.evilgenius.io; an http:// address is refused outside development and test). The calls are: activation and deactivation, when an operator performs them; a refresh of the signed licence token once a day per tenant (at most once every 15 minutes); and a reachability probe, with an empty request, once at scheduler start. The daily refresh and the probe run only in releases that enforce the licence: every release after v1.4.0. v1.4.0 and earlier releases call the licence service only when an operator activates or deactivates.

Egress. The client honours the standard HTTPS_PROXY / NO_PROXY variables, and a custom CA certificate can be configured for a TLS-inspecting proxy. The licence service's name is stable but its IP address is not, so an egress-locked deployment (the Helm NetworkPolicy and the Terraform egress lockdown are both CIDR-based) routes the call through the customer's HTTPS egress proxy and permits that name on the proxy. Neither the chart nor the Terraform modules open this path by default.
If the licence service is unreachable. The signed token is valid for 7 days, so the installation keeps running normally for 7 days without it. After that, a paid licence is degraded until a refresh succeeds: runs are proof-only (counts and hash totals, no field-level diffs) and bringing new configuration into service (for example, creating or enabling a mapping or schedule) is refused. Runs are never stopped, and no data is touched.
Air-gapped installations. The product can read a staff-issued offline activation code, but the licence service cannot yet issue one for Paridity, so an installation with no outbound path cannot be activated today. In a release that enforces the licence it runs degraded, as above, until it is activated.

4. Sub-processor list¶

Core product (reconciliation, evidence, audit): no sub-processor. No customer data reaches Paridity-operated infrastructure or any EGMW vendor. The customer's own cloud provider (AWS / Azure / GCP) is the customer's own infrastructure, not a Paridity sub-processor.

EGMW's sub-processors (licence and support data; no customer data from the product):

Sub-processor	What EGMW uses it for	Data it receives	Customer data?
Railway (cloud hosting)	Hosts EGMW's licence service, which EGMW operates. `license.evilgenius.io`, the default licence address, is a DNS alias for the Railway-hosted service.	The licence data in §2: licence key, machine id, activation id, product name, platform, architecture, and the request's source (IP) address. The licence service also holds the licence and its customer record (the customer the licence was issued to, the licence type, and its term).	No - no source data, object/schema/column names, credentials, evidence values, or tenant or user identities
Microsoft 365 (Microsoft, Exchange Online)	Hosts EGMW's support mailbox, `[email protected]`.	Only what an operator chooses to email. The product's "Contact support" and "Report a Problem" links open the operator's own mail client with a prefilled draft naming the tenant, the Paridity version, the current page and, where relevant, a run id; the operator can edit it, and nothing is sent until the operator sends it. An attached diagnostic bundle carries what §2 lists and nothing else: no column names, evidence values, hashes, connection references, secrets, or SQL text, and object names replaced by opaque tokens unless the operator opts in. The email also carries the sender's address and whatever the operator writes.	Not from the product - only what an operator chooses to write or attach

The hosting region of the licence service is not stated in this document.

Product notifications do not pass through EGMW. Paridity has no EGMW email or messaging service. Notifications go only through the providers the customer configures on each notification channel, with the customer's own endpoints and credentials (the credential is a pointer into the customer's secrets backend): email through the customer's own SMTP server or Microsoft 365 mailbox (Microsoft Graph), Slack, Microsoft Teams, PagerDuty, Jira, ServiceNow, or a webhook. Those providers are the customer's vendors, not Paridity sub-processors.

Every notification send passes the platform's outbound policy before the channel credential leaves the process: https only for HTTP channels, no redirects, no private, loopback, link-local, CGNAT, multicast or unspecified destination address, and the connected address re-checked before a response is read (SMTP: before the password is sent). A destination on the customer's own network (an internal SMTP relay, Jira Data Center, an on-premises ServiceNow, an internal webhook) is sent to only when a platform admin has added its exact host and port to the tenant's private destination allowlist. An entry permits private addresses (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10, fc00::/7) for that one host and port and changes nothing else; loopback, link-local and cloud metadata addresses (169.254.0.0/16, fd00:ec2::254, 100.100.100.200) can never be allowlisted. The allowlist is stored in the customer's own deployment database and every change is audited.

Optional AI features (bring-your-own-key; only if the customer enables them):

External party (customer-contracted, not a Paridity sub-processor)	Purpose	Data exposed	Controls
The AI model provider the customer configures with their own key (e.g. their Anthropic account) - for AI Triage / AI Insights	Failed-run narration, risk/cost explanation, knowledge copilot	Prompt context for the requested explanation - object/schema/column names masked (`mask_object_names`; on this path the masking is absolute - see the note below), prompt context scrubbed, data-residency provider restriction applied; raw evidence values are not sent	Off by default; the provider key lives in the customer's secrets backend and the customer contracts with the provider directly; per-tenant AI settings + quota; deterministic fallback when AI is unavailable

Important: AI is an optional capability, not part of the correctness path - with it disabled there is no external AI traffic at all, and even with it enabled the provider relationship belongs to the customer (there is no Paridity-provided default provider). AI never determines correctness - it explains and triages only.

4.1 The `/mcp` agent surface is a customer egress path, not a Paridity sub-processor¶

If a customer enables the MCP agent surface, their own MCP client - Claude Desktop, Cursor, an internal pipeline agent - authenticates to /mcp with their own credentials and forwards what it reads into a model the customer chose and contracts with directly. Paridity routes nothing to that model and is not a sub-processor for it, but the flow deserves naming because the practical exposure differs from the UI even though the authorization is identical:

What /mcp returns. It is at parity with the REST API: object, schema and column names, run/readiness/drift metadata, and evidence aggregates and masked samples - under the same per-tool RBAC and the same query-time evidence masking (store_raw_diff_values, mask_in_ui) that REST enforces. It is a read-only catalog; arbitrary SQL, shell, secret retrieval, file access and direct database modification are permanently excluded from it.
What mask_object_names does here. It is best-effort on this surface, not absolute. A defined list of object-name fields is tokenized; a field carrying a name that is not on that list is returned in clear text. That is a deliberate design decision, not a defect - deny-by-default masking would make an agent unable to say which column drifted. See §4.2.
Who the disclosing party is. The customer's agent and the customer's model - your agent, your credentials, your disclosure. The customer's own model-provider agreement governs that hop, not Paridity's. One tenant toggle deliberately governs masking on both surfaces; a separate MCP masking control was considered and rejected (plan_mcp_enhancement_v1.md §3, decision record 6).

4.2 Residual risk we state rather than leave to be discovered¶

Same authorization is not the same disclosure surface. A person reading a table name on a Paridity screen does not thereby put that name into an LLM context window, a vendor's request logs, or a retention window. An MCP client does - automatically, on every call, for every field in the response. The permission check is identical to REST; the practical egress is not. "The customer controls the client and its model" is true, and it assumes the customer has considered which model that is - which is exactly the case a compliance control exists to cover when they have not.

We do not resolve this by tightening the masking (that would break the surface's purpose). We state it, and we rely on the controls that bound it:

Control	How it bounds the risk
Off by default, three tiers	`CFG_AI_ENABLED` defaults to false, so `/mcp` returns `403 MCP_DISABLED` on a stock deployment; and in `APP_ENV=production` the API refuses to start with AI enabled unless `CFG_AI_REQUIRE_TENANT_OPT_IN=true` (startup guard in `src/api/middleware/auth.py`), which makes `cfg.tenant_defaults.mcp_enabled` a required second opt-in; per-tool RBAC grants apply on top.
Per-tool RBAC, identical to REST	Every tool declares the same permission string the equivalent REST route requires (`run.read`, `mapping.read`, `evidence.read_masked`, `discovery.execute`), evaluated per call. An agent cannot read anything its principal could not read over REST.
Tenant scope from the principal	Tenant identity always comes from the authenticated principal, never from a tool argument; a cross-tenant lookup returns 404.
Read-only, with permanent exclusions	The shipped tool catalog is read-only. No arbitrary SQL, shell, secret retrieval, file access, or direct database modification tool exists, and none is permitted.
Audited and rate-limited	Every dispatched tool call writes an attributed row to `cfg.audit_log` - successes and failures alike, from a `finally` block, with the agent surface as `actor_type` and the human principal behind it attributed. A per-principal, per-tool ceiling sheds abuse before dispatch; a shed call is refused rather than audited, deliberately, so a rate-limited client cannot flood the audit log.
Data-plane classification	`/mcp` is classified data-plane and runs entirely inside the customer's own environment, like the rest of the platform - nothing on Paridity-operated infrastructure.

The customer-side control that actually closes it is the one only the customer can apply: govern which MCP clients and which models are permitted to hold Paridity credentials, exactly as you would govern REST API keys.

5. Hosted deployment option (internal roadmap)¶

Paridity is customer-hosted today; a hosted deployment option is on our internal roadmap with no committed date. It is being designed under a hard invariant: credentials, object/schema/column names, and evidence field values never leave the customer VPC and never reach Paridity-operated infrastructure - even under a hosted model. If it ships, this disclosure will be updated with the exact control-plane data classes (opaque IDs, counts, status, timings only) and any new sub-processors before a customer contracts against it. Until then, contract against the disclosure above.

Classification: Customer Confidential - reflects the shipping customer-hosted architecture as of the date above; verify against the current Security Whitepaper before relying on it in a signed agreement.

Paridity Business Continuity & Disaster Recovery Summary¶

Verified data. Trusted outcomes.

CDC-Aware Reconciliation Platform

Prepared for: General distribution Version: v0.1 · Date: 2026-10-29

Executive summary¶

Business-continuity reviews of a SaaS vendor usually turn on one question: if the vendor goes down, what happens to us? For Paridity the answer is structural, not contractual. In Paridity's current shipping architecture, the entire platform - API, UI, workflow orchestration, all workers, and the PostgreSQL evidence store - runs inside your own cloud account/VPC, on infrastructure you deploy and operate via a Helm chart and Terraform modules you control. Paridity, the vendor, operates no infrastructure in your reconciliation path. Your backups, your point-in-time recovery, your restore - all execute in your environment, under your cloud provider's managed-database machinery, using procedures Paridity documents and a restore-validation script Paridity ships. A Paridity vendor-side outage cannot take your deployment down, and Paridity's own continuity is not a dependency for the availability of your data or your reconciliation evidence. The one Paridity-operated service your installation uses is the licence service. In releases that enforce the licence (every release after v1.4.0), an installation that cannot reach it keeps running normally for 7 days on the signed licence token it holds; after that a paid licence runs proof-only (counts and hash totals, no field-level diffs, and no new configuration brought into service) until it can reach it again. Runs are never stopped and your data is never touched.

This summary distills Paridity's internal disaster-recovery documentation into the answers a continuity reviewer needs: recovery objectives per deployment tier, the backup and PITR approach, the restore procedure, the quarterly restore-validation playbook, and - per Paridity's own honesty standard - what is a manual runbook today versus what is roadmap, and which of these has actually been exercised.

The continuity model: customer-hosted by design¶

Who owns what. Because the full stack deploys in your VPC, continuity responsibility splits cleanly:

Responsibility	Owner	What it covers
Running the deployment (cluster, database, network)	Customer	Kubernetes cluster, managed Postgres, secrets backend, egress controls - provisioned by the in-repo Terraform modules you apply.
Executing backups and PITR	Customer (via cloud provider)	Managed-Postgres automated backups and point-in-time recovery in your own cloud account. Terraform provisions these by default (see below).
Executing a restore	Customer	Cloud-provider PITR/snapshot restore, then re-pointing the platform per the documented procedure.
Source/target system availability	Customer	Availability of the databases Paridity reconciles (e.g. Oracle, Snowflake) is outside Paridity's recovery scope.
DR procedures, restore runbook, and validation tooling	Paridity (documentation and tooling)	Documented backup/restore procedures, the restore-validation script, Terraform backup defaults, and the RPO/RTO guidance below. Scheduling and executing the drill is the customer's operational task - see "Restore validation" below for what has and has not been exercised.
Software updates and fixes	Paridity	Platform releases, migrations, and the deployment tooling that applies them.

Note: The recovery objectives in this document apply to the Paridity control plane (PostgreSQL + Temporal) running in your environment. Source and target system availability is the customer's responsibility.

Scope note: This document covers platform (deployment-level) continuity only. EGMW is a small, privately held business. It has no formal vendor-side business-continuity arrangement - no organizational BCP, key-person/succession plan, or source-code escrow - and publishes no viability statement or financial statements. The product's only runtime dependency on EGMW is the licence service: without it an installation runs normally for 7 days, then a paid licence runs proof-only (no field-level diffs, no new configuration brought into service) until a refresh succeeds - never a halt. If the licence service never returned, that degraded state would be permanent. See the Trust Center, "What happens without EGMW".

Recovery objectives (RPO / RTO) by deployment tier¶

Paridity documents recovery targets per deployment tier. The tier is a customer deployment choice - larger tiers buy tighter objectives with multi-AZ and replicas:

Tier	Deployment	RPO	RTO
Small	Single-AZ, no replica	1 hour	4 hours
Medium	Multi-AZ, 1 read replica	5 minutes	1 hour
Large	Multi-AZ, 2 read replicas, WAL archive	5 minutes	30 minutes
Global	Per-region with failover	1 minute	15 minutes

These are design targets, not measured recovery times. Paridity has drilled the restore procedure and the integrity of what comes back (once, on 2026-08-17, with an automated quarterly drill scheduled from 2026-11-01) but has not observed a restore at realistic evidence volume, and restore time at volume is what an RTO promises. The reference environment cannot supply that figure: its entire DR backup scope is a 10.6 MB dump. Treat the table as what the architecture is sized for, and ask for the drill transcript rather than inferring an observed number from it.

Important: The Small tier's embedded (in-chart) Postgres option is explicitly non-production - its durability is bounded by the Kubernetes volume's storage class and snapshots, not managed PITR. Any tier with a real RPO target must use external managed Postgres (AWS RDS, Azure Database for PostgreSQL, GCP Cloud SQL).

What must be recovered, in priority order. All platform state lives in one PostgreSQL instance across four schemas:

Schema	Contents	Recovery priority
Configuration	Mapping config, tenant config, schedule config, audit log	Critical - all new runs depend on this
Execution history	Execution history, run status, watermarks	Important - historical record; new runs can restart
Evidence	Proof results, mismatch keys, field diffs, config audit	High - append-only evidence; SOX/ICFR compliance requires it
Projections	Derived run summaries, metric rollups, baselines	Low - fully recomputable from evidence; no custom recovery needed

What does not need recovery. Two categories of state require no custom recovery procedure at all:

In-flight diff state - the in-memory DuckDB diff engine uses local ephemeral storage. If a diff worker dies mid-run, the Temporal workflow retries the diff activity from scratch; there is no persistent diff state to recover. (Known V1 limitation: a long-running diff that fails near completion restarts from the beginning - see Limitations below.)
Workflow state - Temporal persists workflow history durably in its own Postgres database. After a Temporal cluster recovery, in-progress workflows resume from the last recorded event natively. During an extended Temporal outage, new runs cannot be dispatched but no run data is lost - existing run state is preserved.

Backup and point-in-time recovery¶

Standard (Helm/Terraform) deployments. The in-repo Terraform modules for AWS, Azure, and GCP provision managed Postgres with 14-day backup retention plus point-in-time recovery (PITR) enabled, and multi-AZ/high availability on the Medium and Large tiers. The Temporal persistence database (typically 5-10% the size of the application database) is covered by the same managed backups. Paridity's guidance is to verify backup enablement from terraform output after provisioning rather than assume it. For logical, portable backups, the deployment runbook documents a logical pg_dump of the configuration, execution, and evidence schemas.

Self-managed Postgres. For deployments above the Small tier that self-manage Postgres, Paridity documents continuous WAL archiving (archive_mode = on with an object-store archive target), daily base backups via pg_basebackup, and backup integrity verification. On managed platforms the equivalent is: AWS RDS automated backups (the shipped Terraform default is 14-day retention) plus PITR; Azure Database for PostgreSQL geo-redundant backup; GCP Cloud SQL automated backups plus PITR.

All of this runs in the customer's cloud account. There is no Paridity-side backup copy, and no customer data leaves the VPC as part of the backup process.

Restore procedure¶

The documented full-restore procedure (region failure or catastrophic data loss) follows the cloud provider's PITR/snapshot restore, then re-points the platform:

Provision a restored Postgres instance via the cloud provider's point-in-time or snapshot restore to the target timestamp (AWS RDS restore-to-point-in-time, Azure PITR to a new server, GCP clone from backup).
Verify schema integrity - confirm all configuration, execution, and evidence tables are present.
Verify critical row counts against last known values (mapping config, run records, proof results, and the audit log).
Re-point the application - update the database connection secret and roll the API and worker deployments (helm upgrade or kubectl rollout restart). The migration hook re-runs idempotently and is a no-op against a restored, already-migrated schema.
Re-point Temporal to its restored persistence database and restart the Temporal pods.
Run the restore-validation playbook (next section).

Evidence integrity survives restore by construction: evidence tables are append-only and the compare_started_at audit anchor is immutable, so restored evidence is byte-identical to what was originally written - restored as-is, never rewritten.

The licence survives a restore too: the installation id the licence is bound to is a row in the same database, so a restored database (logical dump or PITR) is the same installation and refreshes its licence normally, provided the licence key is still in your secrets backend and the licence service is reachable.

A partial restore procedure is also documented for table-level corruption: restore the affected table from backup into a staging database, compare row counts, and reconcile by inserting missing rows only (using the run identifier as the natural deduplication key) - never updating existing evidence.

Note - rollback vs restore: a helm rollback reverts images and configuration only; database migrations are forward-only and are never reverted by Helm. If a migration caused the incident, the documented recovery is a PITR restore to the pre-upgrade timestamp, not a Helm rollback.

Evidence reconstruction. If a subset of evidence is lost between backups, the documented procedure is to identify affected runs, re-trigger them with the same config version and time window, and log the reconstruction to the audit log (recorded as an evidence-reconstruction action). Stated plainly for audit stakeholders: reconstructed evidence rows carry a different creation timestamp than the originals, and the audit-log entry is the auditor's evidence that reconstruction occurred. Reconstruction must be documented and approved under the customer's SOX/ICFR process.

Restore validation - the quarterly playbook¶

Paridity ships a restore-validation script to be run quarterly, and before every major upgrade, against a scratch database restored from the most recent production backup. It restores the backup and then verifies what came back:

Schema integrity across the configuration, execution, and evidence schemas.
Evidence row counts - every restored evidence table must hold exactly the number of rows the backup artifact itself carries, counted from the dump's own contents rather than from a manifest or from the live database.
Critical table counts - the same artifact comparison for mapping config, schedule config, the audit log, and run records.
Evidence anchors - the immutable compare_started_at audit anchor is present and still constrained, and no restored evidence row references a run the restore did not bring back.
Audit hash-chain - the audit-log tamper-evidence chain recomputes from genesis and its newest signed checkpoint anchors to it, so an auditor can prove the restored trail is the trail that was written.

Any failed check exits non-zero and names the check that failed; a check that could not be executed is reported as unmeasured and counts as a failure, never as a pass. The script only ever writes to a dedicated scratch database and refuses to run against the configured live database. It also carries a --prove-can-fail mode that deliberately corrupts the restored data - deleting an evidence row, rewriting a sealed audit entry - and requires the corresponding checks to go red, so the drill is demonstrably capable of failing before a green result is trusted.

Stated honestly, per Paridity's no-over-claim standard - what is now true, and what is still not:

A restore drill has been executed, and a transcript exists. Restore, PITR, and evidence reconstruction were run against a real backup of the reference environment's live control plane: --prove-can-fail first (all probes red, control green), then the formal run (5/5 checks passed, sealed audit chain verified against its checkpoint anchor and signature), then a PITR leg recovering to a chosen timestamp with a deliberate incident excluded and validated against an end-of-WAL control. The transcript is available to reviewers on request through your Paridity account contact.
The drill is automated and scheduled - it is a workflow in Paridity's continuous-integration pipeline for the reference environment, scheduled quarterly (the 1st of February, May, August, and November), with the --prove-can-fail probes retained ahead of every run. It has executed once to date, by manual dispatch on 2026-08-17; the first scheduled run is 2026-11-01.
Still not true: a drill against a CUSTOMER production backup. Paridity operates no customer production infrastructure, so scheduling and executing the validation in the customer's environment remains the customer's operational task; Paridity supplies the script, the procedure, and now a worked transcript.
Still not true: an RTO observed at realistic evidence volume. The drills prove the procedure and the integrity of what comes back, not how long a restore takes when evidence is large - and that is what the RTO targets promise. The reference environment cannot supply the figure: its whole DR backup scope is a 10.6 MB dump. No number in this document should be read as an observed recovery time.

Limitations & roadmap honesty¶

Per the four-level status vocabulary (Verified / Supported / Roadmap / Evaluating):

Capability	Status	Notes
Customer-hosted deployment - continuity independent of Paridity vendor operations	Verified	Full stack in the customer VPC via Helm + Terraform; Paridity operates no infrastructure that holds customer data. The licence service is the one Paridity-operated dependency (releases after v1.4.0): 7 days of normal running without it, then proof-only for a paid licence until it is reachable; never a halt. An air-gapped installation cannot be activated today, because offline activation is not yet available from the licence service.
Managed-Postgres backups + PITR provisioned by default (14-day retention; multi-AZ on Medium/Large)	Supported	Provisioned by the in-repo Terraform modules; verify enablement via `terraform output` post-deploy.
Documented RPO/RTO targets per deployment tier	Supported	Targets as tabled above; apply to the control plane, not source/target systems.
Full and partial restore procedures (PITR restore + re-point + validate)	Supported	Documented step-by-step, including Temporal recovery and evidence-immutability guarantees.
Restore-validation script	Verified	Shipped and executed end-to-end against a real backup of the reference environment's live control plane, including its `--prove-can-fail` probes (deliberate corruption must turn the checks red).
Quarterly restore drill actually executed	Verified	Performed - restore + PITR + evidence reconstruction, 5/5 checks, sealed audit chain verified; transcript retained and available to reviewers on request. Against Paridity's reference environment; a drill in the customer's own environment remains the customer's operational task.
CI-automated restore validation (automated DR test)	Verified	A continuous-integration workflow on the reference hardware, scheduled quarterly; executed once to date (2026-08-17, manual dispatch, green), first scheduled run 2026-11-01. `--prove-can-fail` probes run first on every execution so a green result is known to be capable of going red.
RTO observed at realistic evidence volume	Roadmap	Not measured. The tier RTO figures are design targets. The reference environment's entire DR backup scope is 10.6 MB, so an at-scale restore timing needs synthesized evidence at volume - outstanding work.
Multi-region failover	Roadmap	Single-region today; full region failure recovery is via restore (RTO per tier). Target V2.0.
Persisted DuckDB diff-state checkpointing	Roadmap	Long-running diff activities restart from scratch on worker failure today; spill-to-object-store checkpointing is on the roadmap with no committed release.
Temporal high availability on the Small tier	Supported	Single-server Temporal on Small is a documented accepted risk for that tier; Medium/Large use an external Postgres-backed cluster.
Paridity vendor-side organizational BCP attestation	Evaluating	Not published; the customer-hosted model removes vendor uptime from the customer's continuity path, and any formal vendor-side attestation is scoped case-by-case.

Appendix¶

The internal disaster-recovery source of truth this summary distills - full RPO/RTO tables, backup commands, restore procedures, the validation playbook, and the limitations register - together with the Helm/Terraform deployment runbook (logical backup and rollback-vs-restore guidance) are available to reviewers on request through your Paridity account contact.
Security Whitepaper - the customer-hosted deployment model, access control, secrets, and evidence-integrity controls in depth.
Trust & Security index - the secure-by-design review starting point and document map.
Paridity glossary - canonical terminology (Mapping, Run, hash_total, compare_started_at, evidence).
Follow-up questions: route through your Paridity account contact.

Paridity Open-Source & Supply-Chain Posture¶

Verified data. Trusted outcomes.

CDC-Aware Reconciliation Platform

Prepared for: General distribution (third-party-risk review) Version: v0.1 · Date: 2026-10-29

Executive summary¶

Paridity is built exclusively on permissively licensed open-source components - MIT, Apache 2.0, and BSD-family licenses (e.g. the PostgreSQL License). No GPL, AGPL, or SSPL component is used anywhere in the platform. This is a deliberate, enforced engineering rule, not an after-the-fact scan result: components with restrictive or source-available licenses (HashiCorp Vault under BSL 1.1, Confluent Platform under the Confluent Community License) were evaluated and excluded by design before they entered the stack. Every shipped Python and JavaScript component is pinned, with exact versions and integrity hashes, in two committed lockfiles that serve as the component manifests of record, and CI gates every change against known-vulnerability databases (pip-audit against the exact pinned Python lockfile; npm audit with a high/critical merge gate on production JavaScript dependencies) plus static analysis (Semgrep) and a full-history secret scan (Gitleaks). Production images are scanned (Trivy) for fixable HIGH/CRITICAL vulnerabilities before every release, and the latest release's images are re-scanned weekly. Since release v1.1.1 (2026-08-17), every release also ships a machine-generated SPDX SBOM per container image, generated (syft) from the published image digests - attached to the GitHub Release and pushed as a cosign in-toto attestation verifiable against the release's signing identity (cosign verify-attestation --type spdxjson) - and every released image and the Helm chart are cosign keyless-signed. The most recent release, v1.4.0 (2026-10-28), carries all of these.

One structural point matters for how you weigh all of this: in Paridity's current shipping model, the entire platform deploys inside your cloud account/VPC. The software supply chain described here is the supply chain of software you run. The one Paridity-operated service, the licence service, runs none of its code in your environment: it returns a signed licence token that your installation verifies against public keys built into the release image. From the first release after v1.4.0 those keys are written into the image at build time by the release pipeline - a source constant in the image, never read from the network, a file, an environment variable, or the database at runtime - and the image gates check the built image's licence mode before it is pushed. This document exists so you can review that supply chain directly.

Problem¶

A third-party-risk review of any data-adjacent platform has to answer three supply-chain questions:

License risk - does any component's license (GPL/AGPL/SSPL, source-available licenses) create copyleft exposure, redistribution constraints, or commercial-use restrictions for the customer deploying the product?
Component transparency - can the vendor say exactly which components, at exactly which versions, ship in the product?
Vulnerability posture - is the dependency set continuously checked against known-vulnerability databases, and does a vulnerable dependency block a release?

Because Paridity deploys into the customer's own environment, these questions are sharper than for a hosted SaaS: the customer's own security tooling will scan the running containers, and any discrepancy between what the vendor claims and what ships is immediately visible. Paridity's posture is written for that level of scrutiny.

Approach: permissive-only by rule, excluded-by-design¶

The license rule¶

The platform's engineering standard, binding on all code changes, states the rule verbatim: "All libraries are MIT or Apache 2.0 licensed. No GPL. No AGPL. No SSPL. This is a commercial product - every dependency must be commercially safe." The full dependency/license inventory is maintained in the platform's technology-stack register, which records for every component its license and commercial suitability; per that register, every dependency was verified in April 2026.

The core stack and its licenses:

Component	Technology	License
Workflow orchestration	Temporal (self-hosted; Python SDK `temporalio`)	MIT
Control-plane API	FastAPI, Pydantic v2, SQLAlchemy 2.x	MIT
In-memory diff engine	DuckDB	MIT (Foundation-protected)
Data interchange	Apache Arrow / PyArrow	Apache 2.0
Evidence + metadata store	PostgreSQL	PostgreSQL License (BSD-style)
Lake SQL access	Trino	Apache 2.0 (Trino Software Foundation)
CDC signals (optional)	Apache Kafka - core only	Apache 2.0
Secrets (self-hosted option)	Infisical	MIT
Secrets (cloud)	AWS Secrets Manager / Azure Key Vault / GCP Secret Manager	Cloud-provider TOS (service, not a bundled library)
Observability	OpenTelemetry SDKs	Apache 2.0 (CNCF-governed)
Frontend	React 18, Vite, TanStack Query/Table/Router, Tailwind CSS, shadcn/ui, Zustand	MIT
Container orchestration	Kubernetes / containerd	Apache 2.0 (CNCF-governed)

Several components were deliberately chosen for governance durability, not just current license text: DuckDB is foundation-protected (MIT in perpetuity), PostgreSQL has no corporate owner able to relicense it, and Trino, Kubernetes, containerd, and OpenTelemetry sit under vendor-neutral foundations. This insulates the stack from the single-vendor relicensing events (BSL, SSPL conversions) that have hit other infrastructure projects.

Excluded by design - proactive supply-chain hygiene¶

The more telling half of the posture is what Paridity refused. These exclusions are recorded in the technology-stack register ("Explicitly Excluded") and enforced as binding engineering rules:

Excluded component	License issue	Paridity's position
HashiCorp Vault (as a default)	BSL 1.1 since August 2023; IBM-owned	Never a default or required dependency. Supported only as an optional fifth secrets adapter for customers who already operate Vault. The default secrets backends are the customer's cloud-native service (AWS SM / Azure KV / GCP SM) or MIT-licensed Infisical.
Confluent Platform components (Schema Registry, ksqlDB, REST Proxy)	Confluent Community License bars competing SaaS offerings	Excluded entirely. Where Kafka is used (optional CDC readiness signals), it is Apache Kafka core only, under Apache 2.0.
Docker Desktop (production)	Paid license required for organizations > 250 employees	Development use only; production runs containerd.
Any GPL / AGPL / SSPL library	Copyleft / source-available restrictions incompatible with commercial distribution	Barred by standing rule at the engineering-standard level.

The pattern to note for risk assessment: these were excluded before adoption, not carved around after an audit finding. License review is an input to component selection, not a remediation activity.

Component manifests of record (pinned dependencies)¶

Paridity does not ask reviewers to trust a hand-maintained list. Two committed, machine-generated lockfiles are the authoritative manifests of exactly what ships, at exactly which versions:

Manifest	Ecosystem	Contents (as of 2026-10-29)
`requirements-ci.lock`	Python (backend: API, workers, connectors, services)	209 exactly pinned packages (`pkg==x.y.z`), each with SHA-256 integrity hashes. Machine-generated from the resolved dependency graph and marked "GENERATED, do not edit by hand". Four native database-driver packages (`ibm-db`, `pyodbc`, `python3-saml`, `xmlsec`) are excluded from this CI manifest because slim CI runners lack their system libraries - see the audit-scope note below.
`frontend/package-lock.json`	JavaScript (React SPA)	npm lockfile v3; 635 resolved package entries (production + development dependencies) with integrity hashes.

Discipline around these manifests is itself an engineering rule: any backend dependency change must regenerate the project's dependency-declaration, resolution, and CI-audit lockfiles together, in one commit, using a pinned resolver version - so the manifest CI audits can never drift from the manifest developers resolve against.

What this means for your review: the exact component inventory is available on request as the two lockfiles themselves (or auditable in-repo under a source-review arrangement). This document intentionally does not transcribe hundreds of package names - the lockfiles are the source of truth, and any transcription would be a staleness risk.

Dependency updates arrive both as Dependabot pull requests (configured for pip, npm, cargo, GitHub Actions, and Docker, weekly) and through this disciplined lockfile-regeneration process, and are reviewed like any other code change. A CI gate fails the build when the three Python dependency files disagree, so a Dependabot update cannot merge with a stale CI lockfile.

Dependency-scanning and static-analysis gates (verified in CI)¶

The following gates exist in the repository's CI pipeline and were re-verified against that pipeline on 2026-10-29. They run on every code change (a documentation-only change skips them); a failure fails the required CI check and blocks merge.

Python: `pip-audit` against the exact pinned lockfile¶

The dependency-audit CI job audits requirements-ci.lock - the exact pinned versions, including all transitive dependencies - against the PyPI advisory database using pip-audit. Auditing the exact pinned lockfile means the audit covers precisely what ships, not what might resolve.

Two scope notes, stated for transparency rather than hidden:

Documented, time-boxed advisory exceptions. An advisory is accepted only explicitly, one advisory id at a time, never by a blanket suppression. Each acceptance is annotated inline in the CI configuration with its rationale and a removal condition, and the more recent acceptances also carry a reachability note (whether the platform reaches the affected code, checked against the source and stated as reachability, not as an exploitability judgement); where the same advisory affects the container-image scan, its .trivyignore entry is removed together with it. As of 2026-10-29 the accepted advisories sit in four packages: python-jose, its ecdsa dependency, and pyjwt, for which no fixed release exists upstream (for the python-jose algorithm-confusion advisory, both of the platform's token-verification paths accept asymmetric algorithms only, so an HMAC-signed token is never accepted and that path is not reached); and filelock, where a fix exists only in a major version the pinned Snowflake driver does not yet accept (the affected lock classes are not used by the platform, or need a second local user that the single-user image does not have; removal is tied to the Snowflake driver upgrade). Exceptions are retired, not left to become permanent: the thrift, anyio, and oauthlib acceptances were each removed when an upgrade made the fixed version resolvable.
Native-driver packages outside the pip-audit job. The four native driver packages excluded from the CI lockfile (ibm-db, pyodbc, python3-saml, xmlsec) are outside this job's scope; the CI configuration documents that they are to be audited separately on a driver-provisioned host, and no recurring, evidenced separate audit is in place. They are installed in the shipped backend image (they appear in its release SBOM), so the Trivy image scan below covers them for fixable HIGH/CRITICAL vulnerabilities; they are not covered by pip-audit's advisory database.

JavaScript: `npm audit` with a production high/critical merge gate¶

The same CI job runs npm audit twice over the frontend:

Informational pass - surfaces all advisories (including dev/test tooling) in the CI log without failing the build, so nothing is invisible.
Merge gate - npm audit --audit-level=high --omit=dev fails the build on any high or critical vulnerability in production (shipping) dependencies. Development-only tooling advisories do not block merge (they do not ship) and are upgraded on their own cadence.

Static analysis and container linting¶

Semgrep SAST (sast CI job): scans the application source with the p/python, p/security-audit, and p/owasp-top-ten open-source rulesets; any finding fails the build (--error). A project-specific blocking rule additionally polices dynamic SQL construction (SQLAlchemy text() usage requires an explicit, justified suppression with all user values bound as parameters).
Hadolint: the fast-gates CI job lints the development Dockerfiles on every code change, and the production-image gate (below) lints the three production Dockerfiles (backend, UI, docs); container images are built from in-repo Dockerfiles rather than pulled as opaque third-party application images.
Secret scan - Gitleaks (a step of the fast-gates CI job): scans every commit a pull request adds for committed secrets; a finding fails the build.
Configuration scan - Trivy (a step of the fast-gates CI job): scans the Dockerfiles, compose files, and the in-repo Terraform modules for HIGH/CRITICAL security misconfigurations; a finding fails the build.

Container images: Trivy vulnerability scan, before release and weekly after¶

Production-image gate. The same gate runs on every pull request that changes the image packaging or the Python dependency manifests, and in the release pipeline before anything is pushed: it builds the three production images (backend/workers, UI, docs) and fails on any fixable HIGH or CRITICAL vulnerability Trivy finds in them (OS and language packages). The pushed release artifact is the exact image that passed the gate, never a rebuild.
Standing re-scan. Every Monday, the images of the latest release are pulled by digest and re-scanned against the current vulnerability feed, so a vulnerability published after release day turns the scan red rather than sitting unseen in the registry. Container-scan acceptances live in one policy file (.trivyignore), read by both the release gate and the weekly re-scan, under the same one-id-at-a-time, removal-condition rule as above.

Running stack: weekly dynamic scan (OWASP ZAP)¶

What runs. Every Monday (and on any change to the scan itself) CI builds the current code into a throwaway stack - the real API with real sign-in, the production UI image - on a network sealed off from everything else, and runs OWASP ZAP against it: a passive baseline scan of the UI, and an authenticated, active scan of the API driven by the API's own OpenAPI document.
What fails it. Any finding in a FAIL-level class (injection, cross-site scripting, path traversal, XML external entities, remote code execution, server-file disclosure, open redirect, and the security headers already in place); a scan error; or an API scan that cannot be shown to have run authenticated (enough protected endpoints must have answered 2xx). Every rule's verdict, and the reason for every ignored rule, is in one file (scripts/dast/zap-rules.tsv).
What it is not. Automated scanning, not a penetration test: it does not chain findings, test business logic or tenant isolation, or test a real deployment's ingress, TLS or identity provider. ZAP (Apache-2.0) runs only in CI and is not part of any shipped image.

Status honesty: what is verified, what is roadmap¶

Per Paridity's positioning standard, an unverified claim is not stated as fact.

License posture - Verified. The permissive-only rule (MIT / Apache 2.0 / BSD-family; no GPL, AGPL, or SSPL) is documented as a binding engineering standard, enforced at component selection, and recorded per-component in the technology-stack register (dependencies verified April 2026 per that register).
Formal SBOM artifact - Verified (since v1.1.1, 2026-08-17). Every release publishes a machine-generated SPDX JSON SBOM per container image (backend/workers, UI, docs), generated by syft from the published image digests - never a local rebuild - so the SBOM describes exactly what customers pull. Delivered two ways per release: as sbom-paridity-{api,ui,docs}-vX.Y.Z.spdx.json assets on the GitHub Release (sbom-dataalign-... for v1.3.1 and earlier, released under the product's former name; most recently v1.4.0, 2026-10-28), and as cosign in-toto attestations on the image digests, verifiable against the release's pinned signing identity (cosign verify-attestation --type spdxjson). The release pipeline fails if SBOM generation degenerates (≤10 packages) or if a wrong-identity attestation verify would pass - both proven can-fail on the live v1.1.1 run. The pinned lockfiles above remain the source-level manifests of record; the SBOM is the shipped-artifact-level manifest.

Limitations & roadmap¶

Capability	Status	Notes
Permissive-license-only dependency posture (no GPL/AGPL/SSPL)	Verified	Binding engineering rule; per-component inventory in the technology-stack register (verified April 2026 per that register).
Restrictive-license components excluded by design (Vault BSL optional-only; Confluent excluded, Kafka core only)	Verified	Recorded in the technology-stack register and enforced as standing rules.
Pinned component manifests with integrity hashes (Python + JS lockfiles)	Verified	`requirements-ci.lock` (209 pinned packages as of 2026-10-29, SHA-256 hashes) + `frontend/package-lock.json` (lockfile v3, 635 resolved entries). Counts move with routine dependency updates; the lockfiles themselves are the always-current record.
Python dependency vulnerability gate (`pip-audit` on exact pins incl. transitives)	Verified	CI `dependency-audit` job; explicit, per-advisory exceptions only, each with a rationale and a removal condition (see above).
JavaScript dependency vulnerability gate (`npm audit --audit-level=high --omit=dev`)	Verified	CI merge gate on production dependencies; all advisories surfaced informationally.
SAST (Semgrep: python / security-audit / owasp-top-ten) + Dockerfile lint (Hadolint)	Verified	CI `sast` job; Hadolint on the development Dockerfiles in the `fast-gates` job and on the production Dockerfiles in the image gate; findings fail the build.
Secret scan (Gitleaks, every commit a pull request adds) + configuration scan (Trivy config)	Verified	`fast-gates` CI job on every code change; findings fail the build.
Container-image vulnerability scan (Trivy, fixable HIGH/CRITICAL)	Verified	Production-image gate before every release push (and on image-packaging pull requests), plus a weekly re-scan of the latest release's images by digest.
Dynamic scan (OWASP ZAP: UI baseline + authenticated API scan, weekly)	Verified	Weekly CI job against a stack built from the current code; FAIL-level findings, scan errors and an unproven-authentication scan fail the job; reports kept as run artifacts. Automated scanning, not a penetration test.
Machine-generated SBOM (SPDX, per image, per release)	Verified (v1.1.1, 2026-08-17)	syft from the published digests; Release assets + cosign in-toto attestations; can-fail proven live.
Automated license-compliance scanning in CI (e.g. license-classifier gate)	Verified	Per-build license gate across all three ecosystems: a Python license check over the exact pins (with a `--prove-can-fail` mode), `license-checker` over production npm dependencies, and `cargo deny` over the Rust crate - each fails the build on a disallowed (GPL/AGPL/SSPL) license. Component-selection review and the technology-stack register still apply on top.
Automated dependency-update automation (Dependabot/Renovate)	Verified	Dependabot is configured for pip, npm, cargo, GitHub Actions, and Docker (weekly); updates arrive as PRs reviewed like any code change, alongside the disciplined lockfile-regeneration process.
Vulnerability audit of the four CI-excluded native driver packages	Roadmap	Outside the `pip-audit` job; prescribed as an out-of-band audit on a driver-provisioned host, with no recurring evidenced process yet. Covered only by the Trivy image scan of the shipped backend image (fixable HIGH/CRITICAL).
Published coordinated vulnerability-disclosure policy	Supported	A coordinated disclosure policy ships as `SECURITY.md` - private reporting via GitHub Security Advisories, plus the account-contact channel for customers. A dedicated monitored `security@` mailbox is a pre-GA operational item, called out in the policy itself.
Formal SLSA build-provenance predicate	Roadmap	cosign keyless signing of every released image and the Helm chart, plus an SPDX-JSON SBOM in-toto attestation per image, ship since v1.1.1 (2026-08-17) - see the Verified SBOM/signing entry above. A formal SLSA provenance predicate on top of that signed-and-attested baseline is not yet generated.

Appendix¶

The full dependency/license inventory and exclusion register (the source of truth for the license posture), the pinned Python and JavaScript component manifests of record (exact pins with SHA-256 integrity hashes), and the CI dependency-audit, SAST, and Dockerfile-lint job definitions cited above are available to reviewers on request through your Paridity account contact.
Security Whitepaper - the broader control set this posture sits within (access control, secrets, data protection).
Trust Center index - the curated entry point for VRM review.
Follow-up questions: route through your Paridity account contact.

Paridity - standardized security questionnaire responses¶

Verified data. Trusted outcomes.

How to read this. Paridity's primary security story is secure by design - see the Trust & Security overview. This document is a reference for teams that must map Paridity to a SIG Lite / CAIQ questionnaire. Because Paridity is customer-hosted and stores no customer data, many rows are legitimately "not applicable" - that is the design working as intended, not a gap. Formal certifications (SOC 2 / ISO) are addressed as applicability, not a roadmap.

CDC-Aware Reconciliation Platform

How to read this document¶

Enterprises send Paridity standardized vendor-risk questionnaires (SIG Lite, CAIQ, or house variants). This document pre-answers the recurring questions, organized by the common framework domains, so a security review can start from written, sourced positions rather than a blank form.

One structural fact answers a large share of the questions up front. In Paridity's current shipping architecture, the entire platform - API, UI, Temporal orchestration, all workers, and the PostgreSQL evidence store - deploys inside the customer's own cloud account/VPC, via a Helm chart and Terraform modules the customer controls. Paridity, the vendor, operates no infrastructure that holds or processes customer data (its one operated service is the licence service, which receives licence data only: the licence key, a machine id derived from a random installation id, the activation id, the product name, and the platform and architecture) and has no network path to a customer's credentials, schema/object names, or evidence values. Many control questions ("where is our data stored?", "who at the vendor can access it?", "what happens if the vendor is breached?") therefore have a structural answer: customer data never leaves the customer's environment, and there is no Paridity-operated environment holding it. Where a control is not in place (e.g. SOC 2, an independent penetration test, cyber insurance), this document says so plainly - per Paridity's own documentation standard, an unverified claim is never stated as fact.

Status vocabulary (used throughout, per the Paridity style guide):

Status	Meaning
Verified	Implemented and validated end-to-end, with evidence traced to code or live test.
Supported	Implemented and operational; broader live verification still expanding.
Roadmap	Planned; not yet built or not yet validated - never presented as shipped.
Evaluating	Candidate under consideration; no committed implementation.

All answers describe the shipping, customer-hosted architecture - the entire platform runs inside the customer's own cloud account. A hosted deployment option is on our internal roadmap with no committed date; it changes none of the answers below.

1. Governance, risk & compliance (GRC)¶

#	Question (typical SIG Lite / CAIQ phrasing)	Paridity response	Status
GRC-1	Do you hold a SOC 2 Type II report or ISO 27001 certification?	No. EGMW holds no SOC 2 report or ISO 27001 certification, and no such audit is planned now or in the immediate future. The reason is applicability: these attest a service organization's controls over the customer data it hosts or processes. Paridity is software the customer runs in its own environment; EGMW does not host it and does not process customer data (its one operated service, the licence service, receives licence data only - see SUB-2). The controls that matter for the deployment - access, encryption, backup, network, monitoring - are the customer's, configured inside the customer's own cloud account and covered by the customer's existing certifications and audits. (Security Whitepaper, "Limitations & roadmap honesty".)	See response (no certification, none planned)
GRC-2	Do you have a documented information-security policy program?	No formal program. EGMW is a small business and has no formal, documented information-security program or published information-security policy set today. What exists: a governance documentation practice (documentation register, style guide with binding no-over-claim rules, classification scheme: Public / Customer Confidential / Under NDA / Paridity Internal), an Acceptable Use Policy, an open-source software policy (draft), a coordinated vulnerability-disclosure policy (`SECURITY.md`), and the automated security gates in AIS-4.	See response (no formal program)
GRC-3	Is your product suitable for SOX/ICFR-relevant processes?	Yes - this is the product's design center. Every run produces an append-only, tamper-evident audit trail: config fingerprint (`config_checksum`) written before any comparison, immutable `compare_started_at` audit anchor, per-phase timings with worker identity, hash-chained `cfg.audit_log`, and cryptographically signed, hash-chained evidence bundles an auditor can verify offline with the customer-held signing key and the shipped fail-closed verifier. (Audit & compliance proof doc.)	Verified
GRC-4	Does AI play any role in your control outcomes?	AI in Paridity assists triage and explanation only - it never determines whether data matches. Correctness is always established by the deterministic `hash_total` / `row_hash` / `field_diff` pipeline. AI never writes to evidence (`ev.*`) or the audit log, and never auto-confirms an operator action.	Verified
GRC-5	Do you perform background checks and security-awareness training for personnel with access to customer data?	Structurally, no Paridity personnel have access to customer data - there is no vendor-side environment holding it, and no vendor network path into the customer VPC. EGMW has no formal, documented background-check or security-awareness training program.	See response
GRC-6	Who is the contracting legal entity?	Evil Genius MindWorks LLC (Paridity is the product name). Contract paper: MSA with Schedules A-D, including a Security Overview (Schedule C) and a Data Processing Addendum (Schedule D) drafted for this architecture.	See response
GRC-7	Could a privileged user manipulate reconciliation outcomes without detection?	Paridity scopes this explicitly rather than claiming blanket tamper-resistance. The proof's `hash_total` aggregate defends against replication faults and corruption (quantified collision bound ≈10⁻¹⁹ per comparison); it is not designed to resist an insider who can deliberately write compensating changes into the databases being compared - no SQL-computable aggregate is, and Paridity says so in its accuracy proof (Threat model section) instead of implying otherwise. That insider case is an access-control and audit problem, covered by: the customer's own database access controls and change auditing on source/target; Paridity's append-only evidence, hash-chained + checkpoint-signed audit log (checkpoints externally anchorable to WORM storage / RFC 3161 TSA / SIEM), and signed evidence bundles verifiable offline with the customer-held key (past results cannot be silently rewritten); and four-role RBAC with actor/IP attribution on every admin action.	Verified (scoping documented)
GRC-8	Do you carry cyber-liability insurance? Provide a certificate of insurance.	No. EGMW holds no cyber-insurance policy and cannot provide a certificate of insurance at this time. If a customer contract requires coverage, EGMW will review the requirement.	See response (no coverage)
GRC-9	Describe your company's financial viability. What happens to us if you cease operating?	EGMW is a small, privately held business; it publishes no formal viability statement and no financial statements. The mitigant is architectural: Paridity runs entirely in the customer's environment, and its only runtime dependency on EGMW is the licence service. If that service becomes unreachable - including if EGMW stopped operating it - the installation runs normally for 7 days on its last signed licence token; after that a paid licence is degraded until a refresh succeeds: runs continue and are never stopped, but they are proof-only (counts and hash totals, no field-level diffs) and bringing new configuration into service (for example, creating or enabling a mapping or schedule) is refused. No data is touched. The risk, stated: if the licence service never returns, that degraded state is permanent. No source-code escrow, perpetual-licence fallback, or offline activation path exists today.	See response

2. Identity & access management (IAM)¶

#	Question	Paridity response	Status
IAM-1	How are users authenticated?	Interactive users sign in through your identity provider over SAML 2.0 or OIDC (IAM-2). All user-facing API endpoints then require an OIDC/JWT bearer token. Bearer-token validation enforces `RS256`/`ES256` signature algorithms only (no `alg=none`) and verifies `aud`, `iss`, and expiry before any request is authenticated. SCIM endpoints require their own dedicated audience and the `/mcp` agent surface another, applied per path, so a user token cannot drive SCIM or `/mcp` and vice versa.	Verified
IAM-2	Do you support SSO with our identity provider?	Yes - two interactive browser sign-in paths. SAML 2.0: Okta and Microsoft Entra ID are both live-verified. OIDC (since v1.4.0): a server-side authorization-code flow with PKCE - the client secret is resolved from your own secrets backend and the ID token must be signed with `RS256`, `PS256`, or `ES256`; live-verified with Microsoft Entra ID, not yet live-tested with Okta. Separately, OIDC bearer tokens from a registered per-tenant issuer are validated for API and service-account access. Identity federates against the customer's own IdP - Paridity is not in the credential path.	Verified (SAML: Okta, Entra; OIDC: Entra); Supported (OIDC: other providers)
IAM-3	Do you support automated provisioning/deprovisioning (SCIM)?	Yes - SCIM user/group provisioning is live-verified end-to-end with IdP-pushed provisioning from Microsoft Entra ID: create, update, and deprovision cycles, including hardening against Entra sending the `active` flag as a string (which would otherwise have bypassed deprovisioning).	Verified
IAM-4	Is role-based access control enforced?	Yes. Four roles in a strict hierarchy: `platform_admin` > `operator` > `analyst` > `auditor`, enforced in the API layer. Access is further narrowed by grants that bind a principal to specific resources (e.g. a business domain), not just a role.	Verified
IAM-5	Are privileged/administrative actions logged?	Yes. Admin-sensitive actions (e.g. `bypass_readiness: true` on run creation, cancellations, retention purges) require `platform_admin` and are written to `cfg.audit_log` with actor, entity, action, timestamp, and `actor_ip` (from `X-Forwarded-For` or direct connection). The audit log itself is hash-chained for tamper-evidence (§4).	Verified
IAM-6	Are there any non-SSO backdoors or development bypasses?	A development-only authentication bypass exists for local use and is production-guarded: the platform refuses to honor it outside a development/test environment, and even then only when no credentials are supplied at all - a presented token is always validated normally so RBAC still applies. Additionally, the login page hides local credentials when SSO is required.	Verified
IAM-7	Can vendor staff access our production instance?	No standing access exists. The deployment runs in the customer's cloud account under the customer's IAM; any Paridity support access is access the customer explicitly provisions in their own environment, on their terms, and can revoke unilaterally.	Verified (structural)

3. Data security & privacy¶

#	Question	Paridity response	Status
DSP-1	Where is our data stored and processed?	Entirely inside your own cloud account/VPC. Source/target row data is read in-VPC by your Paridity workers; evidence (`hash_total`, `row_hash`, `field_diff`), metadata (object/schema/column names, config), and the audit trail all live in your own PostgreSQL instance in your account. No customer data class leaves the customer VPC. Two things can leave, neither of them customer data: the licence check to EGMW's licence service (licence data only), and a support diagnostic bundle, only if your operator downloads it and emails it to support (redacted: no column names, evidence values, hashes, connection references, secrets, or SQL; object names tokenized unless the operator opts in). (Data-Flow Disclosure §2.)	Verified
DSP-2	Is data encrypted in transit and at rest?	Connector transport to source/target databases is fail-closed in the product itself: PostgreSQL connections enforce TLS (`require` or stricter) by default with plaintext requiring an explicit operator opt-out, and MySQL mirrors the same fail-closed posture including defense against driver-level TLS downgrade. Ingress TLS for UI/API and managed-Postgres storage encryption (customer-managed keys available) are deployment-configuration requirements verified at install time (Terraform/Helm defaults + checklist) - stated honestly, the enforcement point for those is the customer's own cloud provider, because the stack runs in the customer's account.	Verified (connector transport defaults); Supported (platform TLS/at-rest via deployment checklist)
DSP-3	Can sensitive field values be prevented from being stored or displayed?	Yes - masking is a chained policy, not a single toggle. `store_raw_diff_values` is an AND-chain across tenant (L2), mapping (L4), and field (L5): any layer set `FALSE` suppresses raw-value storage entirely (row written with `NULL` values, `values_masked = TRUE`). `mask_in_ui` is a per-field display mask: set `TRUE` on a field and display is masked even where values are stored (a tenant-wide display-mask policy is not currently offered). Evidence endpoints apply masking at query time; a masked field never returns raw values regardless of caller role.	Verified
DSP-4	Is evidence protected against tampering?	`ev.` evidence tables are append-only (two narrow, documented lifecycle exceptions; core evidence columns immutable). The one deletion path - retention purge - writes an audit record before* each delete. Evidence bundles carry a SHA-256 `content_hash`, a `previous_bundle_hash` chaining each bundle to the prior one, and a detached signature over exactly the bytes that ship - verifiable offline with the shipped fail-closed verification tool (an unsigned, tampered, or wrong-key bundle never reports as verified). The signing key is the customer's choice: a KMS-backed key (AWS KMS, Azure Key Vault, GCP KMS) or a local ECDSA key signs asymmetrically, so an outside auditor verifies with the public key alone - no customer secret, no platform access, no packages (live-verified end-to-end against AWS KMS on 2026-08-28, including credential-less verification and can-fail legs; Azure/GCP use the same live-verified signer adapters); an HMAC key (customer-held) remains available. Period exports are signed containers over individually signed run bundles. Separately, the audit log's sealed checkpoints use the same asymmetric signers and can be anchored to external WORM storage, an RFC 3161 timestamp authority, and your SIEM.	Verified
DSP-5	How is data retention and deletion handled?	Retention periods are policy-resolved per mapping (`retain_*_days`, mapping value cannot go below the tenant minimum), with evidence-bundle containers governed separately by `bundle_retention_days` (tenant value, platform floor). The retention purge is the sole sanctioned deletion path for evidence and logs each deletion (table, mapping, row count, cutoff, policy) to `cfg.audit_log` before deleting; where a bundle's bytes live in the customer's own object store, the purge records the store's answer per object (deleted / retained by the store / already absent) rather than presuming removal. The purge runs on a built-in schedule that is default-on and dry-run-first, with pause and resume audited. An approval-gated legal hold placed on a tenant, mapping, or bundle freezes the affected evidence from purge; the purge skips it and audits the skip, and placing or releasing a hold requires the `evidence.legal_hold` permission and is itself audited. Because all data resides in the customer's own database, end-of-contract deletion is under the customer's direct control - decommissioning the deployment removes everything; there is no vendor-side copy to chase.	Verified
DSP-6	Do you process personal data as a processor? What about GDPR/data residency?	Paridity (the vendor) holds no customer data of any class - source data, evidence, metadata, and credentials all remain in the customer's environment - so there is no vendor-side processing of customer personal data in the core product. Data residency is wherever the customer deploys. EGMW does receive a small amount of non-customer data through its two sub-processors (SUB-2): licence data, including the request's source IP address, on the licence service, and whatever an operator chooses to send to the support mailbox. The optional AI features are bring-your-own-key: masked, name-scrubbed prompt context goes to the model provider the customer configures and contracts with directly (with a data-residency provider restriction available) - there is no Paridity-provided default provider, so the AI path adds no Paridity sub-processor. If you additionally enable the MCP agent surface, your own agent becomes an egress path to a model you chose - see SUB-5 for what it exposes and why object-name masking is best-effort there.	Verified (structural)
DSP-7	Are encrypted database columns handled safely?	Where a source/target column is itself encrypted (e.g. Oracle TDE), a per-field `decrypt_expression` is injected into generated SQL at query time only; keys are resolved via `SecretsClient` and decrypted values are never persisted outside the normal evidence-masking rules.	Verified

4. Application & infrastructure security¶

#	Question	Paridity response	Status
AIS-1	Describe your deployment/hosting model.	Single-tenant: the full stack runs in the customer's VPC - control plane, UI, orchestrator, all workers, and one PostgreSQL instance. Deployed via the Paridity Helm chart and Terraform reference modules for AWS/Azure/GCP that provision the cluster, managed Postgres with backups/PITR, the customer's secret backend, and an egress-locked network (node egress deny-by-default; cloud services reached over private paths - AWS PrivateLink VPC endpoints, GCP Private Google Access, Azure NSG service tags - with only the operator's declared endpoints allowed outbound; the licence check to EGMW's licence service needs the customer's HTTPS egress proxy, which neither the chart nor the Terraform modules open by default). The packaging is delivered and CI-gated, and the egress lockdown is verified live on all three clouds (August 2026) by a from-empty deployment plus a probe instance confirming that a non-declared destination is refused outbound - with the documented operational consequence that workloads must be pulled from a private registry or declared endpoint (public registries are refused by design). A hosted deployment option is on our internal roadmap with no committed date; its design already enforces, in review, the same invariant - credentials, object/schema/column names, and evidence values never reach Paridity-operated infrastructure.	Verified
AIS-2	How are credentials to our source/target systems managed?	Paridity never stores credentials. `cfg.systems.connection_ref` is always a URI pointer (e.g. `aws-secrets://…`, `azure-keyvault://…`), never a secret. A `SecretsClient` resolves the pointer at runtime against the customer's own configured backend: AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, or Infisical (self-hosted, MIT). HashiCorp Vault is supported as an optional adapter for existing Vault customers only - never a default or required component.	Verified
AIS-3	How is tenant isolation enforced?	Primary enforcement: every tenant-scoped table carries `tenant_id` and every application query filters by it. Defense-in-depth backstop: PostgreSQL row-level security is live - 30+ tables carry a policy that fails closed to zero rows if the session tenant context is unset. Stated honestly: the application's database role currently retains an RLS bypass, so RLS today is a backstop against a missing tenant predicate, not the primary mechanism; binding the owner role with `FORCE ROW LEVEL SECURITY` is intentionally deferred pending a full call-site audit. Note also that deployments are single-tenant per instance, which makes cross-tenant exposure structurally impossible at the deployment boundary.	Supported (RLS backstop); Verified (app-layer predicate)
AIS-4	Do you perform static analysis / dependency scanning in your pipeline?	Yes: on every code change CI runs a blocking Semgrep gate (including SQL-injection-focused rules for dynamic SQL, with a documented, reviewed suppression convention), Python dependency auditing (`pip-audit` against the pinned `requirements-ci.lock`), `npm audit` on production frontend dependencies, a full-history secret scan (Gitleaks), a configuration scan (Trivy), and dependency-license gates. Production images are Trivy-scanned for fixable HIGH/CRITICAL vulnerabilities before release, and the latest release's images are re-scanned weekly. A weekly OWASP ZAP dynamic scan (DAST) runs against a CI-built stack (IRV-1). Migrations are additionally gated by a fresh-database apply job.	Supported
AIS-5	Are health/monitoring endpoints safe?	`/health` performs no queries against evidence tables and no cross-system checks - liveness is process-responding, readiness is DB-pool-initialized. Optional OpenTelemetry telemetry goes to the customer's own observability stack; span attributes never include row values.	Verified
AIS-6	What happens to our security posture if Paridity (the vendor) is breached?	There is no Paridity-operated environment holding customer data, no Paridity store of customer credentials, and no Paridity-held copy of customer data - a vendor-side compromise has no path to customer data or customer environments. The one Paridity-operated service is the licence service: it holds licence keys, machine ids, and the source addresses of licence requests, and your installation accepts only tokens signed by a key whose public half is built into the release image (a release constant, not a runtime setting). It has no path into your environment: your installation calls it, never the reverse, and nothing it returns is executed. Its compromise could at worst withdraw or misstate licences: an unreachable or refusing licence service moves a paid licence to proof-only and never stops runs, but a forged token claiming an expired trial licence would suspend new runs until a correct token is issued. It cannot read or change your data, evidence, or configuration. Software supply-chain integrity (the artifacts you deploy) remains the relevant surface; see §7.	Verified (structural)

5. Data center / hosting & sub-processors¶

#	Question	Paridity response	Status
SUB-1	List your data centers and hosting providers.	Paridity operates no data centers and no hosted production infrastructure for the product. The product runs in your cloud account (AWS, Azure, or GCP - your choice, via the provided Terraform modules); your cloud provider is your own infrastructure, not a Paridity sub-processor. EGMW's own hosting is limited to two services that receive no customer data from the product: the licence service, hosted on Railway, and the support mailbox, hosted on Microsoft 365 (SUB-2).	Verified
SUB-2	Provide your sub-processor list.	None for customer data; two for licence and support data. No customer source data, schema/object names, or evidence values reach EGMW or any third party. EGMW uses two sub-processors for other data: (1) Railway hosts EGMW's licence service and so receives licence data (licence key, machine id, activation id, product, platform, architecture, and the request's source IP address), plus the licence and customer record EGMW holds there; (2) Microsoft 365 hosts EGMW's support mailbox, `[email protected]`, and receives only what an operator chooses to email - the prefilled draft names the tenant, Paridity version, page, and where relevant a run id, and an attached diagnostic bundle is redacted (no column names, evidence values, hashes, connection references, secrets, or SQL; object names tokenized unless the operator opts in). The product sends no email through EGMW: notifications use only the providers the customer configures (its own SMTP server or Microsoft 365 mailbox, Slack, Teams, PagerDuty, Jira, ServiceNow, webhook). The optional AI features are bring-your-own-key (see SUB-3): the AI provider is a vendor the customer configures and contracts with directly, not a Paridity sub-processor. (Data-Flow Disclosure §4.)	Verified
SUB-3	What data does the optional AI capability expose, and how is it controlled?	Only if the customer enables the AI features (AI Triage / AI Insights), prompt context for the requested explanation is sent to the model provider the customer configures with their own key (e.g. their own Anthropic account) - with object/schema/column names masked (`mask_object_names` - on this prompt path the masking is absolute; see SUB-5 for the different guarantee on the MCP agent surface), prompt context scrubbed, and a data-residency provider restriction applied; raw evidence values are not sent. Controls: off by default at the feature level, the provider key lives in the customer's own secrets backend, per-tenant settings and quota, and deterministic fallback when AI is unavailable. There is no Paridity-provided default provider, so the AI path adds no Paridity sub-processor. AI never determines correctness.	Verified
SUB-4	How will we be notified of sub-processor changes?	The sub-processor disclosure (Data-Flow Disclosure §4) is a maintained Trust Center document that names EGMW's current sub-processors (SUB-2) and is updated when one is added; if the hosted deployment option ships, it will be updated with the exact control-plane data classes (opaque IDs, counts, status, timings only) and any new sub-processors before customers contract against it.	See response
SUB-5	If we let our own AI agents query Paridity over MCP, what is exposed and who is the disclosing party?	The `/mcp` agent surface is an authenticated customer API at parity with REST, not a Paridity-routed LLM path: the customer's own MCP client authenticates with the customer's own credentials and forwards what it reads to a model the customer chose and contracts with directly. Paridity is not a sub-processor for that hop. Exposure equals REST exposure - object/schema/column names and run metadata, plus evidence aggregates and masked samples under the same query-time evidence masking (`store_raw_diff_values`, `mask_in_ui`) and the same per-tool permissions. *State plainly: `mask_object_names` is best-effort* here, not absolute** - a defined list of object-name fields is tokenized and a field carrying a name that is not on that list is returned in clear text. That is a deliberate design decision (deny-by-default masking would leave an agent unable to say which column drifted). In short: your agent, your credentials, your disclosure. A single tenant toggle deliberately governs both surfaces - a separate MCP masking control was considered and rejected (decision record 6, same document), because it would present a best-effort mechanism as a guarantee a reviewer could rely on; no independent MCP masking control exists. Residual risk we state rather than leave to be found: same authorization is not the same disclosure surface - a person reading a table name on a screen does not thereby place it in an LLM context window, vendor logs, or a retention window; an MCP client does, automatically, on every call. Controls that bound it: off by default, and in production the API refuses to start with AI enabled unless a tenant has explicitly opted in (a required second opt-in); per-tool RBAC using the same permission strings as REST; tenant scope taken from the authenticated principal and never from a tool argument; a read-only catalog with arbitrary SQL, shell, secret retrieval, file access, and direct database modification permanently excluded; every dispatched tool call audited (success or failure alike); and a per-principal, per-tool rate limit that sheds abuse before dispatch. Classified data-plane and runs entirely inside the customer's own environment, like the rest of the platform. The control only you can apply: govern which MCP clients and models may hold Paridity credentials, as you would REST API keys.	Verified

6. Business continuity & disaster recovery¶

#	Question	Paridity response	Status
BCD-1	What are your RPO/RTO commitments?	Documented targets per deployment tier for the Paridity control plane (Postgres + Temporal), configured in the customer's own account: Small (single-AZ) RPO 1 h / RTO 4 h; Medium (multi-AZ, 1 replica) RPO 5 min / RTO 1 h; Large (multi-AZ, 2 replicas, WAL archive) RPO 5 min / RTO 30 min; Global (per-region failover) RPO 1 min / RTO 15 min. Source/target system availability is the customer's responsibility.	Supported
BCD-2	Are backups automated?	Yes for Helm/Terraform deployments: managed Postgres (RDS / Azure Flexible Server / Cloud SQL) is provisioned with 14-day backup retention + point-in-time recovery (PITR), multi-AZ/HA on medium/large tiers. Temporal's own persistence database is covered by the same managed backups. The runbook instructs verifying via `terraform output` rather than assuming.	Supported
BCD-3	Are restores tested?	Yes - a full restore drill has been executed against Paridity's reference environment (a real backup restored and validated end-to-end, 2026-08-17), and restore validation is scheduled as an automated quarterly CI drill (first scheduled run 2026-11-01): a real backup is restored to a scratch database and five checks run (schema integrity, evidence row counts against the backup artifact, critical-table counts, evidence anchors, and the audit hash chain), preceded on every drill by mandatory prove-can-fail probes so a passing drill is known to be able to fail. A drill against your own production backup is your operational task, using the shipped script.	Verified
BCD-4	What is lost if a worker or orchestrator fails mid-run?	No evidence or audit data: Temporal persists workflow history durably and resumes in-progress workflows after recovery; a failed diff activity retries from scratch (in-flight DuckDB state is ephemeral by design - a documented limitation for very long diffs; checkpointing is on the roadmap with no committed release). Evidence and `compare_started_at` are immutable and restore as-is.	Supported
BCD-5	Does your (the vendor's) availability affect our operations?	Only in a bounded way. The entire stack, including orchestration, runs in the customer's environment. The one Paridity dependency is the licence service: in releases that enforce the licence (every release after v1.4.0) the installation runs normally for 7 days without it, then a paid licence runs proof-only (counts and hash totals, no field-level diffs, no new configuration brought into service) until it is reachable again. Runs are never stopped. Multi-region failover is not yet built (single-region per deployment; V2.0 roadmap item).	Verified (structural; bounded licence dependency)

7. Supply chain / SBOM¶

#	Question	Paridity response	Status
SCM-1	What is your open-source license posture?	Every dependency is MIT, Apache 2.0, or comparably permissive (e.g. PostgreSQL License) - no GPL, AGPL, or SSPL component anywhere in the platform, backed by a full per-dependency license inventory and enforced by blocking CI license gates across all three ecosystems (Python, npm production tree, Rust) - each gate proven able to fail. Two components are deliberately excluded rather than carved around: HashiCorp Vault (BSL 1.1) is optional-only, never default/required; Confluent Platform components are excluded in favor of Apache Kafka core only.	Verified
SCM-2	Can you provide an SBOM?	Yes - every release publishes a machine-generated SPDX-JSON SBOM per container image (backend/workers, UI, docs), generated by syft from the published image digests (never a local rebuild) and shipped both as GitHub Release assets (`sbom-paridity-{api,ui,docs}-vX.Y.Z.spdx.json`) and as cosign in-toto attestations verifiable against the release's signing identity (live since v1.1.1; most recently v1.4.0, 2026-10-28; releases up to v1.3.1 carry the product's former name, `dataalign`, in their file and image names). The fully pinned, committed dependency lockfiles remain the source-level manifests of record.	Verified
SCM-3	Are dependencies scanned for known vulnerabilities?	Yes - CI audits pinned Python dependencies (`pip-audit` on `requirements-ci.lock`) and production npm dependencies (`npm audit --omit=dev`) as blocking gates, alongside a blocking Semgrep static-analysis gate. Production images are Trivy-scanned for fixable HIGH/CRITICAL vulnerabilities before every release and the latest release's images are re-scanned weekly. An advisory is accepted only explicitly, one advisory id at a time, each with a written rationale and a stated removal condition (the more recent ones also with a reachability note); there is no blanket suppression.	Supported
SCM-4	How do customers receive and verify your software?	Deployment is from the Paridity Helm chart and container images plus Terraform reference modules, applied by the customer in their own account. Every released image and the Helm chart are cosign keyless-signed (Sigstore - the signing identity is the release workflow, recorded in the public Rekor transparency log) and carry an SPDX-JSON SBOM attestation. Customers verify signature and digest before mirroring into their own registry (`cosign verify --certificate-identity … --certificate-oidc-issuer https://token.actions.githubusercontent.com`; live since v1.1.1), on top of their own registry-scanning and image-policy controls. A formal SLSA provenance predicate is not yet generated (roadmap).	Verified

8. Incident response & vulnerability management¶

#	Question	Paridity response	Status
IRV-1	Have you had an independent penetration test? Share the report.	No. No third-party penetration test has been performed and none is scheduled. What is done is automated, on every code change in CI: Semgrep static analysis (Python, security-audit, and OWASP Top Ten rule sets), dependency vulnerability audits (`pip-audit`, `npm audit` on production dependencies), a full-history Gitleaks secret scan, and a Trivy configuration scan; release images are Trivy-scanned for fixable HIGH/CRITICAL vulnerabilities before release and the latest release is re-scanned weekly (AIS-4). A dynamic scan (DAST) runs weekly in CI: OWASP ZAP against a throwaway stack built from the current code - a passive baseline scan of the production UI image, and an authenticated, active scan of the API driven by the API's own OpenAPI document. The job fails on any finding in a FAIL-level class (injection, cross-site scripting, path traversal, XML external entities, remote code execution, server-file disclosure, open redirect, and the security headers already in place) and when the API scan cannot be shown to have run authenticated. This is automated scanning, not a penetration test: it does not chain findings, test business logic or tenant isolation, or test a real deployment's ingress, TLS or identity provider. Because Paridity runs in the customer's environment, the customer may test its own deployment with its own tools or testers (Acceptable Use Policy §6.1); testing EGMW-operated systems such as the licence service needs EGMW's prior written authorization (§6.2). Context: there is no Paridity-hosted service holding customer data (the one EGMW-operated service, the licence service, receives licence data only). An internal, AI-assisted review with test-pinned findings is described at IRV-1a; it is not a third-party assessment.	See response (not performed, none scheduled)
IRV-1a	Beyond automated scanning, has the product's code been reviewed for security?	Yes, internally. An internal, AI-assisted code review of the Paridity source (#3357), carried out by EGMW with an AI model in four passes from 2026-10-30 to 2026-11-01. No outside party took part. Scope: tenant isolation and authorization; authentication (bearer tokens, SAML, OIDC, SCIM, API keys); generated SQL; licensing; secrets and logging. Every item in scope was read line by line and either demonstrated as a finding, ruled out with a recorded counter-check, or pinned by a test. A finding required a test, committed to the repository, that failed against the code before the fix. Five findings: three high (#3365, #3369, #3398), one medium (#3383), one low (#3384); all five were fixed by 2026-11-01. This is a secure-development control, not a penetration test and not a third-party assessment: it reviewed source code and a development stack, did not test a customer deployment's ingress, TLS or identity provider, and did not exercise exploit chains against live credentials. The review record (method, coverage, findings, hypotheses not demonstrated, ruled-out items with their counter-checks, follow-ups) is retained by EGMW and available on request.	Verified
IRV-2	Describe your incident-response plan and breach-notification commitments.	No formal plan. EGMW is a small business and has no formal, documented incident-response plan today. Structurally: a Paridity-side incident cannot expose customer data (none is held), and a Paridity-side outage cannot stop customer reconciliation runs (the one runtime dependency, the licence service, degrades a paid licence to proof-only after 7 days, never a halt - see BCD-5; a licence-service compromise is covered in AIS-6); incidents inside the customer's deployment fall under the customer's own IR process, which Paridity's audit trail is built to support (hash-chained audit log, append-only evidence, network attribution via `actor_ip`). Contractual breach-notification terms are addressed in the agreement.	See response (no formal plan)
IRV-3	How would we detect and investigate misuse within the platform?	Every admin action and terminal run status is written to `cfg.audit_log` through a single enrichment chokepoint, with actor, entity, action, timestamp, and source IP; the log is hash-chained (altering, deleting, or reordering any row breaks every subsequent `entry_hash`), with signed sealing checkpoints that can additionally be anchored outside the platform entirely - to WORM object storage (S3 Object Lock / Azure Blob immutability / GCS retention lock), an RFC 3161 timestamp authority, and your SIEM (each anchor live-verified). Evidence bundles and the audit chain are independently verifiable - the audit chain via `GET /audit/verify`, and bundles offline via the shipped fail-closed verification tool with the customer-held signing key.	Verified
IRV-4	How do you become aware of vulnerabilities in your product and dependencies?	Blocking CI gates: dependency audit (Python `pip-audit`, npm production audit), Semgrep static analysis, and a full-history Gitleaks secret scan run on every code change; production images are Trivy-scanned before release and the latest release is re-scanned weekly against the current CVE feed; migrations gate against a fresh database; Dependabot raises weekly update PRs across pip/npm/cargo/GitHub-Actions/Docker. A coordinated vulnerability-disclosure policy ships as `SECURITY.md` (private reporting via GitHub Security Advisories plus the account-contact channel); a dedicated monitored `security@` mailbox and any bug-bounty program are pre-GA.	Supported
IRV-5	How are security fixes delivered to customers?	As versioned releases (chart + images) the customer applies via `helm upgrade` on their own schedule. Database migrations are forward-only and CI-verified to apply cleanly to a fresh database; the DR runbook documents PITR-based rollback for migration-caused incidents (a `helm rollback` reverts images/config only, never applied migrations).	Supported

Appendix - canonical sources¶

Answers in this document distill (never extend) the following canonical documents. For contract-grade reliance, verify against them:

Security Whitepaper - architecture, access control, secrets, data protection, OSS posture, roadmap-honesty table.
Data-Flow & Sub-Processor Disclosure - data classes, data-flow diagram, sub-processor list.
Audit & compliance proof - audit trail, evidence immutability, hash chains, signing, offline verification.
Paridity glossary - canonical terminology (hash_total, row_hash, field_diff, config_checksum, RBAC roles).
Additional materials - disaster-recovery targets and procedures, the pre-production security checklist, and the full dependency/license inventory - are available to reviewers on request through your Paridity account contact.

Follow-up questions or a completed copy of your firm's own questionnaire template: route through your Paridity account contact or the Trust Center index (security/TRUST_CENTER.md, in progress).