Most data quality tools work by copying your data out. You point them at your warehouse, they pull samples or full tables across the network, run their checks on their own infrastructure, and send results back. It works, but it means your data now lives in two places, and the second place is someone else’s.
Data Shepherd takes the opposite approach. Monitoring runs entirely inside your own Snowflake account. The checks execute on your own Snowflake compute, against tables that never leave your environment. There is zero data egress.
What that changes
Three things follow from keeping monitoring in-account:
- Security review is simpler. There is no new system to hold a copy of your data, so there is no new system to procure, secure or audit. Access runs through a dedicated, least-privilege role you grant.
- Billing is predictable. Compute flows through your existing Snowflake contract, rather than a separate per-row or per-table charge for data leaving the building.
- Coverage can be complete. Because there is no egress cost or transfer bottleneck, there is no reason to sample. Checks run against the real tables.
The trade-off we accept
Running in-account means we work within Snowflake’s compute model rather than our own. We think that is the right trade. For data quality, the place to watch your data is where it already lives - close to the tables, inside your own security boundary, around the clock.
This is the first of what will be a regular set of notes on building Data Shepherd. More to come.