Combining Data Build Tool (dbt) with CI/CD is a great way to automate tests during deployments from development environments to production.
This approach ensures higher data quality and provides early feedback about potential issues ππππ¨π«π end users see the data. Thereβs nothing more embarrassing than having an end user point out a data mistake that could have been caught earlier.
By using dbt together with CI/CD, you can automate tests such as checking whether a column:
– Contains unique values
– Only includes specific values (e.g., “yes” or “no”)
– Maintains referential integrity between columns in different tables
– Contains data from the previous day
– And more
How about you? Do you tackle potential data quality issues proactively, or do your end users serve as your data quality control? (No judgment, weβve all been there!)