Python

In my previous article, I presented my first project, in which I assessed the data quality of an Argentine aviation dataset. While it is necessary to perform an initial, thorough analysis when working with a new data source that we consider critical, the reality is that, in day-to-day operations, we need different solutions that can be integrated into a data pipeline. This is because we should never assume that a dataset is of good quality. And this is not only because the data source may have introduced errors or because it may be more or less reliable. Nor is it necessarily because the file may have been unexpectedly modified during the ingestion process. We must never forget that data quality depends on the context, and what we consider to meet the expectations for a data source should be verified continuously. This is where tools such as Great Expectations become especially useful. ...

A practical Data Stewardship project demonstrating how Great Expectations can automate business rule validation for airport data using reusable expectations and Data Docs.

As the DAMA-DMBOK correctly states, the primary objective of Data Management is to enable organizations to derive value from their data. Among its Knowledge Areas, Data Quality plays a fundamental role and has one of the greatest impacts on the value of this digital asset. Just as high-quality data increases business value and creates new opportunities, poor-quality data generates significant costs. According to the MIT Sloan Management Review, these costs account for between 15% and 20% of a company’s revenue. In an increasingly data-driven world, achieving higher standards of data quality is becoming more important than ever. ...

A practical application of Data Stewardship principles, including business rule validation, identity resolution, spatial matching, and external accuracy assessment.