Posts

Hello! I’m back after a month away. During this time, I have not only been training to continue publishing quality articles on my blog, but I have also been dealing with some work-related matters. I’ll share more details about that later — exciting things are coming. But let’s get back to the topic at hand. In my first article, I presented an example of data quality assessment using an aerodrome dataset. In the second one, I showed how to automate that assessment using Great Expectations. ...

In my previous article, I presented my first project, in which I assessed the data quality of an Argentine aviation dataset. While it is necessary to perform an initial, thorough analysis when working with a new data source that we consider critical, the reality is that, in day-to-day operations, we need different solutions that can be integrated into a data pipeline. This is because we should never assume that a dataset is of good quality. And this is not only because the data source may have introduced errors or because it may be more or less reliable. Nor is it necessarily because the file may have been unexpectedly modified during the ingestion process. We must never forget that data quality depends on the context, and what we consider to meet the expectations for a data source should be verified continuously. This is where tools such as Great Expectations become especially useful. ...

As the DAMA-DMBOK correctly states, the primary objective of Data Management is to enable organizations to derive value from their data. Among its Knowledge Areas, Data Quality plays a fundamental role and has one of the greatest impacts on the value of this digital asset. Just as high-quality data increases business value and creates new opportunities, poor-quality data generates significant costs. According to the MIT Sloan Management Review, these costs account for between 15% and 20% of a company’s revenue. In an increasingly data-driven world, achieving higher standards of data quality is becoming more important than ever. ...