A data mesh architecture unites the disparate data sources within an organization through centrally managed data-sharing and governance guidelines. In this post, we highlight the foundational concepts of data architectures, and cover the patterns for designing a data mesh in the AWS Cloud with supporting resources.

Data proliferation has become a norm and as organizations become more data driven, automating data pipelines that enable data ingestion, curation, and processing is vital. In this post, we explain how you can collect telemetry from your data pipeline jobs and use machine learning (ML) to build a lower- and upper-bound threshold to help operators identify anomalies in near-real time.