Delivering data-driven value at scale
Ìý
Many enterprises are investing in a next-generation data lake, hoping to democratize data at scale to provide business insights and ultimately make automated intelligent decisions. In this practical book, author Zhamak Dehghani reveals that, despite the time, money, and effort poured into them, data warehouses and data lakes fail when applied at the scale and speed of today's organizations. A distributed data mesh is a better choice.
Ìý
Dehghani guides architects, technical leaders, and decision-makers on their journey from monolithic big data architecture to a sociotechnical paradigm that draws from modern distributed architecture. A data mesh considers domains as a first-class concern, applies platform thinking to create self-serve data infrastructure, treats data as a product, and introduces a federated and computational model of data governance. This book shows you why and how.
Behind the Scenes
An Interview with Zhamak DehghaniWhat was the writing process like?
It was full of highs and lows. But overall, I thoroughly enjoyed the mental exercise to capture, structure and communicate complex concepts with simplicity and a narrative for the reader to follow. I was fortunate to have a great group of people from diverse backgrounds as early reviewers. They came along with me through the writing process and shared their feedback as conversations live in the draft files. I never felt alone or in an echo chamber having their presence and company.
Ìý
I began the writing process mid-2020 intending to write a series of blogs, but it was soon apparent that there was too much to share and I had to move the writing into the book format. That format I began later that year.
Ìý
Ìý
What did you learn from the process?
Writing a book is the best way to learn about yourself. Through writing and hearing yourself write, you learn about your voice, your hopes and fears, how you approach solving problems, the values that matter to you that underpin your work. Writing a book is the best way to reflect on who you are and change.
Ìý
Of course, having mentors and reviewers likeÌýÌýwas an incredible opportunity to learn to communicate to a technical audience. I could write a book on what I learned about writing from Martin.Ìý
What was the most interesting part of writing a book?
It’s hard to pick one aspect.ÌýFor me, the writing process was divided into two parts - developing and editing. First, developing the ideas, writing them down, and then editing them and in some cases rewriting them again. This cycle repeats itself many times through the book development.Ìý
Ìý
Developing and editing are both enjoyable but completely different activities and use different parts of the brain. I enjoyed content development greatly: it involved learning, researching, creating something new, drawing the complex concepts in simple figures, etc. It had a lot of creative elements. Writing was like coding a new problem for the first time.
Ìý
I also enjoyed editing as it was an opportunity to learn about reviewers’ perspective and feedback, and then reflect on how I could improve and refactor the content. Editing was refactoring the code after a code review.
Ìý
Who will benefit the most from reading your book?
This book is written for people with a diverse set of roles and skill sets.Ìý
Data mesh is a paradigm shift, and it needs the collective effort of many complementary roles and disciplines to make it a reality for any organization, from architects, practitioners and infrastructure engineers to product managers, data leaders, and executives.
What's the book's biggest takeaway?
Adopting Data Mesh requires changes in organizational design, architecture and fundamentally in how we imagine and define what constitutes data - data that can be shared responsibility as a product. We can’t just pick one aspect of data mesh, let’s say technical architecture, and ignore the others like data ownership structure.Ìý
And finally, why now was the perfect time to write a book about Data Mesh?
I perhaps wrote this book a bit earlier than I would have liked. We are still in the early years of a fundamentally different approach in sharing and creating data for analytical and machine learning (ML) use cases. But our industry has the tendency to percolate new concepts and buzzwords beyond recognition. Hence, I decided to write this book to create a common foundation for future evolutions of data mesh implementations. I wanted to make sure, before we get carried away with building new technical solutions, we understand why we need to make a change, what the problems we are trying to address are, and how we are trying to do that.Ìý
Ìý
This book lays the foundation for understanding and applying data mesh, and creates a platform for further refinements and advances without compromising the mission and objectives of data mesh.
Inside the book
Why Data Mesh?
Ìý
- The Inflection Point
- After The Inflection Point
- Before The Inflection Point
Ìý
What is Data Mesh
Ìý
- Principle of Domain ownership
- Principle of Data as a Product
- Principle of Self-Serve Data Platform
- Principle Of Federated Computational Governance
Zhamak Dehghani
Ìý
Zhamak works with ºÚÁÏÃÅ as the director of emerging technologies in North America, with a focus on distributed systems and big data architecture, with a deep passion for decentralized technology solutions - the foundations for democratization: data mesh, decentralized trust and identity, and networking protocols.
Ìý
She founded the concept of Data Mesh in 2018, a paradigm shift in big data management toward data decentralization, and since has been evangelizing the concept with the wider industry.
Ìý
She is a member of ºÚÁÏÃÅ Technology Advisory Board and contributes to the creation of ºÚÁÏÃÅ Technology Radar. Zhamak has worked as a technologist for over 20 years and has contributed to multiple patents in distributed computing communications, as well as embedded device technologies