Member-only story
How to think about Internal Data Products as a Data Engineer
Data Products are all the rage, but why?
About me
I’m Hugo Lu — I started my career working in M&A in London before moving to JUUL and falling into data engineering. After a brief stint back in finance, I headed up the Data function at London-based Fintech Codat. I’m now CEO at Orchestra, which is a data release pipeline tool that helps Data Teams release data into production reliably and efficiently 🚀
Also check out our Substack and our internal blog ⭐️
Introduction
Data Products are the hype at the moment due to a rapid decline in both Data Team population and their ability to drive value from Data. Thinking of Data as a Product, or treating “Data-as-a-Product” is a popular approach for understanding Data, Managing Data, and improving both the velocity and uptake of Data Products produced by Data Engineering Teams for the Business.
In this article we’ll dive into how to think about Internal Data Products as a Data Engineer. We’ll answer some common questions about Data Products. I’ll show you how I think about building your first Data Product too.
What are Data products?
Aside from the myriad of talks, social media posts, blog posts, and thought-leadership pieces, Data Products is a simple concept at its core.
The idea is twofold. The first is to apply principles of Product Management to Data. These are ways of working, and an example of such ideas are listed below:
- Start With Why
- Understand the Problem
- Focus Relentlessly
- Empower the Team
- Embrace Uncertainty
- Balance Inputs, Outputs, Outcomes, and Learning
- Iterate, Iterate, Iterate
Taken from here.
As you can see, these principles are very different from what you might see in a typical Data Engineer Job Description (no…
