Join 📚 Kevin's Highlights
A batch of the best highlights from what Kevin's read, .
When we unpack the common threads of how various people define data engineering, an obvious pattern emerges:
a **data engineer**
*gets data, stores it, and prepares it for consumption*
by **data scientists**, **analysts**, and others.
We define data engineering and data engineer as follows:
**Data engineering** is
the *development*, *implementation*, and *maintenance*
of **systems** and **processes** that take in raw data
and produce high-quality, consistent information
that supports downstream use cases,
such as analysis and machine learning.
**Data engineering** is
the intersection of
*security*,
*data management*,
*DataOps*,
*data architecture*,
*orchestration*, and
*software engineering*.
A **data engineer**
*manages the data engineering lifecycle*,
beginning with getting data from source systems and
ending with serving data for use cases,
such as analysis or machine learning.
Fundamentals of Data Engineering
Reis, Joe;Housley, Matt;
The only controls available to those on board were two push-buttons on the center post of the cabin—
one labeled on and one labeled off.
The on button simply started a flight from Mars.
The off button was connected to nothing.
It was installed at the insistence of Martian mental-health experts, who said that human beings were always happier with machinery they thought they could turn off.
The Sirens of Titan
Kurt Vonnegut
Software companies aren't made of code, much like
bakeries aren't made of bread.
Software companies are made of
*processes that produce and maintain code*.
Software is a by-product of these processes.
It's not even the *final* product — it's a means to an end.
The final product is a solution to a business problem.
The Machine That Makes the Thing Is More Valuable Than the Thing
François Chollet
...catch up on these, and many more highlights
Readwise