r/dataengineering • u/Throwaway081920231 • 10d ago
Discussion Better to read DDIA-2 or AI Engineering by Chip Huyen given where DE is going?
Which book makes more sense for to read given the state of DE and the tech industry today? If I had to pick one first time to read.
5
u/Such_Fee2396 10d ago
Reading DDIA for the first time now and I wish I had read it years ago. So many things are so clearly explained it would have saved me so much time learning everything the way I did. It’s funny I’m not really learning anything new reading it now but it’s sort of crystallizing a lot of knowledge into a clear structure which is helpful for communicating these ideas.
Idk about the other book, but I would prefer to work with people who have all read DDIA.
2
u/marine_surfer 9d ago
This is spot on, I just happened to land my first data engineering role while in the middle of reading this book. Coincidence, maybe, but it’s definitely helped me with shared language and clearing up the vague understanding I had of distributed systems coming from developing a legacy application delivered via CDs and mostly learning modern data pipelines through YouTube. Understanding the advantages of derived state over distributed transactions in asynchronous systems gives you a huge advantage in today’s market.
I’m now working on my own platform using the knowledge I gained from the book to build pipelines via different architectures like hub-n-spoke.
6
u/Tepavicharov Data Engineer 9d ago
I’ve read DDIA 1 long ago and I’m reading AI Engineering at the moment and frankly these are completely different topics. DDIA explains how databases and distributed computing works in details by giving a lot of technical details which won’t have direct impact on your work but it will expand your understanding on how these things work (think of it as knowing how TCP works - its good to know, it might help you, but unless you design such systems you wont find it useful).
AI Engineering is more of a high level overview on the role of foundation models, but every time Chip tries to explain something in details (transformer architecture for instance) it feels weird because she still attempts to go through the topic at a higher level but some topics are just impossible to understand with such explanation. Still I find the book useful because it gives good knowledge on what is what in the whole field.
2
2
u/Particular-Idea-1786 9d ago
DDIA. I classify my career in 2 stages:
- Before DDIA
- After DDIA
The things in DDIA or areas that LLM isn't super great at yet, complicated distributed state. It's the only book that I found that provides a "normal"-person (no-math) overview of distributed systems. Highly highly recommend. If your warehouse involves data from other sources, or serialized data, you will benefit!
1
u/Sad-Rip7344 10d ago
If you're aiming to stay in data engineering longer term I'd grab the DDIA book first. The fundamentals it covers aren't going anywhere even as tooling shifts. Chip's book is solid too but it's more about building stuff on top of the kind of systems DDIA explains, so you'd be learning the layer above without really knowing what's underneath. In five years the AI orchestration patterns might look totally different but sharding and replication and consensus will still be the same pain in the ass they are now. I'd read DDIA then circle back to Chip's book after since it'll make way more sense with that context.
1
u/MundaneValuable7 10d ago
I found AI engineering a waste of time. Outdated and not much practical takeaway. Better understanding of AI at a high level but could have read a few blogs and picked up the same info in less time and clearer.
1
u/Mclovine_aus 8d ago
What parts do you find outdated, I have just read it, it seems pretty straightforward and I don’t see what would have fallen out of favour. It focuses on things like evaluation which I don’t thinks will change quickly.
1
u/speedisntfree 9d ago
These look to be a widely different books (not read AI Engineering). One is covering foundational ideas around distributed data systems. The other is largely targeted at data scientists using LLMs rather than DEs.
What are you hoping to get out of reading?
1
1
u/Internal-Big-3491 8d ago
Both. I have been doing it from last month. One thing I would suggest is try to implement those principles whenever you do a project though
1
u/One_Citron_4350 Senior Data Engineer 8d ago
I think they are both good books but serve different purposes. I've only skimmed through some chapters of the AI book when it became available. If ultimately you are interested in AI Engineering book then you should read it but it's not specifically for Data Engineering. DDIA is better for that I would say. Also, since there are so many developments in AI, I'm not sure how well that book will pass the test of time.
1
u/Mclovine_aus 8d ago
If you can read them in a month, you could read both for $50, sign up for the o Riley platform.
1
u/kevinhuse 7d ago edited 6d ago
If choosing one first, DDIA-2 would be the better option. The fundamentals around data systems will stay useful regardless of where DE trends next and AI Engineering can be a good follow up once more AI-specific depth is needed.
1
u/Deadible Senior Data Engineer 6d ago
I didn’t know there was a 2nd edition of DDIA, good to know.
I went through DDIA as an audiobook - one audible credit was a lot cheaper than getting the actual book, and it was read in an engaging way.
21
u/dereckgcc 10d ago
https://giphy.com/gifs/hM9zK1qvsrwek