The Value Of Big Data Engineering | Jesse Anderson In The Engineering Room Ep. 8
193In this episode of the “Engineering Room” Dave Farley talks to Jesse Anderson about Big Data and Data Engineering exploring what it takes to implement effective big data solutions and explore what data engineering is and why it matters.
Jesse Anderson is a Data Engineer, author and trainer who runs a company called the Big Data Institute where he helps companies all over the world with their Big Data problems.
His company also runs training courses to help people to learn, and extend their skills in Data engineering and data science, targeted at helping software engineers to understand and use big data more effectively.
The Engineering Room is a monthly series of longer-form conversations with industry experts and thought-leaders from the world of software.
_____________________________________________________
🙏The Engineering Room series is SPONSORED BY EQUAL EXPERTS
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ https://bit.ly/3ASy8n0
___________________________________________________
➡️ Links
Jesse Anderson's Blog Site: https://www.jesse-anderson.com
Jesse Anderson's Data Engineering Courses: https://www.jesse-anderson.com/courses/
The Data Dream Team Podcast: https://dreamteam.soda.io
Hidden Tech Debt in Machine Learning Systems: https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf
___________________________________________________
Also from Dave:
🎓 CD TRAINING COURSES
If you want to learn Continuous Delivery and DevOps skills, check out Dave Farley's courses
➡️ https://bit.ly/DFTraining
📧 JOIN CD MAIL LIST 📧
Keep up to date with the latest discussions, free "How To..." guides, events, online courses and exclusive offers. ➡️ https://bit.ly/MailListCD
_____________________________________________________
👕 T-SHIRTS
Do you like the t-shirts that I wear in my videos? YOU CAN SAVE £2 ON EACH ORDER from Qwertee via this link ➡️ https://bit.ly/3yUwvYC.
This Discount is only available for Continuous Delivery viewers!
🚨 DON'T FORGET TO USE OUR DISCOUNT CODE AT CHECKOUT: ContinuousDelivery
-------------------------------------------------------------------------------------
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is now available here ➡️ https://amzn.to/3DwdwT3
📖 "Continuous Delivery Pipelines" by Dave Farley paperback ➡️
https://amzn.to/3gIULlA
ebook version ➡️ https://leanpub.com/cd-pipelines
📖 The award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ https://amzn.to/2WxRYmx
📖 Refactoring Databases: Evolutionary Database Design Scott Ambler & Pramod Sadalage ➡️ https://amzn.to/36BjHrT
📖 Data Mesh: Delivering Data-Driven Value at Scale, by Zhamak Dehghani ➡️ https://amzn.to/3bv4soY
📖 Team Topologies - Matthew Skelton & Manuel Pais ➡️
https://amzn.to/2Y0NdSO
NOTE: If you click on one of these Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-------------------------------------------------------------------------------------
CHAPTERS:
00:00 Welcome
00:56 Introducing Jesse Anderson
02:10 What a SW Engineer Needs to Know About Big Data
04:52 Data Systems Life Cycle
08:57 The Benefits of Binary Formats
11:40 Schema v Records
13:09 What's Makes a Data Engineer?
17:41 Abstraction & Adaptor Patterns
22:45 Patterns for Schema Evolution
24:50 Break Up Shared Schemas
28:25 Cost Driven Architecture
31:37 Data Mesh & Data Teams
41:35 Should Data Engineers be SW Engineers?
45:20 It's a Socio-Technical Thang!
49:30 What Does Big Data Mean to You?
54:45 Big Data & Machine Learning
56:40 From the Perspective of Data Scientists
1:01:20 The Golden Ration 5:1
1:02:55 Key Ideas
1:05:40 Wrap Up
Other Videos By Modern Software Engineering
Other Statistics
ANDERSON Statistics For Modern Software Engineering
Modern Software Engineering currently has 12,842 views spread across 2 videos for ANDERSON. ANDERSON has approximately 1 hour of watchable video on his channel, roughly 1.19% of the content that Modern Software Engineering has uploaded to YouTube.