Duration
4 weeksWeekly study
4 hours100% online
How it works
How to use Databricks Lakehouse and Responsible AI
Master the fundamentals of the Databricks Lakehouse platform
On this four-week course, you’ll delve into the Databricks Lakehouse platform to understand its functionality and architecture.
You’ll gain hands-on experience using the platform to develop practical skills in managing large-scale data operations, developing AI solutions, and implementing ethical AI practices.
Armed with these skills, you’ll be able to build robust, responsible AI systems. This will make you a valuable asset in the rapidly evolving field of AI and data science.
Gain an understanding of data transformation and Data Live tables
You’ll delve into data transformation and pipelines to understand how to develop efficient data pipelines using Data Live tables and Jobs.
Learning the best practices, you’ll discover how to harness data transformation for effective decision-making.
Unpack responsible Generative AI
As the AI industry continues to evolve, it’s important to consider how users can do this responsibly.
On week three of the course, you’ll explore AI ethics in detail and unpack real-world examples so you can evaluate the ethical implications of GenAI technologies.
Gain fundamental skills in Unity Catalog
Finally, you’ll learn how to use data governance strategies with Unity Catalog to help you master advanced skills in Databricks Lakehouse.
Learning from industry experts and through hands-on exercises, you’ll finish the course with the technical skills and ethical considerations needed in the AI industry.
Syllabus
Week 1
Databricks Lakehouse Platform Fundamentals
Databricks Lakehouse Platform
Learn to leverage Databricks and AI for advanced data analytics. Master the Lakehouse platform, build end-to-end ML pipelines, and responsibly integrate LLMs into your workflows. Gain hands-on experience with clusters and more.
Data Transformation with Apache Spark
Explore data transformation using Apache Spark in Databricks. Learn to set up development environments, use CLI tools, and work with notebooks. Covers multi-language support, repos, and practical exercises in various languages.
Data Management with Delta Lake
Learn data management with Delta Lake. Topics: Spark SQL, Catalog Explorer, table creation, querying external sources, Delta Lake pipelines, ACID transactions, Z-Ordering. Includes lab, tutorials, and quiz on RStudio and COPY INTO
Week 2
Data Transformation and Pipelines
Data Pipelines with Delta Live Tables
Explore Delta Live Tables for automated data pipelines. Learn key components, pipeline types, Auto Loader configuration, event querying, and end-to-end examples. Includes vacuum and garbage collection, plus hands-on labs
Workloads with Jobs
Explore Databricks Jobs for orchestrating workloads. Learn about multi-task workflows, dependencies, job history, dashboards, and failure handling. Includes demos, labs, and quizzes to reinforce concepts and practical application.
Data Access with Unity Catalog
Explore Unity Catalog for data access and governance. Learn about catalogs, metastores, and best practices. Hands-on with Python quickstart and object security. Includes external lab, dynamic catalog building, reflection, and quiz
Week 3
Responsible Generative AI
AI Ethics of Generative Models
Explore the ethical implications of generative AI. Learn about profit sharing, tragedy of the commons, game theory, and regulatory challenges. Examine issues of bias, negative externalities, and perfect competition.
Evaluating Real-World Performance of LLMs
Learn to assess Large Language Model performance in real-world scenarios. Implement Elo rating systems in Python, Rust, R, and Julia. Gain hands-on experience through coding labs.
Exploring Production LLM Workflows
Dive into practical LLM deployment using Lorax and Skypilot. Understand Ludwig for model fine-tuning. Apply these tools to fine-tune Mistral-7b and launch Mixtral.
Week 4
Local LLMOps
Getting Started with local models
Explore local AI models with llamafile. Learn to set up and run models like Mixtral, understand system metrics, and get hands-on experience. Dive into key concepts like Whisper.cpp.
Getting Started with Rust Candle
Lesson 2 introduces Rust Candle, covering basic implementation, Starcoder exploration, and Whisper transcription. It delves into remote AWS development, AI security topics like sleeper agents and data poisoning.
Using Rust Candle
Explore Rust Candle for LLM applications. Learn serverless inference, CLI & chat inference, and using Star Coder. Implement Rust Candle on AWS GPU. Includes readings, reflections, and a quiz to reinforce learning.
When would you like to start?
Start straight away and join a global classroom of learners. If the course hasn’t started yet you’ll see the future date listed below.
Available now
Learning on this course
On every step of the course you can meet other learners, share your ideas and join in with active discussions in the comments.
What will you achieve?
By the end of the course, you‘ll be able to...
- Create solutions that use Databricks for data engineering and ML workloads
- Create and design ML pipelines
- Develop solutions with Llamafile and other local LLMs like Mixtral
- Critique ethical issues with the use of Generative AI including the tragegy of the commons and negative externalities.
Who is the course for?
This course is designed for data professionals, software engineers, and AI enthusiasts looking to master the Databricks Lakehouse Platform and explore responsible AI practices.
It’s ideal for those seeking to advance their careers in data engineering, machine learning, and AI ethics.
The course serves professionals in tech, finance, healthcare, and research sectors who want to implement scalable data solutions and understand the ethical implications of generative AI.
Who will you learn with?
Founder of Pragmatic AI Labs & Executive in Residence at Duke MIDS and Duke AIPI. Former Bay Area CTO and author of multiple O'Reilly books.
Who developed the course?
Ways to learn | Buy this course | Subscribe & save | Limited access |
---|---|---|---|
Choose the best way to learn for you! | $79/one-off payment | $244.99 for a whole year Automatically renews | Free |
Fulfill your current learning need | Develop skills to further your career | Sample the course materials | |
Access to this course | tick | tick | Access expires 5 Mar 2025 |
Access to 1,000+ courses | cross | tick | cross |
Learn at your own pace | tick | tick | cross |
Discuss your learning in comments | tick | tick | tick |
Tests to check your learning | tick | tick | cross |
Certificate when you're eligible | Printed and digital | Digital only | cross |
Cancel for free anytime |
Ways to learn
Choose the best way to learn for you!
Subscribe & save
$244.99 for a whole year
Automatically renews
Develop skills to further your career
- Access to this course
- Access to 1,000+ courses
- Learn at your own pace
- Discuss your learning in comments
- Tests to boost your learning
- Digital certificate when you're eligible
Cancel for free anytime
Buy this course
$79/one-off payment
Fulfill your current learning need
- Access to this course
- Learn at your own pace
- Discuss your learning in comments
- Tests to boost your learning
- Printed and digital certificate when you’re eligible
Limited access
Free
Sample the course materials
- Access expires 5 Mar 2025
Find out more about certificates, Unlimited or buying a course (Upgrades) Sale price available until 3 March 2025 at 23:59 (UTC). T&Cs apply. |
Find out more about certificates, Unlimited or buying a course (Upgrades)
Sale price available until 3 March 2025 at 23:59 (UTC). T&Cs apply.
Learning on FutureLearn
Your learning, your rules
- Courses are split into weeks, activities, and steps to help you keep track of your learning
- Learn through a mix of bite-sized videos, long- and short-form articles, audio, and practical activities
- Stay motivated by using the Progress page to keep track of your step completion and assessment scores
Join a global classroom
- Experience the power of social learning, and get inspired by an international network of learners
- Share ideas with your peers and course educators on every step of the course
- Join the conversation by reading, @ing, liking, bookmarking, and replying to comments from others
Map your progress
- As you work through the course, use notifications and the Progress page to guide your learning
- Whenever you’re ready, mark each step as complete, you’re in control
- Complete 90% of course steps and all of the assessments to earn your certificate
Want to know more about learning on FutureLearn? Using FutureLearn
Do you know someone who'd love this course? Tell them about it...
More courses you might like
Learners who joined this course have also enjoyed these courses.
Browse more in IT & Computer Science and Law