Duration
6 weeksWeekly study
4 hours100% online
How it works
Introduction to Data Engineering with Microsoft Azure 2
Prepare for the DP-203: Data Engineering on Microsoft Azure exam
This course has been created in partnership with Microsoft. |
Building on your learning from Introduction to Data Engineering with Microsoft Azure 1, this course will develop your understanding of data engineering processes in Microsoft Azure, further preparing you to take the DP-203 exam and kickstart your career in data engineering.
Explore data services within Microsoft Azure
Using Azure data services and tools, you’ll be able to implement, develop, and optimise data storage, processing and security operations within your organisation.
You’ll be introduced to tools including Azure Synapse, Databricks and Azure Data Lake Storage, learning how each can improve and streamline your processes.
Design hybrid transactional and analytical processing (HTAP) patterns
As businesses continue to move to digital processes, they recognise the value of making faster, well-informed decisions and the impact this can have on gaining a competitive advantage.
You’ll be guided through HTAP architecture and learn how to design HTAP using Azure Synapse Analytics.
With this knowledge, you’ll be able to run analytics in near-real-time, giving you the ability to respond to opportunities at speed.
Discover data operations in Azure Databricks
Azure Databricks, a cloud-based big data and machine learning platform, empowers developers by simplifying enterprise-grade data application production.
You’ll identify the advantages of Azure Databricks over other Big Data platforms, and learn how to spend more time building apps and less time managing infrastructure.
You’ll finish this course understanding how Microsoft Azure can be used to optimise data engineering operations. Having completed both courses, you’ll be equipped to take the DP-203 exam and develop a career as a data professional.
Syllabus
Week 1
Work with Hybrid Transactional and Analytical Processing Solutions using Azure Synapse Analytics
Plan hybrid transactional and analytical processing using Azure Synapse Analytics
In this activity, you will learn about planning hybrid transactional and analytical processing using Azure Synapse Analytics.
Configure Azure Synapse Link with Azure Cosmos DB
During this week, you will learn how to configure Azure Synapse link with Azure Cosmos DB.
Query Azure Cosmos DB with Apache Spark for Azure Synapse Analytics
In this activity, you will learn how to query Azure Cosmos DB with Apache Spark for Azure Synapse Analytics.
Query Azure Cosmos DB with SQL Serverless for Azure Synapse Analytics
In this activity, you will learn about querying Azure Cosmos DB with SQL Serverless for Azure Synapse Analytics.
Week 2
Data engineering with Azure Databricks Part 1
Describe Azure Databricks
In this activity, you will describe Azure Databricks.
Spark architecture fundamentals
In this activity, you will learn about spark architecture fundamentals.
Read and write data in Azure Databricks
During this week, you will learn about reading and writing data in Azure Databricks.
Work with DataFrames in Azure Databricks
During this week, you will learn how to work with DataFrames in Azure Databricks.
Work with DataFrames columns in Azure Databricks
During this week, you will learn how to work with DataFrames columns in Azure Databricks.
Week 3
Data engineering with Azure Databricks Part 2
Describe lazy evaluation and other performance features in Azure Databricks
During this week, you will learn how to describe lazy evaluation and other performance features in Azure Databricks.
Work with DataFrames advanced methods in Azure Databricks
In this activity, you will learn how to work with DataFrames advanced methods in Azure Databricks.
Describe platform architecture, security, and data protection in Azure Databricks
During this week, you will learn how to describe platform architecture, security, and data protection in Azure Databricks.
Build and query a Delta Lake
Learn how to use Delta Lake to create, append, and upsert data to Apache Spark tables, taking advantage of built-in reliability and optimizations.
Process streaming data with Azure Databricks structured streaming
Learn how Structured Streaming helps you process streaming data in real time, and how you can aggregate data over windows of time.
Week 4
Data engineering with Azure Databricks Part 3
Describe Azure Databricks Delta Lake architecture
Use Delta Lakes as an optimization layer on top of blob storage to ensure reliability and low latency within unified Streaming and batch data pipelines.
Create production workloads on Azure Databricks with Azure Data Factory
Azure Data Factory helps you create workflows that orchestrate data movement and transformation at scale. Integrate Azure Databricks into your production pipelines by calling notebooks and libraries.
Implement CI/CD with Azure DevOps
CI/CID isn't just for developers. Learn how to put Azure Databricks notebooks under version control in an Azure DevOps repo and build deployment pipelines to manage your release process.
Integrate Azure Databricks with Azure Synapse
Azure Databricks is just one of many powerful data services in Azure. Learn how to integrate with Azure Synapse Analytics as part of your data architecture.
Describe Azure Databricks best practices
Learn best practices for workspace administration, security, tools, integration, databricks runtime, HA/DR, and clusters in Azure Databricks.
Week 5
Large-Scale Data Processing with Azure Data Lake Storage Gen2
Introduction to Azure Data Lake storage
Learn how Azure Data Lake Storage provides a cloud storage service that is highly available, secure, durable, scalable, and redundant and brings new efficiencies to processing big data analytics workloads.
Upload data to Azure Data Lake Storage
Learn various ways to upload data to Data Lake Storage Gen 2. Upload data through the Azure portal, Azure Storage Explorer, or .NET. Or copy the data in Azure Data Factory.
Secure your Azure Storage account
Learn how Azure Storage provides multilayered security to protect your data. Find out how to use access keys, to secure networks, and to use Advanced Threat Protection to proactively monitor your system.
Week 6
Implement a Data Streaming Solution with Azure Streaming Analytics
Work with data streams by using Azure Stream Analytics
Explore how Azure Stream Analytics integrates with your applications or Internet of Things (IoT) devices to gain insights with real-time streaming data. Learn how to consume and analyze data streams and derive actionable results.
Enable reliable messaging for Big Data applications using Azure Event Hubs
Connect sending and receiving applications with Event Hubs so you can handle extremely high loads without losing data.
Ingest data streams with Azure Stream Analytics
Learn how to create Azure Stream Analytics jobs to process input data, transform it with a query, and return results.
When would you like to start?
Start straight away and join a global classroom of learners. If the course hasn’t started yet you’ll see the future date listed below.
Available now
Learning on this course
On every step of the course you can meet other learners, share your ideas and join in with active discussions in the comments.
What will you achieve?
By the end of the course, you‘ll be able to...
- Explain Hybrid Transactional and Analytical Processing Solutions using Azure Synapse Analytics
- Investigate Data engineering with Azure Databricks
- Explore Large-Scale Data Processing with Azure Data Lake Storage Gen2
- Apply a Data Streaming Solution with Azure Streaming Analytics
Who is the course for?
This course is designed for data professionals preparing for the DP 203: Data Engineering on Microsoft Azure exam.
Before taking this course, learners should take Introduction to Data Engineering with Microsoft Azure 1 to ensure they have covered all topics required for the DP 203 exam.
It’s recommended that you already have a solid understanding of data processing languages, as well as parallel processing and data architecture patterns before taking the exam.
Ways to learn | Buy this course | Subscribe & save | Limited access |
---|---|---|---|
Choose the best way to learn for you! | $79/one-off payment | $244.99 for a whole year Automatically renews | Free |
Fulfill your current learning need | Develop skills to further your career | Sample the course materials | |
Access to this course | tick | tick | Access expires 19 Mar 2025 |
Access to 1,000+ courses | cross | tick | cross |
Learn at your own pace | tick | tick | cross |
Discuss your learning in comments | tick | tick | tick |
Certificate when you're eligible | Printed and digital | Digital only | cross |
Cancel for free anytime |
Ways to learn
Choose the best way to learn for you!
Subscribe & save
$244.99 for a whole year
Automatically renews
Develop skills to further your career
- Access to this course
- Access to 1,000+ courses
- Learn at your own pace
- Discuss your learning in comments
- Digital certificate when you're eligible
Cancel for free anytime
Buy this course
$79/one-off payment
Fulfill your current learning need
- Access to this course
- Learn at your own pace
- Discuss your learning in comments
- Printed and digital certificate when you’re eligible
Limited access
Free
Sample the course materials
- Access expires 19 Mar 2025
Find out more about certificates, Unlimited or buying a course (Upgrades) Sale price available until 3 March 2025 at 23:59 (UTC). T&Cs apply. |
Find out more about certificates, Unlimited or buying a course (Upgrades)
Sale price available until 3 March 2025 at 23:59 (UTC). T&Cs apply.
Learning on FutureLearn
Your learning, your rules
- Courses are split into weeks, activities, and steps to help you keep track of your learning
- Learn through a mix of bite-sized videos, long- and short-form articles, audio, and practical activities
- Stay motivated by using the Progress page to keep track of your step completion and assessment scores
Join a global classroom
- Experience the power of social learning, and get inspired by an international network of learners
- Share ideas with your peers and course educators on every step of the course
- Join the conversation by reading, @ing, liking, bookmarking, and replying to comments from others
Map your progress
- As you work through the course, use notifications and the Progress page to guide your learning
- Whenever you’re ready, mark each step as complete, you’re in control
- Complete 90% of course steps and all of the assessments to earn your certificate
Want to know more about learning on FutureLearn? Using FutureLearn
Do you know someone who'd love this course? Tell them about it...
More courses you might like
Learners who joined this course have also enjoyed these courses.
Browse more in IT & Computer Science