• FutureLearn logo

Introduction to Data Engineering with Microsoft Azure 2

Further your knowledge of Microsoft Azure services and improve your data engineering skills.

A woman with glasses sits gazing at her computer screen surrounded by code, whilst holding a pencil.

Introduction to Data Engineering with Microsoft Azure 2

  • 6 weeks

  • 4 hours per week

  • Digital certificate when eligible

  • Introductory level

Find out more about how to join this course

  • Duration

    6 weeks
  • Weekly study

    4 hours
  • 100% online

    How it works
  • Unlimited subscription

    $244.99 for a whole yearLearn more

Prepare for the DP-203: Data Engineering on Microsoft Azure exam

Microsoft logo This course has been created in partnership with Microsoft.

Building on your learning from Introduction to Data Engineering with Microsoft Azure 1, this course will develop your understanding of data engineering processes in Microsoft Azure, further preparing you to take the DP-203 exam and kickstart your career in data engineering.

Explore data services within Microsoft Azure

Using Azure data services and tools, you’ll be able to implement, develop, and optimise data storage, processing and security operations within your organisation.

You’ll be introduced to tools including Azure Synapse, Databricks and Azure Data Lake Storage, learning how each can improve and streamline your processes.

Design hybrid transactional and analytical processing (HTAP) patterns

As businesses continue to move to digital processes, they recognise the value of making faster, well-informed decisions and the impact this can have on gaining a competitive advantage.

You’ll be guided through HTAP architecture and learn how to design HTAP using Azure Synapse Analytics.

With this knowledge, you’ll be able to run analytics in near-real-time, giving you the ability to respond to opportunities at speed.

Discover data operations in Azure Databricks

Azure Databricks, a cloud-based big data and machine learning platform, empowers developers by simplifying enterprise-grade data application production.

You’ll identify the advantages of Azure Databricks over other Big Data platforms, and learn how to spend more time building apps and less time managing infrastructure.

You’ll finish this course understanding how Microsoft Azure can be used to optimise data engineering operations. Having completed both courses, you’ll be equipped to take the DP-203 exam and develop a career as a data professional.

Syllabus

  • Week 1

    Work with Hybrid Transactional and Analytical Processing Solutions using Azure Synapse Analytics

    • Monitor showing computer application.

      Plan hybrid transactional and analytical processing using Azure Synapse Analytics

      In this activity, you will learn about planning hybrid transactional and analytical processing using Azure Synapse Analytics.

    • Close up of hands of a computer user using typing on laptop notebook with software apps.

      Configure Azure Synapse Link with Azure Cosmos DB

      During this week, you will learn how to configure Azure Synapse link with Azure Cosmos DB.

    • Person monitoring a computer screen

      Query Azure Cosmos DB with Apache Spark for Azure Synapse Analytics

      In this activity, you will learn how to query Azure Cosmos DB with Apache Spark for Azure Synapse Analytics.

    • Robot pointing up to the wall

      Query Azure Cosmos DB with SQL Serverless for Azure Synapse Analytics

      In this activity, you will learn about querying Azure Cosmos DB with SQL Serverless for Azure Synapse Analytics.

  • Week 2

    Data engineering with Azure Databricks Part 1

    • Closeup photo of eyeglasses focused on a screen in background with coding elements on the screen.

      Describe Azure Databricks

      In this activity, you will describe Azure Databricks.

    • Person holding mug in front of a laptop

      Spark architecture fundamentals

      In this activity, you will learn about spark architecture fundamentals.

    • Computer codes on a screen

      Read and write data in Azure Databricks

      During this week, you will learn about reading and writing data in Azure Databricks.

    • Data server illuminated by red and blue LED lighting.

      Work with DataFrames in Azure Databricks

      During this week, you will learn how to work with DataFrames in Azure Databricks.

    • People with computers and headsets in a center

      Work with DataFrames columns in Azure Databricks

      During this week, you will learn how to work with DataFrames columns in Azure Databricks.

  • Week 3

    Data engineering with Azure Databricks Part 2

    • A computer monitor with keyboard and mouse on the table

      Describe lazy evaluation and other performance features in Azure Databricks

      During this week, you will learn how to describe lazy evaluation and other performance features in Azure Databricks.

    • Digital cyberspace with digital data network connections.

      Work with DataFrames advanced methods in Azure Databricks

      In this activity, you will learn how to work with DataFrames advanced methods in Azure Databricks.

    • turned on gray laptop computer with a window in the background and a desklamp next to the computer.

      Describe platform architecture, security, and data protection in Azure Databricks

      During this week, you will learn how to describe platform architecture, security, and data protection in Azure Databricks.

    • Person Typing on Computer Keyboard

      Build and query a Delta Lake

      Learn how to use Delta Lake to create, append, and upsert data to Apache Spark tables, taking advantage of built-in reliability and optimizations.

    • Person wearing a hooded pullover facing a computer

      Process streaming data with Azure Databricks structured streaming

      Learn how Structured Streaming helps you process streaming data in real time, and how you can aggregate data over windows of time.

  • Week 4

    Data engineering with Azure Databricks Part 3

    • A graphic of a person typing on a laptop

      Describe Azure Databricks Delta Lake architecture

      Use Delta Lakes as an optimization layer on top of blob storage to ensure reliability and low latency within unified Streaming and batch data pipelines.

    • Graphic of a man typing on a laptop.

      Create production workloads on Azure Databricks with Azure Data Factory

      Azure Data Factory helps you create workflows that orchestrate data movement and transformation at scale. Integrate Azure Databricks into your production pipelines by calling notebooks and libraries.

    • Graphic of a person typing on a laptop.

      Implement CI/CD with Azure DevOps

      CI/CID isn't just for developers. Learn how to put Azure Databricks notebooks under version control in an Azure DevOps repo and build deployment pipelines to manage your release process.

    • Graphic of a person typing on a laptop.

      Integrate Azure Databricks with Azure Synapse

      Azure Databricks is just one of many powerful data services in Azure. Learn how to integrate with Azure Synapse Analytics as part of your data architecture.

    • Graphic of a person typing on a laptop.

      Describe Azure Databricks best practices

      Learn best practices for workspace administration, security, tools, integration, databricks runtime, HA/DR, and clusters in Azure Databricks.

  • Week 5

    Large-Scale Data Processing with Azure Data Lake Storage Gen2

    • Graphic of a person typing on a laptop.

      Introduction to Azure Data Lake storage

      Learn how Azure Data Lake Storage provides a cloud storage service that is highly available, secure, durable, scalable, and redundant and brings new efficiencies to processing big data analytics workloads.

    • Graphic of a person typing on a laptop.

      Upload data to Azure Data Lake Storage

      Learn various ways to upload data to Data Lake Storage Gen 2. Upload data through the Azure portal, Azure Storage Explorer, or .NET. Or copy the data in Azure Data Factory.

    • Graphic of a person typing on a laptop.

      Secure your Azure Storage account

      Learn how Azure Storage provides multilayered security to protect your data. Find out how to use access keys, to secure networks, and to use Advanced Threat Protection to proactively monitor your system.

  • Week 6

    Implement a Data Streaming Solution with Azure Streaming Analytics

    • Graphic of a person typing on a laptop.

      Work with data streams by using Azure Stream Analytics

      Explore how Azure Stream Analytics integrates with your applications or Internet of Things (IoT) devices to gain insights with real-time streaming data. Learn how to consume and analyze data streams and derive actionable results.

    • Graphic of a person typing on a laptop.

      Enable reliable messaging for Big Data applications using Azure Event Hubs

      Connect sending and receiving applications with Event Hubs so you can handle extremely high loads without losing data.

    • Graphic of a person typing on a laptop.

      Ingest data streams with Azure Stream Analytics

      Learn how to create Azure Stream Analytics jobs to process input data, transform it with a query, and return results.

When would you like to start?

Start straight away and join a global classroom of learners. If the course hasn’t started yet you’ll see the future date listed below.

  • Available now

Learning on this course

On every step of the course you can meet other learners, share your ideas and join in with active discussions in the comments.

What will you achieve?

By the end of the course, you‘ll be able to...

  • Explain Hybrid Transactional and Analytical Processing Solutions using Azure Synapse Analytics
  • Investigate Data engineering with Azure Databricks
  • Explore Large-Scale Data Processing with Azure Data Lake Storage Gen2
  • Apply a Data Streaming Solution with Azure Streaming Analytics

Who is the course for?

This course is designed for data professionals preparing for the DP 203: Data Engineering on Microsoft Azure exam.

Before taking this course, learners should take Introduction to Data Engineering with Microsoft Azure 1 to ensure they have covered all topics required for the DP 203 exam.

It’s recommended that you already have a solid understanding of data processing languages, as well as parallel processing and data architecture patterns before taking the exam.

Who developed the course?

FutureLearn logo

FutureLearn

FutureLearn is a leading social learning platform and has been providing high quality online courses for learners around the world over the last ten years.

Ways to learn

Buy this course

Subscribe & save

Limited access

Choose the best way to learn for you!

$79/one-off payment

$244.99 for a whole year

Automatically renews

Free

Fulfill your current learning needDevelop skills to further your careerSample the course materials
Access to this courseticktick

Access expires 19 Mar 2025

Access to 1,000+ coursescrosstickcross
Learn at your own paceticktickcross
Discuss your learning in commentstickticktick
Certificate when you're eligiblePrinted and digitalDigital onlycross
Continue & Upgrade

Cancel for free anytime

Ways to learn

Choose the best way to learn for you!

Subscribe & save

$244.99 for a whole year

Automatically renews

Develop skills to further your career

  • Access to this course
  • Access to 1,000+ courses
  • Learn at your own pace
  • Discuss your learning in comments
  • Digital certificate when you're eligible

Cancel for free anytime

Buy this course

$79/one-off payment

Fulfill your current learning need

  • Access to this course
  • Learn at your own pace
  • Discuss your learning in comments
  • Printed and digital certificate when you’re eligible

Limited access

Free

Sample the course materials

  • Access expires 19 Mar 2025

Find out more about certificates, Unlimited or buying a course (Upgrades)

Sale price available until 3 March 2025 at 23:59 (UTC). T&Cs apply.

Find out more about certificates, Unlimited or buying a course (Upgrades)

Sale price available until 3 March 2025 at 23:59 (UTC). T&Cs apply.

Learning on FutureLearn

Your learning, your rules

  • Courses are split into weeks, activities, and steps to help you keep track of your learning
  • Learn through a mix of bite-sized videos, long- and short-form articles, audio, and practical activities
  • Stay motivated by using the Progress page to keep track of your step completion and assessment scores

Join a global classroom

  • Experience the power of social learning, and get inspired by an international network of learners
  • Share ideas with your peers and course educators on every step of the course
  • Join the conversation by reading, @ing, liking, bookmarking, and replying to comments from others

Map your progress

  • As you work through the course, use notifications and the Progress page to guide your learning
  • Whenever you’re ready, mark each step as complete, you’re in control
  • Complete 90% of course steps and all of the assessments to earn your certificate

Want to know more about learning on FutureLearn? Using FutureLearn

Do you know someone who'd love this course? Tell them about it...