Sr Azure Data Engineer Lead (Azure Databricks)

Hello Azure Data Professional!

We are looking for an " Sr Azure Data Engineer Lead (Azure Databricks) in Toronto, ON for a (Hybrid Role)."
 
Below is the JD for review.
 
Job Title- :Lead Azure Data Engineer (Azure Databricks)
Location-Toronto, ON (Hybrid)
Role- Full Time
 
Exp- 10-12+ years

Skills:

• Azure Data Lake Storage,
• Azure Data Factory,
• Azure Data Bricks,
• Azure Synapse,
• Azure Analysis Server,
• Azure Devops,
• Spark SQL,
• SQL,
• Data Modelling,
• Data Warehousing,
• Pyspark, Scala

Description:

• Implemented end-to-end solution for migrating on-premises system to Azure Cloud.
• Analysis of the requirements provided by the client and transforming them into functional and technical specifications.
• Built the metadata driven Data Ingestion and Data Transformation pipelines using Azure Data factory
• Have done the performance optimization using delta lake house in databricks
• Built the complex business logic in Databricks using SparkSQL, SCALA and PySpark
• Built the Spark applications for data extraction, transformation and aggregation from multiple file formats for analyzing and
• transforming the data uncover insight into the customer usage patterns.
• Creation of cubes on Visual Studio on top of Azure Synapse and ADLS which works as a consumption source for visualization on Power BI dashboards.
• Created multiple complex and dynamic stored procedures using Azure Synapse and scheduled them Using ADF.
• Have done the performance optimization for Databricks and Stored procedures and reduced the execution time to some extent.
• Deployed all the Components from one environment to another environment Using Azure Dev ops.
• Managed the team and responsible for requirement gathering, data modelling, capacity planning, sprint planning, developing and on time delivery of the project.