Cloud Data -Lead

Job Description

Job Title: Cloud Data Engineer (Azure, Databricks, Data Factory)
Description: Experienced Cloud Data Engineer responsible for designing and developing Data Pipelines to populate a Cloud Data Lakehouse hosted on Azure Databricks.
Location: Offshore to United Kingdom
Reports to: Enterprise Applications Architect / Data Lakehouse Technical Lead
Role Summary:
Cloud Data Engineer required to assist in the ongoing development of a centralised Data Lakehouse hosted on Azure Databricks. This role will work closely with other members of the Data Lakehouse Team as well as the Architecture, Data Strategy and Transformation Teams and will play an important part in the overall Data Lakehouse Team. Ability to progressively learn all aspects of Databricks and to explore new ways of implementing requirements.
Key Characteristics
• Highly passionate, self-motivated and driven individual who has a strong understanding of the core aspects of a Databricks Data Lakehouse
• They will have a strong desire to improve existing solutions and to innovate and introduce new and exciting ways to improve the current way that Data is managed at Christie’s
• They will be good communicators that have an open and transparent style of working that can influence and operate with both technical and business stakeholders effectively. They will have strong interpersonal skills, can demonstrate facilitation skills and the ability to set technical direction when appropriate
Key interactions: Within CTG team, Architects Team, Data Lakehouse/Data Science Team, Infrastructure Team, Within the wider Christie’s business, 
Information Strategy Business Stakeholders
• Business Stakeholders within their business area(s) when required especially when assessing data needs
Key Tasks & Responsibilities (duties include but are not limited to the following)
• Development of highly performant and scalable Data Pipelines from various Sources into a Data Lakehouse
• Applying best practise for Data Modelling the Data Lakehouse Zones
• Implementing CI/CD across all artifact types (Databricks Workflows, Azure Functions, Data Factory, Python Wheel)
• Processing, cleansing, and verifying the integrity of data used for analysis
• Performing ad-hoc analysis and presenting results in a clear manner
• Clarify/explain data to senior business stakeholders in business friendly terms
Education and Professional Membership
• Educated to Degree Level preferably in Computer Science or related subject
• Azure certifications especially Databricks and Spark
Experience/ Skills Requirements
• Strong experience of Azure Databricks and Spark/SQL
• Strong experience in Python development
• Strong Experience of using CI/CD Pipelines within Azure DevOps
• Excellent understanding of the Delta Lake concept and Data Modelling Guidelines
• Excellent Code/Repository/Environment management skills
• Experience of Azure Data Factory
• Experience in Reporting and Visualisation Tools (Tableau, Power BI)
• Experience with Data Warehousing projects
• Experience with Monitoring and supporting/optimizing production workloads
• Experience of Azure Purview/Databricks Unity Catalog advantageous
• Excellent communication and interpersonal skills

Mandatory Competencies