WE ARE DCODED LIMITED
Lead Databricks Data Engineer
Lead Databricks Data Engineer at WE ARE DCODED LIMITED, based in Leeds, with 2 days per week on site, paying £600 per day. This is a contract role.
- Pay
- £600 per day
- Hours and contract
- Contract · On site
- Location
- Leeds, with 2 days per week on site
- Closing date
- Sector
- Data Analyst
About 16% above the going rate for data analyst
Reference job/lead-databricks-data-engineer/WAD4757
About the role
Posted 22 hours ago
Ref: WAD4757
Lead Databricks Data Engineer Leeds £600pd ContractLead Databricks Engineer
Sector: Finance
Contract: 6 months | Outside IR35
Rate: £600
Location: Leeds, with 2 days per week on site
The role
We are looking for a hands-on Lead Databricks Engineer to lead the delivery and improvement of data pipelines within an Azure Databricks environment. The platform uses a medallion architecture across its tables and views, with a strong focus on Python, reliable data processing and engineering quality.
You will combine technical leadership with development, helping the team make sound design decisions, investigate loosely defined requirements and deliver well-tested solutions.
Key responsibilities
Lead the design, development and optimisation of Databricks pipelines using Python, PySpark and SQL.
Build and maintain Delta Live Tables pipelines, managed tables and views across bronze, silver and gold layers.
Define and implement data reconciliation and quality checks, investigating discrepancies between source data and downstream outputs.
Set clear standards for testing, documentation and peer review, providing technical guidance to other engineers.
Use Git feature branches, pull requests and code reviews to manage changes and merge approved work back into main.
Investigate ambiguous tickets, clarify requirements with stakeholders and translate findings into practical delivery tasks.
Work with Azure Data Factory where required and contribute to the adoption of Lakeflow Connect for ingestion.
Explore appropriate uses of Databricks Genie and other AI capabilities within the data platform.
Essential experience
Experience in Payments, Financial Services, Banking and / or Fintech
Strong hands-on Azure Databricks and Python experience, with the ability to lead technical delivery.
Experience with Delta Live Tables, Delta Lake, managed tables and medallion architecture.
Strong SQL skills and experience developing and troubleshooting data transformations.
A practical approach to data reconciliation, pipeline testing, data quality and peer review.
Experience with Git branching and collaborative development workflows.
Ability to work independently through ambiguity and communicate technical decisions clearly.
Working knowledge of Azure Data Factory.
Desirable experience
PySpark experience building and optimising distributed data processing.
Lakeflow Connect experience, particularly for data ingestion.
Practical use of Databricks Genie or other AI tools in data engineering.
Featured Jobs
Technical Support Engineer
Manchester £55000Technology Support Engineer Working arrangement: Hybrid - four days per week in the Cheltenham office Employment type: Full-time Sector: financial services Salary: Up to £55,000 The Opportunity We are looking for a technically capable…
22 hoursRemediation & Treatment Lead
Fully Remote £500-£600pdCustomer Remediation Lead | 6-Month Contract | Outside IR35 | Remote | Occasional London Travel £500-£600 We are supporting a leading financial services organisation undergoing a significant customer remediation programme and are seeking an experi…
4 monthsSoftware Engineer (.NET, JavaScript, SQL)
Cheshire, Fully Remote, Manchester £70,000Full Stack Software Engineer – Stealth InsurTech SaaS Tech: TypeScript / React / React Native / C# / .NET / SQL Location: Remote, but ideally be commutable to North West Salary: £70,000 Recruitment Partner: We Are Dcoded The opportunity We Ar…
2 monthsClient Delivery Manager / Project Manager
Manchester £400pdCustomer Delivery Manager/Project Manager - Fintech £375-£400 per day | Hybrid (1-2 days per week in Central Manchester) Outside IR35 About the Business Our client is a market-leading FinTech organisation - Their cloud-based platforms supp…
4 monthsHead of Platform Engineering
Manchester £110,000-£135,000Head of Platform Engineering Location: Manchester (Hybrid) Type: Permanent Salary: 115-135k DOE The Role We're looking for a visionary cloud and platform engineering leader to take ownership of the entire digital infrastructure estate for a Man…
4 monthsSenior Platform Engineer
Manchester £80,000-£95,000Senior Platform Engineer Location: Manchester (Hybrid) Type: Permanent Salary: 70-95k DOE The Role We're looking for a Senior Platform Engineer to join a Manchester-based SaaS business operating at the heart of the UK financial services market. You'l…
4 months View All JobsReference: job/lead-databricks-data-engineer/WAD4757 · Posted 6 days ago · Closes 27 Nov 2026 · Listed via WE ARE DCODED LIMITED
Apply for this job
This role is listed via WE ARE DCODED LIMITED. Applications are handled on the employer's site.
Apply on employer siteOpens the employer's website in a new tab.
Signed in? Every job you apply for counts toward your weekly goal and badges. How it works
Safe applying: a genuine employer will never ask you to pay for a DBS check, training or equipment, or move you onto WhatsApp before you are hired. If this listing does, report it and do not pay anything.