Senior Data Engineer - Central Government

Role: Senior Data Engineer- 106904
Contract Length: Until 30/04/27
Location: Remote IR35: Inside
Pay Rate to Intermediary: Market Rate Security Clearance: SC requested

About the Role

Spinwell is recruiting for a Commercial Officer for an excellent opportunity within the public sector.

Key Responsibilities

• AI Data Integration & Pipeline Design: Establish client data integration standards and lead the engineering of automated data pipelines, ETL/ELT workflows, and API connections connecting enterprise SaaS and internal databases to AI applications whilst ensuring alignment with existing pipelines and standards.
• AI Data Security & Compliance: Implement identity and access controls, including SSO, conduct data quality, sensitivity and access checks before indexing, and ensure AI applications respect source-system permissions to prevent data leakage.
• AI Data Innovation & Diagnostics: Conduct early-stage data quality diagnostics to test dataset viability for AI automation, configure synthetic test datasets within isolated sandboxes, and structure domain-specific knowledge engines.
• Data Development Process: Establish repeatable, secure, and maintainable data solutions, managing metadata repositories and configuring observability pipelines across client AI workflows, reusing existing repositories and observability pipelines
• where possible.
• Technical & Stakeholder Communication: Act as the key bridge between technical engineering partners and client business leads, interpreting data requirements and translating complex data risks into clear choices for stakeholders.

Essential Skills and Experience

• Technical Mastery & Engineering Practice: Hands-on expertise in building scalable ETL/ELT pipelines, designing robust API integrations, and configuring vector indices or retrieval systems for large-scale enterprise AI architectures.
• Data Governance, Security & Compliance: Expert knowledge of Identity and Access Management, identity federation, data classification standards, and UK GDPR/DPA compliance frameworks within AI environments.
• Analytical Proficiency & Data Diagnostics: Substantial experience conducting quantitative data quality assessments, structuring synthetic testing environments, and evaluating dataset readiness for complex automation or ML models.
• Languages & Software Engineering Standards: Advanced proficiency in AWS, Python,SQL, and query optimisation, alongside demonstrated experience implementing CI/CD pipelines using Terraform, automated testing, and metadata management best practices.
• Strategic Communication & Leadership: Proven ability to translate complex technical data risks into strategic guidance for senior leadership, alongside a track record of leading cross-functional engineering teams in agile environments

 

Disability Confident Scheme

The client supports the Disability Confident Scheme and is committed to providing fair and inclusive recruitment opportunities. Candidates with a disability who meet the minimum essential criteria for the role may be eligible for an interview under the scheme. Where application volumes are particularly high, the client may shortlist those candidates who best meet the role requirements. 

Armed Forces Covenant 

The client supports the Armed Forces Covenant and is committed to helping veterans and military spouses or partners access employment opportunities.Eligible veterans and spouses or partners of Armed Forces personnel who meet the minimum essential criteria may qualify for an interview under the client's applicable recruitment scheme. Where application volumes are particularly high, shortlisting may be necessary. 

How to Apply Under These Schemes 

If you believe you qualify under either scheme, please inform your Spinwell recruitment consultant when applying and quote the relevant vacancy reference. We will explain the applicable process and confirm your request with the client. 

Reasonable Adjustments 

If you require reasonable adjustments at any stage of the recruitment process, or would prefer an alternative or more accessible method of communication, please let us know. We will work with you and the client to accommodate your needs wherever reasonably possible.

Candidate Privacy Notice 

By applying for this vacancy, you acknowledge that Spinwell Global Limited will process your personal information for recruitment purposes and may contact you about other suitable opportunities. Your details may be retained for up to 24 months from your last meaningful interaction with us, in accordance with our Recruitment & Website Privacy Notice, available at www.spinwellglobal.com.