Principal Data Engineer

Job summary

Are you a passionate data engineering leader who can build and evolve data platforms that deliver secure, reliable, and high-quality data services at scale?

Can you lead multidisciplinary teams, establish engineering standards, and create the technical foundations that enable data to become a strategic asset across an organisation?

Do you have the ability to influence senior stakeholders, shape technology roadmaps, and ensure that data engineering delivers measurable outcomes for users and the wider business?

Companies House is building a Cloud Data Platform from the ground up, and we're looking for a Principal Data Engineer to lead its evolution, shaping the data platform utilising Snowflake and dbt building the data systems that underpin our analytical products and lead the platforms adoptions across the organisation. Your work will directly support our ambition to become a data-driven organisation, providing technical leadership for our data engineering capability, guiding the design, development, and optimisation of our data platform while leading a team of engineers and championing modern engineering practices.

Why choose Companies House?

At Companies house we take a proactive approach to wellbeing by promoting a welcoming culture, healthy work life balance and supporting colleagues to be their best in work because we know people are the key to our success.

Our benefits include:

• Flexible working with no core hours (6am – 8pm flexible options)
• 30 days annual leave
• 8 bank holidays per annum + privilege day (minimum offering based on location of contract)
• Enrolment into Civil Service Pension scheme average 28.97%

Find out more about what a great place Companies House is to work

Please note:

• Companies House cannot offer Visa sponsorship to candidates through this campaign.
• Security Clearance is an essential requirement for this campaign. You'll need to have been in the UK for at least 3 out of the last 5 years to be eligible to apply for SC.
• A hybrid employment contract will be provided as the standard offering. However, remote contracts may be considered as an exception whereby commuting to the office location of your team is not reasonably practicable.
• Flexible contract offered for part-time, min 30hrs per week, or full-time 37 per week. Role requires working availability minimum 4 days per week.

Job description

As a Principal Data Engineer, you will:

• Shape the strategic direction and continued evolution of the organisation's data platform, driving adoption across business areas and ensuring it delivers long-term value.
• Lead the design, development and optimisation of scalable data engineering solutions, transforming complex operational and business data into trusted datasets that support strategic decision-making.
• Provide technical leadership for data engineering across Companies House, establishing standards, patterns and best practices that promote quality, automation, security and reliability.
• Drive continuous improvement by identifying opportunities to enhance data architecture, engineering processes, performance and operational effectiveness.
• Ensure appropriate technical documentation, standards, processes and governance arrangements are maintained to support knowledge sharing, service continuity and compliance.
• Build effective relationships with senior stakeholders, influencing decision-making and promoting the value of data-driven approaches across the organisation.
• Lead, coach and develop a team of data engineers, creating an inclusive and supportive environment that enables colleagues to grow, perform and deliver excellent outcomes.
• Manage team performance, resource planning and capability development to ensure the organisation has the skills and expertise required to meet current and future business needs.

Person specification

We are looking for someone who can demonstrate:

• Experience establishing and promoting data engineering best practices, including coding standards, automation, testing, reproducibility, quality assurance and continuous improvement across teams.
• Advanced SQL skills and practical experience with tools such as Python, Git, dbt, Snowflake, Databricks or comparable technologies, including designing solutions for analytical data platforms and data warehouses.
• Strong understanding and practical application of data modelling techniques, including Kimball methodologies, star schemas, fact and dimension modelling, and slowly changing dimensions.
• The ability to take ownership of a data product, platform or service, shaping its strategy and roadmap around the needs of its users, and balancing immediate delivery needs with longer-term objectives and continuous improvement.
• A commitment to fostering an inclusive, collaborative and high-performing team culture, supporting colleagues to develop their skills and achieve successful outcomes.

These key criteria will be the basis for your application form if you choose to apply for this position.

Candidates will be expected to work to the required standards as set out in the Capability framework Data engineer - Government Digital and Data Profession Capability Framework

Behaviours

We'll assess you against these behaviours during the selection process:

• Leadership
• Managing a Quality Service
• Seeing the Big Picture

Technical skills

We'll assess you against these technical skills during the selection process:

• Communicating between technical and non-technical
• Data Modelling
• Programming and build within data & analytics engineering

Benefits

Alongside your salary of £58,831, Companies House contributes £17,043 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides (opens in a new window).

Join Companies House and discover a role that offers more than just a job, with flexibility, career development and a genuinely supportive culture where you can thrive:

• Flexible working from day one – hybrid working, flexible hours and flexitime
• A generous, competitive package – including 30 days annual leave + 8 bank holidays + 1 privilege day, plus family leave and sickness provision
• We invest in your future – funded training, professional qualifications and clear career progression pathways
• A modern and welcoming workplace – collaboration spaces, on-site facilities and an office you’ll enjoy
• Stand-out location benefits – including free parking in Cardiff and on-site amenities across all location
• Wellbeing at the heart of what we do – occupational health support, mental health advocates and wellbeing initiatives
• Real savings and lifestyle perks – myLifestyle discounts, gym discounts and childcare support
• A friendly, social and inclusive culture – staff networks, sports and social groups and volunteering opportunities
• Security you can count on –