Data Engineer (up to £61,340)

Job summary

IMPORTANT - This vacancy is using the Civil Service Numerical and Verbal tests. The deadline for completing the tests is the same as the vacancy closing date. Please ensure you complete the tests with sufficient time to complete the written application.

At DfE, we are proud of the commitment we make to diversity and inclusion and of the progress we have made. We have active & vibrant staff networks, special leave policies and workplace adjustments put in place for those who need them. We are continuing to build a diverse DfE, in an inclusive environment which nurtures and realises potential in all, at all levels.

At our heart, we are the department for Opportunity. We enable children and learners to thrive, by protecting the vulnerable and ensuring the delivery of excellent standards of education, training and care.

Data is integral to achieve this vision and the roles on offer are part of the Data Directorate within DfE. As a directorate, we provide a full range of data sourcing, management, governance, sharing, statistics, and visualisation capabilities and products to users across DfE, the education sector, cross-government, and citizens. We’re seeking highly capable Data Engineers to help us achieve our vision.

We’re striving to create a place that’s great to work in. We offer forward-thinking ways of working and training opportunities, as well as providing support from specialist colleagues. On top of that, you’ll find flexible working, an inclusive culture and a place where your skills can bring tangible change.

Job description

As a Data Engineer, you will be joining a team of talented data engineering professionals, who are committed to delivering high-impact, secure data solutions to achieve departmental and directorate objectives. You will:

• Work as part of a multidisciplinary team in a large and complex data environment to develop and deliver end-to-end data solutions.
• Translate user requirements and data models into effective, robust data engineering solutions in line with build standards and ensure code is production ready.
• Deliver data engineering solutions in line with project goals and delivery cycles, while working openly and collaboratively in our team.
• Be proactive and want to make an impact, and this may sometimes involve challenging current methods.
• Help build and maintain our data engineering community to facilitate knowledge sharing, innovation, and collaboration across the department.
• Work with new technologies such as Azure cloud-based tech stack, Databricks, Azure Data Factory, Agile ways of working, and practices such as CI/CD and DevOps. We use a combination of tooling, data quality processes, scripting, and modern cloud technologies to drive the engineering solutions.

Person specification

Essential Criteria:

We are looking for experience in:

• Designing and building robust and re-usable data engineering solutions on any of the following cloud platforms – Azure, AWS, Google Cloud.
• Working with common data types and industry standard architectural patterns such as data warehouses/lake houses etc. - to service operational and/or analytical business needs.
• Proficiency in one or more of the following – SQL, Python, Apache Spark (Pyspark, Spark SQL)
• Performing troubleshooting, root-cause analysis and improving data quality in the data products.
• Delivering through Agile/DevOps working practices in multi-disciplinary teams – CI/CD, Scrum, Automation.
• Staying up-to-date with latest tools and technologies and building proof-of-concepts using those technologies to solve a business problem.

Desirable Skills:

We are keen to hear about the following desirable skills, although these are not essential:

• Experience with Databricks, Informatica DEI and Azure Data Factory.
• Experience of implementing dimensional models.

Desirable criteria will only be assessed in event of a tie break situation to make an informed decision.

Behaviours

We'll assess you against these behaviours during the selection process:

• Making Effective Decisions

Benefits

Alongside your salary of £44,305, Department for Education contributes £12,835 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides (opens in a new window).

Applicants currently holding a permanent post in the Civil Service should note that, if successful, their salary on appointment would be determined by the Department’s transfer / promotion policies.

As a member of the DfE, you will be entitled to join the highly competitive Civil Service Pension Scheme, which many experts agree is one of the most generous in the UK.

You will have 25 days leave, increasing by 1 day every year to a maximum of 30 days after five years’ service. In addition, all staff receive the King’s Birthday privilege holiday and 8 days’ bank and public holidays.

We offer flexible working arrangements, such as job sharing, term-time working, flexi-time and compressed hours.

Most DfE employees will be working a hybrid pattern, spending at least 60% of their time in an office or work setting. Changes to these working arrangements are available in exceptional circumstances but must be agreed with the line manager and in line with the requirements of the role.

Travel to your primary office location will not be paid for by DfE, but costs for travel to an office which is not your main location will be covered.

As an organisation, which exists to support education and lifelong learning, we offer our staff excellent professional development opportunities.

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours and Experience.

Stage 1: Civil Service Numerical Test (CSNT)

When you have completed your personal information, you will be asked to complete the CSNT. You must achieve the minimum pass mark to progress to the next stage.

Stage 2: Civil Service Verbal Test (CSVT)

When you have completed the CSNT, you will be asked to complete the CSVT. You must achieve the minimum pass mark to progress to the next stage.

Stage 3: Written Application

Applications must include:

• A CV with education and full employment history, giving details of key achievements relevant to the essential criteria outlined in this job description. Please use the template within the application form. When giving details of your employment history, you should include details of the work and projects that you have been involved in, and your role therein.
• A personal statement outlining