Lead Data Engineer 

• You are a seasoned Data Engineer, with a track record of successfully implementing complex solutions. 
• The next step in your career is to move into a highly senior Data Engineering role, but you want to retain the technical aspect of what you do. 
• You have delivered valuable work in diverse, rapidly growing business environments, and with a range of technology appropriately selected for the task at hand. 
• In particular, you have experience with Cloud and Databricks (or similar, eg Snowflake) environments. 
• You relish the challenge of building an ultra-modern Cloud-first Group Data Platform in a highly dynamic setting, working with the best people in the field. 
• You think about how your role contributes to the overall success of the company; this is one of your main guiding principles in decision-making. 
• You are a pragmatic problem-solver with a bias to action and an appetite for getting things done. 
• You are a technical authority within your team: mentoring junior team members, ensuring the highest technical standards of work across the team, and representing as a figurehead in the wider technical community. 
• You’re passionate about data and the ever-evolving technology that is used to leverage it. 
• You are a leading voice in solution design/architecture - thinking proactively about how best to position your team for long-term success. 
• You interface well with your peers in other data pillars, some of whom are your customers, eg, Data Scientists, Data Architects, Data Analysts. 
• You enjoy the “people aspects” of your role, and form good relationships. You are trusted and respected. 
• You’ll be the foremost technical authority in the Data Engineering team within the Data Solutions Centre of Excellence, driving forward the development of the data platform, including developing system architecture and robust, efficient and clearly documented data products and pipelines within an agile working environment.  This is a career-defining opportunity to make a significant impact within a fast-growing organisation. 

Day-to-day, you will: 

• Be doing hands-on engineering, developing our Group Data Platform in partnership with your customers both within and beyond the Data Solutions organisation. 
• Pair with junior team members on their work, with the ultimate goal of elevating them to the same level of technical capability as yourself (or beyond!). 
• Play a key part in the coordination of project work in the team, supervising the work of others. 
• Establish high technical standards in the team (including guidelines for documentation, testing, SDLC, etc), and ensure that these are consistently met (eg, through code reviews). 
• Contribute heavily to a culture of knowledge-sharing, both within the team, and in the wider organisation. 
• Spend time with your stakeholders, forming great relationships with them and becoming a trusted partner. 

• Experience with contemporary data warehousing tools such as Databricks (or similar, eg, Snowflake). 
• Excellent ability to develop and test data pipeline code in Python and/or PySpark. 
• Excellent ability to design and execute complex queries using SQL for ELT processes.   
• Expertise in the use of cloud platform tools such as Data Factory, Integration Services, Azure functions, and Eventhubs.   
• Ability to process complex data structures  such as highly nested JSON, etc.  
• Experience integrating data from variety of typical source systems such as SQL and Oracle databases, and from Salesforce and similar web applications, including both structured and nonstructured data. 
• Ability to clearly document data engineering processes and dataset properties including schema definition and data quality issues. 
• Ability to evaluate and reflect on practice for CI/CD in the data engineering function. 
• Knowledge of tools like Kubernetes for deployment of data products.  
• Expert understanding of requirements related to the storage, management, and processing of personal data as they pertain to the design and management of a cloud data platform. 
• Substantial knowledge of Cloud platforms: e.g. AWS, Azure, GCP and their data services with a preference for Azure, with an Azure data Engineering (especially Databricks) certification a plus.   
• Knowledge of change data capture and processing for delivery of efficient solutions. 
• Ability to work with data platform users and stakeholders to develop user stories and requirements for platform features.