Senior Data Engineer - Batch / Data Lake

Hybrid Working (6 days per month onsite in Farringdon)

Who are we? 

Toyota Connected Europe wants to create a better world through connected mobility for all. We are a new company created to bring big data and a customer focus into all aspects of the mobility experience, so everyone’s experience is more personal, convenient, fun and safe. We create and enable technologies to delight and simplify the lives of everyone who use our products and empower them to think of and use our services in new ways. 

You will be joining us at the beginning of Toyota Connected Europe’s journey of building our team and products. We are building teams to inspire, innovate and build technologies and products that are used by millions of people from all walks of life. We want every member of our team to live and breathe the start-up culture of Toyota Connected Europe and feel and act like an owner everyday. This is an opportunity to have an immediate impact and voice: what you create today, you will see being used tomorrow. 

About the role:  

The Data Engineering team enables and manages the ingestion of low latency, high volume car telemetry data that powers our engineering and data science teams to build smart and insightful data products. We are looking for an experienced Senior Data Engineer to join the team who will have a key role in the design, development, implementation and documentation of large-scale, distributed data applications, systems and services. You will engineer data pipelines that will consume vehicle telemetry data to build insightful data services and products. The features you build will power driving experiences across the world.  

What you will do:  

• Work closely with Data Engineering Lead and Product to shape and deliver features to customers; thrive as a creative thinker that can break out from conventional solutions 
• Help drive adoption of modern principles, techniques and technology to the team, raising software quality, value and delivery 

• Influence team members, encourage engineering practices in accordance with good practice architecture, software engineering and creative thinking to crush expectations 
• Design, implement, and maintain complex data engineering solutions to acquire and prepare data. Create and maintain data pipelines to connect data within and between data stores, applications and organisations. Carry out complex data quality checking and remediation 
• Design system components and complex software applications and modules using appropriate modelling techniques following agreed architecture and software design standards, guidelines, patterns and methodology. Create and communicate multiple design views to balance stakeholders' concerns and to satisfy functional and non-functional requirements. Identifies, evaluates and recommends alternative design options and trade-offs. Model, simulate or prototype the behaviour of proposed software to enable approval by stakeholders and effective construction of the software. Verify software design by constructing and applying appropriate methods 
• Design, code, verify, test, document, amend and refactors complex programs/scripts and integration software services. Contribute to the selection of the software development methods, tools and techniques. Apply agreed standards and tools to achieve well-engineered outcomes. Lead peer reviews of colleagues' work 
• Provide technical expertise to enable the configuration of system components and equipment for systems testing, define complex/new integration builds, ensuring that integration test environments are correctly configured. Recommend and implement improvements to processes and tools 

• Plan and drive development projects which support the organisation's objectives, select adopt and adapt appropriate systems development methods, tools and techniques. Ensures that stakeholders are aware of required resources and manage optimum availability and utilisation. Ensure projects are carried out in accordance with agreed architectures, standards, methods and tools and addresses security and privacy requirements 
• Work side-by-side with other talented engineers in a team-oriented, agile software engineering environment  
• With Engineering Data Lead, coach through continuously sharing knowledge and lookout for improvement opportunities in technology as well as ways of working  
• Love writing code and learning to constantly hone your craft as an engineer  
• Work closely with product owners to shape and deliver features to customers  

• Mentor less experienced team members and help grow an unstoppable team and build awesome, innovative products  

About you:  

• Seasoned technologist with persuasive communication skills who can articulate and deliver best fit solutions covering functional and non-functional requirements within time and capacity constraints  

• Strong expertise and experience of Apache Spark, PySpark, Python based pipeline jobs 
• Solid Data Lake/Data Warehouse principles, techniques and technologies - Star Schema, SQL (AWS EMR, Apache Iceberg, parquet)  
• Strong database skills and experience is required, we have NoSQL databases as well as relational databases in use often with large data volumes 
• Strong data modelling concepts and principles having extensive experience of building data architectures consolidating multiple complex sources   
• Experience developing and delivering systems on at least one major public cloud provider; preferably AWS  

• Excellent knowledge of and experience working with of APIs (designing with OpenAPI is desirable) and web services, CI/CD pipelines (Git-lab desirable) and automated testing (BDD, Performance, Security), Kubernetes and cloud native practices, containerized workloads with tools such as Docker 
• Experience of modern software and data engineering patterns, including those used in highly scalable, distributed, and resilient systems  
• Experience developing microservices-based architectures, including distributed messaging patterns is a plus  
• Experience of high volume IoT domain would be great but not essential  
• Aspiration to take a mentoring role within a team  

• Passion for agile practices, DevSecOps, incremental delivery, continuous improvement and ability to cultivate a strong team culture 
• We would like a self-starter, someone who would reach out to other teams if needed to seek answers and calling out in agile ceremonies blockers 
• Willingness to get involved in problem resolution and initiatives to smooth operational maintenance of production services which might be spread across geographic boundaries 
• We think the knowledge acquired earning an BS in Computer Science, Engineering, Mathematics, or a related field would be of excellent value in this position, but if you are smart and have the experience that backs up your abilities, for us, talent trumps degree every time 

Equal Opportunities, Inclusion & Diversity We’re committed to building a diverse and inclusive group of talent with a broad range of backgrounds, skills and capabilities and will give full and fair consideration to all applicants. We know that flexibility is key to success and our people work flexibly in many ways, so if this is important to you, please let us know. If you have a disability or any other additional need that requires consideration, accommodation or adjustment to the role or recruitment process, please do let us know.