Senior Data Engineer RoleMy client are looking for a Senior Data Engineer to join their commodities trading team in Central London. This project will focus...
Senior Data Software Engineer £600-700/day
My client is a world leading media marketing business and is continuing the progress of an indispensable platform, built by a world-class team. They practice continuous delivery and TDD. They also set time aside for continuous improvement and take the professional development of our team members seriously.
About the role:
As Senior Engineer you will sit within the Data Intelligence function, you'll be responsible for ensuring the development of our robust data platform. You'll lead the implementation of the platform from data collection through storage and on to processing. Your knowhow will inform how we make the right product decisions. You'll make sure that we strike the right balance between efficient and robust, evolving the architecture as the business grows.
- You will come from a software background and have a real passion for delivering innovative software platforms
- You'll understand the architecture trade-offs and have a good understanding of the data tier within a system
- You'll have experience in leading the delivery of complex platforms
Your responsibilities will be to:
- Work with Data Scientists and other software engineers to deliver our platform-based product
- Make the right compromises and leads others to do so
- Lead definition and design of products and projects
- Share knowledge and experience
- Maintain a balance between the short term and the long term
- Contribute to the design and architecture of the platform
- Design and develop batch processing systems, ETLs and data ingestion flows
- Build event driven processing systems
- Build API's to front-end data science models with appropriate levels of security, authentication, logging, and modelling
- Educate the business on how to use the data driven system to meet my client's objectives
- Helps to develop and mentor junior team members
- Contributes to the education of the wider business
What you'll bring to the role:
- Professional experience in developing applications & tools in at least two systems/scripting languages (one of which should be Python, Java or Scala)
- Python will be used for implementations
- Experience in developing platforms to run on AWS
- A good knowledge of UNIX and networking
- Experience working in a continuous delivery environment
- Have a degree in Computer Science, Engineering, Science or a related field, or the equivalent commercial experience
Knowledge of the following systems and technologies is preferable:
Data Modelling - R, Pandas, NumPy, Scikit-learn, Matplotlib
Platforms - Airflow, AWS EMR, Docker, AWS Lambda, AWS SageMaker, AWS DataPipeline, AWS Kinesis, Jupyter, Qubole
Databases/Processing - RedShift, Athena, Postgres, Spark, Hive, Pig
We are an equal opportunities employer and welcome applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation, gender reassignment, marriage and civil partnerships, pregnancy or maternity or age.