This website uses cookies. Read our cookie policy for more information. By continuing to browse this site you are agreeing to our use of cookies.

hidden og tag linkedin

Data Engineer, London, UK

Job Type:
Contract
Job Sector:
Data
Job Category:
Data Science and Big Data
Country:
UK
Area:
London
Location:
London
Salary:
300 to 325 per day
Job Ref:
DaE

DATA ENGINEER / KAFKA / SPARK / HADOOP / SQL / SCALA / DATA INGESTION

My client is looking for a Data Engineer to help provision valuable business data to the Data Platform. You will work closely with Data Platform Engineers, Data Scientists, and InfraDev Engineers across multiple platforms and technologies to create a highly scalable, secure data platform.

As a Data Engineer, you will:

- Build data ingestion / integration pipelines for large scale real-time (streaming) and batch applications
- Build capabilities to efficiently integrate new data sources into common platform
- Develop necessary transformation logic / event logic for data processing across pipeline
- Design and develop all necessary transformation logic, business logic, and preparation to support needs of business
- Care about your craft, be aware of new technologies and be keen to use them when appropriate (e.g. Continuous Integration, Continuous Deployment)
- Feed your insights to the product team to continuously improve the platform
- Be an active contributing part of an agile development team
- Be diligent and positive about tracking (JIRA), system monitoring, security, auditing and other ways of ensuring the quality of the platform and the processes around it

You should have experience with:

- Manipulating data with a variety of tools and programming languages, e.g. SQL, Java, Python
- Building streaming & batch applications with technologies including: JMS, Kafka, Kinesis, Spark Streaming
- Working with data formats including JSON, XML, Flatfiles, Sequence, Avro, Parquet, etc.
- Cloud computing and related challenges around security, bandwidth, etc.
- Understanding statistics and data science to work closely with business analyst

Technologies:

- Kafka, Spark, Hadoop, Hive
- Scala, Java, R, Python
- PostgreSQL, ElasticSearch
- Git, Jira

Contact Details:
Tel: 0203 301 9900
Fax: 0203 301 9901
Contact: James Jarvis

You may return to your current search results by clicking here.

Latest Job Listings