Wunderkind · Feb 3rd 2021
The Data Engineering team is developing the backbone of Wunderkind solutions. To do this, the team is using cutting-edge technologies and frameworks such as BigQuery, Spanner, Bigtable, has & Dataproc to process and analyze data, to help create the pipelines required to handle millions of events daily, and collaborate with the data science team to enable the innovative work in machine learning and AI to be utilized in our production environment.
Responsibilities:
Contribute to the development of Wunderkind Micro Batch/Real Time streaming pipelines (lambda architecture)
Work closely with the data science team to build our machine learning platform infrastructure
Work closely with the engineering team on data preprocessing and data structure
Help mentoring the data engineering team and leading the data engineering initiatives
Participate in architecture reviews
Qualifications:
BA/BS or MS degree in Computer Science preferred, or equivalent work experience
Understanding of distributed data systems
Previous experience building lambda architecture
Previous experience working with big data tools such as Bigtable, Apache Spark, Apache Spark Streaming, Apache Heron, BigQuery
Preferred:
Google Cloud, Kubernetes
Experience running production Golang systems, Infrastructure background, Low Latency / High Throughput Architectural System Designs
Strong interest in data engineering
Eagerness to learn new technologies on the fly and ship to production
Knowledge in data science is a plus