Senior Data Engineer

CircleCI · Apr 10th 2018

Apply on GitHub Jobs

At the heart of the CircleCI product offerings is the flow of data. You will be the key person that creates robust data pipelines that are high volume, sometimes extremely sensitive to completeness and often require integration with sources that require programming skills. Comfortable with databases, AWS, Python, cmd line, product management, engineering, and marketing, this is a role that will require cross-functional skills and cross-department communication. Expect to spend time designing and architecting next-generation solutions to solve engaging challenges as CircleCI usership grows and products evolve at a terrific pace.

About CircleCI CircleCI is the best platform for software teams looking to rapidly build quality projects, at scale. Our intelligent continuous integration and delivery tools are simple yet powerful. Our aim is to provide the wisdom of a connected development ecosystem to every team member making technology decisions.

We run 7M+ builds a month on our platform for companies like Spotify, Kickstarter, Sony, and Coinbase. Over 25,000 organizations and 300,000 developers actively build, test, and deploy on CircleCI. We’ve raised $59.5M in venture capital from Industry Ventures, Top Tier Capital, Scale Venture Partners, DFJ, Harrison Metal Capital, and Baseline Ventures.

** About Data at CircleCI** The Data team is growing from analysis, projections and research to data as a product. Today, the Data team works closely with Product Management, Engineering, Marketing, and User Experience to create actionable insights from the millions of builds performed every week on CircleCI. The Data team is critical to CircleCI success. As a result, the team supports highly cross-functional initiatives that rely heavily on data and user-research to drive decision making and deliver features or improvements as a continuous flow. We are looking for a smart, passionate person to help create the backbone of our long term data strategy.

Joining the Team After introductory discussions with recruiting and a member of the technical staff, you will work through a brief data/programming assignment that we use during an on-site interview to get to know more about each other. This helps you and CircleCI ensure that our approach to problems and skills are a good fit.

Responsibilities:

  • Establish, build, and maintain a data management platform and ETL pipeline
  • Automate big data flow and processing
  • Maintain the data warehouse and data architecture
  • Connect to new APIs, new data sources and build new data pipelines
  • Comfortable building and deploying api / Microservice architectures
  • Design and deploy visualization/insights sharing applications for internal customers
  • Work within engineering teams to integrate new capability into analytics infrastructure

Essential Qualifications:

  • 5+ years experience in Data Engineering role
  • Prefers automation to being a hero
  • Ability to design and implement robust data pipelines and work in a team environment on a production codebase using version control system
  • Strong familiarity with data warehousing concepts - dimensional data warehousing (e.g. Kimball Group), star and snowflake schema, etc.
  • Comfortable with data infrastructure concepts (e.g. MPP databases, CAP theorem) and platforms/providers (e.g. AWS, DigitalOcean, GCP)
  • Experience building pipelines that run within cloud-based environments
  • Proficiency in Python (or similar programming language) for transforming data
  • Exposure to functional programing
  • Experience working on cross-functional, feature-driven engineering teams
  • Intermediate understanding of Linux/Docker
  • Demonstrated database schema and design experience

Preferred Qualifications:

  • Experience with the following technologies: PostgreSQL, MongoDB, Docker, Spark, Hive
  • Advanced knowledge and experience with AWS
  • Advanced knowledge of Linux/Docker in deployments
  • Motivated to learn and empower others, and when meeting obstacles, not afraid to test new concepts or ask for guidance

If you’re interested in joining the team at CircleCI, please send a resumé and cover letter describing what inspires you and why you think you are a good fit for the team. If you contribute to an open source project, write a blog, or have a presence on the web (Twitter, GitHub, LinkedIn, etc.) we would love to hear about it.

We care deeply about diversity and inclusivity. We’re hiring at all experience levels, and seek talented teammates from a wide variety of backgrounds and experiences who are equally committed to cultivating a work environment of respect and kindness. We carefully consider every applicant that takes the time to apply.

Apply on GitHub Jobs