Data Pipeline Engineer (Top Secret Clearance)

Job Description Job description:

Data Pipeline Engineer

Clearance Requirement:TS

Location:Washington D.C. or NYC

Onsite Requirement:5 Days/week onsite

The Role

Data Pipeline engineers ensure that our customers work with updated and accurate data in Foundry. You are the first to respond to data health failures on key pipelines. You are technical, with a passion for system support. You are comfortable reading code to identify fixes and making changes to improve the performance and stability of production data pipelines. Working alongside pipeline engineers from other customer accounts, your insights into the product and process will form an essential part of feedback to our product engineering team, and other user and product support teams.

Core Responsibilities

• Maintaining and building schedules so that pipelines run.
• Setting up and maintaining health checks on different pipelines.
• Responding to, triaging, and debugging pipelines when there is a problem (usually this is when health checks fail). This will include limited out of hours support for critical issues (e.g. a critical subset of alerts can page you overnight or during the weekend*).
• Reading code and writing code changes and/or modifying the monitoring set-up where necessary.
• Knowing and understanding how to navigate the pipelines and documentation.
• Following SOPs to contact other teams and data providers when data is incorrect or not received on time.
• Communicating outages with the end users of a pipeline.
• Contributing to and monitoring tooling improvements (where feasible).
• Active work not required on weekends or out of hours, however due to the nature of on-call, the person must be available to respond if there is a critical outage during their assigned on-call weeks. After an on-call weekend the engineer will receive 2 days off.

What We Require

• TS clearance or higher.

Qualifications:

What We Value

• Comfortable reading and writing code in SQL, Python, Pyspark and Java.
• Basic understanding of Spark and familiar/interested in learning the basics of tuning Spark jobs.
• Practical experience with performing root cause analysis and documenting lessons learned from production incidents (e.g. creating post-mortem reports).
• Ability to work within an agile team.
• Strong written and verbal communication skills with the ability to skillfully engage with customers on complex, sensitive topics.
• Strong organizational skills and attention to detail through effective prioritization.

Why is This a Great Opportunity:

Great salaries

Lots of upward mobility

401K, Sign On Bonuses, Tuition Reimbursements, Covered Healthcare

Salary Type : Annual Salary

Salary Min : 160000

Salary Max : 200000

Currency Type : US Dollars

]]>