Data Engineer

Summary: Duration: 8 months Responsibilities: Design and maintain Palantir Foundry pipelines for OEIS compliance reporting Build and maintain data checker/validation frameworks within Foundry Develop PySpark and Spark SQL transformations for large-scale datasets Implement compliance-driven checks (completeness, consistency, reconciliation, exception flags) Ensure full data lineage, auditability, and reproducibility of compliance datasets Partner with compliance SMEs and stakeholders to validate logic and reporting outputs Support operational monitoring, issue triage, and data quality remediation Requirements: Strong Python skills Experience with PySpark/Spark SQL Advanced SQL knowledge Hands-on experience with Palantir Foundry Experience building data validation, QA, or data quality frameworks Understanding of compliance, audit, or regulatory reporting requirements Preferred Skills: Experience with OEIS or regulatory compliance reporting Foundry Workshop experience Experience working with ontology-driven or curated datasets Utility, infrastructure, or asset-based domain experience