This job is with Allianz Commercial, an inclusive employer and a member of myGwork - the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. We are looking for an experienced Principal Data Engineer to work hands-on on the re-engineering of an existing enterprise data platform built on Azure Synapse Analytics. The role requires strong technical depth to audit, understand, and validate a complex end-to-end data architecture spanning source ingestion through to consumption - and to help deliver the migration of validated workloads to Databricks. This is hands on role and demands the ability to reverse-engineer existing implementations, assess their correctness, and build to an agreed migration strategy. Key Responsibilities Contribute to the technical assessment and re-engineering of an existing enterprise data platform, spanning all layers from source ingestion through to data consumption Reverse-engineer, document, and validate existing pipeline logic, data models, transformation frameworks, and data governance controls Identify gaps, defects, and technical debt across the platform and remediate where implementations are incorrect or sub-optimal Ensure correctness of data processing patterns including change data capture, slowly changing dimensions, deduplication, and business reconciliation Implement target-state designs aligned to modern lakehouse principles, ensuring feature parity and business logic fidelity during transitions Support platform evolution initiatives, including parallel-run phases where multiple implementations operate simultaneously, validating output consistency before cutover Execute migration of existing workloads to modern data platforms, preserving existing governance and control framework semantics Re-implement ingestion, transformation, and orchestration pipelines on target platforms, maintaining audit, quality, and reconciliation standards Collaborate with business, data governance, and architecture stakeholders to validate embedded business rules and data quality requirements Mentor less experienced engineers, review code and designs, and support decommission planning for legacy components Core Technical Skills Azure Synapse & Data Platform Mandatory hands-on expertise with: Azure Synapse Analytics (Pipelines, Spark Pool, Dedicated SQL Pool) Azure Data Lake Storage Gen2 (ADLS Gen2) Delta Lake on Azure (Synapse Lakehouse patterns) Oracle Golden Gate Replication for real-time source integration Azure Analysis Services and Power BI consumption layer patterns Deep understanding of medallion architecture: Raw / Harmonized / Conformed / Consumption layers Strong knowledge of SCD Type 0/1/2, CDC patterns, soft/hard delete, and retroactive change processing Experience with Synapse SQL Pool - stored procedures, control tables, and data quality validation patterns Experience with audit, balance, and control frameworks - parameterized, modular pipeline governance at enterprise scale Familiarity with config-driven and automation-first pipeline patterns (YAML, PySpark, SQL-driven generation from mapping documents) Databricks & Lakehouse Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred) Strong Apache Spark skills (PySpark / Spark SQL) Experience migrating workloads from legacy data warehouse or Synapse environments to a Databricks Lakehouse Ability to re-implement governance and control frameworks natively in Databricks (audit logging, reconciliation, DQ checks) Experience with Delta Lake features: MERGE, CDC, time travel, schema enforcement Data Engineering & Development Strong Python and SQL programming skills Experience with ETL/ELT at scale: denormalization, surrogate keys, directory tables, curated data models Experience integrating complex data sources: Oracle DB, SQL Server, Azure SQL DB, file systems, Salesforce, APIs Strong data modelling skills: relational, dimensional, and lakehouse-oriented DevOps & Automation CI/CD pipelines for data engineering (Azure DevOps / GitHub Actions) Infrastructure as Code (Terraform or ARM) Containerization (Docker) Experience with automated testing frameworks for data pipelines (unit testing, reconciliation-based validation) Nice to Have Experience with Unity Catalog for data governance and lineage Familiarity with Azure Purview for data cataloguing and governance Exposure to real-time and streaming pipelines (Event Hub / Kafka / Kinesis) Experience with GenAI or ML platform integration (MLOps, feature engineering pipelines) Familiarity with monitoring and observability tools (e.g., Dynatrace) Exposure to BI tools (Power BI, Tableau) Experience & Profile 7+ years of hands-on experience in Data Engineering, including platform migration or re-engineering work Proven track record working on existing, complex enterprise data platforms - not just building from scratch Deep knowledge of enterprise data governance patterns: audit trails, reconciliation, data quality controls, SCD versioning Strong analytical mindset: ability to read existing implementations, identify intent versus defect, and make sound re-engineering decisions Primarily hands-on, while comfortable contributing to technical design discussions Strong communication skills - able to engage business, governance, and engineering stakeholders with clarity Experience working in regulated or enterprise-scale environments (financial services a plus) What we offer We offer a hybrid work model which recognizes the value of striking a balance between in-person collaboration and remote working We believe in rewarding performance and our compensation and benefits package includes health insurance, company bonus scheme, 401(K) with company match, company paid holidays, paid time off, paid volunteer days, tuition reimbursement, paid parental leave, an employee shares program and employee discounts From career development and digital learning programs to international career mobility, we offer lifelong learning for our employees worldwide and an environment where innovation, delivery and empowerment are fostered. The annualized base pay range for this role is $110,000 - 150,000. The annual base salary range represents a nationwide market range. The actual salary for this position will be determined by a number of factors, including the scope, complexity and location of the role, and the skills, education, training, credentials and experience of the candidate. The base pay is just one component of the ATA total compensation package. As part of our comprehensive compensation and highly rated benefits programs, ATA also offers eligibility for an incentive-based annual bonus. About Allianz Technology Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges is what makes us a unique employer. We are united by a shared commitment: to put our customers first and at the centre of everything we do. Their needs inspire our thinking and guide our actions. Together, we can build an environment where everyone feels empowered and confident to explore, grow and shape a better future - for our customers and for the world around us. At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. The foundation for this is our inclusive workplace, where people and performance both matter, and where integrity, fairness, inclusion and trust are at the heart of our culture. We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations. Join us. Let's care for tomorrow. You.IT
10/26/2026
Full time
This job is with Allianz Commercial, an inclusive employer and a member of myGwork - the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. We are looking for an experienced Principal Data Engineer to work hands-on on the re-engineering of an existing enterprise data platform built on Azure Synapse Analytics. The role requires strong technical depth to audit, understand, and validate a complex end-to-end data architecture spanning source ingestion through to consumption - and to help deliver the migration of validated workloads to Databricks. This is hands on role and demands the ability to reverse-engineer existing implementations, assess their correctness, and build to an agreed migration strategy. Key Responsibilities Contribute to the technical assessment and re-engineering of an existing enterprise data platform, spanning all layers from source ingestion through to data consumption Reverse-engineer, document, and validate existing pipeline logic, data models, transformation frameworks, and data governance controls Identify gaps, defects, and technical debt across the platform and remediate where implementations are incorrect or sub-optimal Ensure correctness of data processing patterns including change data capture, slowly changing dimensions, deduplication, and business reconciliation Implement target-state designs aligned to modern lakehouse principles, ensuring feature parity and business logic fidelity during transitions Support platform evolution initiatives, including parallel-run phases where multiple implementations operate simultaneously, validating output consistency before cutover Execute migration of existing workloads to modern data platforms, preserving existing governance and control framework semantics Re-implement ingestion, transformation, and orchestration pipelines on target platforms, maintaining audit, quality, and reconciliation standards Collaborate with business, data governance, and architecture stakeholders to validate embedded business rules and data quality requirements Mentor less experienced engineers, review code and designs, and support decommission planning for legacy components Core Technical Skills Azure Synapse & Data Platform Mandatory hands-on expertise with: Azure Synapse Analytics (Pipelines, Spark Pool, Dedicated SQL Pool) Azure Data Lake Storage Gen2 (ADLS Gen2) Delta Lake on Azure (Synapse Lakehouse patterns) Oracle Golden Gate Replication for real-time source integration Azure Analysis Services and Power BI consumption layer patterns Deep understanding of medallion architecture: Raw / Harmonized / Conformed / Consumption layers Strong knowledge of SCD Type 0/1/2, CDC patterns, soft/hard delete, and retroactive change processing Experience with Synapse SQL Pool - stored procedures, control tables, and data quality validation patterns Experience with audit, balance, and control frameworks - parameterized, modular pipeline governance at enterprise scale Familiarity with config-driven and automation-first pipeline patterns (YAML, PySpark, SQL-driven generation from mapping documents) Databricks & Lakehouse Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred) Strong Apache Spark skills (PySpark / Spark SQL) Experience migrating workloads from legacy data warehouse or Synapse environments to a Databricks Lakehouse Ability to re-implement governance and control frameworks natively in Databricks (audit logging, reconciliation, DQ checks) Experience with Delta Lake features: MERGE, CDC, time travel, schema enforcement Data Engineering & Development Strong Python and SQL programming skills Experience with ETL/ELT at scale: denormalization, surrogate keys, directory tables, curated data models Experience integrating complex data sources: Oracle DB, SQL Server, Azure SQL DB, file systems, Salesforce, APIs Strong data modelling skills: relational, dimensional, and lakehouse-oriented DevOps & Automation CI/CD pipelines for data engineering (Azure DevOps / GitHub Actions) Infrastructure as Code (Terraform or ARM) Containerization (Docker) Experience with automated testing frameworks for data pipelines (unit testing, reconciliation-based validation) Nice to Have Experience with Unity Catalog for data governance and lineage Familiarity with Azure Purview for data cataloguing and governance Exposure to real-time and streaming pipelines (Event Hub / Kafka / Kinesis) Experience with GenAI or ML platform integration (MLOps, feature engineering pipelines) Familiarity with monitoring and observability tools (e.g., Dynatrace) Exposure to BI tools (Power BI, Tableau) Experience & Profile 7+ years of hands-on experience in Data Engineering, including platform migration or re-engineering work Proven track record working on existing, complex enterprise data platforms - not just building from scratch Deep knowledge of enterprise data governance patterns: audit trails, reconciliation, data quality controls, SCD versioning Strong analytical mindset: ability to read existing implementations, identify intent versus defect, and make sound re-engineering decisions Primarily hands-on, while comfortable contributing to technical design discussions Strong communication skills - able to engage business, governance, and engineering stakeholders with clarity Experience working in regulated or enterprise-scale environments (financial services a plus) What we offer We offer a hybrid work model which recognizes the value of striking a balance between in-person collaboration and remote working We believe in rewarding performance and our compensation and benefits package includes health insurance, company bonus scheme, 401(K) with company match, company paid holidays, paid time off, paid volunteer days, tuition reimbursement, paid parental leave, an employee shares program and employee discounts From career development and digital learning programs to international career mobility, we offer lifelong learning for our employees worldwide and an environment where innovation, delivery and empowerment are fostered. The annualized base pay range for this role is $110,000 - 150,000. The annual base salary range represents a nationwide market range. The actual salary for this position will be determined by a number of factors, including the scope, complexity and location of the role, and the skills, education, training, credentials and experience of the candidate. The base pay is just one component of the ATA total compensation package. As part of our comprehensive compensation and highly rated benefits programs, ATA also offers eligibility for an incentive-based annual bonus. About Allianz Technology Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges is what makes us a unique employer. We are united by a shared commitment: to put our customers first and at the centre of everything we do. Their needs inspire our thinking and guide our actions. Together, we can build an environment where everyone feels empowered and confident to explore, grow and shape a better future - for our customers and for the world around us. At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. The foundation for this is our inclusive workplace, where people and performance both matter, and where integrity, fairness, inclusion and trust are at the heart of our culture. We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations. Join us. Let's care for tomorrow. You.IT
Job Description Job Description Company Description About AbbVie AbbVie's mission is to discover and deliver innovative medicines and solutions that solve serious health issues today and address the medical challenges of tomorrow. We strive to have a remarkable impact on people's lives across several key therapeutic areas including immunology, oncology and neuroscience - and products and services in our Allergan Aesthetics portfolio. For more information about AbbVie, please visit us at . on LinkedIn, Facebook, Instagram, X and YouTube. Job Description While the AI innovation race in Biopharma is focused on Drug discovery, Product Development/ CMC represents the next barrier/ bottleneck. The complexity of biological systems, the rigor of regulatory expectations, the pace of pipeline growth, and the enormous value at stake make this one of the highest-leverage domains for applied data science and AI in the entire pharmaceutical value chain. We here at BTS - PDST, are building a dedicated, AI-native team that is driving cutting edge programs across early stage, late stage and commercial product development to accelerate E2E product development and launch, maximize yields of block buster products. Through our deep collaboration with PDST scientists we are boldly reimagining how Abbvie can bring our pipeline products and lifesaving drugs to patients faster, safer and in cost effective manner fueled by AI. Principal Data Engineer is a highly technical, AI-native role responsible for designing, building, and operating production-grade data pipelines and data products that power AI/ML, analytics, and automation across AbbVie's CMC and manufacturing ecosystem. This role is embedded inside PDST and works at the frontier of pharmaceutical data engineering. You will integrate and harmonize data from the full spectrum of manufacturing and development systems - including MES, historians, LIMS, QMS, ERP, and instrument platforms - and transform it into reliable, governed, semantically rich data assets that data scientists, process engineers, and AI systems can actually use. Enterprise-scale scope: Enterprise-scale biologics portfolio spanning clinical, commercial, and lifecycle stages Building AI playbook for the future: First-in-AbbVie and first-in-biologics analytical approaches; you build the AI playbook for the future Growth and Impact: Direct impact on regulatory submissions, commercial readiness, and manufacturing decisions through deep cross-functional exposure to manufacturing, quality, regulatory, and scientific leadership Mission: Every model you build helps ensure safe, reliable medicines reach patients at scale Responsibilities Data Ingestion & Integration Design and implement scalable, robust data ingestion pipelines that connect CMC and manufacturing source systems - including MES (Manufacturing Execution Systems), process historians, LIMS, QMS, ERP platforms, and instrument data sources - to centralized and federated data environments. Build connectors, adapters, and integration layers that handle the heterogeneous data formats, protocols, and latency profiles characteristic of pharmaceutical manufacturing environments. Support both batch and real-time/streaming data patterns, selecting appropriate architectures based on use case requirements. Data Harmonization & Semantic Modeling Develop and maintain harmonized data models and ontologies that bring consistency to CMC and manufacturing data across sites, systems, and modalities. Execute semantic mapping efforts that align source system fields, units, and identifiers to enterprise data standards and scientific meaning. Collaborate with process scientists, analytical chemists, and manufacturing engineers to ensure data models accurately reflect domain reality. Data Quality, Observability & Governance Implement automated data quality controls, validation frameworks, and anomaly detection mechanisms across pipeline layers. Build and maintain data lineage documentation and metadata infrastructure, enabling full traceability from source system to AI model input. Establish pipeline observability practices - monitoring, alerting, SLA tracking - to ensure data product reliability in production. Support data governance practices aligned with GxP requirements, 21 CFR Part 11, and AbbVie data standards. AI/ML Enablement & Data Product Development Architect and deliver governed, versioned, reusable data products purpose-built for AI/ML consumption, including feature stores, curated datasets, and vector-ready data layers for RAG and LLM applications. Partner closely with data scientists, ML engineers, and process modelers to understand model data requirements and translate them into reliable, scalable data infrastructure. Accelerate AI program delivery by eliminating data bottlenecks - not by workarounds, but by solving root causes structurally. Platform & Operational Enablement Contribute to the design and evolution of PDST's cloud-based data platform, including lakehouse architecture, data cataloging, access control, and compute infrastructure. Write and maintain infrastructure-as-code, CI/CD pipelines, and automated testing frameworks for data systems. Support platform onboarding of new CMC data domains and manufacturing sites, ensuring consistent application of standards and patterns. Provide operational support for production data pipelines, maintaining uptime and data freshness commitments. Stakeholder Engagement & Scientific Leadership Influence technical decision-making without formal authority - earning trust through scientific rigor, transparent methodology, and demonstrated business impact. Qualifications Required: Bachelor's Degree Computer Science, Data Engineering, Information Systems, Software Engineering, Bioinformatics, or a closely related technical field plus 6 years' experience; Master's Degree plus 5 years' experience; PhD plus 0 years' experience. Respective years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Expert-level proficiency in Python for data engineering tasks - pipeline development, transformation logic, data validation, and automation. Strong SQL skills across modern analytical and transactional databases; comfort with both ANSI SQL and platform-specific dialects. Demonstrated experience with cloud data platforms (AWS, Azure, or GCP) and modern data stack components - including tools such as dbt, Spark, Airflow, Databricks, Snowflake, or equivalents. Develop ETL/ELT pipelines using tools such as Informatica, Talend, Apache NiFi, and cloud-native services (e.g., AWS Glue, Azure Data Factory). Implement master data management (MDM), metadata management, and data cataloging solutions to ensure proper data lineage, accessibility, and compliance. Set and enforce standards for API development and data integration (REST, GraphQL, OData), enabling seamless integration using microservices architectures. Ownership orientation: you define your own problem space, drive solutions to completion, and hold yourself accountable to outcomes - not just outputs. Solution-architect instinct: you think before you build, consider the full landscape of available approaches, and choose tools based on fit-for-purpose reasoning rather than familiarity or trend. Scientific integrity: you build models you can explain, defend, and improve - and you apply the same standard to the work of others. Influence through credibility: you earn the confidence of scientists, engineers, and quality professionals by being right, being clear, and being useful - not by title or volume. Bias for impact: you are drawn to problems where the stakes are high and the analytical opportunity is real, and you are energized rather than intimidated by ambiguity. Preferred: 3+ years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Familiarity with technology transfer workflows, process characterization study design, or commercial process validation (PPQ/PV) in a biologics or pharmaceutical context. Experience in pharmaceutical, biotech, or other regulated life sciences manufacturing environments. Familiarity with GxP data principles, 21 CFR Part 11 compliance, or data integrity requirements in regulated industries. Prior exposure to manufacturing source systems such as MES, process historians (e.g., OSIsoft PI/AVEVA), LIMS, QMS, or ERP platforms Experience building data infrastructure for AI/ML programs - including feature engineering pipelines, model training datasets, or vector/embedding data layers for RAG architectures. Knowledge of biologics manufacturing processes (e.g., upstream cell culture, downstream purification, fill-finish) or CMC development workflows. Familiarity with data mesh, data fabric, or federated data architecture patterns. Experience with graph databases, knowledge graphs, or ontology frameworks applied to scientific or manufacturing data. Contributions to open-source data tooling or demonstrated engagement with the modern data engineering community. Design logical, physical, and conceptual data models using modeling tools (e.g., Erwin, PowerDesigner, dbt). . click apply for full job details
10/19/2026
Full time
Job Description Job Description Company Description About AbbVie AbbVie's mission is to discover and deliver innovative medicines and solutions that solve serious health issues today and address the medical challenges of tomorrow. We strive to have a remarkable impact on people's lives across several key therapeutic areas including immunology, oncology and neuroscience - and products and services in our Allergan Aesthetics portfolio. For more information about AbbVie, please visit us at . on LinkedIn, Facebook, Instagram, X and YouTube. Job Description While the AI innovation race in Biopharma is focused on Drug discovery, Product Development/ CMC represents the next barrier/ bottleneck. The complexity of biological systems, the rigor of regulatory expectations, the pace of pipeline growth, and the enormous value at stake make this one of the highest-leverage domains for applied data science and AI in the entire pharmaceutical value chain. We here at BTS - PDST, are building a dedicated, AI-native team that is driving cutting edge programs across early stage, late stage and commercial product development to accelerate E2E product development and launch, maximize yields of block buster products. Through our deep collaboration with PDST scientists we are boldly reimagining how Abbvie can bring our pipeline products and lifesaving drugs to patients faster, safer and in cost effective manner fueled by AI. Principal Data Engineer is a highly technical, AI-native role responsible for designing, building, and operating production-grade data pipelines and data products that power AI/ML, analytics, and automation across AbbVie's CMC and manufacturing ecosystem. This role is embedded inside PDST and works at the frontier of pharmaceutical data engineering. You will integrate and harmonize data from the full spectrum of manufacturing and development systems - including MES, historians, LIMS, QMS, ERP, and instrument platforms - and transform it into reliable, governed, semantically rich data assets that data scientists, process engineers, and AI systems can actually use. Enterprise-scale scope: Enterprise-scale biologics portfolio spanning clinical, commercial, and lifecycle stages Building AI playbook for the future: First-in-AbbVie and first-in-biologics analytical approaches; you build the AI playbook for the future Growth and Impact: Direct impact on regulatory submissions, commercial readiness, and manufacturing decisions through deep cross-functional exposure to manufacturing, quality, regulatory, and scientific leadership Mission: Every model you build helps ensure safe, reliable medicines reach patients at scale Responsibilities Data Ingestion & Integration Design and implement scalable, robust data ingestion pipelines that connect CMC and manufacturing source systems - including MES (Manufacturing Execution Systems), process historians, LIMS, QMS, ERP platforms, and instrument data sources - to centralized and federated data environments. Build connectors, adapters, and integration layers that handle the heterogeneous data formats, protocols, and latency profiles characteristic of pharmaceutical manufacturing environments. Support both batch and real-time/streaming data patterns, selecting appropriate architectures based on use case requirements. Data Harmonization & Semantic Modeling Develop and maintain harmonized data models and ontologies that bring consistency to CMC and manufacturing data across sites, systems, and modalities. Execute semantic mapping efforts that align source system fields, units, and identifiers to enterprise data standards and scientific meaning. Collaborate with process scientists, analytical chemists, and manufacturing engineers to ensure data models accurately reflect domain reality. Data Quality, Observability & Governance Implement automated data quality controls, validation frameworks, and anomaly detection mechanisms across pipeline layers. Build and maintain data lineage documentation and metadata infrastructure, enabling full traceability from source system to AI model input. Establish pipeline observability practices - monitoring, alerting, SLA tracking - to ensure data product reliability in production. Support data governance practices aligned with GxP requirements, 21 CFR Part 11, and AbbVie data standards. AI/ML Enablement & Data Product Development Architect and deliver governed, versioned, reusable data products purpose-built for AI/ML consumption, including feature stores, curated datasets, and vector-ready data layers for RAG and LLM applications. Partner closely with data scientists, ML engineers, and process modelers to understand model data requirements and translate them into reliable, scalable data infrastructure. Accelerate AI program delivery by eliminating data bottlenecks - not by workarounds, but by solving root causes structurally. Platform & Operational Enablement Contribute to the design and evolution of PDST's cloud-based data platform, including lakehouse architecture, data cataloging, access control, and compute infrastructure. Write and maintain infrastructure-as-code, CI/CD pipelines, and automated testing frameworks for data systems. Support platform onboarding of new CMC data domains and manufacturing sites, ensuring consistent application of standards and patterns. Provide operational support for production data pipelines, maintaining uptime and data freshness commitments. Stakeholder Engagement & Scientific Leadership Influence technical decision-making without formal authority - earning trust through scientific rigor, transparent methodology, and demonstrated business impact. Qualifications Required: Bachelor's Degree Computer Science, Data Engineering, Information Systems, Software Engineering, Bioinformatics, or a closely related technical field plus 6 years' experience; Master's Degree plus 5 years' experience; PhD plus 0 years' experience. Respective years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Expert-level proficiency in Python for data engineering tasks - pipeline development, transformation logic, data validation, and automation. Strong SQL skills across modern analytical and transactional databases; comfort with both ANSI SQL and platform-specific dialects. Demonstrated experience with cloud data platforms (AWS, Azure, or GCP) and modern data stack components - including tools such as dbt, Spark, Airflow, Databricks, Snowflake, or equivalents. Develop ETL/ELT pipelines using tools such as Informatica, Talend, Apache NiFi, and cloud-native services (e.g., AWS Glue, Azure Data Factory). Implement master data management (MDM), metadata management, and data cataloging solutions to ensure proper data lineage, accessibility, and compliance. Set and enforce standards for API development and data integration (REST, GraphQL, OData), enabling seamless integration using microservices architectures. Ownership orientation: you define your own problem space, drive solutions to completion, and hold yourself accountable to outcomes - not just outputs. Solution-architect instinct: you think before you build, consider the full landscape of available approaches, and choose tools based on fit-for-purpose reasoning rather than familiarity or trend. Scientific integrity: you build models you can explain, defend, and improve - and you apply the same standard to the work of others. Influence through credibility: you earn the confidence of scientists, engineers, and quality professionals by being right, being clear, and being useful - not by title or volume. Bias for impact: you are drawn to problems where the stakes are high and the analytical opportunity is real, and you are energized rather than intimidated by ambiguity. Preferred: 3+ years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Familiarity with technology transfer workflows, process characterization study design, or commercial process validation (PPQ/PV) in a biologics or pharmaceutical context. Experience in pharmaceutical, biotech, or other regulated life sciences manufacturing environments. Familiarity with GxP data principles, 21 CFR Part 11 compliance, or data integrity requirements in regulated industries. Prior exposure to manufacturing source systems such as MES, process historians (e.g., OSIsoft PI/AVEVA), LIMS, QMS, or ERP platforms Experience building data infrastructure for AI/ML programs - including feature engineering pipelines, model training datasets, or vector/embedding data layers for RAG architectures. Knowledge of biologics manufacturing processes (e.g., upstream cell culture, downstream purification, fill-finish) or CMC development workflows. Familiarity with data mesh, data fabric, or federated data architecture patterns. Experience with graph databases, knowledge graphs, or ontology frameworks applied to scientific or manufacturing data. Contributions to open-source data tooling or demonstrated engagement with the modern data engineering community. Design logical, physical, and conceptual data models using modeling tools (e.g., Erwin, PowerDesigner, dbt). . click apply for full job details
Engineers and Constructors International Inc.
Houston, Texas
Job Description Job Description Join the ECI Group's Professional Technical Services Network as a contract Principal Data Engineer. This is a Houston TX based contract role with an anticipated 7-month duration with an estimated start date of October 5, 2026. Note: You must be a US Citizen or Permanent US Resident to qualify for this position. About the Role We are seeking a visionary Principal Data Engineer to lead the delivery of innovative data analytics solutions, with a focus on technical excellence, governance, and operational performance in a high-volume data environment. This role involves strategic and technical leadership in building Azure-based data lake solutions, driving maturity in data management, and engaging with cross-functional teams to leverage data for organizational success. Key Responsibilities: Architect, design, and implement scalable Azure-based data lake solutions. Establish roadmaps for capability-building across technologies such as Qlik Replicate, Databricks Delta Live Tables, and Unity Catalog. Provide technical leadership, ensuring adherence to best practices and quality standards. Lead the development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming. Optimize Databricks resources for high-volume data processing without exceeding budget. Oversee the use of Databricks Unity Catalog for data governance, lineage, and metadata management. Develop and enforce robust data governance policies, ensuring compliance, security, and data quality. Monitor data pipeline performance and proactively resolve production issues. Collaborate with cross-functional teams to align project objectives with business goals. Translate technical challenges into actionable insights for executive teams. Mentor and develop a high-performing team skilled in Azure, Databricks, and data streaming technologies. Facilitate knowledge transfer sessions to empower internal teams with cutting-edge methodologies. Education and Experience: Bachelor's or Master's degree in IT or a related discipline. 15-17 years of experience in the IT industry with expertise in delivering cost-effective cloud-based data analytics solutions. Proven track record in building data lake architectures and optimizing performance in big data environments. Certification in Azure Data Engineering or Databricks (preferred). Technical Skills: Strong knowledge of Azure technologies, including ADF, ADLS, Databricks, Stream Analytics, Delta Live Tables, and Synapse Analytics. Advanced proficiency in SQL and programming languages such as Python, Scala, or Spark-SQL. Experience in designing and developing scalable ETL pipelines and optimizing data workflows. Deep understanding of data migration from on-premises to cloud, cloud to cloud. Leadership and Soft Skills: Proven ability to lead diverse teams and manage end-to-end analytics projects. Exceptional communication and collaboration skills for engaging executives and stakeholders. Strong sense of ownership, accountability, adaptability, and commitment to results. Expertise in identifying risks and implementing effective mitigation strategies. Nice to Have: Familiarity with big data tools like Hadoop, Kafka, and Spark. Experience with visualization tools such as PowerBI or QlikSense. Knowledge of AI/ML, NLP, and GenAI concepts. Familiarity with AGILE and KANBAN methodologies. Portfolio: Shell IDT/Downstream & Renewables IDT/Trading & Supply/TS - Crude & Products/TS - Products/TS - Products Investments Program: Project: T&S Future Freight Platform - Jigsaw Description: Build a new Freight Platform (replacing RADAR) to serve the freight business and its customers. This includes a new industry standard Voyage Management System (VIP) alongside other components to close capability gaps. Optimize business processes and define master data maintenance processes. The overall Freight Platform will serve as a hub to facilitate seamless integration of necessary data, supporting an ecosystem of IT tools that are aligned with Shell's business growth roadmap. Freight Platform Project will begin a detailed design phase through Stage Gate 2 to accomplish the following: An approved (Solution Architecture) that provides a solution for Industry standard Voyage Management System (VIP) alongside other IT components to close capability gaps - identified during the Study Phase Optimize business processes and define master data maintenance processes Establish how well the platform can meet the needs for capturing Freight exposure management across the house, with the potential to use existing ETRMs for this if required, What IMOS reporting capability to utilise and avoid paying for functionality in IMOS we don't need and can do internally in a cost-efficient manner, Identify MI data use-cases that provide Shell a competitive edge and develop them in-house e.g. with Qliksense offering-up alternate solutions for exposure reporting, VaR, Carbon Emissions reporting. Identify a middleware integration solution to replace Mulesoft to tie Shell systems to VESON VIP Develop a data platform solution to ingest, host and distribute data between multiple integration points Business Application: RADAR Bulk Upload, RADAR Mail, RADAR-SAFE, RadarMsgServer, RADAR_app, VESON VIP T&S Pr Data Modelling, Data Engineering, Data Integration Please Note: The ECI Group uses AI-assisted tools in the early stages of application review, with all decisions guided by human oversight and fully compliant with General Data Protection Regulation (GDPR) and ECI Group's applicable data protection policies and Privacy Notice. Powered by JazzHR aBI8is58bZ
09/29/2026
Full time
Job Description Job Description Join the ECI Group's Professional Technical Services Network as a contract Principal Data Engineer. This is a Houston TX based contract role with an anticipated 7-month duration with an estimated start date of October 5, 2026. Note: You must be a US Citizen or Permanent US Resident to qualify for this position. About the Role We are seeking a visionary Principal Data Engineer to lead the delivery of innovative data analytics solutions, with a focus on technical excellence, governance, and operational performance in a high-volume data environment. This role involves strategic and technical leadership in building Azure-based data lake solutions, driving maturity in data management, and engaging with cross-functional teams to leverage data for organizational success. Key Responsibilities: Architect, design, and implement scalable Azure-based data lake solutions. Establish roadmaps for capability-building across technologies such as Qlik Replicate, Databricks Delta Live Tables, and Unity Catalog. Provide technical leadership, ensuring adherence to best practices and quality standards. Lead the development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming. Optimize Databricks resources for high-volume data processing without exceeding budget. Oversee the use of Databricks Unity Catalog for data governance, lineage, and metadata management. Develop and enforce robust data governance policies, ensuring compliance, security, and data quality. Monitor data pipeline performance and proactively resolve production issues. Collaborate with cross-functional teams to align project objectives with business goals. Translate technical challenges into actionable insights for executive teams. Mentor and develop a high-performing team skilled in Azure, Databricks, and data streaming technologies. Facilitate knowledge transfer sessions to empower internal teams with cutting-edge methodologies. Education and Experience: Bachelor's or Master's degree in IT or a related discipline. 15-17 years of experience in the IT industry with expertise in delivering cost-effective cloud-based data analytics solutions. Proven track record in building data lake architectures and optimizing performance in big data environments. Certification in Azure Data Engineering or Databricks (preferred). Technical Skills: Strong knowledge of Azure technologies, including ADF, ADLS, Databricks, Stream Analytics, Delta Live Tables, and Synapse Analytics. Advanced proficiency in SQL and programming languages such as Python, Scala, or Spark-SQL. Experience in designing and developing scalable ETL pipelines and optimizing data workflows. Deep understanding of data migration from on-premises to cloud, cloud to cloud. Leadership and Soft Skills: Proven ability to lead diverse teams and manage end-to-end analytics projects. Exceptional communication and collaboration skills for engaging executives and stakeholders. Strong sense of ownership, accountability, adaptability, and commitment to results. Expertise in identifying risks and implementing effective mitigation strategies. Nice to Have: Familiarity with big data tools like Hadoop, Kafka, and Spark. Experience with visualization tools such as PowerBI or QlikSense. Knowledge of AI/ML, NLP, and GenAI concepts. Familiarity with AGILE and KANBAN methodologies. Portfolio: Shell IDT/Downstream & Renewables IDT/Trading & Supply/TS - Crude & Products/TS - Products/TS - Products Investments Program: Project: T&S Future Freight Platform - Jigsaw Description: Build a new Freight Platform (replacing RADAR) to serve the freight business and its customers. This includes a new industry standard Voyage Management System (VIP) alongside other components to close capability gaps. Optimize business processes and define master data maintenance processes. The overall Freight Platform will serve as a hub to facilitate seamless integration of necessary data, supporting an ecosystem of IT tools that are aligned with Shell's business growth roadmap. Freight Platform Project will begin a detailed design phase through Stage Gate 2 to accomplish the following: An approved (Solution Architecture) that provides a solution for Industry standard Voyage Management System (VIP) alongside other IT components to close capability gaps - identified during the Study Phase Optimize business processes and define master data maintenance processes Establish how well the platform can meet the needs for capturing Freight exposure management across the house, with the potential to use existing ETRMs for this if required, What IMOS reporting capability to utilise and avoid paying for functionality in IMOS we don't need and can do internally in a cost-efficient manner, Identify MI data use-cases that provide Shell a competitive edge and develop them in-house e.g. with Qliksense offering-up alternate solutions for exposure reporting, VaR, Carbon Emissions reporting. Identify a middleware integration solution to replace Mulesoft to tie Shell systems to VESON VIP Develop a data platform solution to ingest, host and distribute data between multiple integration points Business Application: RADAR Bulk Upload, RADAR Mail, RADAR-SAFE, RadarMsgServer, RADAR_app, VESON VIP T&S Pr Data Modelling, Data Engineering, Data Integration Please Note: The ECI Group uses AI-assisted tools in the early stages of application review, with all decisions guided by human oversight and fully compliant with General Data Protection Regulation (GDPR) and ECI Group's applicable data protection policies and Privacy Notice. Powered by JazzHR aBI8is58bZ
Job Description Job Description About Medical Guardian: Medical Guardian is a fast-growing digital health and safety company on a mission to help people live a life without limits. With 13 consecutive years on the Inc. 5000 list of Fastest Growing Companies, we are redefining what it means to age confidently and independently. We support over 625,000 members nationwide with life-saving emergency response systems and remote patient monitoring solutions. Trusted by families, healthcare providers, and care managers, our work is powered by a culture of innovation, compassion, and purpose. Mission: This role is focused on building and leading the data engineering foundation that powers real-time decisioning, operational applications, analytics, ML/AI model development, and data services across Medical Guardian. The Principal Data Engineer will own the design, delivery, and maturity of production-grade data pipelines and data platforms, with a primary emphasis on real-time streaming, IoT telemetry, Databricks, Azure, data services for APIs and microservices, and reliable data products for downstream consumption. Role Summary: We are looking for a Principal Data Engineer to serve as a hands-on technical and people leader for data engineering, data platform architecture, real-time streaming, and production data services. This role will focus on designing, building, operating, and improving data pipelines and data products while also bringing principal-level judgment to architecture, stakeholder shaping, delivery priorities, team management, and production readiness. This is a hands-on engineering leadership role first. The ideal candidate should be comfortable spending significant time working directly with Databricks, Spark, SQL, Python/PySpark, Azure services, streaming architectures, data quality frameworks, pipeline automation, CI/CD, and production troubleshooting. They should also be able to operate with the maturity of a principal-level leader: shaping unclear requirements, making pragmatic technical decisions, managing and mentoring engineers, and driving work forward without waiting for perfect specifications. This is a fast-moving, startup-like environment. Requirements may be incomplete, priorities may evolve, and the right candidate will help create clarity while building quickly. We need someone who can move from ambiguous business need to reliable data capability with urgency, discipline, and ownership. Stakeholder shaping is a critical part of this role. The Principal Data Engineer should be able to work directly with business, product, software engineering, analytics, ML/AI, operations, and leadership stakeholders to define what data needs to exist, how it should be consumed, what production guarantees are required, and how success should be measured. A background in commercial software, SaaS, digital products, healthtech, fintech, IoT, data platforms, or other product-driven environments is strongly preferred. We want someone who understands that data pipelines and data services are not just technical artifacts. They are product capabilities that support real users, real workflows, operational decisions, ML/AI systems, APIs, analytics, and measurable business outcomes. Key Responsibilities: Hands-On Data Engineering and Platform Development Design, build, optimize, and operate production-grade batch and streaming data pipelines on Azure and Databricks, with a primary focus on real-time IoT and telemetry use cases within a Medallion architecture. Develop ETL/ELT workflows to ingest, transform, validate, and serve large volumes of structured, semi-structured, unstructured, and streaming data. Build and maintain reliable data products, data services, APIs, and microservices that support operational applications, analytics, software engineering, and ML/AI teams. Use Python, PySpark, Spark SQL, SQL, Delta Lake, Databricks Workflows, CI/CD, and related tools to build maintainable, testable, and observable data systems. Troubleshoot complex production pipeline issues across Databricks, Azure, streaming systems, APIs, and source systems, including root cause analysis, corrective action, and prevention planning. Move quickly from rough business need to prototype, pilot, and production-ready data capability while maintaining appropriate engineering discipline. Real-Time Streaming, IoT Telemetry, and Operational Data Services Lead the design and delivery of real-time streaming ingestion and processing patterns for connected medical device telemetry, event data, and operational data feeds. Implement streaming solutions using Azure Event Hubs, Azure Stream Analytics, Databricks, Delta Lake, and related Azure integration patterns. Design cost-effective throughput, partitioning, delivery, retention, and replay strategies for high-volume event and telemetry workloads. Create consumption patterns that support APIs, microservices, operational applications, near-real-time decisioning, analytics, and ML/AI use cases. Define reliability, latency, quality, observability, and supportability expectations for production streaming systems. Databricks, Lakehouse, and Data Platform Architecture Set direction for Databricks-based data engineering patterns, including Medallion architecture, Delta Lake, Spark optimization, data modeling, data quality, and reusable pipeline design. Optimize production Databricks pipelines using PySpark, Spark SQL, Delta Lake, partitioning strategies, caching, shuffle optimization, cluster/job configuration, and cost-aware design. Establish practical standards for pipeline structure, code organization, testing, deployment, monitoring, documentation, and ownership. Partner with data platform, security, infrastructure, and engineering teams to ensure the data platform is scalable, secure, reliable, and aligned with enterprise architecture. Make pragmatic architecture tradeoffs between speed, durability, cost, governance, performance, and downstream business impact. Stakeholder Shaping and Cross-Functional Partnership Work directly with business, product, analytics, ML/AI, operations, software engineering, and leadership stakeholders to clarify what data is needed, why it matters, how it will be used, and what success looks like. Translate ambiguous business needs into concrete data requirements, data product definitions, architecture options, delivery priorities, and implementation plans. Ask practical questions early: who will use the data, what decision or workflow does it support, what latency and quality are required, what happens if the data is wrong or late, and how will we know the capability is creating value? Help the organization avoid becoming a data ticket factory by shaping solutions, not just executing requests. Communicate architecture decisions, tradeoffs, risks, dependencies, and delivery options clearly to technical and non-technical stakeholders. Team Management and Principal-Level Technical Leadership Manage, mentor, and develop data engineers, providing clear expectations, technical guidance, prioritization support, feedback, and accountability. Provide technical leadership through hands-on example, strong engineering judgment, clear recommendations, and pragmatic decision-making. Lead design reviews, code reviews, production readiness reviews, incident reviews, and architecture discussions across data engineering initiatives. Establish and improve engineering standards for data quality, testing, CI/CD, observability, documentation, runbooks, cost management, privacy, and security. Proactively identify platform risks, data gaps, unclear ownership, operational weaknesses, and opportunities to improve reliability, scalability, and delivery speed. Influence without relying only on formal authority by building trust, framing tradeoffs, and helping cross-functional teams get to decisions. ML/AI, Analytics, and GenAI Enablement Partner with ML engineers, data scientists, analytics engineers, and analysts to deliver reliable data pipelines, feature pipelines, training datasets, scoring inputs, and feedback loops. Support the data foundation for predictive models, risk scores, operational decisioning, GenAI workflows, RAG, document intelligence, summarization, and AI-enabled automation. Help define data contracts, model-ready datasets, feature definitions, lineage, and monitoring expectations for ML/AI and analytics use cases. Ensure downstream consumers understand the meaning, freshness, quality, limitations, and appropriate use of the data products they depend on. Governance, Data Quality, Security, and Production Operations Apply privacy-first, security-aware, and governance-aligned practices for regulated, sensitive, and operationally critical data. Design and implement data quality checks, validation rules, anomaly detection, schema expectations, alerting, and operational monitoring. Ensure production pipelines and services are supportable, observable, documented, recoverable, and aligned with business continuity needs. Drive incident response and continuous improvement for data platform and pipeline issues, including root cause analysis and preventative remediation. Balance innovation with reliability, compliance, privacy, cost discipline, and operational usefulness. Required Qualifications: 10+ years of professional experience in data engineering . click apply for full job details
09/29/2026
Full time
Job Description Job Description About Medical Guardian: Medical Guardian is a fast-growing digital health and safety company on a mission to help people live a life without limits. With 13 consecutive years on the Inc. 5000 list of Fastest Growing Companies, we are redefining what it means to age confidently and independently. We support over 625,000 members nationwide with life-saving emergency response systems and remote patient monitoring solutions. Trusted by families, healthcare providers, and care managers, our work is powered by a culture of innovation, compassion, and purpose. Mission: This role is focused on building and leading the data engineering foundation that powers real-time decisioning, operational applications, analytics, ML/AI model development, and data services across Medical Guardian. The Principal Data Engineer will own the design, delivery, and maturity of production-grade data pipelines and data platforms, with a primary emphasis on real-time streaming, IoT telemetry, Databricks, Azure, data services for APIs and microservices, and reliable data products for downstream consumption. Role Summary: We are looking for a Principal Data Engineer to serve as a hands-on technical and people leader for data engineering, data platform architecture, real-time streaming, and production data services. This role will focus on designing, building, operating, and improving data pipelines and data products while also bringing principal-level judgment to architecture, stakeholder shaping, delivery priorities, team management, and production readiness. This is a hands-on engineering leadership role first. The ideal candidate should be comfortable spending significant time working directly with Databricks, Spark, SQL, Python/PySpark, Azure services, streaming architectures, data quality frameworks, pipeline automation, CI/CD, and production troubleshooting. They should also be able to operate with the maturity of a principal-level leader: shaping unclear requirements, making pragmatic technical decisions, managing and mentoring engineers, and driving work forward without waiting for perfect specifications. This is a fast-moving, startup-like environment. Requirements may be incomplete, priorities may evolve, and the right candidate will help create clarity while building quickly. We need someone who can move from ambiguous business need to reliable data capability with urgency, discipline, and ownership. Stakeholder shaping is a critical part of this role. The Principal Data Engineer should be able to work directly with business, product, software engineering, analytics, ML/AI, operations, and leadership stakeholders to define what data needs to exist, how it should be consumed, what production guarantees are required, and how success should be measured. A background in commercial software, SaaS, digital products, healthtech, fintech, IoT, data platforms, or other product-driven environments is strongly preferred. We want someone who understands that data pipelines and data services are not just technical artifacts. They are product capabilities that support real users, real workflows, operational decisions, ML/AI systems, APIs, analytics, and measurable business outcomes. Key Responsibilities: Hands-On Data Engineering and Platform Development Design, build, optimize, and operate production-grade batch and streaming data pipelines on Azure and Databricks, with a primary focus on real-time IoT and telemetry use cases within a Medallion architecture. Develop ETL/ELT workflows to ingest, transform, validate, and serve large volumes of structured, semi-structured, unstructured, and streaming data. Build and maintain reliable data products, data services, APIs, and microservices that support operational applications, analytics, software engineering, and ML/AI teams. Use Python, PySpark, Spark SQL, SQL, Delta Lake, Databricks Workflows, CI/CD, and related tools to build maintainable, testable, and observable data systems. Troubleshoot complex production pipeline issues across Databricks, Azure, streaming systems, APIs, and source systems, including root cause analysis, corrective action, and prevention planning. Move quickly from rough business need to prototype, pilot, and production-ready data capability while maintaining appropriate engineering discipline. Real-Time Streaming, IoT Telemetry, and Operational Data Services Lead the design and delivery of real-time streaming ingestion and processing patterns for connected medical device telemetry, event data, and operational data feeds. Implement streaming solutions using Azure Event Hubs, Azure Stream Analytics, Databricks, Delta Lake, and related Azure integration patterns. Design cost-effective throughput, partitioning, delivery, retention, and replay strategies for high-volume event and telemetry workloads. Create consumption patterns that support APIs, microservices, operational applications, near-real-time decisioning, analytics, and ML/AI use cases. Define reliability, latency, quality, observability, and supportability expectations for production streaming systems. Databricks, Lakehouse, and Data Platform Architecture Set direction for Databricks-based data engineering patterns, including Medallion architecture, Delta Lake, Spark optimization, data modeling, data quality, and reusable pipeline design. Optimize production Databricks pipelines using PySpark, Spark SQL, Delta Lake, partitioning strategies, caching, shuffle optimization, cluster/job configuration, and cost-aware design. Establish practical standards for pipeline structure, code organization, testing, deployment, monitoring, documentation, and ownership. Partner with data platform, security, infrastructure, and engineering teams to ensure the data platform is scalable, secure, reliable, and aligned with enterprise architecture. Make pragmatic architecture tradeoffs between speed, durability, cost, governance, performance, and downstream business impact. Stakeholder Shaping and Cross-Functional Partnership Work directly with business, product, analytics, ML/AI, operations, software engineering, and leadership stakeholders to clarify what data is needed, why it matters, how it will be used, and what success looks like. Translate ambiguous business needs into concrete data requirements, data product definitions, architecture options, delivery priorities, and implementation plans. Ask practical questions early: who will use the data, what decision or workflow does it support, what latency and quality are required, what happens if the data is wrong or late, and how will we know the capability is creating value? Help the organization avoid becoming a data ticket factory by shaping solutions, not just executing requests. Communicate architecture decisions, tradeoffs, risks, dependencies, and delivery options clearly to technical and non-technical stakeholders. Team Management and Principal-Level Technical Leadership Manage, mentor, and develop data engineers, providing clear expectations, technical guidance, prioritization support, feedback, and accountability. Provide technical leadership through hands-on example, strong engineering judgment, clear recommendations, and pragmatic decision-making. Lead design reviews, code reviews, production readiness reviews, incident reviews, and architecture discussions across data engineering initiatives. Establish and improve engineering standards for data quality, testing, CI/CD, observability, documentation, runbooks, cost management, privacy, and security. Proactively identify platform risks, data gaps, unclear ownership, operational weaknesses, and opportunities to improve reliability, scalability, and delivery speed. Influence without relying only on formal authority by building trust, framing tradeoffs, and helping cross-functional teams get to decisions. ML/AI, Analytics, and GenAI Enablement Partner with ML engineers, data scientists, analytics engineers, and analysts to deliver reliable data pipelines, feature pipelines, training datasets, scoring inputs, and feedback loops. Support the data foundation for predictive models, risk scores, operational decisioning, GenAI workflows, RAG, document intelligence, summarization, and AI-enabled automation. Help define data contracts, model-ready datasets, feature definitions, lineage, and monitoring expectations for ML/AI and analytics use cases. Ensure downstream consumers understand the meaning, freshness, quality, limitations, and appropriate use of the data products they depend on. Governance, Data Quality, Security, and Production Operations Apply privacy-first, security-aware, and governance-aligned practices for regulated, sensitive, and operationally critical data. Design and implement data quality checks, validation rules, anomaly detection, schema expectations, alerting, and operational monitoring. Ensure production pipelines and services are supportable, observable, documented, recoverable, and aligned with business continuity needs. Drive incident response and continuous improvement for data platform and pipeline issues, including root cause analysis and preventative remediation. Balance innovation with reliability, compliance, privacy, cost discipline, and operational usefulness. Required Qualifications: 10+ years of professional experience in data engineering . click apply for full job details
Job Description Job Description Our mission is to create the Experience of a Lifetime for our employees, so they can, in turn, create the Experience of a Lifetime for our guests. We own and operate the most renowned destination resorts in the world as well as regional and local ski areas outside major cities, and connect them all through one unrivaled network. We are looking for ambitious leaders, innovators and creators to join our talented team. If you're ready to pursue your fullest potential, we want to get to know you! Candidates for year-round positions are reviewed on a rolling basis. Applications will be accepted up to 90 days after the posting date, or until the position is filled (whichever is first). Job Summary: We are looking for a curious, driven, innovative machine learning engineer who takes initiative to solve problems and create environments that accelerate the development, deployment, and usage of data science models and AI to drive greater organizational impact. The Data Science & Data Engineering team within the Enterprise Analytics organization builds data assets, predictive models, analytical applications, and platforms across the organization. Our team collaborates with business stakeholders, analysts, and technology teams to tackle high-impact use cases with state-of-the-art models and tools to grow the business, streamline costs, and improve guest experiences. Job Specifications: Starting Wage: $140,000 - $185,000 + Annual Bonus Employment Type: Year Round Shift Type: Full Time hours Minimum Age: At least 18 years of age Housing Availability: No Job Responsibilities: Productionize ML models developed by data science into reliable, monitored, maintainable systems. Build model data foundations that ensure training, inference, monitoring, and analytics data are trustworthy and scalable. Architect ML platform patterns in Databricks that bring reliability, consistency, governance, performance, and cost discipline to ML and data workflows. Identify and scope opportunities for ML engineering across the business for high-impact. Develop reusable tools , libraries, standards, documentation, and production-readiness practices to enable data science and data engineering teams. Develop analytical and model-powered applications that turn data and ML outputs into usable business workflows for end users. Prepare the platform for future AI engineering , including LLM and agent-based systems, as the organization matures. Provide technical leadership and mentoring across engineering, architecture, and development including design and code reviews. Job Requirements: Technical Skills: Quantitative Foundation : B.S. degree in a quantitative field (e.g., Computer Science, Mathematics, Statistics, Economics, Operations Research, Engineering). Software Engineering Fundamentals: write clean, modular, testable, maintainable code and understand how to structure production-grade systems rather than one-off notebooks or scripts. Python and SQL Proficiency: strong in Python and SQL for building data pipelines, automation, model integrations, analytical workflows, and production services. Data Modeling and Pipeline Design: understand how to design reliable, well-structured data assets, including curated tables, feature datasets, batch pipelines, orchestration, data quality checks, and lineage. ML Lifecycle Fluency: understand the full model lifecycle: data collection, exploration, model development, validation, deployment, monitoring, retraining, and retirement. Production ML Patterns: understand core MLOps patterns such as model registries, feature/data versioning, reproducible environments, testing/validation, monitoring, and rollback. Cloud and Platform Engineering: You are comfortable working in cloud-based data and ML environments and understand the foundations of permissions, environments, jobs, services, storage, networking, and cost-aware architecture. Databricks Expertise : You're familiar and experienced with the core parts of Spark, Unity Catalog, Delta Lake, Databricks Workflows, MLflow, model registry patterns, job/cluster optimization, and governance. DevOps Practices: You use modern engineering practices such as Git, CI/CD, automated testing, code review, dependency management, environment management, and observability. Application Development : You can build applications, APIs, dashboards, or workflow tools that sit on top of data and model outputs. System Design: You can reason through tradeoffs across reliability, latency, scale, cost, governance, maintainability, and ease of use. Soft Skills: Curious : bring intellectual curiosity, an inquisitive nature, and a desire to deepen your knowledge and continue learning. Ownership : take responsibility to proactively advance projects, contribute to the organization, and develop the best solutions. Communication: explain technical concepts, risks, tradeoffs, and recommendations clearly to technical and non-technical audiences. Collaboration: work effectively cross-functionally with data scientists, data engineers, analysts, application engineers, product partners, and business stakeholders. Pragmatism : You know how to balance ideal architecture with business urgency, team maturity, operational constraints, and the need to ship. Preferred qualifications: A graduate degree (Masters or PhD) in a quantitative field Experience with dbt (Core) for modular data modeling, including testing, documentation, and dependency management Experience with AI engineer to use, build, and monitor agentic solutions The expected Total Compensation for this role is $140,000 - $185,000 + Annual Bonus. Individual compensation decisions are based on a variety of factors. Job Benefits Ski/Mountain Perks! Free passes for employees, employee discounted lift tickets for friends and family AND free ski lessons MORE employee discounts on lodging, food, gear, and mountain shuttles 401(k) Retirement Plan Employee Assistance Program Excellent training and professional development Full Time roles are eligible for the above, plus: Health Insurance; Medical Insurance, Dental Insurance, and Vision Insurance plans (for eligible seasonal employees after working 500 hours) Free ski passes for dependents Critical Illness and Accident plans Employees can work remotely from British Columbia, Washington D.C., and the 16 U.S. states in which we currently operate. This includes: California, Colorado, Indiana, Michigan, Minnesota, Missouri, New Hampshire, New York, Nevada, Ohio, Pennsylvania, Utah, Vermont, Washington State, Wisconsin, and Wyoming. Please note that the ability to work in person or off-site, and the particulars related to such work, are subject to change at any time; and, accordingly, the Company reserves the right to change its policies and/or require in-person/in-office work or off-site work at any time in its sole discretion. In completing this application, and when submitting related documentation, applicants may redact information that identifies their age, date of birth, and/or dates of attendance at or graduation from an educational institution. We follow all federal, state, and local laws including restrictions on child/minor labor. Minors hired into this position will not be asked or permitted to engage in any activities restricted to adult workers. Vail Resorts is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, protected veteran status or any other status protected by applicable law. Requisition ID 517322 Reference Date: 09/05/2026 Job Code Function: Data Science
09/28/2026
Full time
Job Description Job Description Our mission is to create the Experience of a Lifetime for our employees, so they can, in turn, create the Experience of a Lifetime for our guests. We own and operate the most renowned destination resorts in the world as well as regional and local ski areas outside major cities, and connect them all through one unrivaled network. We are looking for ambitious leaders, innovators and creators to join our talented team. If you're ready to pursue your fullest potential, we want to get to know you! Candidates for year-round positions are reviewed on a rolling basis. Applications will be accepted up to 90 days after the posting date, or until the position is filled (whichever is first). Job Summary: We are looking for a curious, driven, innovative machine learning engineer who takes initiative to solve problems and create environments that accelerate the development, deployment, and usage of data science models and AI to drive greater organizational impact. The Data Science & Data Engineering team within the Enterprise Analytics organization builds data assets, predictive models, analytical applications, and platforms across the organization. Our team collaborates with business stakeholders, analysts, and technology teams to tackle high-impact use cases with state-of-the-art models and tools to grow the business, streamline costs, and improve guest experiences. Job Specifications: Starting Wage: $140,000 - $185,000 + Annual Bonus Employment Type: Year Round Shift Type: Full Time hours Minimum Age: At least 18 years of age Housing Availability: No Job Responsibilities: Productionize ML models developed by data science into reliable, monitored, maintainable systems. Build model data foundations that ensure training, inference, monitoring, and analytics data are trustworthy and scalable. Architect ML platform patterns in Databricks that bring reliability, consistency, governance, performance, and cost discipline to ML and data workflows. Identify and scope opportunities for ML engineering across the business for high-impact. Develop reusable tools , libraries, standards, documentation, and production-readiness practices to enable data science and data engineering teams. Develop analytical and model-powered applications that turn data and ML outputs into usable business workflows for end users. Prepare the platform for future AI engineering , including LLM and agent-based systems, as the organization matures. Provide technical leadership and mentoring across engineering, architecture, and development including design and code reviews. Job Requirements: Technical Skills: Quantitative Foundation : B.S. degree in a quantitative field (e.g., Computer Science, Mathematics, Statistics, Economics, Operations Research, Engineering). Software Engineering Fundamentals: write clean, modular, testable, maintainable code and understand how to structure production-grade systems rather than one-off notebooks or scripts. Python and SQL Proficiency: strong in Python and SQL for building data pipelines, automation, model integrations, analytical workflows, and production services. Data Modeling and Pipeline Design: understand how to design reliable, well-structured data assets, including curated tables, feature datasets, batch pipelines, orchestration, data quality checks, and lineage. ML Lifecycle Fluency: understand the full model lifecycle: data collection, exploration, model development, validation, deployment, monitoring, retraining, and retirement. Production ML Patterns: understand core MLOps patterns such as model registries, feature/data versioning, reproducible environments, testing/validation, monitoring, and rollback. Cloud and Platform Engineering: You are comfortable working in cloud-based data and ML environments and understand the foundations of permissions, environments, jobs, services, storage, networking, and cost-aware architecture. Databricks Expertise : You're familiar and experienced with the core parts of Spark, Unity Catalog, Delta Lake, Databricks Workflows, MLflow, model registry patterns, job/cluster optimization, and governance. DevOps Practices: You use modern engineering practices such as Git, CI/CD, automated testing, code review, dependency management, environment management, and observability. Application Development : You can build applications, APIs, dashboards, or workflow tools that sit on top of data and model outputs. System Design: You can reason through tradeoffs across reliability, latency, scale, cost, governance, maintainability, and ease of use. Soft Skills: Curious : bring intellectual curiosity, an inquisitive nature, and a desire to deepen your knowledge and continue learning. Ownership : take responsibility to proactively advance projects, contribute to the organization, and develop the best solutions. Communication: explain technical concepts, risks, tradeoffs, and recommendations clearly to technical and non-technical audiences. Collaboration: work effectively cross-functionally with data scientists, data engineers, analysts, application engineers, product partners, and business stakeholders. Pragmatism : You know how to balance ideal architecture with business urgency, team maturity, operational constraints, and the need to ship. Preferred qualifications: A graduate degree (Masters or PhD) in a quantitative field Experience with dbt (Core) for modular data modeling, including testing, documentation, and dependency management Experience with AI engineer to use, build, and monitor agentic solutions The expected Total Compensation for this role is $140,000 - $185,000 + Annual Bonus. Individual compensation decisions are based on a variety of factors. Job Benefits Ski/Mountain Perks! Free passes for employees, employee discounted lift tickets for friends and family AND free ski lessons MORE employee discounts on lodging, food, gear, and mountain shuttles 401(k) Retirement Plan Employee Assistance Program Excellent training and professional development Full Time roles are eligible for the above, plus: Health Insurance; Medical Insurance, Dental Insurance, and Vision Insurance plans (for eligible seasonal employees after working 500 hours) Free ski passes for dependents Critical Illness and Accident plans Employees can work remotely from British Columbia, Washington D.C., and the 16 U.S. states in which we currently operate. This includes: California, Colorado, Indiana, Michigan, Minnesota, Missouri, New Hampshire, New York, Nevada, Ohio, Pennsylvania, Utah, Vermont, Washington State, Wisconsin, and Wyoming. Please note that the ability to work in person or off-site, and the particulars related to such work, are subject to change at any time; and, accordingly, the Company reserves the right to change its policies and/or require in-person/in-office work or off-site work at any time in its sole discretion. In completing this application, and when submitting related documentation, applicants may redact information that identifies their age, date of birth, and/or dates of attendance at or graduation from an educational institution. We follow all federal, state, and local laws including restrictions on child/minor labor. Minors hired into this position will not be asked or permitted to engage in any activities restricted to adult workers. Vail Resorts is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, protected veteran status or any other status protected by applicable law. Requisition ID 517322 Reference Date: 09/05/2026 Job Code Function: Data Science
Job Description Job Description Essential Responsibilities Define enterprise-wide platform strategy, vision, and target-state architectures for platforms such as Microsoft Fabric, Foundry/OpenAI, Databricks, and Synapse. Own the end-to-end lifecycle of cloud platforms. Establish standards and guardrails across all platform engineering domains. Drive innovation in cloud, data, and AI platforms (Fabric, OpenAI/Foundry, etc.). Lead large-scale transformation initiatives (e.g., platform modernization, DC exit, AI adoption). Act as the highest-level escalation point for technical decisions and complex issues. Influence enterprise architecture boards, governance, and regulatory alignment. Drive cost optimization strategies and large-scale efficiency improvements. Foster platform product mindset across the organization (self-service, reuse, enablement). Mentor senior and lead engineers; shape engineering culture and technical direction. Partner with executives and business leaders to translate strategy into platform capabilities. Education & Years of Experience Minimum - 4 Year Bachelors Degree in Computer Science, Engineering or related Minimum - 10 Years of Platform engineering, cloud infrastructure, SRE roles or related In Lieu of Education 12 Years of Supporting enterprise-level environments in platform engineering, cloud infrastructure or SRE roles or related License/Certifications/Training Preferred - Azure DevOps Engineer Expert (or equivalent DevOps focused certification) Preferred - Fabric Analytics Engineer Associate (or equivalent data platform focused certification) Preferred - Azure AI Engineer Associate (or equivalent AI focused certification) Preferred - HashiCorp Certified: Terraform Professional (or equivalent IaC focused certification) Preferred - Azure Solutions Architect Expert (or equivalent Cloud focused certification) Compensation & Benefits: Typical hiring range: $119,400.00 to $204,600.00 Annually. Actual compensation will be determined using factors such as experience, skills & knowledge. Benefits: Alliant provides a benefits package including health care, vision, dental, and 401k with employer match including: Annual performance bonus Work from home up to 3 days a week Paid parental leave Employee discount programs Time off including paid personal and sick days 11 paid holidays Education reimbursement Note that eligibility and cost of benefits can vary depending on the number of regularly scheduled hours, and job status such as regular full-time, regular part-time, or temporary employment. Adhere to and ensure compliance of all business transactions with policy and process of the Bank Secrecy Act. Ensures compliance with all applicable state and federal laws, company procedures and policies. Maintains integrity and ethics in all actions and conversations with or regarding credit union members and their accounts; complies with Privacy Act directives. The responsibilities listed do not contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this position. Duties, responsibilities and activities may change at any time with or without notice.
09/28/2026
Full time
Job Description Job Description Essential Responsibilities Define enterprise-wide platform strategy, vision, and target-state architectures for platforms such as Microsoft Fabric, Foundry/OpenAI, Databricks, and Synapse. Own the end-to-end lifecycle of cloud platforms. Establish standards and guardrails across all platform engineering domains. Drive innovation in cloud, data, and AI platforms (Fabric, OpenAI/Foundry, etc.). Lead large-scale transformation initiatives (e.g., platform modernization, DC exit, AI adoption). Act as the highest-level escalation point for technical decisions and complex issues. Influence enterprise architecture boards, governance, and regulatory alignment. Drive cost optimization strategies and large-scale efficiency improvements. Foster platform product mindset across the organization (self-service, reuse, enablement). Mentor senior and lead engineers; shape engineering culture and technical direction. Partner with executives and business leaders to translate strategy into platform capabilities. Education & Years of Experience Minimum - 4 Year Bachelors Degree in Computer Science, Engineering or related Minimum - 10 Years of Platform engineering, cloud infrastructure, SRE roles or related In Lieu of Education 12 Years of Supporting enterprise-level environments in platform engineering, cloud infrastructure or SRE roles or related License/Certifications/Training Preferred - Azure DevOps Engineer Expert (or equivalent DevOps focused certification) Preferred - Fabric Analytics Engineer Associate (or equivalent data platform focused certification) Preferred - Azure AI Engineer Associate (or equivalent AI focused certification) Preferred - HashiCorp Certified: Terraform Professional (or equivalent IaC focused certification) Preferred - Azure Solutions Architect Expert (or equivalent Cloud focused certification) Compensation & Benefits: Typical hiring range: $119,400.00 to $204,600.00 Annually. Actual compensation will be determined using factors such as experience, skills & knowledge. Benefits: Alliant provides a benefits package including health care, vision, dental, and 401k with employer match including: Annual performance bonus Work from home up to 3 days a week Paid parental leave Employee discount programs Time off including paid personal and sick days 11 paid holidays Education reimbursement Note that eligibility and cost of benefits can vary depending on the number of regularly scheduled hours, and job status such as regular full-time, regular part-time, or temporary employment. Adhere to and ensure compliance of all business transactions with policy and process of the Bank Secrecy Act. Ensures compliance with all applicable state and federal laws, company procedures and policies. Maintains integrity and ethics in all actions and conversations with or regarding credit union members and their accounts; complies with Privacy Act directives. The responsibilities listed do not contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this position. Duties, responsibilities and activities may change at any time with or without notice.
Job Description Job Description Company Description About AbbVie AbbVie's mission is to discover and deliver innovative medicines and solutions that solve serious health issues today and address the medical challenges of tomorrow. We strive to have a remarkable impact on people's lives across several key therapeutic areas including immunology, oncology and neuroscience - and products and services in our Allergan Aesthetics portfolio. For more information about AbbVie, please visit us at . on LinkedIn, Facebook, Instagram, X and YouTube. Job Description While the AI innovation race in Biopharma is focused on Drug discovery, Product Development/ CMC represents the next barrier/ bottleneck. The complexity of biological systems, the rigor of regulatory expectations, the pace of pipeline growth, and the enormous value at stake make this one of the highest-leverage domains for applied data science and AI in the entire pharmaceutical value chain. We here at BTS - PDST, are building a dedicated, AI-native team that is driving cutting edge programs across early stage, late stage and commercial product development to accelerate E2E product development and launch, maximize yields of block buster products. Through our deep collaboration with PDST scientists we are boldly reimagining how Abbvie can bring our pipeline products and lifesaving drugs to patients faster, safer and in cost effective manner fueled by AI. Principal Data Engineer is a highly technical, AI-native role responsible for designing, building, and operating production-grade data pipelines and data products that power AI/ML, analytics, and automation across AbbVie's CMC and manufacturing ecosystem. This role is embedded inside PDST and works at the frontier of pharmaceutical data engineering. You will integrate and harmonize data from the full spectrum of manufacturing and development systems - including MES, historians, LIMS, QMS, ERP, and instrument platforms - and transform it into reliable, governed, semantically rich data assets that data scientists, process engineers, and AI systems can actually use. Enterprise-scale scope: Enterprise-scale biologics portfolio spanning clinical, commercial, and lifecycle stages Building AI playbook for the future: First-in-AbbVie and first-in-biologics analytical approaches; you build the AI playbook for the future Growth and Impact: Direct impact on regulatory submissions, commercial readiness, and manufacturing decisions through deep cross-functional exposure to manufacturing, quality, regulatory, and scientific leadership Mission: Every model you build helps ensure safe, reliable medicines reach patients at scale Responsibilities Data Ingestion & Integration Design and implement scalable, robust data ingestion pipelines that connect CMC and manufacturing source systems - including MES (Manufacturing Execution Systems), process historians, LIMS, QMS, ERP platforms, and instrument data sources - to centralized and federated data environments. Build connectors, adapters, and integration layers that handle the heterogeneous data formats, protocols, and latency profiles characteristic of pharmaceutical manufacturing environments. Support both batch and real-time/streaming data patterns, selecting appropriate architectures based on use case requirements. Data Harmonization & Semantic Modeling Develop and maintain harmonized data models and ontologies that bring consistency to CMC and manufacturing data across sites, systems, and modalities. Execute semantic mapping efforts that align source system fields, units, and identifiers to enterprise data standards and scientific meaning. Collaborate with process scientists, analytical chemists, and manufacturing engineers to ensure data models accurately reflect domain reality. Data Quality, Observability & Governance Implement automated data quality controls, validation frameworks, and anomaly detection mechanisms across pipeline layers. Build and maintain data lineage documentation and metadata infrastructure, enabling full traceability from source system to AI model input. Establish pipeline observability practices - monitoring, alerting, SLA tracking - to ensure data product reliability in production. Support data governance practices aligned with GxP requirements, 21 CFR Part 11, and AbbVie data standards. AI/ML Enablement & Data Product Development Architect and deliver governed, versioned, reusable data products purpose-built for AI/ML consumption, including feature stores, curated datasets, and vector-ready data layers for RAG and LLM applications. Partner closely with data scientists, ML engineers, and process modelers to understand model data requirements and translate them into reliable, scalable data infrastructure. Accelerate AI program delivery by eliminating data bottlenecks - not by workarounds, but by solving root causes structurally. Platform & Operational Enablement Contribute to the design and evolution of PDST's cloud-based data platform, including lakehouse architecture, data cataloging, access control, and compute infrastructure. Write and maintain infrastructure-as-code, CI/CD pipelines, and automated testing frameworks for data systems. Support platform onboarding of new CMC data domains and manufacturing sites, ensuring consistent application of standards and patterns. Provide operational support for production data pipelines, maintaining uptime and data freshness commitments. Stakeholder Engagement & Scientific Leadership Influence technical decision-making without formal authority - earning trust through scientific rigor, transparent methodology, and demonstrated business impact. Qualifications Required: Bachelor's Degree Computer Science, Data Engineering, Information Systems, Software Engineering, Bioinformatics, or a closely related technical field plus 6 years' experience; Master's Degree plus 5 years' experience; PhD plus 0 years' experience. Respective years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Expert-level proficiency in Python for data engineering tasks - pipeline development, transformation logic, data validation, and automation. Strong SQL skills across modern analytical and transactional databases; comfort with both ANSI SQL and platform-specific dialects. Demonstrated experience with cloud data platforms (AWS, Azure, or GCP) and modern data stack components - including tools such as dbt, Spark, Airflow, Databricks, Snowflake, or equivalents. Develop ETL/ELT pipelines using tools such as Informatica, Talend, Apache NiFi, and cloud-native services (e.g., AWS Glue, Azure Data Factory). Implement master data management (MDM), metadata management, and data cataloging solutions to ensure proper data lineage, accessibility, and compliance. Set and enforce standards for API development and data integration (REST, GraphQL, OData), enabling seamless integration using microservices architectures. Ownership orientation: you define your own problem space, drive solutions to completion, and hold yourself accountable to outcomes - not just outputs. Solution-architect instinct: you think before you build, consider the full landscape of available approaches, and choose tools based on fit-for-purpose reasoning rather than familiarity or trend. Scientific integrity: you build models you can explain, defend, and improve - and you apply the same standard to the work of others. Influence through credibility: you earn the confidence of scientists, engineers, and quality professionals by being right, being clear, and being useful - not by title or volume. Bias for impact: you are drawn to problems where the stakes are high and the analytical opportunity is real, and you are energized rather than intimidated by ambiguity. Preferred: 3+ years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Familiarity with technology transfer workflows, process characterization study design, or commercial process validation (PPQ/PV) in a biologics or pharmaceutical context. Experience in pharmaceutical, biotech, or other regulated life sciences manufacturing environments. Familiarity with GxP data principles, 21 CFR Part 11 compliance, or data integrity requirements in regulated industries. Prior exposure to manufacturing source systems such as MES, process historians (e.g., OSIsoft PI/AVEVA), LIMS, QMS, or ERP platforms Experience building data infrastructure for AI/ML programs - including feature engineering pipelines, model training datasets, or vector/embedding data layers for RAG architectures. Knowledge of biologics manufacturing processes (e.g., upstream cell culture, downstream purification, fill-finish) or CMC development workflows. Familiarity with data mesh, data fabric, or federated data architecture patterns. Experience with graph databases, knowledge graphs, or ontology frameworks applied to scientific or manufacturing data. Contributions to open-source data tooling or demonstrated engagement with the modern data engineering community. Design logical, physical, and conceptual data models using modeling tools (e.g., Erwin, PowerDesigner, dbt). . click apply for full job details
09/26/2026
Full time
Job Description Job Description Company Description About AbbVie AbbVie's mission is to discover and deliver innovative medicines and solutions that solve serious health issues today and address the medical challenges of tomorrow. We strive to have a remarkable impact on people's lives across several key therapeutic areas including immunology, oncology and neuroscience - and products and services in our Allergan Aesthetics portfolio. For more information about AbbVie, please visit us at . on LinkedIn, Facebook, Instagram, X and YouTube. Job Description While the AI innovation race in Biopharma is focused on Drug discovery, Product Development/ CMC represents the next barrier/ bottleneck. The complexity of biological systems, the rigor of regulatory expectations, the pace of pipeline growth, and the enormous value at stake make this one of the highest-leverage domains for applied data science and AI in the entire pharmaceutical value chain. We here at BTS - PDST, are building a dedicated, AI-native team that is driving cutting edge programs across early stage, late stage and commercial product development to accelerate E2E product development and launch, maximize yields of block buster products. Through our deep collaboration with PDST scientists we are boldly reimagining how Abbvie can bring our pipeline products and lifesaving drugs to patients faster, safer and in cost effective manner fueled by AI. Principal Data Engineer is a highly technical, AI-native role responsible for designing, building, and operating production-grade data pipelines and data products that power AI/ML, analytics, and automation across AbbVie's CMC and manufacturing ecosystem. This role is embedded inside PDST and works at the frontier of pharmaceutical data engineering. You will integrate and harmonize data from the full spectrum of manufacturing and development systems - including MES, historians, LIMS, QMS, ERP, and instrument platforms - and transform it into reliable, governed, semantically rich data assets that data scientists, process engineers, and AI systems can actually use. Enterprise-scale scope: Enterprise-scale biologics portfolio spanning clinical, commercial, and lifecycle stages Building AI playbook for the future: First-in-AbbVie and first-in-biologics analytical approaches; you build the AI playbook for the future Growth and Impact: Direct impact on regulatory submissions, commercial readiness, and manufacturing decisions through deep cross-functional exposure to manufacturing, quality, regulatory, and scientific leadership Mission: Every model you build helps ensure safe, reliable medicines reach patients at scale Responsibilities Data Ingestion & Integration Design and implement scalable, robust data ingestion pipelines that connect CMC and manufacturing source systems - including MES (Manufacturing Execution Systems), process historians, LIMS, QMS, ERP platforms, and instrument data sources - to centralized and federated data environments. Build connectors, adapters, and integration layers that handle the heterogeneous data formats, protocols, and latency profiles characteristic of pharmaceutical manufacturing environments. Support both batch and real-time/streaming data patterns, selecting appropriate architectures based on use case requirements. Data Harmonization & Semantic Modeling Develop and maintain harmonized data models and ontologies that bring consistency to CMC and manufacturing data across sites, systems, and modalities. Execute semantic mapping efforts that align source system fields, units, and identifiers to enterprise data standards and scientific meaning. Collaborate with process scientists, analytical chemists, and manufacturing engineers to ensure data models accurately reflect domain reality. Data Quality, Observability & Governance Implement automated data quality controls, validation frameworks, and anomaly detection mechanisms across pipeline layers. Build and maintain data lineage documentation and metadata infrastructure, enabling full traceability from source system to AI model input. Establish pipeline observability practices - monitoring, alerting, SLA tracking - to ensure data product reliability in production. Support data governance practices aligned with GxP requirements, 21 CFR Part 11, and AbbVie data standards. AI/ML Enablement & Data Product Development Architect and deliver governed, versioned, reusable data products purpose-built for AI/ML consumption, including feature stores, curated datasets, and vector-ready data layers for RAG and LLM applications. Partner closely with data scientists, ML engineers, and process modelers to understand model data requirements and translate them into reliable, scalable data infrastructure. Accelerate AI program delivery by eliminating data bottlenecks - not by workarounds, but by solving root causes structurally. Platform & Operational Enablement Contribute to the design and evolution of PDST's cloud-based data platform, including lakehouse architecture, data cataloging, access control, and compute infrastructure. Write and maintain infrastructure-as-code, CI/CD pipelines, and automated testing frameworks for data systems. Support platform onboarding of new CMC data domains and manufacturing sites, ensuring consistent application of standards and patterns. Provide operational support for production data pipelines, maintaining uptime and data freshness commitments. Stakeholder Engagement & Scientific Leadership Influence technical decision-making without formal authority - earning trust through scientific rigor, transparent methodology, and demonstrated business impact. Qualifications Required: Bachelor's Degree Computer Science, Data Engineering, Information Systems, Software Engineering, Bioinformatics, or a closely related technical field plus 6 years' experience; Master's Degree plus 5 years' experience; PhD plus 0 years' experience. Respective years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Expert-level proficiency in Python for data engineering tasks - pipeline development, transformation logic, data validation, and automation. Strong SQL skills across modern analytical and transactional databases; comfort with both ANSI SQL and platform-specific dialects. Demonstrated experience with cloud data platforms (AWS, Azure, or GCP) and modern data stack components - including tools such as dbt, Spark, Airflow, Databricks, Snowflake, or equivalents. Develop ETL/ELT pipelines using tools such as Informatica, Talend, Apache NiFi, and cloud-native services (e.g., AWS Glue, Azure Data Factory). Implement master data management (MDM), metadata management, and data cataloging solutions to ensure proper data lineage, accessibility, and compliance. Set and enforce standards for API development and data integration (REST, GraphQL, OData), enabling seamless integration using microservices architectures. Ownership orientation: you define your own problem space, drive solutions to completion, and hold yourself accountable to outcomes - not just outputs. Solution-architect instinct: you think before you build, consider the full landscape of available approaches, and choose tools based on fit-for-purpose reasoning rather than familiarity or trend. Scientific integrity: you build models you can explain, defend, and improve - and you apply the same standard to the work of others. Influence through credibility: you earn the confidence of scientists, engineers, and quality professionals by being right, being clear, and being useful - not by title or volume. Bias for impact: you are drawn to problems where the stakes are high and the analytical opportunity is real, and you are energized rather than intimidated by ambiguity. Preferred: 3+ years of hands-on experience designing and building enterprise-grade data pipelines, integration workflows, and data products in complex, multi-source environments. Familiarity with technology transfer workflows, process characterization study design, or commercial process validation (PPQ/PV) in a biologics or pharmaceutical context. Experience in pharmaceutical, biotech, or other regulated life sciences manufacturing environments. Familiarity with GxP data principles, 21 CFR Part 11 compliance, or data integrity requirements in regulated industries. Prior exposure to manufacturing source systems such as MES, process historians (e.g., OSIsoft PI/AVEVA), LIMS, QMS, or ERP platforms Experience building data infrastructure for AI/ML programs - including feature engineering pipelines, model training datasets, or vector/embedding data layers for RAG architectures. Knowledge of biologics manufacturing processes (e.g., upstream cell culture, downstream purification, fill-finish) or CMC development workflows. Familiarity with data mesh, data fabric, or federated data architecture patterns. Experience with graph databases, knowledge graphs, or ontology frameworks applied to scientific or manufacturing data. Contributions to open-source data tooling or demonstrated engagement with the modern data engineering community. Design logical, physical, and conceptual data models using modeling tools (e.g., Erwin, PowerDesigner, dbt). . click apply for full job details