Job Description
Requires 5 years of experience in a related occupation. Active or past AWS and Databricks certifications required. Requires experience with: Working in/with clients in the healthcare or pharmacy benefit management industry; Migrating enterprise data pipelines from on-premises Hadoop to AWS Databricks; Transforming legacy oozie workflows into modular Databricks jobs and notebooks; Migrating hive tables to Delta Lake on S3, incorporating schema evolution, ACID guarantees, and time travel; Designing and implementing event-driven pipelines using Amazon EventBridge and Step Function; Building and optimizing Spark workloads on EMR and Databricks; Engineering serverless automated validation framework using AWS Lambda, SQS and EMR; Developing and automating CI/CD pipelines using Terraform and Jenkins; Creating and maintaining Apache Iceberg tables from full and incremental loads leveraging schema evolution and versioning; Designing automation scripts for resource cleanup and cost optimization across AWS and Databricks environments; and Using: Kafka, NIFI, Hadoop, Java, Jenkins, Docker, Ansible, AWS, Spark, Python, Scala, Golang, Apache Kafka, Apache Camel, and Spring Boot. Requires a Bachelor's degree or foreign equivalent in any engineering field Please send us your resume via email at candidates (at) placementservicesusa (dot) com with reference . Do not copy and paste your resume in the body email. Please attach it as a Word or PDF document. Thank you.