Senior Data Engineer - AWS
Charlotte, NC - Onsite
6 Months+
Required Skills & Qualifications
• Data engineering, backend development, or software engineering, with demonstrated hands-on development experience.
• Strong hands-on experience with AWS data and cloud services, including services such as S3, Glue, EMR, Lambda, and Redshift.
• Proven experience developing and maintaining production-grade APIs and backend services using Java, with a strong understanding of RESTful API design, integration patterns, and error handling.
• Strong programming skills in Python, with hands-on experience building reusable modules, backend components, or API integrations.
• Hands-on experience with PySpark and distributed data processing, including developing scalable data transformations and optimising data pipelines.
• Proficiency in SQL, including complex queries, data transformations, and query performance optimisation.
• Strong understanding of object-oriented programming, software design principles, data structures, and maintainable application architecture.
• Practical exposure to MCP, LLMs, generative AI, or AI application integration, with an understanding of how AI capabilities can be integrated into APIs and enterprise workflows.
• Experience with API testing, debugging, authentication and authorisation, and application integration.
• Familiarity with Git, CI/CD pipelines, automated testing, and Agile development practices.
• Strong problem-solving, communication, and collaboration skills, with the ability to work independently on complex technical requirements.
Preferred Qualifications (Nice to Have)
• Experience building AI-enabled applications, LLM-powered workflows, or MCP-based integrations.
• Exposure to LLM orchestration frameworks, tool calling, retrieval-augmented generation (RAG), or enterprise AI integration patterns.
• Experience with AWS-native AI services and deployment of AI-enabled applications in cloud environments.
• Familiarity with API gateways, containerisation, microservices architecture, and application observability.
• Experience with data warehousing, distributed computing, Spark performance tuning, and data quality frameworks.
Job Responsibilities
Key Responsibilities
• Design, develop, and optimise scalable data pipelines and data processing workflows using Python, PySpark, and SQL.
• Develop, enhance, and maintain production-grade APIs and backend services using Java and Python, with a strong focus on API design, integration, performance, and reliability.
• Build and integrate RESTful APIs and microservices to support data-driven applications and enterprise integrations.
• Leverage AWS services such as S3, Glue, EMR, Lambda, Redshift, and other relevant cloud-native services to build and manage scalable data engineering solutions.
• Work with Model Context Protocol (MCP), Large Language Models (LLMs), and generative AI technologies to develop or integrate AI-enabled applications and workflows.
• Implement reusable, maintainable, and scalable code using object-oriented programming principles, design patterns, and established engineering best practices.
• Write and optimise complex SQL queries and implement efficient ETL/ELT processes for large-scale data transformations.
• Collaborate with data engineers, application developers, architects, data scientists, and business stakeholders to translate requirements into robust technical solutions.
• Troubleshoot and resolve issues across APIs, data pipelines, integrations, and cloud-based applications.
• Participate in code reviews, automated testing, CI/CD implementation, performance tuning, and production support while adhering to engineering standards.