Data Engineer

Role summary / purpose AWS Data Engineer III Key responsibilities • Collaborate with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead) to understand requirements and use cases, outline technical scope, and lead delivery of technical solutions • Confirm required developers and skill sets specific to the product • Collaborate with Data and Solution architects on key technical decisions • Develop data pipelines with focus on long-term reliability and maintaining high data quality • Design data warehousing solutions with the end-user in mind, ensuring ease of use without compromising on performance • Manage and resolve issues in production data warehouse environments on AWS • Perform hands-on development and peer review for certain components and tech stack • Stand up development instances and establish migration paths with required security and access controls • Develop components and related processes including data pipelines, ETL processes, and workflows • Build new data pipelines, identify existing data gaps, and provide automated solutions to deliver analytical capabilities and enriched data to applications • Implement data pipelines with attentiveness to durability and data quality Requirements – education & experience • 5+ years of AWS experience • Proven track record in the design and implementation of data warehouse solutions using AWS • Experience with RDBMS platforms and strong proficiency with SQL • Extensive understanding of database management fundamentals Required skills • AWS services: RedShift, S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, QuickSight • Data modeling and ETL processes tailored for data warehousing • Developing and refining data pipelines within AWS • Real-time and batch data processing • Python, Spark, PySpark, and Pandas • Infrastructure as Code: Terraform/CloudFormation • DevOps pipeline (CI/CD): Bitbucket, Concourse • Rest APIs and API gateway • Deep knowledge of IAM roles and policies • AWS monitoring services: CloudWatch, CloudTrail, and CloudWatch events • Networking fundamentals: DNS, TCP/IP, and VPN • Creating alerts and automated solutions for handling production problems Nice-to-have / preferred skills • Experience with Kafka/Messaging, preferably Confluent Kafka • Experience with EMR databases such as Glue Catalog, Lake Formation, Redshift, DynamoDB, and Aurora • Secrets Management Platform experience such as Vault and AWS Secrets Manager • Event Driven Architecture • AWS workflow orchestration tools like Airflow or Step Functions Employment details – type, location, hours Location: Charlotte, hybrid required. Remote work EST hours may be considered for exceptional candidates. Local Charlotte candidates must work hybrid.