Lead Data Engineer

Currently, we are looking for talented resources for one of our listed clients. If interested please reply to me with your updated resume or feel free to reach out to me for more details at (949) 482-4218.

Title: Lead Data Engineer (Hybrid Role)
Location: Atlanta, GA
Duration: Long Term
 

Job Description:

• We are seeking a talented and experienced Lead Data Engineer to lead our data engineering team in building and maintaining a robust and scalable data platform on AWS. The ideal candidate is a hands-on technical leader who possesses a deep understanding of data engineering concepts, excellent Python development skills, and extensive experience with various AWS services.

Responsibilities:

• Lead a team of data engineers in designing, developing, and maintaining our data infrastructure and pipelines.
• Collaborate with cross-functional teams including data scientists, analysts, and software engineers to understand data requirements and deliver high-quality solutions.
• Design, implement, and optimize data pipelines using Python and relevant frameworks to ensure efficient data processing and ingestion.
• Utilize AWS CloudFormation for infrastructure provisioning and management to ensure consistent and repeatable deployments.
• Set up monitoring and alerting using AWS CloudWatch to proactively identify and address any issues in the data pipeline.
• Design and implement data storage solutions using AWS DynamoDB, AWS S3, and other appropriate AWS services.
• Implement real-time data processing and streaming using AWS Kinesis and AWS Lambda to support event-driven architecture.
• Build and maintain data lake architecture using AWS Lake Formation, AWS Glue, and Athena for efficient data storage, cataloging, and querying.
• Define and enforce data modeling standards using Pydantic models to ensure data consistency and quality.
• Collaborate with the DevOps team to ensure proper CI/CD integration for data pipelines and applications.
• Stay current with industry trends and best practices in data engineering, cloud technologies, and Python development.

Required Skills & Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 10+ years of professional experience in data engineering and 3+ years in a leadership role.
• Proficient in Python application development with a strong understanding of Python frameworks.
• Extensive experience with AWS services including CloudFormation, CloudWatch, DynamoDB, EventBridge, Kinesis, Lambda, and S3.
• Hands-on experience with building and maintaining data lakes using AWS Lake Formation, Glue, and Athena.
• Strong knowledge of real-time data processing, event-driven architecture, and streaming technologies.
• Solid understanding of data modeling concepts and practical experience with Pydantic models.
• Proven track record of leading and mentoring data engineering teams to deliver high-quality solutions.
• Excellent problem-solving skills and the ability to work effectively in a collaborative team environment.
• Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to non-technical stakeholders.

Preferred Qualifications:

• AWS certification(s) in relevant areas such as AWS Certified Data Analytics - Specialty, AWS Certified Solutions Architect, or AWS Certified DevOps Engineer.
• Experience with other data engineering tools and technologies outside of the AWS ecosystem.
• Familiarity with big data technologies such as Hadoop, Spark, or Presto.

About us:
At our organization, we take our mission and values to heart! We are on a mission to offer more and better jobs all over the world! Our goal is to care for you while you care for our clients and get you paid the highest pay possible. All our associates working with us are expected to embrace our RACE values: R - Results Matter, A- Approachable, C - Care, and E - Emergency i.e. work with a sense of urgency.
For more relevant job opportunities please visit our website: Denken Solutions Careers