Senior Data Engineer

Role Overview: As a Senior Data Engineer, you will play a crucial role in optimizing our data architecture and pipelines specifically focusing on ETL and ELT processes using AWS Glue. Your expertise will drive the integration and transformation of data to enhance business analytics and reporting while ensuring efficient DevOps practices with a focus on AWS, Azure, CloudWatch, Kubernetes, and Docker.

12-Month Success: Within your first year, you will have successfully improved data processing speeds by at least 30% and implemented robust data quality checks. Your contributions will lead to more accurate data insights across the organization, significantly enhancing data-driven decision making.

Key Responsibilities: Design and implement scalable ETL and ELT pipelines and architectures utilizing AWS Glue. Collaborate with data scientists and analysts to ensure data accessibility. Optimize data retrieval processes and ensure data integrity. Provide mentorship to junior engineers and participate in code reviews to maintain code quality. Ensure DevOps practices are integrated into data engineering processes, including CI/CD automation and monitoring using AWS, Azure, Kubernetes, and Docker.

Required Qualifications: Strong experience in data engineering with proficiency in SQL, Python, and AWS Glue. Expertise in cloud-based data services and ETL tools. Solid understanding of data modeling and database design principles. Demonstrated ability to work collaboratively in a hybrid work environment. Experience with DevOps practices such as continuous integration and deployment.

Preferred Qualifications: Experience with big data technologies such as Hadoop or Spark. Familiarity with machine learning concepts and deployment practices. Knowledge of data governance and compliance frameworks.