Senior Data Engineer

Job Description
Summary
Our client is seeking a highly experienced Senior Data Engineer to lead the design, development, and optimization of data pipelines, APIs, and data platforms. This role will focus on ETL/ELT processes using Matillion and Snowflake, API development, and integration with machine learning workflows and Databricks. The ideal candidate will have a strong background in data engineering, cloud platforms, and modern data architecture.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines using Matillion and Snowflake.
• Develop and manage RESTful APIs for data access and integration.
• Collaborate with data scientists and ML engineers to integrate data pipelines with machine learning workflows.
• Optimize Snowflake data warehouse performance and manage data models.
• Implement data quality, governance, and security best practices.
• Work with Databricks for data processing, transformation, and ML model support.
• Automate data validation and reconciliation processes.
• Document data architecture, pipelines, and integration processes.

Qualifications

• Bachelors or Masters degree in Computer Science, Engineering, or a related field.
• 8+ years of experience in a data modeling, data warehousing, and data integration role.

Required Skills

• Strong experience with Matillion ETL and Snowflake.
• Proficiency in Databricks
• Experience in data management requirements, integration platforms, and have expertise with APIs.
• Experience with API development using frameworks.
• Familiarity with Databricks and integrating Machine Learning workflows.
• Solid understanding of data warehousing, data modeling, and ELT best practices.
• Experience with CI/CD pipelines, version control (Git), and DevOps practices.
• Strong proficiency in SQL and at least one programming language (e.g., Python, Java, or Scala).
• Experience with data modeling, data warehousing, and data integration.
• Familiarity with cloud platforms and data services.
• Understanding of data governance, security, and compliance requirements