Data Quality Engineer

Job Summary:

We are seeking a meticulous and proactive Data Quality Engineer to join our dynamic DataEngineering team. The ideal candidate will possess a deep understanding of data quality,validation, and data reconciliation concepts. They will collaborate closely with our dataengineers to ensure the integrity and accuracy of our data. This role requires expertise inPySpark, Python, data pipelines, ETL processes, and Snowflake.Key Responsibilities:• Data Quality Management: Develop and implement data quality checks, validationprocedures, and reconciliation processes to ensure data integrity and consistency.• Collaboration: Work closely with data engineers to design and optimize datapipelines and ETL processes, ensuring data quality throughout the data lifecycle.• Data Validation: Create automated scripts and processes to validate data againstbusiness rules and requirements using PySpark and Python.• Data Reconciliation: Develop and maintain data reconciliation frameworks toidentify and resolve data discrepancies in a timely manner.• Snowflake Expertise: Leverage Snowflake for data warehousing and data qualitymanagement, ensuring efficient and effective data storage and retrieval.• Process Improvement: Identify areas for improvement in existing data qualityprocesses and propose innovative solutions to enhance data accuracy andreliability.• Documentation: Maintain comprehensive documentation of data qualityprocedures, validation rules, and reconciliation processes.• Reporting: Generate regular reports on data quality metrics, highlighting issues andtrends to inform stakeholders and drive continuous improvement.Qualifications:• Education: Bachelor’s degree in Computer Science, Information Technology, or arelated field. Master’s degree is a plus.• Experience: Minimum of 4 years of experience in a data quality engineering role,with a strong focus on data validation, reconciliation, and quality management.• Technical Skills:o Proficiency in PySpark and Python for data processing and automation.o Extensive experience with data pipelines and ETL processes.o Strong knowledge of Snowflake for data warehousing and qualitymanagement.o Familiarity with SQL and other database technologies.Soft Skills:o Excellent problem-solving and analytical skills.o Strong attention to detail and commitment to data accuracy.o Effective communication skills, with the ability to collaborate with cross-functional teams.o Ability to work independently and manage multiple priorities in a fast-paced environment.