Descripción del puesto

We are seeking a highly skilled Senior Data Engineer with expertise in Databricks and PySpark to design, develop, and optimize scalable data processing solutions.

Key Responsibilities:

• Design, develop, and maintain scalable ETL pipelines within Databricks.
• Optimize data processing workflows for performance and cost efficiency.
• Implement and fine-tune distributed computing solutions.
• Ensure data quality, integrity, and consistency across platforms.
• Orchestrate data pipelines using airflow.
• Work closely with data scientists, analysts, and business stakeholders to understand data requirements.
• Translate business needs into technical solutions that align with industry best practices.
• Monitor and troubleshoot performance bottlenecks in Databricks
• Develop and maintain technical documentation for data workflows and solutions.

Requisitos

Required Qualifications:

• Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or a related field.
• 5+ years of experience in data engineering and development.
• Proficiency in Databricks and PySpark, python for large-scale data processing.
• Some knowledge of scala, airflow and SQL databases.
• Experience working with cloud data platforms (e.g., AWS, Azure, Google Cloud).
• Strong understanding of distributed computing and big data frameworks.
• Excellent problem-solving, analytical, and communication skills.

Preferred Qualifications:

• Databricks certifications (e.g., Databricks Certified Developer for Apache Spark).
• Experience with machine learning workflows and MLOps in Databricks.
• Familiarity with CI/CD pipelines and DevOps practices for data engineering.
• Experience in an Agile development environment.

Beneficios

- Fulltime and long term position

- USD Salary

- Paid Time Off

- Hardware when needed

- Corporate gifts on special dates