• At least 3 years of experience in Data Engineering
• Good communication in English
• Degree in Computer Science or related field
• Experience with designing database structures
• Analytical mindset and good problem-solving skills
• Good organizational skills
• Ability to work both independently and as part of a team
• Strong knowledge of Linux/Unix operating system and command line (bash script)
• Proficiency in programming languages, databases: (Python or Java), SQL, PostgreSQL, MySQL and building and maintaining ETL workflows
• Proficiency in writing clean and well-tested code
• Solid experience in software design, modeling, web scraping and data crawling, cleaning, integration, processing, and analysis.
• Experienced with tools like Docker, Conda, Singularity, and Elasticsearch, Airflow and how to use ci/cd tools to include custom compiled code if needed
• Familiar with distributed computing engines (Spark, Trino, Presto, Flink) & streaming processing (Flink, kafka)

Plus

• Deep understanding of data formats, data lake abstractions and optimizations to ensure best performance at query stage (partitioning, bucketing, file sizes and how this relates to concurrency that can be achieved)
• Keen eye on data quality, data lineage and documenting data with a focus on creating a self service environment for data scientists to explore data independently where needed
• Feature engineering experience from previous data science jobs, help data scientist with optimizing compute and to act as a guide for what is possible when data volumes are high or skew in distribution

Skills

• Ability to quickly learn and adapt to new technologies, tools, or programming languages
• Strong problem-solving skills for high-tech applications
• Excellent team collaboration and communication skills
• Ability to work independently and handle pressure effectively
• Strong desire to bring positive values to the community

Why you will love working here