30 set
|
Merkleinnovation
|
Asia
30 set
Merkleinnovation
Asia
We are looking for a skilled and self-motivated ETL Development Engineer with strong hands-on experience in high-throughput data pipelines and offline batch processing. In this role, you will design, develop, and optimize scalable data warehouse models and Spark-based ETL workflows. You will work closely with cross-functional teams to ensure high data availability, seamless task orchestration, and reliable data architecture delivery.
Responsibilities
- ETL Pipeline Development: Design, build, and maintain high-throughput offline batch ETL pipelines utilizing Apache Spark (Core/SQL).
- Performance Tuning: Troubleshoot, benchmark, and optimize Spark jobs, memory allocation, and query execution for high performance and cost efficiency.
- Data Warehouse Modeling: Architect and implement scalable data warehouse dimensional models (Kimball/Inmon methodologies, ODS/DWH/ADS layering) supporting analytical workflows.
- Data Warehouse Modeling: Architect and implement scalable data warehouse dimensional models (Kimball/Inmon methodologies, ODS/DWH/ADS layering) supporting analytical workflows.
- Pipeline Orchestration: Manage DAG workflows, incremental processing, failure retry logic, and real-time monitoring/alerting using orchestration platforms such as Apache Airflow.
- Code Quality & Best Practices: Maintain high software engineering standards through clean code writing, robust documentation, code reviews, and automated unit testing.
- Migration & Modernization: Assist in or lead large-scale ETL re-engineering, platform migrations, and optimization initiatives across legacy and modern data stacks.
- Cross-functional Collaboration: Read technical documentation, communicate seamlessly with international tech teams in English, and deliver features independently from requirements to production.
Requirements
- Bachelor's degree or above in a computer-related major
- Experience: 3+ years in PySpark offline ETL development
- Technical Skills: Spark Core/SQL tuning, Airflow orc
📌 ETL Developer (Asia)
🏢 Merkleinnovation
📍 Asia