MAIN RESPONSIBILITIES
Leading Strategy & Roadmap for Data Engineering
- Build and lead the development roadmap for Data Engineering, transforming business objectives into technical plans, prioritized initiatives, and measurable outcomes (OKRs/KPIs).
- Serve as the primary technical point of contact in critical data architecture decisions, ensuring alignment with the overall data strategy.
- Proactively identify and manage technical risks, ensuring the progress, quality, and long-term sustainability of the data platform.
Data Architecture & Platform
- Lead the design and implementation of the Data Lakehouse architecture on Azure, including the selection and optimization of technologies:
- Azure Data Lake Storage Gen2
- Delta Lake
- Azure Synapse Analytics / Azure Databricks
- Design highly scalable data architecture optimized for both batch processing and streaming.
- Pioneer the implementation of strategies to optimize large data storage and querying, balancing performance, scalability, and operational costs.
Develop & Operate Data Pipelines
- Design, develop, and optimize high-performance ETL/ELT pipelines from various data sources: on-premises, cloud, streaming, API, file system.
- Ensure pipelines are fault-tolerant, self-healing, well-monitored, and meet SLA requirements in a production environment.
- Build real-time/near-real-time pipelines to directly serve AI/ML systems and advanced analytics.
Integration & Cross-Team Coordination
- Act as a technical bridge between Data Engineering and AI/ML, BI, DevOps, and Product teams.
- Support data integration into:
- Machine Learning models in a production environment
- BI systems, dashboards, and predictive analytics
- Complex data applications
- Coordinate with infrastructure and security teams to design and operate a secure data system, complying with security standards and internal regulations.
Data Governance & Technical Standards
- Establish and enforce data governance standards, including:
- Data Quality Framework
- Centralized Metadata Management
- Automated Data Lineage and Data Observability
- Ensure compliance with best practices in data design, coding standards, and documentation.
Lead the Team & Share Knowledge
- Train, mentor, and develop technical capabilities for Data Engineers.
- Conduct code reviews, architecture design reviews, and technical documentation reviews.
- Build a high-quality technical documentation system to facilitate knowledge sharing and effective operations among teams.
JOB REQUIREMENTS
- Graduated with a Bachelor's or Master's degree in fields such as: Information Technology, Data Science, Information Systems, or equivalent.
- Minimum of 8+ years of experience in the field of Data Engineering, including 2–3 years in a Technical Lead/Team Lead role.
- Experience in designing and implementing end-to-end data solutions at scale in a production environment.
- In-depth and practical experience in deploying and operating data infrastructure on Azure Cloud, including:
- Azure Data Lake Storage Gen2
- Azure Synapse Analytics, Azure Data Factory, Azure Databricks
- Streaming systems: Azure Event Hub, Azure Stream Analytics, or Apache Kafka
- Proficient in SQL and at least one programming language such as Python or PySpark.
- Solid understanding of distributed computing principles and big data processing.
- Deep experience in designing advanced data models (Star Schema, Snowflake Schema), mastering Lakehouse architecture and Delta Lake best practices.
- Mandatory experience in operating large-scale production data systems, stable real-time pipelines, and direct integration with the AI/ML layer.
- High ownership of work, system thinking, and a structured approach to problem-solving.
- Excellent communication and cross-departmental collaboration skills with AI, DevOps, Product teams, and technical stakeholders.
- Strong analytical and problem-solving mindset; confident in decision-making in complex and ambiguous contexts.
- Growth mindset, continuous learning spirit, flexibility, and willingness to experiment with new technologies.
BENEFITS & WELFARE