Senior Data Engineer

Responsibilities

• Design, build, and maintain scalable data pipelines on Azure and Databricks
• Develop batch and streaming ETL with Spark; optimize for reliability, cost, and performance
• Partner with business stakeholders to translate trading and risk use cases into data solutions
• Ensure data quality, governance, and security across environments
• Operate and support production-grade pipelines; monitor, troubleshoot, and root-cause issues
• Implement CI/CD, testing, and version control for data engineering workflows
• Produce documentation and reusable frameworks/best practices

Requirements

• 6+ years of experience in data engineering or related roles
• Expertise in Azure Data Factory and Azure Databricks
• Strong proficiency in Python for data engineering tasks
• Solid SQL and relational database skills
• Exposure to real-time processing (e.g., Spark Streaming, Kafka)
• Experience supporting production-grade data pipelines
• Familiarity with trading domain: market data, trade lifecycle, financial instruments
• Experience with Git and CI/CD pipelines
• Excellent problem-solving skills and attention to detail
• Based in Dubai, UAE with a hybrid work setup

Preferred Qualifications

• Experience with Delta Lake and Lakehouse patterns
• Knowledge of Azure Synapse, Azure Data Lake Storage, and Azure Monitor
• Infrastructure as Code (e.g., Terraform) and Databricks Jobs/Workflows
• Data modeling, unit testing for data (e.g., pytest, Great Expectations)
• Performance tuning for Spark and SQL workloads

Benefits

• Hybrid work model in Dubai
• Learning and professional growth culture

About the Company

We are a global technology consultancy with offices in the Middle East, Asia, Europe, and the USA. We deliver business value through innovation, from sustainable software development to cloud services and operational excellence. We have deep expertise in Capital Markets, particularly Trading and Risk Management functions.