BBGC

Senior Data Engineer

BBGC

Posted 1 day ago

Employment Type

Full Time

Location

Dubai

Requirements

6+ years, Azure Data Factory, Azure Databricks, Python, SQL, Apache Spark, Spark Streaming, Kafka, Production pipelines, Git, CI/CD, Trading domain, Market data, Financial instruments

Required Skills

Azure Azure Data Factory Azure Databricks Python SQL Apache Spark Spark Streaming Kafka Delta Lake CI/CD Git Data Engineering Trading Market Data Financial Instruments

Job Description

Responsibilities

  • Design, build, and maintain scalable data pipelines on Azure and Databricks
  • Develop batch and streaming ETL with Spark; optimize for reliability, cost, and performance
  • Partner with business stakeholders to translate trading and risk use cases into data solutions
  • Ensure data quality, governance, and security across environments
  • Operate and support production-grade pipelines; monitor, troubleshoot, and root-cause issues
  • Implement CI/CD, testing, and version control for data engineering workflows
  • Produce documentation and reusable frameworks/best practices

Requirements

  • 6+ years of experience in data engineering or related roles
  • Expertise in Azure Data Factory and Azure Databricks
  • Strong proficiency in Python for data engineering tasks
  • Solid SQL and relational database skills
  • Exposure to real-time processing (e.g., Spark Streaming, Kafka)
  • Experience supporting production-grade data pipelines
  • Familiarity with trading domain: market data, trade lifecycle, financial instruments
  • Experience with Git and CI/CD pipelines
  • Excellent problem-solving skills and attention to detail
  • Based in Dubai, UAE with a hybrid work setup

Preferred Qualifications

  • Experience with Delta Lake and Lakehouse patterns
  • Knowledge of Azure Synapse, Azure Data Lake Storage, and Azure Monitor
  • Infrastructure as Code (e.g., Terraform) and Databricks Jobs/Workflows
  • Data modeling, unit testing for data (e.g., pytest, Great Expectations)
  • Performance tuning for Spark and SQL workloads

Benefits

  • Hybrid work model in Dubai
  • Learning and professional growth culture

About the Company

We are a global technology consultancy with offices in the Middle East, Asia, Europe, and the USA. We deliver business value through innovation, from sustainable software development to cloud services and operational excellence. We have deep expertise in Capital Markets, particularly Trading and Risk Management functions.

How to Apply

Similar Jobs You Might Be Interested In

Join Dubai's Remote Work Revolution.

Stay ahead in your career with Dubai's first platform dedicated to remote and hybrid job opportunities. Subscribe for weekly insights and job alerts directly to your inbox.

Thank you for subscribing! Check your inbox for confirmation.
Weekly Job Alerts
Subscribe to receive curated lists of the best remote and hybrid job opportunities in Dubai, tailored to your skills and interests.
Weekly Blog Newsletter
Get the latest insights, trends, and advice on remote work every week to help you thrive in the evolving work environment.