D

Data Engineer

DeepLight AI

Posted 2 months ago

Employment Type

Full Time

Location

Dubai

Job Listing No Longer Available

This job posting is no longer accepting applications. It may be more than 30 days old or the position has been filled.

Requirements

Lakehouse expertise, Spark, Kafka, SQL, Python, Scala, MLOps, CI/CD, Docker, Kubernetes, Cloud platforms

Required Skills

Databricks Delta Lake Spark Kafka SQL Python Scala MLOps MLflow Docker Kubernetes AWS Azure GCP Model Monitoring ETL/ELT Data Lakehouse CI/CD

Job Description

Responsibilities

  • Design, build and optimise scalable data solutions using Lakehouse architectures (data lake + data warehouse) and advise stakeholders on architectural trade-offs.
  • Develop, maintain and monitor reliable data ingestion and ETL/ELT pipelines for batch, streaming and real-time data sources.
  • Ensure ACID properties, schema evolution, high data quality and implement data governance (security, privacy, integrity, compliance, auditing).
  • Optimise storage, compute and query performance for BI and ML workloads using partitioning, indexing and parallel processing.
  • Implement CI/CD for data and ML lifecycles, automate data validation, model training, deployment and infrastructure provisioning.
  • Deploy, manage and scale machine learning models into production using MLOps best practices; implement model serving and rollout strategies.
  • Establish monitoring for model performance, data drift and model decay; implement alerting and remediation workflows.
  • Maintain version control and tracking for code, datasets and model artifacts (eg. MLflow or equivalent).
  • Produce clear technical documentation, data flow diagrams and operational runbooks.

Requirements

  • Lakehouse expertise
  • Spark experience
  • Kafka experience
  • SQL proficiency
  • Python / Scala
  • ETL / ELT
  • MLOps experience
  • CI/CD pipelines
  • Docker / Kubernetes
  • Cloud platforms

Preferred Qualifications

  • Hands-on experience with Databricks and Delta Lake.
  • Experience with model serving frameworks (TensorFlow Serving, SageMaker or equivalent).
  • Familiarity with experiment tracking tools such as MLflow.
  • Prior work in financial services or enterprise AI projects.
  • Strong communication skills and experience working in multidisciplinary teams.

Benefits

  • Competitive salary and performance bonuses
  • Comprehensive health insurance
  • Annual leave and paid leave
  • Professional development and certification support
  • Flexible / hybrid working arrangements
  • Career progression on fast-growing AI projects

About the Company

DeepLight AI is a specialist AI and data consultancy delivering intelligent enterprise systems across multiple industries with particular depth in financial services. The team combines data science, statistical modelling, AI/ML technologies, workflow automation and systems integration to build practical, production-grade solutions for complex business operations. This role is based in the UAE and offers the opportunity to work on cutting-edge AI initiatives with a collaborative team.

How to Apply

Similar Jobs You Might Be Interested In

Join Dubai's Remote Work Revolution.

Stay ahead in your career with Dubai's first platform dedicated to remote and hybrid job opportunities. Subscribe for weekly insights and job alerts directly to your inbox.

Thank you for subscribing! Check your inbox for confirmation.
Weekly Job Alerts
Subscribe to receive curated lists of the best remote and hybrid job opportunities in Dubai, tailored to your skills and interests.
Weekly Blog Newsletter
Get the latest insights, trends, and advice on remote work every week to help you thrive in the evolving work environment.