Data Engineer

Responsibilities

• Design, build and optimise scalable data solutions using Lakehouse architectures (data lake + data warehouse) and advise stakeholders on architectural trade-offs.
• Develop, maintain and monitor reliable data ingestion and ETL/ELT pipelines for batch, streaming and real-time data sources.
• Ensure ACID properties, schema evolution, high data quality and implement data governance (security, privacy, integrity, compliance, auditing).
• Optimise storage, compute and query performance for BI and ML workloads using partitioning, indexing and parallel processing.
• Implement CI/CD for data and ML lifecycles, automate data validation, model training, deployment and infrastructure provisioning.
• Deploy, manage and scale machine learning models into production using MLOps best practices; implement model serving and rollout strategies.
• Establish monitoring for model performance, data drift and model decay; implement alerting and remediation workflows.
• Maintain version control and tracking for code, datasets and model artifacts (eg. MLflow or equivalent).
• Produce clear technical documentation, data flow diagrams and operational runbooks.

Requirements

• Lakehouse expertise
• Spark experience
• Kafka experience
• SQL proficiency
• Python / Scala
• ETL / ELT
• MLOps experience
• CI/CD pipelines
• Docker / Kubernetes
• Cloud platforms

Preferred Qualifications

• Hands-on experience with Databricks and Delta Lake.
• Experience with model serving frameworks (TensorFlow Serving, SageMaker or equivalent).
• Familiarity with experiment tracking tools such as MLflow.
• Prior work in financial services or enterprise AI projects.
• Strong communication skills and experience working in multidisciplinary teams.

Benefits

• Competitive salary and performance bonuses
• Comprehensive health insurance
• Annual leave and paid leave
• Professional development and certification support
• Flexible / hybrid working arrangements
• Career progression on fast-growing AI projects

About the Company

DeepLight AI is a specialist AI and data consultancy delivering intelligent enterprise systems across multiple industries with particular depth in financial services. The team combines data science, statistical modelling, AI/ML technologies, workflow automation and systems integration to build practical, production-grade solutions for complex business operations. This role is based in the UAE and offers the opportunity to work on cutting-edge AI initiatives with a collaborative team.