Data Engineer

Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for ingesting, transforming, and loading a variety of data sources, including structured, unstructured, streaming, and real-time data.
• Implement robust data integration to ensure consistent, reliable, and high-quality data for analytics and AI use cases.
• Optimize data storage solutions, including relational databases, NoSQL databases, data lakes, and cloud storage services for efficient management of large volumes of data.
• Enhance system scalability, reliability, and efficiency via performance optimization techniques such as indexing, partitioning, caching, and parallel processing, especially for AI model training and inference.
• Implement data governance frameworks covering security, privacy, integrity, and compliance including encryption, access controls, and auditing.
• Monitor data pipelines and infrastructure, proactively identify and resolve issues, and perform routine maintenance.
• Collaborate with data scientists, ML engineers, architects, and domain experts for integrated production-ready data solutions.
• Create comprehensive documentation including technical specifications and data flow diagrams to facilitate collaboration and knowledge sharing.

Requirements

• Proven experience as a Data Engineer with a track record of designing and implementing complex data pipelines and infrastructure.
• Expertise in data modeling, ETL processes, and data warehousing with proficiency in SQL and scripting languages (e.g., Python, Scala).
• Strong hands-on experience with big data technologies and frameworks such as Hadoop, Spark, Kafka, Flink, and cloud platforms (AWS, Azure, GCP).
• Familiarity with containerization and orchestration technologies, such as Docker and Kubernetes, and DevOps practices for CI/CD, including ML model deployment and pipeline automation.
• Experience supporting AI/ML initiatives including dataset preparation, model input/output data flows, MLOps integration, and experimentation tracking.
• Excellent analytical, problem-solving, and communication skills.
• Ability to work effectively in a fast-paced, collaborative environment with a passion for innovation and continuous learning.

Benefits

• Impact: Be part of a team shaping the future of AI and influencing industries and society.
• Innovation: Work on cutting-edge projects in AI, data engineering, and machine learning, using the latest technologies and methodologies.
• Collaboration: Collaborate with diverse experts, fostering creativity, learning, and career growth.
• Opportunity: Ample opportunities for professional development, advancement, and leadership in a rapidly growing company.
• Culture: Join a culture of curiosity, excellence, and collaboration where your contributions are recognized and valued.

About the Company

DeepLight is a pioneering AI company at the forefront of artificial intelligence research and development. The company's mission is to harness the power of data and machine learning to revolutionize industries and create a brighter future. With a dynamic team and a culture of relentless innovation, DeepLight operates globally from its UAE base, focusing on impactful AI solutions.