Senior Data Engineer

Responsibilities

• Design, develop, test, and maintain scalable data pipelines and data products supporting Enterprise Credit Risk initiatives
• Build and optimize batch and real-time data processing solutions using modern cloud and lakehouse technologies
• Partner with Risk, Product, Data Science, and Engineering teams to understand business requirements and translate them into technical solutions
• Develop reusable frameworks, components, and engineering patterns that improve development efficiency and platform consistency
• Implement data quality controls, monitoring, alerting, and observability capabilities to ensure data reliability and trustworthiness
• Support the migration and modernization of existing data assets and workloads into Databricks and cloud-native platforms
• Build and maintain curated datasets that support lending decisioning, portfolio risk management, and and more
• Participate in architecture discussions and contribute technical recommendations for platform enhancements
• Troubleshoot production issues and drive root cause analysis to improve platform reliability and performance
• Follow engineering best practices for code quality, testing, deployment automation, $\text{\ old}$ security, and operational excellence
• Mentor junior engineers through code reviews, technical guidance, and knowledge sharing
• Continuously evaluate opportunities to improve scalability, performance, cost efficiency, and maintainability of data platforms

Requirements

• Strong experience developing and maintaining enterprise-scale data engineering solutions
• Hands-0n experience with Databricks, Apache Spark, PySpark, Hadoop, SQL, and Python
• Experience building ETL/ELT pipelines and large-scale data processing applications
• Experience working with cloud-
• Strong understanding of data modeling concepts and analytical data structures
• Experience implementing automated testing, monitoring, and data quality practices
• Experience with Data formats ( Parquet, Avro, ORC )
• Knowledge of CI/CD pipelines, Git-based development workflows, and DevOps principles
• Experience with Workflow orchestration Tools like Airflow
• Strong analytical and problem-solving skills with the ability to ability to work independently on complex technical challenges
• Effective communication skills with the ability to collaborate across technical and business teams
• Experience mentoring or supporting less experienced engineers
• Knowledge of Java Based application development is a huge Plus

Preferred Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related STEM field or alternative minimum of 5 years of experience in a related field
• Experience with several of the following technologies: Databricks, Apache Spark / PySpark, Airflow, SQL, Python, Delta Lake, Azure Data Services, Data Factory, Kafka, GitHub, CI/CD automation, Data Quality and Observability tools

About the Company

N/A