Senior Databricks Platform Engineer

Responsibilities

• Design, configure, and maintain the enterprise Databricks Lakehouse Platform, including workspaces, Unity Catalog, and scalable data architectures
• Administer and optimize Databricks compute, clusters, SQL warehouses, and serverless capabilities
• Develop and optimize data pipelines, ETL/ELT processes, and ingestion frameworks using Apache Spark, Delta Lake, and Databricks Workflows
• Automate platform provisioning and deployments using Terraform, Python, and Databricks APIs
• Implement platform monitoring, logging, alerting, and observability
• Perform Spark and SQL performance tuning and troubleshooting
• Implement cost optimization strategies and DBU consumption forecasting
• Integrate Databricks with enterprise data sources, governance tools, and BI platforms
• Provide technical guidance and mentorship to data engineers and platform users

Requirements

• MA/MS degree with 12+ years of experience (or equivalent experience in lieu of degree)
• Strong, hands-on experience implementing and administering the Databricks Lakehouse Platform
• Expertise in SQL, Spark performance tuning, and workload management
• Proficiency in Python and Bash for automation and scripting
• Experience with CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions, Azure DevOps)
• Experience operating Databricks in AWS-based or hybrid cloud environments
• Familiarity with containerized environments like Docker and Kubernetes
• Knowledge of data ingestion, ETL/ELT, and distributed data processing
• Understanding of cloud security, IAM, and resource governance within Databricks and Unity Catalog

About the Company

GDIT is a global technology and professional services company that delivers technology solutions and mission services to every major agency across the U.S. government, defense and intelligence community.