Senior Databricks Engineer

Responsibilities

• Manage Databricks workspaces, clusters, jobs, security configurations, and overall platform health
• Build and optimize data pipelines and transformations using PySpark and T-SQL
• Create automation and tooling using Python, Bash, and PowerShell
• Implement and automate data governance, access controls, and compliance best practices
• Translate conversational ideas into clear solution architectures and actionable tasks in Azure DevOps
• Participate in a 24x7x365 on-call rotation to support Databricks and SQL Server
• Produce high-quality documentation including design specs, build docs, and test plans

Requirements

• Solid experience with Databricks administration
• Strong proficiency in Python, Bash, and PowerShell
• Experience with PySpark and/or T-SQL for data processing
• Strong understanding of data governance and information security best practices
• Minimum of 12 years of relevant experience
• Minimum of 3 years experience managing people, processes, or projects

Preferred Qualifications

• Familiarity with Terraform for infrastructure as code
• Experience creating AI/BI dashboards
• Knowledge of CI/CD pipelines using Azure DevOps or GitHub
• Hands-on experience with AWS data ingestion tools like Glue and Lambda
• Experience with MS SQL Server administration

About the Company

Geisinger is a healthcare system serving more than 1 million people, comprising nine hospital campuses, a health plan, two research centers, and the Geisinger Commonwealth School of Medicine.