Senior Platform Data Engineer

Responsibilities

• Stream data from Epic SDE, ADT feeds, and clinical sources into Databricks
• Curate shared clinical feature tables in Databricks/Unity Catalog for AI model training and monitoring
• Own and operate RAG (Retrieval-Augmented Generation) infrastructure, including ingestion, chunking, and embedding pipelines
• Administer vector databases, including schema design, indexing, and performance tuning
• Build and maintain retrieval pipelines using hybrid search and reranking
• Manage Databricks workspace configuration, Unity Catalog governance, and cost monitoring
• Administer the Feature Store and manage user access controls

Requirements

• 5+ years of experience in data engineering with batch and streaming pipelines
• Expert proficiency with Databricks (Delta Live Tables, PySpark, Unity Catalog, Feature Store)
• Hands-on experience with real-time data ingestion frameworks like Kafka or Spark Structured Streaming
• Strong SQL and Python (pandas, PySpark) skills
• Experience administering Databricks workspaces and compute management
• Understanding of data governance, lineage, and quality monitoring
• Bachelor's degree in a related field; Master's degree preferred

Preferred Qualifications

• Familiarity with clinical data models and healthcare data sources (EHR, ADT, claims)
• Experience with Epic data extraction methods (SDE, FHIR, epic-ws)

Benefits

• Healthcare benefits including vision and dental from day one
• Benefits for domestic partners

About the Company

Geisinger is a healthcare system serving over 1 million people, comprising nine hospital campuses, a health plan, two research centers, and the Geisinger Commonwealth School of Medicine.