Business Intelligence Analyst Senior (AI Evaluation & Monitoring)

Responsibilities

• Independently consult with AI stakeholders to gather evaluation and data requirements
• Execute evaluation work for the AI portfolio including validation studies and production monitoring
• Collaborate with internal partners on reporting solutions and analytic studies
• Develop, validate, and execute complex queries and reports
• Use statistical methods to analyze data and visualize results
• Mentor junior BI developers and participate in code reviews
• Troubleshoot data quality issues through profiling and testing

Requirements

• Minimum of 8 years of relevant experience
• High School Diploma or Equivalent (GED)
• Proficiency in SQL and Python for data analysis
• Knowledge of applied statistics including hypothesis testing and regression
• Experience with LLM and generative AI evaluation

Preferred Qualifications

• Healthcare, clinical, or regulated-industry data experience
• Experience with production monitoring design (drift detection, performance decay)
• Proficiency with pandas, numpy, scikit-learn, and statsmodels

About the Company

Geisinger is a large healthcare system serving over 1 million people, including hospital campuses, a health plan, and research centers.