Senior Engineer (Python)

Responsibilities

• Build and design large scale, distributed crawling bots and infrastructure
• Develop and maintain data pipelines to extract data from web pages, documents, PDFs, and APIs
• Unify heterogeneous documents into a coherent data schema
• Preprocess and normalize raw data for downstream classification, ML/NLP, and search indexing
• Build APIs to expose structured, classified data via ElasticSearch/OpenSearch
• Collaborate with ML/NLP teams to integrate classification models into the pipeline
• Use LLMs to find precise answers in OEM repair procedures
• Automate workflows using Apache Airflow and deploy solutions in Kubernetes on AWS
• Optimize and scale data pipelines using Spark (EMR)

Requirements

• 5+ years of experience in Python with building crawling/scraping solutions at scale
• Experience working with APIs (REST)
• Experience with PDF processing (OCR, Tesseract, PyMuPDF etc.)
• Proficiency in data processing and search technologies (ElasticSearch/OpenSearch, NoSQL/SQL databases)
• Hands-on experience with Airflow and Spark (EMR) or similar distributed systems
• Strong problem-solving skills in handling anti-scraping mechanisms and data scaling challenges
• Hands-on experience with AWS or GCP

Preferred Qualifications

• Familiarity with NLP and Machine Learning
• Experience with LLMs, NLP models, or ML frameworks (Hugging Face, spaCy, TensorFlow, PyTorch)
• Prior experience in automated document classification
• Experience working in high-scale, production environments with petabytes of data
• Hands-on experience with Kubernetes

About the Company

Tekmetric is the all-in-one, cloud-based platform helping auto repair shops run smarter, grow faster, and serve customers better.