Data Engineering Technical Supervisor (Medior) – Databricks & Python

Work Schedule

Standard (Mon-Fri)

Environmental Conditions

Office

Job Description

Thermo Fisher Scientific

Data Engineering Technical Supervisor

Position Summary

We are seeking a Data Engineering Technical Supervisor to provide hands-on technical leadership and day-to-day guidance to a team of data engineers building scalable, reliable, and high-quality data solutions. This role does not include people management responsibilities.

The role combines hands-on data engineering, technical supervision, mentoring, and project leadership, with a strong focus on Databricks, Python/PySpark, SQL, Delta Lake, and modern data engineering practices.

As part of the Data & Analytics team, the Technical Supervisor will collaborate with data engineers, BI developers, analysts, product owners, and business stakeholders while supporting the reliability, maintainability, and operational continuity of production ETL/ELT processes in Databricks.

Key Responsibilities

• Provide technical leadership and mentoring to data engineers and technical oversight for Databricks solutions.

• Lead and contribute hands-on to the design and implementation of scalable ETL/ELT pipelines using Databricks, Python/PySpark, SQL, and Delta Lake.

• Guide engineers on architecture, design patterns, coding standards, testing, and performance optimization.

• Review technical designs and code to ensure solutions are scalable, maintainable, secure, and aligned with engineering standards.

• Troubleshoot complex production issues, support root-cause analysis, reduce technical debt, and improve platform reliability.

• Promote engineering best practices including Git-based source control, automated testing, CI/CD, peer review, and documentation.

• Develop reusable Python/PySpark components and standardized patterns for data ingestion, transformation, validation, and processing.

• Work with Delta Lake, Unity Catalog, and lakehouse architecture patterns, and support Databricks workflow orchestration and automation.

• Implement appropriate data quality, monitoring, observability, and error-handling mechanisms.

• Enable BI developers to build ETL/ELT pipelines within governed engineering patterns, including approved templates, standards, guardrails, and code reviews.

• Facilitate technical discussions, design reviews, knowledge sharing, and technical onboarding.

• Foster a collaborative engineering culture focused on ownership, continuous improvement, and high-quality delivery.

Delivery & Stakeholder Collaboration

• Partner with BI developers, analysts, product owners, and business stakeholders to translate data requirements into scalable technical solutions.

• Provide technical estimates, identify dependencies and risks, and support prioritization and delivery of engineering initiatives.

• Communicate technical concepts, trade-offs, risks, and recommendations clearly to technical and non-technical stakeholders.

• Ensure solutions meet agreed functional, performance, security, quality, and operational requirements.

Required Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Software Engineering, Information Technology, or a related discipline, or equivalent professional experience.

• 4+ years of professional experience in data engineering, software engineering, or a related technical field.

• Strong hands-on experience with Python/PySpark, Databricks, Apache Spark, and SQL in production data engineering.

• Experience designing and operating production-grade ETL/ELT pipelines using Delta Lake and lakehouse architecture concepts.

• Solid understanding of data modeling, data quality, partitioning, performance optimization, and distributed data processing.

• Experience with Git-based development, automated testing, CI/CD, and code review.

• Demonstrated ability to troubleshoot complex technical issues and provide constructive engineering guidance.

• Experience mentoring engineers or providing technical leadership.

• Strong analytical, problem-solving, communication, and stakeholder-management skills.

Preferred Qualifications

• Experience with Databricks on AWS/GCP/Azure

• Experience with Unity Catalog and Databricks governance capabilities.

• Experience with data observability, monitoring, and pipeline automation.

• Experience optimizing Spark/Databricks workloads for performance and cost.