Senior Data Engineer
Devoteam is an AI-driven tech consulting firm specialized in cloud platforms, cyber, data, and sustainability. Tech native for almost 30 years, we guide businesses through sustainable digital transformation to deliver value.Present in more than 25 countries across Europe, the Middle East, and Africa with over 11,500 collaborators, Devoteam is committed to using technology to serve people.Part of a regional Benelux entity, Luxembourg has a local workforce of almost 150 professionals, including an Innovative Tech Department: a focused group of around 40 specialists in cutting-edge technologies. Embracing a vendor-agnostic approach, this team cultivates deep expertise across diverse cloud providers, open-source technologies, and other innovative tools, enabling us to craft tailored solutions that best fit our clients' needs. We are looking for a Senior Data Engineer to join our Innovative Tech department. In this role, you will be responsible for designing, building, and optimizing robust data architectures that empower our clients to leverage their data effectively. You will work in a dynamic environment, tackling complex data challenges and collaborating with cross-functional teams to deliver scalable, high-performance solutions.What you will do:Build: Develop and optimize ETL/ELT pipelines handling both batch and streaming data processing;Bridge the sources: Integrate diverse data flows seamlessly from various structured and unstructured sources;Design: Contribute to the design of scalable solutions using medallion architecture to ensure high quality and performance;Modernize the stack: Implement advanced cloud-based data architectures like Data Lakehouse and Delta Lake;Innovate: Evaluate emerging technologies and architectural patterns to continuously upgrade our infrastructure;Guarantee quality: Ensure strict adherence to data security, governance, and performance standards;Solve the complex: Tackle difficult data challenges to ensure the reliability of multiple distributed systems;Share the knowledge: Animate the Data Community by sharing documentation, training, and tech updates;Collaborate: Work closely with Data Science, BI, Product, and Architecture teams to align on goals. What we are looking forTechnical skills:Core languages: Proficient in Python with strong SQL expertise;Distributed processing: Spark (batch & streaming), Hadoop;Orchestration: Workflow orchestration and GitOps: Airflow, Kubernetes-native pipelines;Lakehouse expertise: Iceberg, Delta Lake, Parquet, S3 / MinIO, Polaris Catalog;Real-time data processing: Kafka, Spark Streaming, Flink;Modern architectures: trong knowledge of real-time architectures (e.g., Debezium) and databases like PostgreSQL, MongoDB, or Cassandra;Cloud native: Strong experience with Databricks, Snowflake, GCP, or AWS is a major asset.Fluent in both English and French (spoken and written)Softskills : Collaborative | Attention to detail | Strong communication | Analytical | AutonomyNice ...