← Wszystkie ofertySenior Data Engineer, Databricks & AI Automation
Senior7000 SGD · okres nie podanySingapore, SG2 dn. temu
Aplikuj
Odkryj firmę i link powyżej — Twoja aplikacja trafia prosto do systemu rekrutacyjnego firmy, bez pośredników.
Opis
Senior Data Engineer Databricks and Automation
Data and AI Innovation Team
O stanowisku
About the role
Build reliable data pipelines and trusted datasets for analytics, business applications and AI use cases. This hands-on role centres on Databricks, Python, PySpark and SQL. You will also simplify recurring engineering and support activities through automation, using AI where it provides a clear benefit.
Obowiązki
Key responsibilities
- Design, build and maintain production ETL/ELT pipelines using Databricks, PySpark, SQL and Delta Lake.
- Integrate data from databases, APIs and files, with streaming ingestion where required.
- Develop curated datasets and data models that meet agreed business definitions, access requirements and consumption needs.
- Implement data quality checks, reconciliation, incremental processing, schema handling and recovery procedures.
- Monitor pipeline reliability and data freshness; investigate failures, resolve root causes and improve recovery.
- Optimise processing performance and job compute usage, partnering with the Platform Engineer on shared settings and controls.
- Apply version control, automated testing, CI/CD and documentation to support reliable releases and maintainable solutions.
- Automate repetitive engineering and support tasks; assess AI-assisted approaches against accuracy, security and manual effort saved.
Essential skills and experience
- Typically 5 or more years of hands-on data engineering experience, including building and supporting productio
Pełny opis, firma i link do aplikacji — po zalogowaniu.
Zaloguj sięKontakty pracodawcy (e-mail/telefon/Telegram) są ukryte w publicznym podglądzie.