← Tutte le offerteData Engineer
Senior6500 SGD · periodo non indicatoSingapore, SG1 g fa
Dettagli
- Experience
- 5+ years
- Practices
- CI/CD
Candidati
Apri l'azienda e il link qui sopra: la candidatura arriva direttamente nel sistema di selezione dell'azienda, senza intermediari.
Descrizione
Job Overview
Design, develop and maintain data engineering solutions, including large-scale batch and streaming pipelines, distributed processing and containerized data workloads.
Responsabilità
Key Responsibilities
- Design, develop and maintain ingestion, transformation and validation pipelines using Apache Spark, Apache Flink and Apache Kafka.
- Implement distributed processing using Java or Scala and SQL, integrating data sources and S3-compatible object storage.
- Maintain schemas, metadata, data lineage and data-quality controls across processing stages, supporting layered data processing where applicable.
- Deploy and scale workloads on Kubernetes, and implement automated tests, GitLab CI/CD, monitoring and operational troubleshooting.
- Implement checkpointing, restart recovery, retention and lifecycle controls.
- Support high availability, backup, disaster recovery and inter-site synchronization in line with agreed requirements.
Requirements & Skills
- 5–8 years of data engineering experience, including building and deploying large-scale data systems.
- Strong hands-on experience designing batch and streaming pipelines using Apache Spark, Apache Flink and Apache Kafka.
- Proficiency in Java or Scala for distributed data processing, together with SQL.
- Experience deploying and scaling data workloads in containerised Kubernetes environments.
- Experience with S3-compatible object storage, data-quality controls, schema and lineage management, checkpointing and recovery, GitLab and CI/CD practices.
How t
Descrizione completa, azienda e link per candidarti: dopo l'accesso.
AccediI contatti del datore di lavoro (email/telefono/telegram) sono nascosti nell'anteprima pubblica.