Role Overview The Senior Data Engineer is responsible for designing, building, deploying and maintaining reliable data pipelines and reusable data products on a cloud-based data platform using AWS and Databricks.
The role
provides technical leadership across data ingestion, transformation, modelling, quality, lineage, orchestration, observability and performance, ensuring data remains dependable for analytics, reporting and ML/AI workloads. Key
Responsibilities
Data Pipeline Architecture and Delivery: Lead the design and implementation of maintainable batch and streaming pipelines that ingest, validate, transform and publish structured, semi-structured and unstructured data across AWS and Databricks. Lakehouse and Data Product Engineering: Design Bronze, Silver and Gold data layers using Delta Lake or equivalent patterns; develop reusable domain data models, curated datasets and data products with explicit ownership, definitions and service expectations. Source Integration and Incremental Processing: Implement reliable ingestion from databases, files, APIs and event streams using full-load, incremental, change-data-capture and replayable processing patterns appropriate to each source.
Data Quality and Contracts: Establish schema enforcement, data contracts, automated validation, reconciliation and anomaly checks covering completeness, validity, consistency, uniqueness, timeliness and referential integrity. Orchestration and Deployment Automation: Build parameterised workflows