Senior Data Engineer

About the role

The Senior Data Engineer is responsible for bridging business objectives and technical execution, leading the delivery of data engineering initiatives spanning data lakes, data platform modernization, and reporting solutions. The role requires strong expertise in cloud data architectures and hands‑on delivery experience, combined with the ability to collaborate effectively with business and technology stakeholders.

Key responsibilities

• Partner closely with business unit leaders, product owners, and IT stakeholders to capture business requirements, define project scopes, and establish clear delivery timelines.
• Perform workload and effort estimation; break down complex data initiatives into detailed technical epics, user stories, and execution roadmaps.
• Drive project delivery from architecture design through to testing, deployment, and operational handoff while proactively managing risks, dependencies, and scope creep.
• Serve as the primary technical interface between business stakeholders and the data engineering team, translating business needs into high‑performing data solutions.
• Design, build, and optimize scalable batch and real‑time data pipelines, data lakehouse architectures, and data warehousing solutions.
• Provide technical leadership for enterprise data platforms, driving architecture, engineering best practices, platform optimization, and innovation across Azure Cloud, Databricks (Spark), DataWorks, MaxCompute, OSS, and Hologres.
• Implement robust data orchestration, data ingestion, cleansing, transformation, augmentation, and data quality control processes.
• Establish and enforce technical quality standards, data governance frameworks, and best practices across data ingestion, storage, and processing.
• Conduct thorough code reviews, lead technical troubleshooting, and ensure data pipelines are secure, resilient, and cost‑optimized.
• Mentor and coach mid/junior data engineers, fostering a collaborative, continuous‑learning environment.

Required profile

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Quantitative Engineering, or a related field.
• Minimum 10+ years of IT experience, with at least 3+ years specifically in data engineering delivery leadership or technical lead roles.
• Solid domain understanding of the Insurance or Financial Services industry (e.g., policy administration, claims management, actuarial metrics, agency analytics).
• Proven track record in delivery leadership—demonstrated ability in effort estimation, scope decomposition, sprint planning, and managing cross‑functional stakeholder expectations.
• Hands‑on experience with Azure Cloud Services (Azure Data Lake Storage, Azure Data Factory, Azure SQL/Synapse) and database technology (Oracle, SQL Server, or PostgreSQL).
• Strong proficiency with Azure Databricks, Apache Spark (PySpark / Spark SQL), and modern ETL/ELT pipeline design.
• Deep expertise in enterprise data design methodologies, dimensional modeling (Kimball), data lakehouse architecture, and data warehousing concepts.
• Excellent verbal and written communication skills with the ability to articulate technical visions, architectural trade‑offs, and project updates to non‑technical stakeholders.
• Practical experience or familiarity with AliCloud big data tools, specifically DataWorks, MaxCompute, OSS, and Hologres.
• Experience with FineBI, Power BI, or similar enterprise reporting tools.

Required skills

• Azure Cloud Services
• Azure Data Lake Storage
• Azure Data Factory
• Azure SQL / Synapse
• Oracle
• SQL Server
• PostgreSQL
• Azure Databricks
• Apache Spark (PySpark, Spark SQL)
• ETL/ELT pipeline design
• Dimensional modeling (Kimball)
• Data lakehouse architecture
• Data warehousing
• DataWorks (AliCloud)
• MaxCompute
• OSS
• Hologres
• FineBI
• Power BI