We are looking for a Senior Data Engineer with deep expertise in Databricks and Apache Spark to join a high-growth global retail enterprise. You will own the design and delivery of scalable data pipelines across the Medallion Architecture (Bronze, Silver, Gold layers), enabling analytics and AI workloads at enterprise scale. Role & Responsibilities: • Design, build and maintain production-grade data pipelines on Databricks using PySpark and Delta Lake • Implement and govern Medallion Architecture across Bronze, Silver and Gold layers in Azure Data Lake Storage Gen2 • Collaborate with data scientists and ML engineers to productionise model training pipelines • Optimise Spark jobs for performance, cost and reliability across large-scale batch and streaming workloads • Define data quality standards and implement monitoring using Great Expectations or Delta Live Tables • Work closely with Azure DevOps for CI/CD of notebook and pipeline code • Mentor junior engineers and contribute to architectural decisions Required Skills & Experience: • 4+ years of data engineering experience, with at least 2 years on Databricks • Strong Python and PySpark programming skills • Hands-on experience with Delta Lake, Unity Catalog and Databricks Workflows • Proficiency with Azure services: ADLS Gen2, Azure Data Factory, Azure Key Vault • Experience with Medallion Architecture and layered data modelling • Familiarity with dbt or similar transformation frameworks is a plus • Databricks Certified Data Engineer Associate or Professional preferred What We Offer: • Fully remote role based in India with flexible working hours aligned to IST • Competitive salary of ₹35–50L depending on experience • Opportunity to work on AI-first data infrastructure for a global brand • Fast-paced, collaborative environment with direct exposure to senior leadership This role is ideal for engineers who have built production Lakehouse platforms and want to take ownership of data infrastructure that directly powers AI and analytics products.
Remote · India | ₹35–50L