- Data Pipeline Development
· Design, build, and optimize scalable ETL/ELT pipelines using Azure Databricks (PySpark, Spark SQL)
· Develop automated workflows using Databricks Workflows, Jobs, and Delta Live Tables (DLT) Implement efficient data ingestion across batch and streaming sources using Kafka, Event Hub, ADLS, or APIs
· Deep hands‑on expertise with Apache Spark for distributed data processing
· Proficiency with Azure ecosystem including ADLS, Azure Data Factory, Key Vault, Azure Functions, and Event Hub
· Strong coding skills in Python, SQL, and PySpark
· Implement CI/CD pipelines for Databricks using Azure DevOps, GitHub Actions, or Databricks Repos.
