Azure Databricks Developer

Infosys Limited is seeking a highly skilled and result-driven Azure Databricks Developer to join our elite Data & Analytics Unit in Bangalore. This core data engineering role centers on developing high-performance data processing pipelines, leveraging distributed systems, and architecting modern data lakehouse solutions on the Microsoft Azure cloud platform.

Key Responsibilities & Scope of Work

• Pipeline Architecture & Engineering: Design, build, and optimize scalable automated data pipelines (ETL/ELT) using the native capabilities of the Azure Databricks platform.
• Distributed Data Processing: Implement and fine-tune complex cloud transformation and analytical tasks using PySpark and core Apache Spark frameworks.
• Data Quality & Governance: Enforce strict data quality, validation, and metadata integrity checks across disparate relational and non-relational database sources.
• Cloud Security Adherence: Apply enterprise data governance frameworks and cloud infrastructure security best practices to protect sensitive data assets.
• Team Mentorship: Collaborate with cross-functional software architects and business analysts while providing proactive technical guidance and mentoring to junior data engineers.

Required Qualifications & Technical Stack

• Professional Experience: 2 to 5 years of solid hands-on data engineering track record specifically focused on distributed cloud environments and big data management, with a minimum of 2 years dedicated to Databricks environments.
• Core Technical Toolkit: Advanced technical proficiency in Python programming, PySpark scripting, and writing optimized SQL database queries.
• Preferred Assets: An active Azure Databricks Certification and prior experience successfully delivering data engineering solutions within a technology consulting environment are highly preferred.
• Core Soft Skills: Exceptional problem-solving capabilities, clear communication skills, and the proven ability to drive technical deliverables independently or as part of an agile scrum squad.