PySpark Developer

Infosys Limited is seeking a highly skilled and analytical PySpark Developer to join our elite Data & Analytics Unit in Bangalore. This comprehensive data engineering and delivery leadership role focuses on architecting modern big data processing pipelines, managing cloud-based distributed system infrastructures, and guiding engineering teams to deliver transformative digital solutions.

Key Responsibilities & Scope of Work

• Team Management & Collaboration: Lead, mentor, and govern a cross-functional squad of technical professionals—including Data Engineers and Application Developers—to ensure optimal collaboration, upskilling, and delivery productivity.
• Client Interfacing & Project Delivery: Act as the primary technical point of contact for enterprise clients, meticulously mapping business requirements into scalable architectures and overseeing the full project lifecycle from planning to execution within scope and budget boundaries.
• Data Pipeline Engineering: Build, scale, and optimize high-volume automated data pipelines (ETL/ELT) using PySpark and core Apache Spark environments to ingest, transform, and clean massive structured and unstructured datasets.
• Status Reporting & Risk Management: Prepare and present highly functional, detailed operational progress reports to key corporate stakeholders, proactively highlighting project milestones, potential engineering risks, and definitive mitigation plans.
• Technical Problem Solving & Troubleshooting: Isolate backend database roadblocks, execute complex root-cause diagnostics across cloud frameworks, and maintain smooth project execution through stable configuration management toolsets.
• Process Innovation & Financial Sizing: Assess current computing processes, identify operational performance gaps, recommend modern technology enhancements to increase business profitability, and apply solid knowledge of enterprise pricing models.

Required Qualifications & Technical Stack

• Core Tech Domain Mastery: 2 to 5 years of proven hands-on track record operating inside data engineering or cloud analytics landscapes with specialized depth in PySpark and Big Data processing.
• Cloud Platform Infrastructure: Practical experience working with unified Databricks environments and deploying components inside public cloud networks, specifically Amazon Web Services (AWS) layers.
• Core Soft Skills: Strong logical thinking skills, advanced software configuration management awareness, multi-industry domain knowledge, exceptional communication capabilities, and natural team leadership attributes.