Databricks Engineer

Infosys Limited is seeking a highly skilled and results-driven Databricks Engineer to join our elite Data & Analytics Unit in Bangalore. This core data engineering role centers on developing high-performance data processing pipelines, leveraging distributed systems, and architecting modern data lakehouse solutions on the Databricks unified analytics platform.

Key Responsibilities & Scope of Work

• Databricks Platform Engineering: Design, implement, and maintain highly scalable data workloads and compute environments utilizing the native capabilities of the Databricks platform.
• Distributed Data Processing: Architect and optimize complex big data processing scripts using Apache Spark frameworks, writing proficient code across PySpark, Scala, and Spark SQL interfaces.
• Pipeline Architecture (ETL/ELT): Build, orchestrate, and manage end-to-end automated data ingestion and transformation pipelines to integrate diverse, large-scale structural and unstructured datasets.
• Data Warehousing & Modeling: Formulate and deploy advanced data modeling techniques to construct robust enterprise data warehouses, ensuring schema optimization, reliability, and low-latency storage.
• Performance Optimization: Monitor distributed execution workflows, tune Spark configurations, and optimize query workloads to achieve superior resource utilization and cost efficiency.

Required Qualifications & Technical Stack

• Professional Experience: 3 to 5 years of solid hands-on data engineering experience specifically focused on distributed cloud environments and big data management.
• Core Technical Stack: Mastery of the Databricks architecture combined with high proficiency in Python programming and advanced SQL database scripting.
• Domain Capabilities: Deep foundational knowledge of enterprise distributed computing topologies, data lake principles, and analytical problem-solving to translate business logic into high-velocity production data streams.