Big Data - Senior Engineer

• Design scalable Big Data solutions using Apache Spark, Hadoop, Azure Databricks, and modern data platform technologies.

• Define data processing architecture, transformation strategies, and engineering standards aligned with business objectives.

• Lead development of distributed data processing pipelines using Spark (Scala or PySpark) and Hadoop ecosystem technologies.

• Design and optimize SQL and Hive-based data processing solutions to improve performance and scalability.

• Architect and optimize Azure Databricks solutions supporting large-scale data engineering and analytics workloads.

• Design and implement data Lakehouse solutions leveraging Apache Hudi or Apache Iceberg.

• Establish data ingestion, transformation, validation, and reconciliation frameworks to improve data reliability.

• Drive performance tuning initiatives across Spark jobs, Databricks workloads, Hive queries, and Hadoop processing environments.

• Review data engineering solutions to ensure adherence to architecture, performance, and engineering standards.

• Troubleshoot complex data processing, performance, and platform issues through detailed root cause analysis.

• Mentor team members on Spark, Hadoop, Databricks, Hudi/Iceberg, SQL optimization, and Big Data engineering best practices.

• Collaborate with various teams and stakeholders to support end-to-end data platform delivery.

• Drive continuous improvement initiatives focused on scalability, performance, reliability, and operational efficiency.

• Demonstrates strong ownership while driving Big Data Engineering excellence.

• Collaborate effectively with various teams and business stakeholders to ensure smooth delivery.

• Promotes quality-focused engineering through proactive validation, optimization, and continuous improvement.

• Applies strong analytical thinking to evaluate complex data engineering and platform challenges.

• Demonstrate adaptability while managing evolving technologies, data ecosystems, and business requirements.

• Communicates effectively regarding delivery status, risks, dependencies, and improvement opportunities.

• Maintains high attention to detail across data architecture, processing design, testing, and implementation activities.

• Encourages continuous improvement in data engineering practices and platform operations.

• Supports knowledge sharing and mentoring to strengthen team capabilities.

• Balances scalability, performance, reliability, and business priorities while driving delivery excellence.