About
Highly skilled Data Engineer with 3+ years of experience in designing, developing, and optimizing Azure-based ETL pipelines using ADF, Databricks (PySpark), and SQL. Proven ability to drive significant performance improvements, evidenced by reducing pipeline runtime by 50% and optimizing PySpark jobs by 40%, while delivering near real-time analytics. Expertise spans end-to-end pipeline ownership, including incremental loads, robust data quality checks, and proactive monitoring, ensuring scalable and reliable data workflows.
Work
CheqIT Pvt Ltd
|Data Engineer
Pune, Maharashtra, India
→
Summary
Led the migration of FMCG Sales Analytics workflows from Alteryx to Azure Databricks, significantly enhancing data ingestion, processing, and real-time insights for 5+ regional sources.
Highlights
Led migration of 15+ Alteryx workflows to Azure Data Factory (ADF) and Databricks pipelines, automating ingestion of ~500GB/day of sales and inventory data from 5+ regional sources.
Replaced multiple Alteryx macros with parameterized ADF datasets, reducing maintenance effort by 80% and enabling a single reusable pipeline.
Optimized Databricks workflows via `repartition()` and `filter pushdown`, converting Alteryx transformations into PySpark jobs for data cleaning, which improved transformation performance by 40%.
Implemented incremental loads with watermarking, decreasing data ingestion runtime by 50% (from 6 hours to 3 hours), and established robust pipeline monitoring & alerting with Azure Monitor + MS Teams, reducing failure recovery time from 2 hours to 20 minutes.
Achieved significant operational improvements, reducing manual interventions by 80%, data failures by 30%, and reporting delays by 85% for near real-time FMCG sales insights.
IBM Pvt Ltd
|Application Developer
Bengaluru, Karnataka, India
→
Summary
Designed and deployed robust SQL Server ETL workflows using SSIS, efficiently handling millions of daily transactions and optimizing database performance.
Highlights
Designed and deployed SQL Server ETL workflows utilizing SSIS to handle millions of daily transactions efficiently.
Optimized SQL queries and stored procedures, enhancing data retrieval speed by 30% and supporting faster reporting.
Accenture Pvt Ltd
|Associate Software Engineer
Bengaluru, Karnataka, India
→
Summary
Contributed to enhancing application reliability and code quality by implementing JUnit test cases and resolving SonarQube issues within agile development cycles.
Highlights
Improved application reliability by implementing JUnit test cases, effectively reducing bugs by 30%.
Resolved SonarQube issues across agile sprints, significantly increasing overall code quality.
Education
Truba Institute of Engineering and Information Technology
→
Bachelor of Engineering
Information Technology
Languages
English
Certificates
Azure Fundamentals (AZ-900)
Issued By
Microsoft Certified
Storytelling Using Power BI
Issued By
Analytics Vidhya
Storytelling Using Tableau
Issued By
Analytics Vidhya
Microsoft Excel
Issued By
Microsoft
SQL for Data Science
Issued By
Online Course
Skills
Software Development & Testing
JUnit, SonarQube, Application Reliability, Agile Methodologies.
Cloud & Big Data
Azure Data Factory, Azure Databricks, ADLS Gen2, Data Migration, Workflow Automation, Pipeline Development.
Programming & Scripting
Python, PySpark, SQL.
Databases
SQL Server, Azure SQL Database.
ETL & Data Integration
ADF Pipelines, Parameterization, Incremental Loads, Schema Drift Handling, SSIS.
Big Data Processing & Optimization
PySpark Transformations, Partitioning, Caching, Optimization, Filter Pushdown, Performance Tuning, Real-time Analytics.
DevOps & Monitoring
Git, Azure DevOps, Azure Monitor, Log Analytics, MS Teams Alerts, Pipeline Monitoring.
Data Quality & Governance
Data Cleaning, Data Quality Checks, Error Handling, Data Consistency, Watermarking.
Reporting & Visualization
Power BI, Tableau, Microsoft Excel.