I am a Software Engineer by profession.
I write code for my living
When I am not doing all these, I can be found watching movies, playing gully cricket
Working on scalable data platforms, Data Warehousing, ETL, large scale web scraping, stream processing
Designed and developed data collection pipeline to crawl fashion websites. Developed ETL pipelines to ingest the crawl data to data lake. Trained and deployed machine learning models using google cloud vision for multi-label classification of fashion clothing. Product was appreciated from investors and advisors
Developed Data Cleaning Pipeline processing gigabytes of data. Implemented Daily Automated Report Generation using SQL. Reduced Human Effort by 15%
Designed and developed end to end small scale crawler platform crawling 30-40 websites daily. Developed ETL workflows for the extraction and transformation of the crawled data. Developed REST API for the 3rd party services to consume the crawled data.
Website Link - Samarpana Charitable Trust