Scraping Archives | devharish

AWS Azure Data Engineering Laravel Postgres Python Scraping Spark SQL

Data Integration Platform

This is a comprehensive data integration and ETL platform designed to serve as a backend pipeline for a Business Intelligence (BI) platform. The system automates the process of ingesting, cleaning, and loading large volumes of data from various sources into a format optimized for analytics consumption. A key component of this platform is a custom-built […]

News aggregator platform

This is a robust news aggregator platform designed to gather and present industry-specific news from various online publishers. The platform leverages Google News RSS feeds to fetch daily news articles tailored to specific industries or keywords. Core Features: News Scraping: Automatically scrapes news articles from the RSS feeds of various publishers. News Cleaning: Processes raw […]