Published onOctober 14, 2023 (1y ago)Web Crawling at Scale: Navigating Billions of URLs with EfficiencyKubernetesWeb-CrawlerGolangNodejsDistributed-SystemDive into the world of distributed web crawling with Golang, Docker, and Redis. Learn the logic behind efficient code, use Bloom filters for...
Published onOctober 13, 2023 (1y ago)The Architecture of a Web Crawler: Building a Google-Inspired Distributed Web Crawler. Part 1KubernetesWeb-CrawlerGolangNodejsDistributed-SystemUnlock the potential of the web with a Google-inspired distributed web crawler. Explore scalable solutions using Kubernetes, Golang, Python,...
Published onJune 11, 2023 (1y ago)A Step-by-Step Guide to Building a Scalable Distributed Crawler for Scraping Millions of Top TikTok ProfilesKubernetesWeb-CrawlerGolangNodejsDistributed-SystemEmbark on a comprehensive journey to construct a powerful TikTok scraper using Golang, Docker, and Kubernetes. Gain insights into website an...