Lead-Data Scraper & Exporter for E-commerce Store Intelligence
A social storytelling / lead-gen platform
Overview
A set of Node.js scripts that harvest e-commerce store leads from a store-intelligence data source, segment them by platform, and export structured CSV datasets, feeding the lead-generation side of a social storytelling / lead-gen platform.
Why It Exists
Building a lead pipeline meant pulling lists of online stores (with their tech stack and metadata) from a store-intelligence service, then turning the paginated, filtered results into clean, usable CSVs segmented by e-commerce platform.
What We Built
A small Node.js (ES modules) toolset: shopify.js and prestashop.js page through the source service’s authenticated JSON domains endpoint, sending the appropriate headers, filters (e.g. country, platform) and pagination parameters, to collect store records. Companion scripts summarise-shopify.js/sumarise.js aggregate and summarize the harvested data, and the results are written out as platform-segmented CSV exports (domains-shopify.csv, domains.csv) plus per-platform output folders. The design isolates each storefront platform (Shopify, PrestaShop) into its own collection and summarization path.
Technologies & Approach
Plain Node.js with fetch and async pagination kept the scraper lightweight and easy to run. The work focused on correctly driving a filtered, paginated data API, handling its session/auth requirements, and shaping the output into segmented CSV datasets ready for downstream lead-gen use.
Outcome / Impact
Produced reusable store-intelligence datasets, segmented by e-commerce platform, to power lead generation, demonstrating practical web-data extraction and dataset-engineering capability.
Capabilities Demonstrated
- Authenticated, paginated web-data harvesting
- Platform-segmented dataset generation
- CSV export and summarization pipelines
- Lightweight Node.js automation/RPA scripting