Feed ETL & Data-Ingestion Worker
An influencer-marketing media-intelligence platform
Overview
A focused ETL worker for the platform that pulls and parses content feeds and pushes normalised data downstream, packaged for Kubernetes deployment.
The Challenge
Content arrives as heterogeneous feeds that must be fetched, parsed and normalised reliably before the core engine can index it. This job isolates that extraction concern into a small, independently deployable worker.
What We Built
A Node.js worker (do.js) built around the @gaphub/feed parser, with kubeconfig.yaml and a get_helm.sh bootstrap for cluster deployment. Deliberately minimal, a single-purpose ETL step in the larger ingestion pipeline.
Technologies & Approach
Plain Node.js for low overhead, a dedicated feed-parsing library, and Helm/Kubernetes for deployment alongside the rest of the platform infrastructure.
Outcome / Impact
Provided a clean, isolated feed-ingestion stage that fed the core engine, keeping extraction concerns decoupled from the main backend.
Capabilities Demonstrated
- Building small, single-purpose ETL workers
- Feed/RSS parsing and normalisation
- Packaging Node jobs for Kubernetes/Helm deployment