Text-Classification Inference Microservice (spaCy)

Overview

A lightweight Python inference microservice that serves the platform’s trained text-classification models. It loads multiple spaCy models from disk and exposes a small HTTP API to score arbitrary text, returning per-category confidence so the media-intelligence product can auto-classify content.

The Challenge

The product needed to apply trained NLP classifiers to incoming text at runtime, ideally across several models at once, behind a simple, deployable API that fit into a containerized AWS pipeline.

What We Built

A hug-based HTTP service (index.py) that walks a ./models directory, loads every spaCy model it finds, and exposes three endpoints: /input (score text against a named model), /input-all (run text through every loaded model and return a per-model result map), and /models (list available models). The service returns spaCy doc.cats text-classification scores. It ships with a Dockerfile, a Gunicorn run script, a Makefile, an .aws deployment config, and a GitLab CI pipeline.

Technologies & Approach

Python with spaCy 3.0 for inference, hug for the API layer, and Gunicorn for serving. Packaged as a Docker image and wired for AWS deployment through GitLab CI. The multi-model loading pattern lets new classifiers be dropped in without code changes.

Outcome / Impact

Provided the runtime serving layer that turned the project’s trained spaCy classifiers (see the model-training repo) into a callable service, enabling automated content categorization within the platform.

Capabilities Demonstrated

• Building and deploying an ML model-serving microservice
• Dynamic multi-model loading and batched inference over a single API
• Containerized, CI-driven deployment of Python ML services to AWS
• Productionizing spaCy text classifiers behind a clean HTTP interface