Crawl Result Models and Import Pipeline

Now that we've covered Web Scraping with Scrapy and Django, our spider can extract data from a webpage.

That is useful, but it is still temporary. If the result only appears in the terminal or a JSON file, our Django app cannot review it later, display it to a user, or build product features on top of it.

In this lesson, we'll add a small import pipeline:

Scrapy spider
   ↓ exports crawl result
JSONL file
   ↓ Django import command
Django models
   ↓ admin / future UI
Reviewable crawl history

This is the startup-engineer move: turn a script into product data.