URL and column-spec scraper
The problem
You need one site scraped into a spreadsheet, with the columns named up front and proof that no column came back silently empty.
The artefact
A small CLI that takes a URL and a JSON column spec, fetches the page with Playwright, extracts one row per element matched by the spec row selector, and writes CSV plus a short run log. The log records rows found, empty-field counts per column, and failures. The artefact README, spec example, and source are copied under files on this page.
Input and output sample
Input spec:
{
"row_selector": ".book",
"columns": {
"title": {"selector": ".title", "required": true},
"price": {"selector": ".price"},
"link": {"selector": "a", "attribute": "href"}
}
}
Output CSV, from the bundled fixture with three rows:
title,price,link Alpha Title,10.50,/books/alpha Beta Title,12.00,/books/beta Gamma Title,,/books/gamma
Output run log:
rows found: 3 empty fields per column: title: 0 price: 1 link: 0 failures: none
What can break
- Selector drift after a site change, so a column matches nothing
- A required column returning empty while the run still exits clean
- Values moving between text content and attributes
- Pagination that loops or stops early
The fix or design choice
A column marked required fails the run when any row leaves it empty, so a silent selector break shows up as a failed run with exit code 2 instead of a blank column shipped downstream. Exit codes: 0 success, 2 required column empty, 1 other failure. Before pointing the CLI at a real site, each selector gets verified against the live DOM.
Verification log
- 2026-11-19: test suite run from a clean shell, 6 passed
- Tests cover parsing, empty-field counts, the required-field guard both ways, CSV output, and a full CLI run, all offline against the bundled fixture
Verified 2026-11-19
Files
Price shape
Scrapers from $300 fixed. A paid diagnostic at $150 to $300 covers triage first: selectors that still match, a sample CSV, and a failure log, with the acceptance check in writing before work starts.
Start here
Email the target URL and the exact columns you need. I reply with the fixed quote and the first sample plan.