140 lines
6.1 KiB
Markdown
140 lines
6.1 KiB
Markdown
# kompress e-shop scrape
|
||
|
||
Snapshot of the live catalogue at <http://e-shop.kompress.sk> (PrestaShop 1.5/1.6),
|
||
taken 2026-07-29. Only the product catalogue is captured — no CMS pages, no orders.
|
||
|
||
Re-run with `python3 scrape.py` (stdlib only, no dependencies). It is idempotent:
|
||
already-downloaded images are skipped.
|
||
|
||
## Files
|
||
|
||
| File | Contents |
|
||
| --- | --- |
|
||
| `categories.json` | 30 categories, flat list ordered parent-before-child |
|
||
| `products.json` | 118 products, ordered by PrestaShop id |
|
||
| `images/<product_id>/<image_id>.jpg` | 181 product images, original uploads |
|
||
| `images/categories/<category_id>.jpg` | 30 category thumbnails, original uploads |
|
||
|
||
## Category schema
|
||
|
||
```jsonc
|
||
{
|
||
"id": 8, // PrestaShop id_category, kept as the stable key
|
||
"name": "Gáza, role, štvorce, prírezy",
|
||
"title": "…", // link title attr; often a longer marketing blurb
|
||
"url": "http://e-shop.kompress.sk/8-gaza-role-…",
|
||
"parent_id": 7, // null for top-level
|
||
"path": ["Zdravotnícke", "Gáza, role, štvorce, prírezy"],
|
||
"depth": 1, // 0 = top level
|
||
"description_html": "<p>…</p>",
|
||
"description": "…", // plain-text rendering
|
||
"image": { "url": "…/img/c/8.jpg", "file": "images/categories/8.jpg" },
|
||
"product_ids": [59, 62, …],
|
||
"product_count": 8
|
||
}
|
||
```
|
||
|
||
The tree is 2 levels deep: 3 roots (`Zdravotnícke`, `Somatické`, `Pre invalidov`),
|
||
27 children. The roots hold a few products directly; `Somatické` and `Pre invalidov`
|
||
hold none of their own.
|
||
|
||
## Product schema
|
||
|
||
```jsonc
|
||
{
|
||
"id": 73, // PrestaShop id_product
|
||
"name": "Návlek na obuv",
|
||
"url": "http://e-shop.kompress.sk/-netkane-textie-…/73-navlek-na-obuv.html",
|
||
"reference": "6811", // the "Kód:" on the page; "" when the shop has none
|
||
"price": 0.04, // tax included, EUR — this is the displayed price
|
||
"price_display": "0,04 €",
|
||
"price_tax_excluded": 0.033333,
|
||
"price_without_reduction": 0.04,
|
||
"tax_rate": 20.0,
|
||
"currency": "EUR",
|
||
"quantity_available": 1000,
|
||
"available_for_order": true,
|
||
"has_attributes": false, // true when the product has variant combinations
|
||
"attribute_groups": [], // see "Variants" below
|
||
"variants": [],
|
||
"short_description_html": "<p>…</p>",
|
||
"short_description": "…",
|
||
"description_html": "<p>…</p><ul><li>…</li></ul>",
|
||
"description": "…", // plain text, list items prefixed with •
|
||
"breadcrumb": [{ "id": 7, "name": "Zdravotnícke", "url": "…" }, …],
|
||
"category_path": ["Zdravotnícke", " Netkané textíe, Pervin, SMS - …"],
|
||
"categories": [{ "id": 10, "name": "…", "path": [...] }], // every category listing it
|
||
"default_image_id": 279,
|
||
"images": [{ "id": 279, "url": "…/img/p/2/7/9/279.jpg",
|
||
"file": "images/73/279.jpg", "bytes": 53443 }]
|
||
}
|
||
```
|
||
|
||
`breadcrumb`/`category_path` is the product's primary category trail. `categories`
|
||
lists every category the product is actually listed in — 7 products appear in more
|
||
than one, so treat product↔category as many-to-many.
|
||
|
||
## Variants
|
||
|
||
20 of the 118 products sell in several sizes/packagings — 74 combinations in total,
|
||
across 4 attribute groups: `Rozmer`, `Veľkosť`, `Balenie`, `Počet útržkov v balení`.
|
||
Each is its own purchasable unit with its own code, price and stock, so these map to
|
||
`product_variants` rather than to plain products.
|
||
|
||
```jsonc
|
||
"attribute_groups": [
|
||
{ "id": 4, "name": "Rozmer", "slug": "rozmer",
|
||
"attributes": [ { "id": 21, "label": "10cm x 10cm", "default": false }, … ] }
|
||
],
|
||
"variants": [
|
||
{
|
||
"id": 30, // id_product_attribute
|
||
"attribute_ids": [23],
|
||
"label": "5cm x 5cm", // groups joined with " / " when there are several
|
||
"attributes": [{ "group": "rozmer", "label": "5cm x 5cm" }],
|
||
"reference": "1211", // per-variant Kód; "" on some products
|
||
"quantity_available": 500,
|
||
"price_impact_tax_excluded": -2.35,
|
||
"price_tax_excluded": 1.04,
|
||
"price": 1.14, // tax included — what the shop shows when selected
|
||
"minimal_quantity": 1,
|
||
"image_id": 100, // null when the variant has no own image
|
||
"is_default": true
|
||
}
|
||
]
|
||
```
|
||
|
||
`price` is computed as `(product.price_tax_excluded + price_impact_tax_excluded) *
|
||
(1 + tax_rate/100)`, matching `themes/default/js/product.js`. Verified: every
|
||
product's displayed price equals its default variant's computed price, and every
|
||
product's `quantity_available` equals the sum of its variants' stock.
|
||
|
||
On variant products the top-level `price`/`reference` describe the **default**
|
||
variant, and `quantity_available` is the total across variants. 14 of the 74
|
||
variants have an empty `reference`.
|
||
|
||
## Images
|
||
|
||
`images[].url` points at PrestaShop's untouched original upload
|
||
(`/img/p/<digits split by slash>/<image_id>.jpg`), not a generated thumbnail. For
|
||
product 73 that is 53 KB versus 10 KB for the `-large_default` the page shows. Every
|
||
downloaded file was verified to be a real JPEG; sizes run 3 KB – 634 KB.
|
||
|
||
Product 270 (`Plachta na posteľ z PVC`) has no image in the shop at all — its
|
||
`images` array is empty and `default_image_id` is null.
|
||
|
||
## Coverage notes
|
||
|
||
- Product URLs came from crawling every category listing, unioned with `sitemap.xml`.
|
||
All category listings fit on a single page, so nothing was lost to pagination.
|
||
- `sitemap.xml` is stale: it advertises 182 products, but 83 of those (ids 8–55,
|
||
72, 86, 96, 103–105, 109, 118–197 among them) return HTTP 404 — they are
|
||
disabled/deleted products, verified individually via
|
||
`index.php?id_product=<id>&controller=product`. 118 live products remain.
|
||
- Some products are served under both an old and a new slug (e.g. product 59 as
|
||
`…59-gaza-hydrofilna-skladana-zlozky-z-gazy.html` and `…59-gaza-hydrofilna-skladana.html`).
|
||
These are deduplicated by id, keeping the URL the category listing links to.
|
||
- 29 products have an empty top-level `reference` — that is genuine, the shop leaves
|
||
the Kód field blank for them (the block is rendered `display:none`). 20 of those 29
|
||
12 of those 29 are variant products, where the code lives on each variant instead.
|