American Eagle Outfitters — американский ретейлер одежды и аксессуаров со штаб квартирой в Питтсбурге, Пенсильвания. Бренд был основан в 1977 году братьями Джеем и Марком Сильверман. Используя этот бесплатный парсер, вы сможете собрать информацию о товарах, представленных на сайте ae.com.
Примерное количество товаров: 10000
Примерное количество запросов: 20000
Рекомендуемый план подписки: X-Small
ВНИМАНИЕ! Количество запросов может превышать количество товаров, потому что данные о вариациях, изображениях и др. могут парсится используя запросы к дополнительным ресурсам. Также часть данных о товаре может доставляться с помощью XHR запросов, что также увеличивает общее количество необходимых запросов.
Для его использования вы должны иметь учетную запись в нашем сервисе Diggernaut.
- Пройдите по этой ссылке для регистрации в сервисе Diggernaut
- После регистрации и подтверждения email адреса войдите в свою учетную запись
- Создайте проект с любый именем и описанием, если вы не знаете как, обратитесь к нашей документации
- Войдите во вновь созданный проект и создайте в нем диггер с любым именем, если вы не знаете как, обратитесь к нашей документации
- Скопируйте в буфер обмена приведенный ниже сценарий диггера и вставьте его в созданный вами диггер, если вы не знаете как, обратитесь к нашей документации
- Переключите режим работы диггера с Debug на Active, если вы не знаете как, обратитесь к нашей документации
- Запустите ваш диггер и дождитесь окончания его работы, если вы не знаете как, обратитесь к нашей документации
- Скачайте собранный набор данных в нужном вам формате, если вы не знаете как, обратитесь к нашей документации
В дальнейшем вы можете установить расписание для запуска вашего парсера и забирать информацию регулярно.
Сценарий парсера:
---
config:
debug: 2
do:
- pool_clear: pages
- walk:
to: https://www.ae.com
do:
- find:
path: a.third-level,a.second-level-item
do:
- parse:
attr: href
- normalize:
routine: url
- link_add:
pool: catalog
- walk:
to: links
pool: catalog
do:
- find:
path: div.product-tile>div.product-details-container>a[data-qa-link-to="pdp"]
do:
- parse:
attr: href
filter: ^([^\?]+)
- normalize:
routine: url
- link_add:
pool: pages
- walk:
to: links
pool: pages
do:
- sleep: 3
- find:
path: div.container-fluid[itemtype="http://schema.org/Product"]
do:
- variable_clear: pid
- object_new: product
- eval:
routine: js
body: '(function (){var d = new Date(); return d.toISOString()})();'
- object_field_set:
object: product
field: date
- static_get: url
- object_field_set:
object: product
field: url
- find:
path: h1.psp-product-name
do:
- parse
- space_dedupe
- trim
- object_field_set:
object: product
field: name
- find:
path: span.pdp-about-cs
do:
- parse
- space_dedupe
- trim
- object_field_set:
object: product
field: sku
- find:
in: doc
path: script:contains("var $AWP_ENV")
do:
- parse:
filter: var\s+\$AWP_ENV\s+\=\s+(.+)\s*$
- normalize:
routine: json2xml
- to_block
- find:
path: body_safe>viewdata>availableproducts
do:
- find:
path: brandname
slice: 0
do:
- parse
- space_dedupe
- trim
- object_field_set:
object: product
field: brand
- find:
path: colorname
do:
- parse
- space_dedupe
- trim
- if:
match: \w+
do:
- object_field_set:
object: product
joinby: "|"
field: variations
- find:
path: largeimages
do:
- parse
- space_dedupe
- trim
- if:
match: \w+
do:
- normalize:
routine: url
- variable_set: iurl
- register_set: "<%iurl%>?scl=1"
- object_field_set:
object: product
joinby: "|"
field: images
- find:
path: span.psp-product-regularprice[itemprop="priceCurrency"]
do:
- parse:
attr: content
- object_field_set:
object: product
field: currency
- find:
path: span.psp-product-regularprice[itemprop="price"]
do:
- parse:
attr: content
- object_field_set:
object: product
type: float
field: price
- find:
path: span.psp-product-saleprice[itemprop="priceCurrency"],span.psp-product-sale-currency[itemprop="priceCurrency"]
do:
- parse:
attr: content
- object_field_set:
object: product
field: currency
- find:
path: span.psp-product-saleprice[itemprop="price"]
do:
- parse:
attr: content
- object_field_set:
object: product
type: float
field: price
- find:
in: doc
path: div.pdp-about-details-equit
slice: 0
do:
- parse
- space_dedupe
- trim
- object_field_set:
object: product
field: description
- find:
in: doc
path: ol.breadcrumb>li.bc-item
slice: 0:-2
do:
- variable_clear: list_item
- parse
- space_dedupe
- trim
- if:
match: \w{2,}
do:
- object_field_set:
object: product
joinby: "|"
field: category
- object_save:
name: product
Ниже приведен пример датасета с несколькими товарами в формате JSON (для наглядности). Датасет может быть скачан и как CSV, XLSX, XML, и любой другой текстовый формат используя темплейтный подход.
[{
"product": {
"brand": "AEO",
"category": "Women|Featured|Best Sellers",
"currency": "USD",
"date": "2017-12-05T09:15:17.642Z",
"description": "Sweater weather forever.",
"images": "https://s7d2.scene7.com/is/image/aeo/0341_7976_020_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_020_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_020_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_020_b?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_020_d1?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_410_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_410_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_410_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_410_b?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_410_d1?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_639_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_639_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_639_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_639_b?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_951_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_951_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_951_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_951_b?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_7976_951_d1?scl=1",
"name": "AE Chenille Balloon-Sleeve Sweater",
"price": 26.97,
"url": "https://www.ae.com/women-gifts-ae-chenille-balloon-sleeve-sweater-light-gray/web/s-prod/0341_7976_951",
"variations": "Gray|Navy|Rose|Light Gray"
}
}
,{
"product": {
"brand": "AEO",
"category": "Women|Featured|Best Sellers",
"currency": "USD",
"date": "2017-12-05T09:15:37.475Z",
"description": "Sweater weather forever.",
"images": "https://s7d2.scene7.com/is/image/aeo/0341_8032_401_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_401_os?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_401_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_401_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_401_b?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_401_d1?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_500_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_500_os?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_500_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_500_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_500_b?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_500_d1?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_639_of?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_639_os?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_639_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_639_f?scl=1|https://s7d2.scene7.com/is/image/aeo/0341_8032_639_b?scl=1",
"name": "AE Soft & Pretty Sweater",
"price": 24.99,
"url": "https://www.ae.com/women-gifts-ae-soft-pretty-sweater-rose/web/s-prod/0341_8032_639",
"variations": "Light Blue|Purple|Rose"
}
}
,{
"product": {
"brand": "AEO",
"category": "Women|Featured|New Arrivals|Tops|Sweaters",
"currency": "USD",
"date": "2017-12-05T09:15:58.845Z",
"description": "Sweater weather forever.",
"images": "https://s7d2.scene7.com/is/image/aeo/1341_7951_020_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_020_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_020_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_020_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_401_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_401_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_401_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_401_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_410_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_410_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_410_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_410_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_580_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_580_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_580_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_613_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_613_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_613_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7951_613_b?scl=1",
"name": "AE Waffle Knit Turtleneck Tunic",
"price": 32.97,
"url": "https://www.ae.com/women-gifts-ae-waffle-knit-turtleneck-tunic-gray/web/s-prod/1341_7951_020",
"variations": "Gray|Light Blue|Navy|Lavender|Burgundy"
}
}
,{
"product": {
"brand": "AEO",
"category": "Women|Featured|New Arrivals|Tops|Sweaters",
"currency": "USD",
"date": "2017-12-05T09:16:41.537Z",
"description": "Sweater weather forever.",
"images": "https://s7d2.scene7.com/is/image/aeo/1341_7947_286_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7947_286_os?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7947_286_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7947_286_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7947_286_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7947_286_d1?scl=1",
"name": "AE Destroyed Cutout-Collar Sweater",
"price": 26.97,
"url": "https://www.ae.com/women-gifts-ae-destroyed-cutout-collar-sweater-oatmeal/web/s-prod/1341_7947_286",
"variations": "Oatmeal"
}
}
,{
"product": {
"brand": "AEO",
"category": "Women|Featured|Statement Sleeves",
"currency": "USD",
"date": "2017-12-05T09:17:09.343Z",
"description": "Sweater weather forever.",
"images": "https://s7d2.scene7.com/is/image/aeo/1341_7819_008_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_008_os?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_008_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_008_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_008_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_106_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_106_os?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_106_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_106_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_106_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_309_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_309_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_628_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_628_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_628_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_628_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_639_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_639_os?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_639_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_639_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7819_639_b?scl=1",
"name": "AE Zip-Back Bell Sleeve Sweater",
"price": 26.97,
"url": "https://www.ae.com/women-ae-zip-back-bell-sleeve-sweater-neon-pink/web/s-prod/1341_7819_628",
"variations": "Charcoal|Cream|Olive|Neon Pink|Rose"
}
}
,{
"product": {
"brand": "AEO",
"category": "Women|Featured|New Arrivals|Tops|Sweaters",
"currency": "USD",
"date": "2017-12-05T09:17:29.820Z",
"description": "Sweater weather forever.",
"images": "https://s7d2.scene7.com/is/image/aeo/1341_7808_001_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7808_001_b?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7808_300_of?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7808_300_os?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7808_300_ob?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7808_300_f?scl=1|https://s7d2.scene7.com/is/image/aeo/1341_7808_300_b?scl=1",
"name": "AE Chenille Side-Tie Sweater",
"price": 26.97,
"url": "https://www.ae.com/women-ae-chenille-side-tie-sweater-green/web/s-prod/1341_7808_300",
"variations": "Black|Green"
}
}
]