Skip to main content

/scrape-js examples

Make screenshot of a web page​

By default, /scrape-js engine makes screenshot of the target web page and it can be accessed through .info.screenshot property of the response JSON.


curl https://scrapeninja.p.rapidapi.com/scrape-js \
-H "Content-Type: application/json" \
-H "X-Rapidapi-Key: YOUR-KEY" \
-d '{
  "url": "https://example.com/product", 
  "waitForSelector":".price"
  }'  | jq '.info.screenshot'

https://cdn.scrapeninja.net/screenshots/website-screenshot.png

Screenshot functionality can be disabled by passing screenshot: 0 into the payload (see high performance example).

Intercept AJAX call made by website and dump headers​

curl --request POST \
    --url https://scrapeninja.p.rapidapi.com/scrape-js \
    --header 'X-RapidAPI-Host: scrapeninja.p.rapidapi.com' \
    --header 'X-RapidAPI-Key: YOUR-KEY' \
    --header 'content-type: application/json' \
    --data '{
    "url": "https://www.virginatlantic.com/flight-search/book-a-flight",
    "waitForSelector": ".advanced-search-heading",
    "geo": "de",
    "retryNum": 2,
    "catchAjaxHeadersUrlMask": "air-shopping/advancesearch.data.json",
    "blockImages": 1
}'

Dump iframe data​

curl --request POST \
    --url https://scrapeninja.p.rapidapi.com/scrape-js \
    --header 'X-RapidAPI-Host: scrapeninja.p.rapidapi.com' \
    --header 'X-RapidAPI-Key: YOUR-KEY' \
    --header 'content-type: application/json' \
    --data '{
    "url": "https://jobs-airdc.icims.com/jobs/search?ss=1",
    "geo": "us",
    "waitForSelectorIframe": ".iCIMS_PagingBatch",
    "dumpIframe": "icims_content_iframe",
    "extractorTargetIframe": true,
    "extractor": "function(input, cheerio) {\n    // return object with extracted values              \n    let c = cheerio.load(input);\n    return { jobs: c('\''.iCIMS_JobsTable .row'\'').map(function() { let r2 = c(this).find('\''.row .title'\''); return { title: r2.find('\''h2'\'').text().trim(), href: r2.find('\''a'\'').attr('\''href'\'') } } ).toArray() };  \n}",
    "timeout": 20
}'

High performance​

/scrape-js is generally much slower than the /scrape engine, as /scrape-js by default downloads all images and media available on a page, potentially executing 50-100 HTTP requests before the page is considered fully loaded, while /scrape makes exactly one HTTP request. The difference becomes even greater when using private proxies, which in many cases are not super fast. To reduce the latency and traffic used by /scrape-js, disable screenshots, images, and media downloads:

curl --request POST \
    --url https://scrapeninja.p.rapidapi.com/scrape-js \
    --header 'X-RapidAPI-Host: scrapeninja.p.rapidapi.com' \
    --header 'X-RapidAPI-Key: YOUR-KEY' \
    --header 'content-type: application/json' \
    --data '{
    "url": "https://news.ycombinator.com/",
    "retryNum": 1,
    "blockImages": true,
    "blockMedia": true,
    "screenshot": 0
}'