Get started right from your browser: execute scraping requests, both JS and non-JS, submitting the form below.
Quick examples: auto.de results scraping    HackerNews basic    Airlines dump AJAX headers    POST JSON    Click & dump JSON
 
 
Advanced. Can be used to dump sub-request. See airlines scraper example. Use with caution: bad URL mask will result in timeouts because scraper will be waiting for the AJAX call until the timeout, and won't be able to dump required data! Explore .info.catchedAjax of the ScrapeNinja response to retrieve the dumped request. This is a global handler, you can also specify XHR request catcher per-step (see "Interact with page" sectin below).
 
Advanced. Interact with page: fill in forms, and click links!
  Activate if the action triggers page redirect.
 
Catched XHR response body will be available within .info.log events array from the returned ScrapeNinja response (find this particular event by filtering let xhr = body.info.log.find(e => e.type == 'xhr' && e.stepIdx == {{idx+1}})).
New Step [+]
 
 
Custom Proxy
Wait for DOM selector to appear

Examples of valid payloads:

  • JSON payload: {\"fefe\":\"few\"}
  • www-encoded payload: key1=val1&key2=val2
Add Content-Type: application/x-www-form-urlencoded to headers in case of www-encoded POST and Content-Type: application/json

Raw ScrapeNinja Response:

Latency: {{ responseLatency }}ms HTTP Status: {{ responseBody.info.statusCode }}
Submit the form to scrape the URL.
                {{ responseBodyFormatted }}
                

Screenshot

Click to open full size in a new tab.

Unescaped target website response

Access it in your code via responseJson.body property. Use {{cmdKey}}+F for a quick search in the response body.

Generated code for ScrapeNinja:

                

Launching the scraper in your local Node.js:

Make sure you are using Node.js v16 node -v

Step #1. Create project folder and install node-fetch

mkdir your-project-folder && \
  cd "$_" && \
  npm i -g create-esnext && \
  npm init esnext && \
  npm i node-fetch -y
                  

                  

Step #2. Copy&paste the code above

Create new empty file like scraper.js and paste the code to this file.

Step #3. Launch

node ./scraper.js