Skip to main content

/scrape examples

Send form as www-encoded data​

By default, ScrapeNinja interprets data property as raw data for POST request. It's perfect for sending JSON blobs to target website. To make it interpret the data as www-form-urlencoded data, pass "Content-Type: application/x-www-form-urlencoded" into headers array:

curl --request POST \
    --url https://scrapeninja.p.rapidapi.com/scrape \
    --header 'X-RapidAPI-Host: scrapeninja.p.rapidapi.com' \
    --header 'X-RapidAPI-Key: YOUR KEY' \
    --header 'content-type: application/json' \
    --data '{
    "url": "https://website.com/post.php",
    "headers": [
        "Content-Type: application/x-www-form-urlencoded"
    ],
    "method": "POST",
    "data": "key1=val1&key2=val2"
}'

Send POST as multipart data​

upload property value of multipart is the name of the input to attach the multipart data to.


curl --request POST
    –-url https://scrapeninja.p.rapidapi.com/scrape
    –-header 'accept: application/json'
    –-header 'content-type: application/json'
    --header 'content-type: application/json' \
    –-data '{ 
    "multipart": "upload", 
    "url": "https://apiroad.net/post-json.php", 
    "method": "POST", 
    "data": "key1=val1&key2=val2&ewfew=23f23f23f32&wgewllfwejfewijewj33mme=ewfewfjwjeje33233232&key2"
}'

Retry on string occurence​

Use textNotExpected which accepts an array of strings. ScrapeNinja already has some certain Cloudflare string occurences hardcoded, but you can add your own. If the string is found in the response, ScrapeNinja will retry the request up to retryNum attempts (default retryNum value is 2).


curl --request POST \
--url https://scrapeninja.p.rapidapi.com/scrape \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '{
    "url": "https://website.com/post.php",
    "textNotExpected": ["<div id=\"p\">Your request was blocked</div>", "Some other unique text to retry"],
    "retryNum": 3
}'