Skip to main content

Using ScrapeNinja with Make.com

Make.com (former Integromat) is a no-code automation platform that allows you to build automation scenarios without writing code. It is very similar to Zapier. It is a great tool for non-technical people who want to automate their business processes. It is also a great tool for technical people who want to automate their business processes without writing code.

There are two major ways to integrate ScrapeNinja into Make: 1) using official ScrapeNinja module ( https://www.make.com/en/integrations/scrapeninja ) 2) using regular HTTP API module in Make.com

Option #1: Using official ScrapeNinja module​

This is the easiest way to use ScrapeNinja in Make.com. You can find the module in the Make.com marketplace: https://www.make.com/en/integrations/scrapeninja . The only con of using official integration is that some of freshest ScrapeNinja params may be available a bit later in ScrapeNinja Make module because it requires ScrapeNinja maintainers to manually sync the Make.com module specification with latest ScrapeNinja API features.

ScrapeNinja Make official integration is designed to work with RapidAPI marketplace subscription only (not APIRoad). If you want to use APIRoad, you can use the HTTP API module instead (read on). So, get your RapidAPI key and subscribe to ScrapeNinja on RapidAPI: https://rapidapi.com/restyler/api/scrapeninja before you start integrating web scraping into your Make scenario.

Option #2: Using regular HTTP API module in Make.com​

This is the most flexible way to use ScrapeNinja in Make.com. You can find the HTTP module in the Make.com scenario builder. The best thing about using HTTP module of Make is that you will understand better how APIs work in general, which will make you a better engineer: you can use this knowledge with any other API in the wild, which does not even have Make.com integration. The con is that you will need a bit of extra typing to get things working. Essentially we just need to convert the API call shown in RapidAPI code generator (or APIRoad code generator) into Make.com HTTP module format.

Step1. Start with adding new module to scenario. Choose HTTP module

Step1. Start with adding new module to scenario. Choose HTTP module.

Screen2

Step2. Use "Make an API Key Auth request" action.

Step3. Locate code example generators on ScrapeNinja RapidAPI or APIroad marketplaces

Step3. Locate code example generators on ScrapeNinja RapidAPI or APIroad marketplaces.
tip

We are just copying everything into Make.com HTTP module format

Step4. Copy API parameters into Make "API request" action.

Note that auth headers (x-rapidapi-key for RapidAPI and x-apiroad-key for APIRoad) are usually not copied into regular headers of every particular HTTP request (though you can definitely do this if you want - as it's just another HTTP header), instead they are usually copied into "Authentication" tab of Make.com HTTP module for later easier re-use. It is more convenient to have auth headers in one place.

Step5. Copy API key into Make "API Auth" action.

Step5. Copy API key into Make "API Auth" action.

Copy&pasting ScrapeNinja payload​

Payload of ScrapeNinja API request is a JSON structure. Here is an example of this payload:

{
    "geo": "eu",
    "url": "https://website.com/post.php",
    "headers": [
        "Content-Type: application/x-www-form-urlencoded"
    ],
    "method": "POST",
    "data": "key1=val1&key2=val2"
}

copy this into Make.com HTTP module "Request content" field. Set "Yes" for "Parse response" flag so you can later use ScrapeNinja JSON response as an Make.com object with properties.

danger

Please do not confuse this payload with the payload of the HTTP request that you are going to make to APIRoad or RapidAPI. This payload is the payload of the ScrapeNinja API request itself. It is a JSON structure that contains all the information that ScrapeNinja needs to make a web scraping request to target scraped website, on your behalf.

So, we basically send a HTTP request to API Marketplace and attach the payload of ScrapeNinja API request to it.

Understanding the nesting of ScrapeNinja payload into API request to API marketplace.

Understanding the nesting of params.

Now click "Run this module" on HTTP module to test it.

Step 6. Make.com example ScrapeNinja output.

Step 6. Make.com example ScrapeNinja output.

Fixing the "ern: Error: SyntaxError: Invalid or unexpected token" in extractor​

When you copy&paste Javascript code of the extractor into Make, once you've tested that the extractor works perfectly well in the Cheerio Sandbox, make sure to remove all whitespace and new lines before and after the extract function definition.

Otherwise, you might get "SyntaxError" instead of the extractor results.