Лайфхаки

Маленькие, полезные хитрости

5 Best Proxies for Puppeteer in 2023. Why Should You Use Proxies for Puppeteer?

15.10.2023 в 05:04

5 Best Proxies for Puppeteer in 2023. Why Should You Use Proxies for Puppeteer?

First, let us get straight with something: proxies are not a hard prerequisite for using Puppeteer.

However, doing literally anything that requires sending multiple test requests to a server bears the risk of getting yourself banned.

Bots, in particular, can be easily identified due to this, so disguising the traces of your activity and hiding your IP is very important.

Another thing is that, in spite of its well-rounded potential, Puppeteer as a mere Node.js library still lacks any means of giving your bots specific geolocation.

When given a sufficiently large scope, developers will want to test Chrome scripts’ and bots’ performance from various locations that they most likely will not have access to.

Cue proxies! Specifically targeted content cannot be accessed or retrieved from wherever so proxies can allow developers to review data meant for end-users around the world directly from their location.

Similarly, scraping SERP (search engine results page) for keywords would be impossible without proxies. Just be careful to opt for the proxy that has IPs in the location you need.

Every proxy on this list allows prospective Puppeteer users all this and much more. Without further ado, here are the 5 best proxies for Puppeteer.

Puppeteer proxy. Practical Puppeteer: Using proxy to browse a page

Today Puppeteer topic will be related to proxy. Using proxy when browse a page is useful when we want to hide our origin access location. That's only one reason. Another reason it can be used to protect our privacy and other use case is to open a website with geographical restriction.

According to Wikipedia,

In computer networks, a proxy server is a server (a computer system or an application) that acts as an intermediary for requests from clients seeking resources from other servers. A client connects to the proxy server, requesting some service, such as a file, connection, web page, or other resource available from a different server and the proxy server evaluates the request as a way to simplify and control its complexity. Proxies were invented to add structure and encapsulation to distributed systems.

In Puppetter we can use a proxy when we browse a page on internet. I will use several sample of proxy, such as SOCKS4, SOCKS5 and HTTP proxy.

Let's start.

Preparation

Install Puppeteer

We also need some proxy sample. For this I will use list of free proxy fromand we can pick several proxy from there.

The code

We will use SOCKS4 proxy and IP location of this proxy at Cambodia. Proxy IP address 96.9.77.192 and port 55796. I hope the proxy address still working when you try the example.

Fileproxy_with_puppeteer.js

const puppeteer = require ( ' puppeteer ' ); ( async () => { // set some options (set headless to false so we can see // this automated browsing experience) let launchOptions = { headless : false , args :

Run it with

It will open websiteand it will show like below.

Ow nice, it means the proxy works.

How about SOCKS5 proxy? It's easy, just change the code that set the proxy like below.

For HTTP or HTTPS proxy we can do like below.

If the proxy need authentication, we can add this code to support authentication. Put it beforepage.goto()part.

That's it.

Thank you and I hope you enjoy it.

Iproyal Puppeteer. Best Puppeteer Proxies of 2023


Are you looking forward to using Puppeteer in your script for automation and do you need the best proxies for the job? Then come in now and discover the best proxies you can use together with Puppeteer to avoid getting blocked.

The modern web is rich in Javascript which makes libraries and frameworks that do not support Javascript obsolete even though they are actively in use for sites that do not rely heavily on Javascript. Puppeteer is one of the most popular web automation tools that you can use to develop bots to automate your tasks on all kinds of sites including websites that rely heavily on Javascript and it is developed for NodeJS developers.

Bots developed with Puppeteer are quite fast and as such, they can easily be identified by the target website. To avoid getting detected, you will need to make use of proxies which would provide you alternative IP addresses to hide your IP footprint.

There are many proxies you can use together with Puppeteer. The tool does not even force you to use proxies and even if you use them, it accepts it even if the proxies are not functional — It is only at the point of usage that you discover. Unfortunately, not all of the proxies in the market that claim to be high quality are actually high quality.

A good number of them are not and wouldn’t be good for use as Puppeteer proxies. We have here to provide you recommendations on the proxies you can use as Puppeteer proxies and get your tasks done without getting blocked or experiencing other technicality problems.

Recommended Proxy Providers for Puppeteer Proxies

It might interest you to know that proxies do not need to be labeled Puppeteer proxies in other to be used with it. In fact, for those doing it, they are doing it for marketing reasons. Any high-quality proxy server can be used with Puppeteer.

All you need to make sure is that the proxies are compatible and undetectable to the website you want to make use of them on as most websites do not support the use of proxies. Below are some of the providers you can buy high-quality proxies to use in your NodeJS based Puppeteer projects.

Puppeteer change ip. IP Rotation with Puppeteer

IP rotation refers to using multiple IP addresses in a rotation to perform web scraping or automation tasks. This technique helps prevent IP blocking or detection by websites and allows for more extensive data collection or automation without being easily identified.

To set up IP rotation with Puppeteer using a proxy server, follow these steps :

  1. Choose a Proxy Server : Select a reliable proxy provider offering rotating IP addresses. Rotating proxies assign a new IP address for each request or after a certain time interval. Choosing a proxy service that provides the desired protocol (HTTP, HTTPS, or SOCKS) and supports IP rotation is essential.
  2. Setting up the Proxy Server : Sign up for the chosen proxy server and obtain the necessary credentials, including the IP address, port, username, password, and authentication method. These credentials will be used to connect to the proxy server.
  3. Testing the Proxy Server : Before integrating the proxy server with Puppeteer, it is advisable to ensure its functionality. You can use tools like cURL or browser extensions like FoxyProxy to verify that you can connect to the proxy server and receive responses.
  4. Creating a new Puppeteer instance with Proxy settings : In your Puppeteer code, use the puppeteer.launch() method to create a new Puppeteer instance with the appropriate proxy settings. Pass the proxy server’s IP address and port as command-line arguments to Puppeteer. Additionally, if the proxy server requires authentication, provide the username and password as part of the arguments.

Bright data Puppeteer. Puppeteer proxy integration

Puppeteer is a Node library created to control and automate headless and non-headless Chrome and Chromium browsers with its high-level API. Though it wasn't originally designed to be used as a testing platform, it has become a very popular alternative to Selenium among JavaScript users and features some additional stealth extra plug-ins .

  • First, access your Bright Data control panel and click ‘add zone’ .
  • Select your preferred proxy network type - Datacenter, ISP, Residential, Mobile, etc. and click 'add zone' again.
  • Go to Puppeteer, add theProxy IP:Portin theproxy-servervalue, for instancezproxy.lum-superproxy.io:22225.
  • Under thepage.authenticateinsert your Bright Data account ID and proxy zone name in theusernamevalue, this way:lum-customer-CUSTOMER-zone-YOURZONEand your proxy zone password found in the proxy zone settings.
  • Here is an example how what your Puppeteer code should look like:
  • Access your Bright Data control panel and create a zone with the proxy network type, IP type, and number of IPs you require.
  • Install the Proxy Manager on your device or access it via the cloud on your Bright Data control panel.
  • Click ‘add new proxy’ and choose the zone and settings you require, click ‘save’ .
  • In Puppeteer, under theproxy-server, insert your local IP and Bright Data Proxy Manager port (i.e. 127.0.0.1:24000).
  • The local host IP is 127.0.0.1
  • Don't type username and password into the fields — the Bright Data Proxy Manager is already authenticated with the Super Proxy server.
  • Here is an example how what your Puppeteer code should look like:

Start using the Bright Data Puppeteer proxy integration here .