Is there a way to distinguish between a request from prerender.io (crawlers) and a real user (browser) using JavaScript?

I needed to identify whether requests were coming from prerender.io (a library used for rendering AngularJS applications for web crawlers) or from real users. If the request was from prerender, I had to redirect to a page specifically designed for SEO purposes.

I initially tried using cookies for detection, but it didn't work because prerender.io executes JavaScript code and even cookie/session storage functions within prerender.io.

After conducting some research, I discovered that we could detect the user agent, as prerender.io calls the site in a headless browser like PhantomJS.

if (/PhantomJS/.test(window.navigator.userAgent)) {
          //  console.log("PhantomJS environment detected.");
 } else {
        //  console.log("PhantomJS environment not detected.");    
  }

However, I wondered if this method was a permanent/proper/best solution for the issue, or if there were any alternative approaches?

javascriptangularjsseoprerender

Answer №1

Check out their FAQ to find the UserAgent they utilize:

What is the user agent of their crawler?

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3159.5 Safari/537.36 Prerender (+https://github.com/prerender/prerender)

You can use userAgent to determine if the request is from their crawler.

Answer №2

If you want to identify GoogleBot requests:

One option is to utilize IP range and UserAgent, as recommended by Google in the article linked below, to implement a redirect specifically for GoogleBot.

https://support.google.com/webmasters/answer/80553

For other scenarios, using UserAgent detection alone, similar to your current approach, may be sufficient.

Answer №3

Looking for the presence of "Prerender" in the user agent is one way to identify whether a Prerender.io browser is being used, but altering the page content based on this detection could be viewed as cloaking.

Instead, it is recommended to provide crawlers with the raw HTML version of the page that is visible to users.

Answer №4

Prerender detection

It is recommended in the Prerender documentation to search for "Prerender" in the user-agent instead of other parts of the string due to frequent Chrome version updates.

var isPrerender = navigator.userAgent.toLowerCase().indexOf('prerender') !== -1;

Permanent/Proper/Best Fix

In addition to GoogleBot detection, it is important to implement proper Prerender setup as follows:

  • Check the user-agent when your web app is accessed. If it's a bot, call Prerender with the URL of the requested page.

  • Prerender will render the page and return HTML for response delivery.

  • Implement strategies for prerender detection, addressing issues specific to headless browsers and optimizing site performance as per best practices: .

  • If the user-agent is neither a bot nor Prerender, serve the Angular app normally without modifications.

Server-side or intermediary solutions can handle bot detection and Prerender invocation before reaching your app. Some commonly used methods include:

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Tips for adding an array to an array of objects with AngularJs

I'm facing an issue with the array structure in my code. Here's what I currently have: $scope.arrayList=[{FirstName:"",LastName:""}]; $scope.Address=[{address:"",PhoneNumber:""}]; What I want to achieve is to push the $scope.Address array into ...

javascriptarraysangularjsarraylist

The div element is supposed to only be visible when the user scrolls down and then back up, rather than when the

Looking for some help with a Javascript issue on my website (http://www.pjsmusic.com). I need a div of 200px to appear when the user scrolls down 40px. I have tried using the following Javascript code: $(document).scroll(function() { $('#jerkBox& ...

javascripthtmlcssscroll

Retrieving PHP variable within a JavaScript file

I have a PHP script running on my server that tracks the number of files in a specific directory. I want to retrieve this file count ($fileCount) in my JavaScript code. numberOfImages.php: <?php $dir = "/my/directory/"; $fi = new FilesystemIterator($d ...

javascriptphpajax

Having trouble with jQuery after adding more content?

I recently started delving into the world of javascript, incorporating it into my website. Just last week, I came across a script that utilizes jQuery to load extra content onto my page. Initially, everything seemed to be functioning smoothly until I reali ...

javascriptjquerydocument-ready

AngularJS offers a full calendar feature that includes various views such as year, month, and week

I want to showcase a Full Calendar on my web page using AngularJS. I am looking for detailed steps to achieve this. Set up the UI calendar with full functionality Add events to populate the calendar Implement actions for the events Since I am new to Ang ...

angularjs

Tips for updating the selected date format in a Material Table component using React

In the context of using a material table in React, the columns set up are as follows: columns={[ { title: 'Name', field: 'name', type: 'string', }, ...

javascriptreactjsmaterial-uimaterial-table

Instructions on how to activate scrolling for a div element that becomes visible upon clicking a button

When a button in the navbar is clicked, a div appears with a height that exceeds the viewport/page size. How can I enable scrolling for the entire page when this happens? I attempted to use overflow: scroll, but it only applied scrolling to the internal d ...

javascriptjqueryhtmlcss

AngularJS function orderBy reverses the array instead of sorting it

I encountered an issue where, after clicking the 'order button', the table does not order as expected. Instead, it reverses all the td elements. For example, 'A', 'C', 'B' becomes 'B', 'C', "A". I ...

javascripthtmlangularjsangularjs-orderby

Is it possible that the method of wrapping options using an array is not functioning correctly in the quiz app being managed in React?

I need your help in figuring out where I've made a mistake. The following line is causing an error: const choices = Array.forms(document.querySelectorAll('.choice')); console.log(choices); ...

javascriptreactjs

What is the best way to conceal certain choices in a dropdown menu?

How can I display only the cities in Australia in a dropdown list? I have tried to find "Australia" in the options and hide everything before and after it, but I have been unsuccessful! Here is my fiddle. <select class="dropdown" id="dropdown"> ...

javascriptjqueryhtml

Enhance your browsing experience by inputting valuable information into the

I am looking to enhance a text box by adding values. The text box already has a default value of 10, and I want to create a button that will add new text boxes with a limit of 4. My goal is to continuously add values from text box 1 to text box 4. For exa ...

javascripthtml

Encountering an Unexpected Identifier while trying to adjust the scope within an Angular.js Directive

Within Angular.js, I am attempting to update a $scope value located in my controller using a directive with two-way binding "=" functionality. The goal is for the directive to update the scope variable. However, I am encountering an error message stating ...

angularjs

Creating a dynamic data filter for a table dynamically

I am seeking help with an issue I've been struggling with for the past week. I am attempting to add a feature that allows the filtering of items based on vendor names. Here are the steps I need assistance with: Retrieve all vendor names from the dat ...

javascriptjqueryjspdatatable

Minimizing the gap between icon and label text

I have a React form that I need help with. The issue is that I want to reduce the space between the list icon and the label. Here is the CSS I am using: .form__container { display: flex; flex-wrap: wrap; } .form__container input { color: rgb(115, 0, ...

javascripthtmlcssreactjstwitter-bootstrap

An error popped up while using the fetch API due to an unexpected end of input

fetch('http://www.freegamesforyourwebsite.com/feeds/games/?tag=platform&limit=100&format=json', { method:'GET', mode:'no-cors', dataType: 'json', headers: { 'Accept': 'a ...

javascriptjsonfetch-api

Tips for positioning elements within an ng-repeat list using absolute positioning?

Here is the HTML code: <div class = "container"> <form ng-submit = "createSticky()"> <input ng-model="formData.data" type="textarea" name="sticky_content" placeholder="add sticky text" required="true"/> <input ng-model= ...

htmlcssangularjs

What is the best way to have setState function properly within setInterval? (currently functioning somewhat)

function timerFunction(){ const [time, setTime] = useState(10); var timeRemaining = 10; const myInterval = setInterval(() => { if (timeRemaining > 0) { timeRemaining = timeRemaining - 1; setTime(timeRemaining); } els ...

javascriptreactjsreact-hookssetinterval

The issue with the dispatch function not working in the Component props of React Redux

I'm struggling with my colorcontrol issue. I've been attempting to use this.props.dispatch(triggerFBEvent(fbID, method, params)) without success. Interestingly, it seems to work fine if I just use triggerFBEvent(fbID, method, params). However, I ...

javascriptecmascript-6reduxreactjs

The Angular web application utilizing an ASP.NET Web API that is hosted on AppHarbor is showing the incorrect webpage

Working on a web application using Angular and ASP.NET Web API in Visual Studio 2015 has been quite the journey for me. Upon running the solution in VS2015, I encountered two tabs opening in Chrome - one for my Angular client-side application and another ...

javascriptc#asp.netangularjsappharbor

Utilizing JSON encoding in PHP to populate a dropdown menu

How can I update a select dropdown with dynamic data every few seconds using PHP and JavaScript? I have a PHP script that retrieves an array of numbers from 1 to 10 and returns it as a JSON response. However, when I try to update the select dropdown with ...

javascriptphp