How can you tell if a request should render server side? (ReactJS)

Question

I am writing an application that uses ReactJS to render the page. ReactJS components have front end code, but it's important to render them server side for SEO. How can I tell if the request is coming from a web crawler to render server-side?

Answer 1

To answer the question directly you can test the user-agent string of the request:

var crawlerUserAgents = [
  'googlebot',
  'yahoo',
  'bingbot',
  'baiduspider',
  'facebookexternalhit',
  'twitterbot',
  'rogerbot',
  'linkedinbot',
  'embedly',
  'quora link preview',
  'showyoubot',
  'outbrain',
  'pinterest',
  'developers.google.com/+/web/snippet',
  'slackbot',
  'vkShare',
  'W3C_Validator',
  'redditbot',
  'Applebot'
];

var userAgent = req.headers['user-agent'];

function isBot() {
  return crawlerUserAgents.some(function(crawlerUserAgent) {
    return userAgent.toLowerCase().indexOf(crawlerUserAgent.toLowerCase()) !== -1;
  });
}

app.use(function(req, res, next) {
  var userAgent = req.headers['user-agent'];

  if (isBot(userAgent)) {
    // It's a user agent
  }

  next();

});

(Adapted from prerender and using Express middleware)

However the alternative solution is to make your app render universally/isomorphically - in that the initial request renders the same on the server and client. There are many isomorphic tutorials and repositories out there. It means that the user also gets delivered the first lot of html without loading spinners. It's worth noting that it's also probably the harder solution to get set up and comfortable with though.

If your app is working universally then you don't need to detect if a request is a bot, or from the server so long as your router and fetching is isomorphic and you haven't got client specific code in places that the server will execute.

Probably a good idea to use a solution that handles the complexities for you eg https://nextjs.org/