Got confused? When you start a blog, all you want is traffic to your website and Google indexing your site is like a dream come true. Then why would anyone in their right mind want to discourage search engines from indexing their site?
Why Would You Want To Stop Search Engines From Indexing Your Website?
When you have just started out, and you are still putting things up on your page, chances are pretty fine that you are directly working on your site and publishing the changes. At this stage, your site and your content are not perfect and you would not want anyone see it that way. Hence, at this juncture, you want to remove the website from Google search until such time that you are ready for people to have a look at your site.
You might think that even though you have started work on your website but have not put up links in order to get the search engines to find you, you are mistaken. Search engines don’t work just on links, they also work on content. But fret not; it is very easy to hide WordPress website from search engines.
So how do you block or stop search engines from crawling your site? Here are a few tips and tricks that may be adopted to achieve this. Go through the tutorial to find out how to hide WordPress site from search engines.
Table of Contents
Step 1 – Discouraging Search Engines From Indexing WordPress
The simplest way to stopping search engines from indexing your website is by preventing them from crawling it. Here are a few ways to achieve that:
Method 1 – Using the Inbuilt Feature on The WordPress site
Removing WordPress site from search engines is easy, all you have to do is use the inbuilt feature of the WordPress site to block the search engines from crawling:
- Access WordPress admin area and go to Settings and select Reading.
- There you will find an option called Search Engine Visibility. Check the box that says Discourage search engines from indexing this site. After enabling it, WordPress will edit the robots.txt file and apply disallow rules which discourage search engines from crawling and indexing your site.
- Press Save button to apply the changes.
Method 2 – Editing robots.txt File
If you prefer doing things manually, you can achieve the same result by editing the robots.txt file:
- Use File Manager or FTP client and access your website files.
- Locate robots.txt file. It should be in the same folder as WordPress (usually it is public_html folder). If you cannot find it, create a new blank file.
- Enter the following syntax that prevents search engines from crawling your site:
User-agent: * Disallow: /
Entering the above syntax will prevent search engines from crawling your site and make them stay away from it altogether. Having this syntax in the robots.txt file is like saying “no, I don’t want my site to be crawled” for search engines.
Step 2 – Password Protecting Your WordPress Website
Search engines and web crawlers do not have access to password protected files. You can password protect your files through the following methods:
Method 1 – Password Protecting Your Site Using Hosting Control Panel
If you are Hostinger client, password protection can be easily enabled by using Password Protect Directories tool:
- Access Hostinger control panel and click on Password Protect Directories icon.
- At the left side choose the directory you want to protect. In our case WordPress is installed in public_html.
- Once the directory is selected, enter your username and password in the right panel and click Protect button.
If you happen to be using cPanel, the process is quite similar:
- Log in to your cPanel account and click on Directory Privacy.
- Select the folder in which WordPress is installed. Usually it is public_html.
- Now select Password protect this directory option and enter the name for the directory you want to protect. Click Save button. Use form and create a user account which will be able to access protected directory. Press Save button once you are done.
Method 2 – Using Plugin to Password Protect WordPress
You can also install a WordPress plugin to achieve the same result. There are various plugins which could help you in achieving this. Some of these reliable names include Password Protected Plugin, WordFence and an array of others. Choose a plugin that has been recently updated and install it, once installed you will have to make your way to settings and set your password for the site. When your site is password protected, the search engines and web crawlers can’t access and further, can’t index your website.
Step 3 – Removing Already Indexed Page From Google
However, if your site has been already indexed by Google, you can still manage to remove it from Google by following the given steps:
- Set up Google Search Console for your website.
- Access Google Search Console, choose your newly added website and click on Google Index -> Remove URLs.
- Enter the web page URL you want to remove and press Continue button.
- On a new window choose Temporary hide page from search results and remove from cache and press Submit Request.
Google will temporary remove the URL from search results. Be sure to apply methods from Step 1 and password protect your WordPress site to prevent Google from indexing it again.
No matter what your reasons are to keep the search engines at bay, you can follow any of the above tricks to prevent search engines from indexing your website. While a few of them might not give you the best results, they would definitely serve the purpose.