{"id":13697,"date":"2021-08-23T12:49:38","date_gmt":"2021-08-23T12:49:38","guid":{"rendered":"https:\/\/www.milesweb.in\/hosting-faqs\/?p=13697"},"modified":"2026-02-27T12:29:03","modified_gmt":"2026-02-27T06:59:03","slug":"how-to-block-bots-with-robot-txt-file-via-cpanel","status":"publish","type":"post","link":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/","title":{"rendered":"How to Block Bots with robot.txt File via cPanel?"},"content":{"rendered":"<p>Bot traffic is non-user traffic that can hurt your web performance significantly. It is essential to safeguard your website from abusive bot traffic and, there are a few ways to do it. In this article, you will learn to block bots using the robot.txt file with your cPanel account.<\/p>\n<p>Located in the root directory of your website, robot.txt is a text file that enables the search engine bots to crawl and index your website. Apparently, it also decides what files should be accessible to the bots making its configuration essential. It is also why you can restrict these bots from indexing your website in the search engines using the robot.txt file.<\/p>\n<h2>To Block Bots Using Robot.txt File<\/h2>\n<p>1. Log in to your cPanel account.<\/p>\n<p>2. Go to the <strong>FILES<\/strong> section and click on\u00a0<strong>File Manager<\/strong>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12494\" src=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png\" alt=\"\" width=\"1360\" height=\"318\" srcset=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png 1360w, https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager-300x70.png 300w, https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager-768x180.png 768w, https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager-1024x239.png 1024w\" sizes=\"auto, (max-width: 1360px) 100vw, 1360px\" \/><\/p>\n<p>The File Manager interface will open. You will have to create the\u00a0robot.txt file.<\/p>\n<p>3. Click the <strong>+File<\/strong> button on the top left corner of the interface.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12577\" src=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_file_manager_add_file.jpg\" alt=\"\" width=\"1598\" height=\"317\" \/><\/p>\n<p>4. Enter the file name as<strong> robot.txt<\/strong> and click the\u00a0<strong>Create new file<\/strong> button.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12578\" src=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_robot_txt.png\" alt=\"\" width=\"505\" height=\"255\" \/><\/p>\n<p>5. Locate and right-click on the <strong>robot.txt<\/strong> file. Select the <strong>Edit<\/strong> option.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12580\" src=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_edit_robot_txt_censored.jpg\" alt=\"\" width=\"797\" height=\"318\" \/><\/p>\n<p>The file will open for you to enter\/edit its information.<\/p>\n<p>6. Copy this code in the file.<\/p>\n<pre class=\"lang:default decode:true\">RewriteEngine On\r\nRewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]\r\nRewriteRule .* \u2013 [F,L]<\/pre>\n<p>This code will block the bot from the Baidu search engine.<\/p>\n<p>7. Click the <strong>Save Changes<\/strong> button and <strong>Close<\/strong> the file.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12581\" src=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/cp_robot_txt_save.jpg\" alt=\"\" width=\"1599\" height=\"158\" \/><br \/>\nWith this robot.txt file, the Baidu search engine cannot use bots on your website.<\/p>\n<div class=\"vlt-box vlt-green\">\n<div class=\"box-title\">Note<\/div>\n<div class=\"box-content\">\n<p>You can also add the bot crawler of any other search engine to the code preventing it from indexing your website.<\/p>\n<\/div>\n<\/div>\n<p>In short, you can even block bots from other search engines with the robot.txt file by editing it via cPanel File Manager.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Bot traffic is non-user traffic that can hurt your web performance significantly. It is essential to safeguard your website from abusive bot traffic and, there are a few ways to do it. In this article, you will learn to block bots using the robot.txt file with your cPanel account. Located in the root directory of [&hellip;]<\/p>\n","protected":false},"author":27,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3046],"tags":[2414,2412,2411,152,1729,2413,2415],"class_list":["post-13697","post","type-post","status-publish","format-standard","placeholder-for-hentry","category-files-and-ftp-management","tag-block-bots","tag-block-bots-via-cpanel","tag-block-bots-with-robot-txt","tag-cpanel","tag-cpanel-settings","tag-how-to-block-bots","tag-robot-txt"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.1.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Block Bots with robot.txt via cPanel - Web Hosting FAQs by MilesWeb<\/title>\n<meta name=\"description\" content=\"This article will show you how to block bots with the robot.txt file that you can create using the File Manager tool via cPanel.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Block Bots with robot.txt via cPanel - Web Hosting FAQs by MilesWeb\" \/>\n<meta property=\"og:description\" content=\"This article will show you how to block bots with the robot.txt file that you can create using the File Manager tool via cPanel.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\" \/>\n<meta property=\"og:site_name\" content=\"Web Hosting FAQs by MilesWeb\" \/>\n<meta property=\"article:published_time\" content=\"2021-08-23T12:49:38+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-02-27T06:59:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1360\" \/>\n\t<meta property=\"og:image:height\" content=\"318\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Prasad Khartadkar\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Prasad Khartadkar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\"},\"author\":{\"name\":\"Prasad Khartadkar\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/c9138a9488172a8d0aa2d4f40878d83e\"},\"headline\":\"How to Block Bots with robot.txt File via cPanel?\",\"datePublished\":\"2021-08-23T12:49:38+00:00\",\"dateModified\":\"2026-02-27T06:59:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\"},\"wordCount\":297,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png\",\"keywords\":[\"Block Bots\",\"Block Bots via cPanel\",\"Block Bots with robot.txt\",\"cPanel\",\"cPanel settings\",\"How to Block Bots\",\"robot.txt\"],\"articleSection\":[\"Files &amp; FTP Management\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\",\"url\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\",\"name\":\"Block Bots with robot.txt via cPanel - Web Hosting FAQs by MilesWeb\",\"isPartOf\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png\",\"datePublished\":\"2021-08-23T12:49:38+00:00\",\"dateModified\":\"2026-02-27T06:59:03+00:00\",\"author\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/c9138a9488172a8d0aa2d4f40878d83e\"},\"description\":\"This article will show you how to block bots with the robot.txt file that you can create using the File Manager tool via cPanel.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage\",\"url\":\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png\",\"contentUrl\":\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.milesweb.in\/hosting-faqs\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to Block Bots with robot.txt File via cPanel?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/#website\",\"url\":\"https:\/\/www.milesweb.in\/hosting-faqs\/\",\"name\":\"Web Hosting FAQs by MilesWeb\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.milesweb.in\/hosting-faqs\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/c9138a9488172a8d0aa2d4f40878d83e\",\"name\":\"Prasad Khartadkar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/Prasad-K.-1-150x150.jpg\",\"contentUrl\":\"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/Prasad-K.-1-150x150.jpg\",\"caption\":\"Prasad Khartadkar\"},\"description\":\"Prasad is a business grad specialized in Marketing. He has garnered experience as a technical content writer and a digital marketer that he brings out in his work. He likes reading classics and travel in his free time.\",\"sameAs\":[\"https:\/\/www.milesweb.in\"],\"url\":\"https:\/\/www.milesweb.in\/hosting-faqs\/author\/prasad-khartadkar\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Block Bots with robot.txt via cPanel - Web Hosting FAQs by MilesWeb","description":"This article will show you how to block bots with the robot.txt file that you can create using the File Manager tool via cPanel.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/","og_locale":"en_US","og_type":"article","og_title":"Block Bots with robot.txt via cPanel - Web Hosting FAQs by MilesWeb","og_description":"This article will show you how to block bots with the robot.txt file that you can create using the File Manager tool via cPanel.","og_url":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/","og_site_name":"Web Hosting FAQs by MilesWeb","article_published_time":"2021-08-23T12:49:38+00:00","article_modified_time":"2026-02-27T06:59:03+00:00","og_image":[{"width":1360,"height":318,"url":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png","type":"image\/png"}],"author":"Prasad Khartadkar","twitter_misc":{"Written by":"Prasad Khartadkar","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#article","isPartOf":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/"},"author":{"name":"Prasad Khartadkar","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/c9138a9488172a8d0aa2d4f40878d83e"},"headline":"How to Block Bots with robot.txt File via cPanel?","datePublished":"2021-08-23T12:49:38+00:00","dateModified":"2026-02-27T06:59:03+00:00","mainEntityOfPage":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/"},"wordCount":297,"commentCount":0,"image":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage"},"thumbnailUrl":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png","keywords":["Block Bots","Block Bots via cPanel","Block Bots with robot.txt","cPanel","cPanel settings","How to Block Bots","robot.txt"],"articleSection":["Files &amp; FTP Management"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/","url":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/","name":"Block Bots with robot.txt via cPanel - Web Hosting FAQs by MilesWeb","isPartOf":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage"},"image":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage"},"thumbnailUrl":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png","datePublished":"2021-08-23T12:49:38+00:00","dateModified":"2026-02-27T06:59:03+00:00","author":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/c9138a9488172a8d0aa2d4f40878d83e"},"description":"This article will show you how to block bots with the robot.txt file that you can create using the File Manager tool via cPanel.","breadcrumb":{"@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#primaryimage","url":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png","contentUrl":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/03\/file_manager.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/how-to-block-bots-with-robot-txt-file-via-cpanel\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.milesweb.in\/hosting-faqs\/"},{"@type":"ListItem","position":2,"name":"How to Block Bots with robot.txt File via cPanel?"}]},{"@type":"WebSite","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/#website","url":"https:\/\/www.milesweb.in\/hosting-faqs\/","name":"Web Hosting FAQs by MilesWeb","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.milesweb.in\/hosting-faqs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/c9138a9488172a8d0aa2d4f40878d83e","name":"Prasad Khartadkar","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.milesweb.in\/hosting-faqs\/#\/schema\/person\/image\/","url":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/Prasad-K.-1-150x150.jpg","contentUrl":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-content\/uploads\/2021\/08\/Prasad-K.-1-150x150.jpg","caption":"Prasad Khartadkar"},"description":"Prasad is a business grad specialized in Marketing. He has garnered experience as a technical content writer and a digital marketer that he brings out in his work. He likes reading classics and travel in his free time.","sameAs":["https:\/\/www.milesweb.in"],"url":"https:\/\/www.milesweb.in\/hosting-faqs\/author\/prasad-khartadkar\/"}]}},"_links":{"self":[{"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/posts\/13697","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/users\/27"}],"replies":[{"embeddable":true,"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/comments?post=13697"}],"version-history":[{"count":1,"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/posts\/13697\/revisions"}],"predecessor-version":[{"id":13702,"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/posts\/13697\/revisions\/13702"}],"wp:attachment":[{"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/media?parent=13697"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/categories?post=13697"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.milesweb.in\/hosting-faqs\/wp-json\/wp\/v2\/tags?post=13697"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}