Use the Search Engine Robots Feature of Magento 2 SEO

Magento 2 SEO comes with numerous features and functionalities. 'Search Engine Robots' is one of the features of the Magento 2 SEO addon that allows an admin to create and manage instructions for web crawlers and bots. These bots index the website.