This is useful for when you have identified your Preliminary listing of backlink prospective buyers with Google or another tool that lacks Website positioning metrics and details.txt to disallow almost everything or unique endpoints (hackers can nonetheless look for robots.txt for endpoints) which helps prevent Google bots from crawling sensitive