How does ChatGPT index my site? No jargon, the very basics.
Ok, let's keep this basic. Simples. Back in the day, Google would index every website, I remember reading once Google had read every page on the Internet, that was its job, a spider, a crawler, I even remember the first website I fell in love with, to find information, was called MetaCrawler.
So No, GPT doesn't crawl "every .com."
GPT works in a slightly different way
First of all GPT doesn't just index every single website, in the way that Google does. Now this quick blog is going to be about GPT visiting your site, but I have to mention that GPT crawls high authority websites that do not belong to you, such as Reddit, Wikipedia, Quora whereby GPT understands that this data is relevant, mainly authentic and from trusted sources. Think how long Redditors on Reddit spend building up their Karma for example.
GPT looks at your site in a more "opportunistic" / "dynamic" way
For example if someone asks GPT "where do I find the best Baby Heart Monitor" and you sell Baby Heart Monitors, then it may just look. If someone asks "can you tell me about XYZ brand or product" which is your brand or product, then GPT will look at your website. Not exactly rocket science.
And to add to this, GPT telemetry shows GPT traffic is growing, meaning that GPT is looking more and more at your website than it did before.
Your website MUST allow AI crawlers (spiders) to visit
This is super simple and we believe EZY is one of the first platforms to not just simplify, but to automate this process. You simply connect to your hosting provider through EZY and we will update the files on your website so that GPT can access and understand your website.
If you don't have the budget for EZY, login to your hosting provider and amend the robots.txt to allow both GPT bots (mentioned below) and Schema (JSON-LD) to allow GPT and AI engines to crawl your website.
If you are a charity or a non-profit, working for an objectively GOOD cause, send us an email to see if we can onboard you for free.
There are a couple of little clever robots, perfectly named
GPTBot and ChatGPT-User that swing by your website. Now, we can measure when these little critters run by your website, and inform you, so you will be able to see through your EZY dashboard, if and when your site is getting busier from an AI perspective.
The Future of AI Search Optimization
If Google adds Trillions of dollars to the economy, it's fairly likely that GPT is already adding hundreds of billions to businesses like yours. And we think this is only set to rise. By 2028, it's most likely that the best growth of your business, will come down to AI engines telling people about you, your product, service or business.