GEO technical optimization: how page structure, load speed, and structured data affect AI crawling
1. L1 Get in: can bots actually read your pages?
The first layer of technical optimization is not optimization, it is clearing mines. The most common GEO technical problem on B2B websites is that bots never got in at all. There are three reasons: robots.txt blocks them by mistake, the firewall blocks by UA, or pure JS rendering returns an empty shell.
| Issue | How to detect it | Fix actions |
|---|---|---|
| robots.txt blocks AI bots by mistake | Check robots.txt and bot status codes | Explicitly Allow GPTBot, OAI-SearchBot, and PerplexityBot |
| WAF blocks unknown UAs | Test with curl using a bot UA | Whitelist these bot UAs |
| Pure JS rendering leaves the first screen blank | Fetch the HTML with curl and grep for body text | Switch to SSR or SSG |
| Chinese and English versions of the page are inconsistent | curl both versions separately | Make sure both versions serve the body text directly |
The check is very specific. In your server logs, filter by the UAs of these bots over the last 30 days: GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, Googlebot. If the status codes are all 4xx, they are being blocked; if you see 200s but only for the homepage, your internal links or sitemap are not letting bots crawl deeper. For how to write a sitemap, see Site rebuilds and domain changes the link structure section in
A real example: a client running a standalone WordPress site had installed a security plugin that blocked all unknown bots by default. Six months after launch, we checked the logs and found PerplexityBot had never visited once. After we whitelisted it, the bot started visiting product pages the following week, and 4 weeks later the company began showing up as a citation in Perplexity answers. This problem needs no content changes, just one line of config.
2. L2 Get fast: TTFB and LCP determine how often bots come back
Traditional SEO treats page speed as a user experience issue, and Core Web Vitals were designed for real people. But for AI bots, speed determines crawl priority and frequency. Bots sort the queue by response time, pages with TTFB over 2 seconds get pushed to the back, and the recrawl cycle stretches from 3 days to 2 weeks.
| Metrics | Pass line | Impact of failing the threshold | What to optimize |
|---|---|---|---|
| TTFB | ≤ 1.5 seconds | Lower crawl priority | CDN caching, PHP upgrade |
| LCP | ≤ 2.5s | Incomplete first-screen chunk | Image compression, inline critical CSS |
| INP | ≤ 200ms | Little impact on bots, big impact on real users | Lazy-load third-party scripts |
| CLS | ≤ 0.1 | Little impact on GEO | Fixed image aspect ratio |
For GEO, TTFB and LCP matter most. INP and CLS mainly serve real users and have little effect on bot crawling, so there is no need to over-optimize them. For specific thresholds and optimization priorities, see Core Web Vitals. CDN node selection has the biggest impact on TTFB. For B2B overseas sites, configure nodes in both North America and Europe, see Global CDN acceleration.
Counterintuitive take: a slow page is not a “user experience problem,” it is a GEO crawl frequency problem. A product page with a 3-second TTFB may have content as good as your competitors’, but if the bot only visits once every two weeks, your content updates won’t be seen by the model for two weeks, and your citation rate falls behind. Push TTFB under 1.5 seconds and the bot visits 4 to 5 times more often.
3. L3 Comprehension: make page structure so models don’t have to guess
Once bots can read your page and read it fast, they still need to understand it. This layer depends on page structure and structured data. Models use HTML heading hierarchy, tables, and link anchors to judge what a page is about. When the structure is messy, they have to guess from context, and the odds of guessing wrong go up.
| Structural item | Correct markup | Incorrect markup |
|---|---|---|
| H1 | Unique per page, contains the main keyword | Multiple H1s or no H1 |
| H2/H3 hierarchy | Progresses logically with the content | Skips levels, uses H4 as a subheading |
| Tables | Has a thead, first row is the threshold | Builds a table out of images |
| Anchor | Every H2 has an id | No id, so models can’t point to a section |
| Internal links | Anchor text is the keyword | Anchor text is “click here” |
The structured data trio (Organization, WebSite, BreadcrumbList) is mandatory site-wide; it is the entity ID card you hand to models. Product and article pages also need Product, Article, and FAQPage. Fields must match the visible content; mismatches can trigger a Google manual action. For the full markup checklist, see Structured data (Schema). For how entities get recognized in AI answers, see Knowledge graphs and entity markup.
Scenario example: if a product spec table is built out of images, the bot sees an image, and OCR may not read the numbers in the table. If you use a real HTML table with “PN16 butterfly valve max operating temperature” in the thead, the model can pull that row out directly as a fact. For the limits of image localization, see Charts and tables on multilingual sites.
4. Check order and launch deadlines
Don’t change all three layers at once. Go in order, and each layer has clear verification tools and deadlines.
| Steps | Tools | Timeframe | Pass criteria |
|---|---|---|---|
| 1. Check robots logs | Filter bot UAs in server logs | Same day | All 5 AI bots return a 200 status code |
| 2. Test TTFB/LCP | Lighthouse、PageSpeed | Fix within 3 days | TTFB ≤ 1.5s，LCP ≤ 2.5s |
| 3. Check Schema | Rich Results Test | Same day | No errors in the trio |
| 4. Check H hierarchy | Browser outline extension | Same day | Unique H1, no skipped levels |
| 5. Wait for bots to recrawl | Keep watching server logs | 1 to 2 weeks | Bot visit frequency rises |
Don’t expect citations to jump the day after you make changes. It takes 1–2 weeks for bots to recrawl, and another 1–2 weeks for model indexes to update. So after technical optimization, wait at least 4 weeks, then run a fixed question set to see how citation rates have changed. This lag is something you have to account for in GEO experiment design. See GEO experiment design.
A common mistake is to start writing new content right away. If L1 and L2 aren’t passing, new content won’t reach the model any more than old content will. Clear the mines first, then speed things up, and only then talk about content. Get these three layers out of order and every step pays less.
One more thing that’s easy to overlook: technical layers on multilingual sites need to be checked separately. If your German, Spanish, and Arabic versions share the same robots file but your CDN nodes are only in the US, TTFB in non-English markets will run high, and AI citation rates in those languages won’t climb.Multilingual GEO covers the German, Spanish, and Arabic markets. Technically, that means getting three things in place first: local nodes, local hreflang tags, and local caching. Content comes after.
A real-world example: a client doing multilingual expansion had normal citation rates on the English version, but the German version was nearly zero in Gemini. Logs showed Googlebot hitting the German version from European nodes with a TTFB of 3.2 seconds, so it got pushed to the back of the queue. After adding a European CDN node and caching the German pages in Frankfurt, TTFB dropped to 1.1 seconds, and three weeks later the German citation rate started climbing. This had nothing to do with content. It was purely a missing region in the technical layer.
One last reminder: don’t change URLs frequently after technical optimization is done. Every redesign needs a 301 handoff, or the old links bots just got familiar with will break and new links will have to queue up again. For how to map out 301s during a site migration, see Site rebuilds and domain changes. The technical layer is slow work. Once it’s stable, leave it alone.
Make the three-layer technical check a fixed monthly task, alongside the other routine items in Website Maintenance Schedule: run bot logs, Lighthouse, and Rich Results Test once a month. Technical debt isn’t as visible as content debt, but it determines whether models can read your content at all. A stable, fast, clearly structured site is the foundation of all GEO content work.
FAQ
Which layer of GEO technical optimization comes first?
Why does slow TTFB affect AI citations?
How does a purely JS-rendered site affect GEO?
How much does structured data (Schema) help with AI citations?
How long after technical fixes do citations improve?
Should spec tables on a page be images or HTML tables?
Final thoughts
Content teams often underestimate the technical side of GEO. Everyone is writing better articles, but no matter how good the article is, if bots can’t get in, come slowly, or can’t understand it, it’s all for nothing. The three layers of technical optimization are essentially paving the road for the model: open the gate first, then smooth the road, and finally paint the signs clearly.
Three steps to implement: this week, check the status codes of 5 AI bots in your server logs and whitelist the ones that are blocked. Next week, test TTFB and LCP with Lighthouse and get TTFB under 1.5 seconds. The third week, check your Schema trio with the Rich Results Test and fill in sitewide markup. Once these three things are done, wait 4 weeks and run your question set. You’ll see the first jump in citation rate.
One line worth remembering: GEO isn’t about pleasing users, it’s about paving the way for bots. Open the door, smooth the path, mark the signs clearly, and only then does your content reach the model.