My site looked finished. AI tools saw almost none of it.

My pages looked done in a browser. With scripts off, a full page showed about 4,300 characters and no title. Here’s the fix, and how to check your own site.

Etching-style drawing of a sheet of paper with only a few lines printed at the top and the rest blank, with a rust-colored pencil beside it

Scarcity’s pages looked done in my browser. Charts, tables, a short answer at the top. Then I turned scripts off and loaded the same page. It showed about 4,300 characters of text, and the page title was empty.

That matters because people aren’t the only ones reading your site. Google runs scripts before it indexes a page. The main AI crawlers don’t. As of the latest study of crawler traffic, none of the major AI crawlers render JavaScript.

My pages built most of their text in the browser with JavaScript. So to an AI tool, a page with a full analysis on it was mostly blank.

The fix was to write the text into the page when the site is built, not when someone opens it. On November 4, each profession page started shipping its text, tables, and charts as plain HTML. The same page now shows 12,600 to 14,600 characters with scripts off. The scripts still run for people. I also set the build to fail if the text changes after a second run, so the two versions can’t drift apart.

Text a crawler could read on one page, with scripts off

Characters of text on a Scarcity profession page

BeforeText built in the browser
About 4,300
After Nov 4Text written in at build
12,600 to 14,600
The striped part shows the range across profession pages. The page title was empty before the fix. Source: Scarcity’s change log, November 4, 2026.

How to check your own site

  1. Open one of your most important pages in Chrome. Open developer tools, press Ctrl+Shift+P (Cmd+Shift+P on a Mac), type “Disable JavaScript,” and reload.
  2. Look at what’s left. Is the main text there? The headline? The prices, program details, or answer the page exists to give?
  3. Check the title in the browser tab.
  4. Or view the page source and search for a sentence you can see on the page. If it isn’t in the source, a crawler that doesn’t run scripts won’t see it either.

If the text is missing, ask whoever builds your site about server-side rendering or prerendering. It’s a known problem with a known fix, and most site tools support it.

Check your three most important pages this week.

More on this: Scarcity, the project this came from, and Paid media and AI search, my working view on where search is going.

Written by Aaron Edgell

Growth executive. Builder. Writer.

Get new essays and notes by email.

One email when I publish something new.

Previous:
Next: