One of the 14 ecommerce homepages in my test returned a CloudFront 403 to the research request. SEO Evaluator v2 diagnosed the error document as if it were the store.
It reported 13 actionable SEO problems: missing metadata, canonical, images, schema, hreflang, and more. It then claimed the homepage was blocking all users and crawlers and that existing rankings would decay.
The request had been blocked. That was all the evidence showed.
This case changed the direction of the evaluator. Before an AI system scores a page, it has to prove that it retrieved the intended page. Before it calls an observation an issue, it has to connect the observation to an applicable requirement and a reproducible consequence.
I ran the original evaluator and a revised v2.1 prompt on the same 14 homepage snapshots with the same model. The revised version cut actionable claims by 77.3% and removed every fixed-threshold finding. Be careful with that number. It does not mean v2.1 became 77.3% more accurate because I did not complete full human ground truth; it measures how much output disappeared when the original scoring rules were removed.
My conclusion: an AI ecommerce SEO audit should give the operator a review queue backed by evidence, not a grade that invites blind fixes.
The test setup
The sample covered 14 public ecommerce sites: 10 Shopify stores, two WooCommerce stores, and two headless Shopify implementations. I selected sites from public platform showcases and did not use private Search Console, analytics, Merchant Center, server-log, or admin data.
The crawler attempted 105 pages across home, collection or category, product, and content templates. Assessable content came back for 103. The two exceptions were different: one homepage returned 403 to the research request, and one product URL returned 404.
For each attempt, I saved the status, final URL, HTML size, title, meta description, canonical, H1 and heading structure, word count, links, images, structured-data types, and platform signals. The paired model test evaluated the 14 homepages only. Both versions received the same deterministic snapshot and were prohibited from browsing for extra evidence.
The model was DeepSeek v4 Pro, and each evaluator ran in a separate call. That produced 28 outputs. I then normalized every reported row into one of three groups:
- actionable claim;
- unavailable or dependent on private data;
- appendix item, including accessibility, security, or social-preview observations that were not SEO findings.
This classification tells me what the evaluator claimed. It is not a human truth label.
What changed between v2 and v2.1
| Measure | SEO Evaluator v2 | SEO Evaluator v2.1 | |---|---:|---:| | Total finding rows | 164 | 65 | | Actionable claims | 154 | 35 | | Urgent claims | 31 | 1 | | Fixed-threshold findings | 41 | 0 | | Unavailable/private-data rows | 8 | 9 | | Appendix/non-SEO rows | 2 | 21 |
The actionable count fell from 154 to 35, a reduction of 119 claims. Urgent claims fell from 31 to one. v2.1 also moved more observations into an appendix rather than letting them affect an SEO diagnosis.
Fewer findings are not automatically better. An evaluator can become quiet and miss real problems. I have not published precision or recall because the 229 combined finding rows do not have complete independent human labels.
What I can verify is narrower: 41 v2 findings depended on fixed thresholds that should not have been cross-site SEO rules. v2.1 produced none of those.
The original score rewarded unsupported certainty
The first evaluator included rules such as:
- commercial pages should contain 900–1,500 words;
- visible text should exceed 5% of raw HTML;
- meta descriptions should stay within a fixed 150–160-character range;
- homepages should remain below unsupported HTML-size, response-time, or link-count limits;
- Open Graph image absence should affect SEO severity;
- one raw HTML fetch could support claims about crawl budget, field Core Web Vitals, index efficiency, or mobile performance.
These rules made scoring easy. They did not make the diagnosis defensible.
Google's SEO Starter Guide says there is no magical minimum or maximum content length, nor an ideal heading number or order for Search. Meta descriptions have no fixed limit in Google's snippet documentation; snippets are truncated as needed and may come from page content. Keyword stuffing is prohibited. A universal keyword-density percentage is not recommended.
Page weight, response time, heading structure, and social-preview metadata can still deserve attention. The mistake is turning a diagnostic observation into a ranking problem because it crossed a generic number.
Once every category contributes to a total score, the model is encouraged to fill every category. The score turns uncertainty into missing points.
The 403 page was the boundary test
The blocked homepage revealed a more serious failure than a bad word-count rule.
v2 treated the CDN error document as the target page, then inferred consequences it could not observe. All I knew was that the research user agent had been blocked. That says nothing conclusive about ordinary shoppers, Googlebot, indexing status, historical rankings, or ranking decay.
The revised evaluator kept one finding from the same snapshot: the requested URL returned a 403 to this test. Then it stopped. Nine checks that depended on the homepage content were marked unavailable because the homepage had never been retrieved.
That is the behavior I want. A useful audit can stop.
My first gate now asks:
- Did the request reach the intended final URL?
- Is the response a real page, a consent wall, a bot challenge, a soft 404, an error template, or a cached intermediary?
- Does the content identify the expected site and page type?
- Was the page rendered when the check depends on JavaScript?
- Which conclusions remain unavailable from this evidence?
If the answer to the first three questions is uncertain, content and schema findings should not proceed.
Raw HTML cannot answer every SEO question
The original evaluator often crossed the boundary between observation and inaccessible state.
A public HTML snapshot can show the canonical element delivered in that response. It cannot show which canonical Google selected. It can reveal Product structured data in source HTML. It cannot establish Merchant Center consistency without the feed and account. A lab run can diagnose performance under a controlled condition, but it is not field Core Web Vitals. web.dev explains why lab and field data answer different questions, and Lighthouse does not directly measure field INP.
The revised version explicitly marked these areas unavailable when it lacked the required source:
- Google Search Console indexing and selected canonical;
- field Core Web Vitals;
- server-log crawl behavior;
- full-site orphan and internal-link analysis;
- rendered DOM when only raw HTML was supplied;
- Merchant Center feed consistency;
- traffic, ranking, and conversion impact.
An unavailable row is not a weakness in the report. It is a boundary around the evidence.
What v2.1 did better
The revision removed the universal total score and letter grade. In their place, it separated observations from diagnostics, unavailable checks, and appendix items. A finding now needed a traceable chain from evidence to an applicable requirement and then to a consequence.
That changed several common findings:
- multiple H1s and heading skips were usually editorial or accessibility observations unless a concrete content problem was shown;
- missing Open Graph assets moved to social-preview notes;
- response time and HTML size stayed diagnostic unless measured with an appropriate method;
- Search Console, rendered-page, Merchant Center, and field-performance claims became unavailable rather than guessed;
- a store could receive no verified SEO issue from the supplied homepage snapshot.
One WooCommerce homepage produced no v2.1 SEO problem that warranted action; it had three appendix notes. Another site also received no verified SEO problem, while its H1 observation stayed in the accessibility appendix. Zero must be a valid answer.
The revised prompt still made mistakes
I would not ship v2.1 unchanged.
It sometimes described missing Organization or WebSite schema on a homepage as an opportunity without showing that the markup was required, eligible, or useful for that page. A weaker model reintroduced “typical” ranges for HTML size and homepage word count even though the prompt prohibited unsupported benchmarks.
The output format also allowed passed checks to appear inside the findings array. In some rows, verified meant only that a value had been observed, not that the value was a verified problem.
The model choice mattered. DeepSeek Flash was much looser with ranges and inference during the pilot. DeepSeek Pro followed the boundaries more consistently, but prompt compliance was still not validation.
The next version needs a schema validator. A finding should be rejected unless it includes:
- the observed evidence and source;
- the page or scope to which it applies;
- the applicable requirement or documented best practice;
- the reason the observation creates a problem in this case;
- the information still unavailable;
- a verification step before remediation.
A passed item cannot also be a finding. Likewise, dependent checks cannot be verified when retrieval itself is unverified. I would enforce both conditions in code rather than ask the model to remember them.
How I would use an AI SEO evaluator in store operations
I would place the evaluator before human investigation, as the intake step—not as the final approver.
Its job is to collect evidence, group related observations, identify checks that require another data source, and prepare a small review queue. The operator then verifies the template, rendered page, Search Console, analytics, crawler output, or Merchant Center as needed.
I would not let it bulk-edit titles, canonicals, robots rules, schema, or internal links from a score. Those changes can affect thousands of URLs when they live in a shared Shopify theme or app.
For ecommerce specifically, the review should follow page type:
- Homepage: brand/entity clarity, crawlable routes into categories, and only the structured data that fits the page.
- Collection or category: crawlable product links, pagination, filter strategy, canonical behavior, and useful category context.
- Product: price and availability consistency, variants, canonical behavior, Product structured data, media, and merchant-policy context.
- Content: intent coverage, paths into commercial pages, authorship where relevant, and duplication.
Google's ecommerce site-structure guidance emphasizes crawlable links between navigation, categories, subcategories, and products. A homepage-only score cannot verify that system. It can only suggest the next crawl.
A practical acceptance checklist
Before acting on an AI-generated SEO finding, I would ask:
- Was the intended page actually retrieved?
- Is the evidence raw HTML, rendered DOM, lab data, field data, Search Console, a feed, or a full crawl?
- Does the recommendation depend on a made-up universal threshold?
- Is the item SEO, accessibility, security, social sharing, or general quality?
- Does the cited requirement apply to this page type?
- Can the claimed consequence be reproduced?
- Is the fix local to one page or shared across a template?
- What would prove the issue is resolved?
If the evaluator cannot answer the first five, I would keep the item as an observation, not open a remediation ticket.
My verdict
SEO Evaluator v2.1 produced a smaller, more honest review queue in this paired test. The best change wasn't a schema check or a more detailed score. It was one word: unavailable.
The result fits an operator who already knows how to verify findings with crawls, rendered pages, Search Console, and platform settings. It is a poor fit for anyone looking for a one-click grade followed by automatic fixes. The most dangerous output in this test was not a missed title. It was a confident diagnosis of the wrong document.
Run the retrieval gate first. Then take five findings, trace each one to its evidence and requirement, and reject every item that cannot survive that review. The number left is more useful than the score you started with. If the team also evaluates tools with a weighted score, the same evidence discipline belongs in the EcomAgentTools scoring methodology.
Frequently asked questions
Can an AI perform a Shopify SEO audit?
It can collect public-page evidence and prepare a review queue. A complete Shopify audit also needs template-level crawling, rendered-page checks, Search Console, field performance data, analytics, and sometimes Merchant Center or app configuration. Treat the AI output as triage until those sources confirm it.
What causes false positives in AI SEO audits?
Common causes include fixed thresholds presented as universal rules, scoring categories that force the model to find something, analysis of an error or bot-challenge page, claims about private data from public HTML, and recommendations that ignore page type or platform behavior.
Is a lower number of SEO findings better?
Not by itself. A quiet evaluator may miss real issues. Measure accuracy against human-reviewed ground truth, track false positives and harmful remediation advice, and keep unavailable checks visible. In this test, the reduction proved that fixed-threshold noise was removed; it did not prove overall precision.
Should AI automatically fix canonical tags or structured data?
Not from a single-page score. Canonical and schema changes often live in a shared theme, template, or app and can affect many URLs. Verify the issue across representative templates, test the generated markup, review platform defaults, and require approval before deployment.