Meta's Oct 30 business terms: scraping clause and robots.txt

If a Facebook Page, Instagram profile, WhatsApp channel or ad account is tied to your website, Meta’s new business terms matter to you. They take effect on October 30, 2026, and include a clause that lets Meta send crawlers to sites associated with your account. Meta has already reworded it once, and legal experts say it is still broad. Below are the clause, the change, who says what, and which crawlers you can name in robots.txt. This is not legal advice.

What does Meta’s new scraping clause say?

It authorizes Meta, or anyone acting for it, to run bots and crawlers on your sites. The text in Meta’s own preview page says you authorize “automated software (such as web scrapers, bots, robots, spiders, crawlers and user agents)” to access, index, cache, collect or crawl “the domains, web pages or URLs” that are “associated with your products or services,” and to “periodically pull updates” from them. It also lets Meta create test credentials. The Commercial Terms apply to business or commercial use of Meta products, which Meta’s page says includes using ads or marketing tools. Nieman Lab notes that virtually every news organization has at least one business account. PPC Land points out that the terms do not define “associated with your products or services” or say what test credentials are.

What changed after Nieman Lab’s report?

Meta removed the word “train” and narrowed the purposes. Nieman Lab reported the original text on Thursday, October 15. In a revision published Friday afternoon, October 16, Meta cut “train” from the list of allowed uses and limited those uses to “advertising and business products” instead of all Meta products, according to Nieman Lab’s October 17 follow-up. The original text allowed use to “provide, develop, improve, research, train, and maintain the integrity of” Meta Company Products. The revised text covers “provide, develop, improve, research, and maintain advertising and business products” and the integrity and security of Meta products.

Nieman Lab says the current terms allow this kind of scraping only when an account uses Meta’s generative AI features to run ads, and then only to “optimize your ads.” The new text moves it into the general terms.

What do Meta and legal experts say?

Meta says the change is a consolidation, and the experts say the wording still reaches far. A Meta spokesperson told Nieman Lab the update folds the “Ad Creative Generative AI Terms” into the general terms, that “robots.txt preferences are respected,” and that the revision was made because that was not clear. The spokesperson also said the update is not about scraping news sites. PPC Land reports that Meta’s Andy Stone called the reports false on X on October 16, which we did not see directly.

Three legal experts told Nieman Lab the revision does not fix the problem:

• Matt Topic, media and IP attorney at Loevy & Loevy: “business product” is not a defined term, and “develop” is broad enough to include “train.”
• James Grimmelmann, Cornell Law School: the “associated with your products or services” scope would still let Meta scrape entire sites.
• Benjamin Sobel, University of Wisconsin law professor: robots.txt states a preference, and unless it is legally enforceable it can be ignored.

The European Publishers Council said it would petition EU regulators, the UK News Media Association called on ministers to intervene, and the US News/Media Alliance called the terms “outrageous and probably illegal,” all as reported by Nieman Lab. These are advocacy positions, not rulings.

Can you opt out, and does robots.txt help?

Not before October 30, and robots.txt is Meta’s stated answer rather than a contract term. Nieman Lab reports there is no option for business accounts to opt out of the new terms before they take effect, and that the revised terms contain no language about respecting robots.txt. PPC Land, reading the terms, says continuing to use Meta’s business products after notice counts as acceptance and that the only way to reject them is to stop using those products. Meta says site owners can still disable its crawler through robots.txt. Whether that preference would override the contract is a legal question nobody has tested.

Which Meta crawlers can you block?

Meta’s developer page lists five agents and says its crawlers respect robots.txt, with two exceptions. We read it through a summarizing fetch, so check the page itself before editing a live file.

• facebookexternalhit fetches shared links to build previews. Meta says it may bypass robots.txt for security checks such as malware scanning.
• meta-webindexer improves Meta AI search results. Allowing it helps Meta cite and link to your content.
• meta-externalads supports advertising and business products.
• meta-externalagent supports training foundation AI models and improving products by indexing content.
• meta-externalfetcher fetches links at a user’s request for agentic features. Meta says it may bypass robots.txt.

A rule for a crawler looks like this:

User-agent: meta-externalagent
Disallow: /

Meta says crawlers may cache robots.txt for up to 24 hours. Blocking has trade-offs: blocking facebookexternalhit would likely break link previews when your pages are shared, and blocking meta-webindexer means Meta AI cannot cite you through it. Advertisers should think about meta-externalads before blocking it. Which crawlers the new clause covers is our question, not something Meta has said.

What is still unknown about the terms?

• What “associated with your products or services” covers. The text does not define it.
• Whether robots.txt overrides the contract. The revised text does not say.
• What counts as a “business product.” Meta currently lists ads, Meta Business Tools and Meta Business Suite as examples, per Nieman Lab.
• Whether the text is the same in every region. See the note below.
• Whether Meta will change the terms again before October 30, or how regulators will respond.

What to keep in mind

These are our observations:

• The word “train” is gone, the reach is not. The scope is the same, and the purposes are broader than ads alone.
• A stated preference is not a contract term. Meta’s robots.txt statement comes from a spokesperson, not from the revised text.
• Check which domains your business accounts point to. A site linked to a Page or ad account is the clearest example of what “associated” could mean.
• We saw the old wording on the preview page. When we opened Meta’s preview page in a browser from Türkiye at about 17:40 local time on October 18, the clause still read “research, train and maintain the integrity of” and used British spellings. Nieman Lab quotes the revised text with US spellings. This may be a regional version or a delay in updating. We could not view the US version, so we do not claim Meta has not updated it.

Editorial review: October 18, 2026 (AI). Prepared from Meta’s Commercial Terms preview page (opened in a browser), Nieman Lab’s October 17 follow-up (opened in a browser), PPC Land and Meta’s crawler page (each read through a summarizing fetch). Nieman Lab’s first story was not opened. We did not contact Meta, and we did not verify how Meta’s systems actually behave. This is not legal advice.