Does faymas.in Let AI Agents In? Its robots.txt and llms.txt Disagree

Yes and no — on the same host, in two files that contradict each other. faymas.in serves /llms.txt (HTTP 200, 1,907 bytes, text/plain) telling AI agents to fetch /api/feed, /api/slug-prompt and /api/health. The same morning its robots.txt (200, 407 bytes) carried the line Disallow: /api/ for every crawler. We requested all four endpoints its OpenAPI file documents: three answered 200 with JSON, /api/saves answered 401, and /api/ itself answered 404. On the other three faymas domains there is no contract to contradict — faymas.net and faymas.com.in return 404 for every machine path we tried, and faymas-in.com returns 200 with 66,899 bytes of homepage HTML.

Updated October 24, 2026 · Prompt-text analysis

This guide analyses published prompt text. No generation tests were run.

What each domain actually publishes

All four domains answer /robots.txt with 200. That is where the similarity ends. faymas.in writes two crawler groups, faymas.net writes one, and the two WordPress sites ship the same default file — 117 bytes each, byte-identical apart from the Sitemap line.

None of the four names a single AI crawler. We searched all four files for 28 agent strings — GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, anthropic-ai, Claude-User, Claude-SearchBot, Google-Extended, CCBot, PerplexityBot, Applebot-Extended, Bytespider, Amazonbot, meta-externalagent, Diffbot, Timpibot, omgilibot, YouBot, AI2Bot, cohere-ai and others. Zero hits, on every domain. Search engines are not named either; neither are SEO crawlers or social scrapers.

The only agents any of these sites addresses by name are two Google advertising bots, and both appear on faymas.in. They get one extra restriction the wildcard group does not: /user/, the creator profile directory. For an AI prompt library, that is a strange inversion. The crawlers that pay are told to stay out of the author pages; everyone else is not told anything at all.

Where the two files contradict each other

faymas.in is the only one of the four with a machine-readable contract, and it is a thorough one. /llms.txt is 1,907 bytes of plain text describing the site, its API and its URLs. Then come three files underneath it.

The OpenAPI file declares four paths: /api/feed, /api/slug-prompt, /api/saves and /api/health. All four sit under /api/. The RFC 9727 catalogue is blunter still — its anchor is https://faymas.in/api, the forbidden directory itself, with llms.txt listed as the human-readable service documentation.

So the same host says two opposite things about the same URLs. A crawler that honours robots.txt will not fetch them. An agent that reads llms.txt is pointed straight at them. Which instruction applies depends entirely on which file the visitor happens to read, and nothing on the site reconciles them.

One more detail is worth stating plainly, because it changes how much weight the contradiction carries. robots.txt governs crawling; llms.txt is usually consulted at a different stage, by a model rather than by a crawler. So this is not proof that agents are being blocked. It is proof that two machine-facing files on one host were written without reference to each other.

What the endpoints return when you ask

We did not take the contract's word for anything. Every documented path was requested once, with a browser user agent and no credentials.

Three of four work. The fourth does not, and it is the one llms.txt never mentions — /api/saves appears only in the OpenAPI file. Meanwhile /api/ itself is a 404. An agent that follows the llms.txt instruction literally and types the directory gets nothing; one that reads the OpenAPI paths gets data every time except on /api/saves.

The feed also contradicts the prose above it. llms.txt introduces the site as a place for "ChatGPT, Gemini, Claude, Midjourney, Veo, Runway, FLUX, Kling and more". The first 20 items the feed returns name three models: chatgpt 15, gemini 4, midjourney 1. No Claude, no Veo, no Runway, no FLUX, no Kling. Twenty items is a small window, so we are not claiming those models are absent from the catalogue — only that the description and the data do not currently describe the same place.

The contract is otherwise well built, and that is worth saying. The agent-skills index lists three skills, each with a SHA-256. We fetched all three SKILL.md files. All three returned 200, and all three hashes matched the index exactly. That is more verification than most sites publish. The failure is not sloppiness in the contract — it is that the contract and the crawler policy were never introduced to each other.

The paths robots.txt guards mostly are not there

Here is the other half of the picture. faymas.in forbids seven paths. We requested each one.

Three of the seven return 404. The four that return 200 carry between 1 and 47 visible words — pages that are four and six times larger than a full article, rendering almost nothing. A crawler barred from /creator/dashboard is being kept away from three words of text.

Then the arithmetic that makes the point cleanly. We pulled the sitemap: 21,432 URLs. We checked every one against all seven disallow prefixes. The number of sitemap URLs that fall under a robots.txt restriction is zero. Not a small share — zero out of 21,432.

Read together, those two measurements say the same thing from opposite ends. robots.txt forbids a directory that contains the site's only machine-readable content, and it forbids six other directories that contain nothing. The one path worth protecting is the one that got protected by accident.

Content negotiation: one claim out of three holds

llms.txt states that home, prompt and profile pages "return Markdown when requested with Accept: text/markdown". We sent that header at all three page types and compared the answer against a plain request.

The profile page is the interesting one. It returns Markdown to an ordinary browser request with no header at all, which means the negotiation is not negotiation — that route is simply Markdown now. The two pages the claim is mostly about, home and prompt, ignore the header completely and return the same HTML they would have returned anyway.

And the third skill in the agent-skills index — "Read Faymas pages as Markdown" — publishes a recipe for a capability that reproduced on one of the three page types we tested. The recipe file itself is fine. We fetched it: 200, 677 bytes, hash matching. It describes something the site only partly does.

The other three domains, where the answer is simply no

faymas.net has the most interesting robots.txt of the four and the least to back it up. It disallows /_next/*. Its own homepage references 53 distinct /_next/ URLs; we fetched one of them, a JavaScript chunk, and got 200 with 18,837 bytes of text/javascript. So the file asks crawlers not to retrieve 53 assets the page needs in order to render. Whether that hurts depends on how a given crawler treats blocked subresources, which we did not test — we are reporting the overlap, not the consequence.

Its other two rules point at near-empty pages. /settings/ returns 200 with 0 visible words and no title. /dashboard/ returns 200 with 34 words and the title "Login | Faymas". It is forbidding crawlers from its own login page, which every site does, and from a settings shell that renders nothing.

The two WordPress domains are a different failure. faymas-in.com answers 200 for /llms.txt, /.well-known/openapi.json, /.well-known/api-catalog and /.well-known/agent-skills/index.json — and every one of those answers is the same 66,899 bytes of homepage HTML, with the home page's title. An agent probing for a contract gets a 200 and a homepage. That is worse than a 404, because 200 looks like success. faymas.com.in does return 404 for all four, but its 404 page is 185,092 bytes of HTML, and its declared sitemap — the one line in robots.txt that differs from its twin — is also 404.

Both WordPress sites answer /api/feed with a 673-byte RSS document. Not JSON. An agent following the same convention that works on faymas.in gets a feed format nobody asked it for.

How to check this yourself

Three requests tell you almost everything, and none of them needs a login or a tool.

First, fetch /robots.txt and read only the names. If no AI agent is named anywhere, the site has no stated position on agents — which, for an AI prompt library, is a position in itself. Second, fetch /llms.txt and check the content type. text/plain means someone wrote it; text/html means you have been handed a page. Third, take any path llms.txt advertises and see whether it appears in robots.txt as a Disallow. That one comparison is the whole test, and on faymas.in it fires immediately.

If you want the fourth check, request the endpoints anyway. Documentation is a claim. Status codes are a measurement. On faymas.in the claim and the measurement agreed three times out of four, and the disagreement was a 401 on an endpoint the plain-text guide never mentioned.

What would change this reading

This is one snapshot, taken on 2026-10-24, of server-side responses only. We did not execute JavaScript, so anything rendered client-side is outside what we measured. We did not crawl, so we cannot say what any crawler actually did with these files — only what the files say.

Two things would move the conclusion. If faymas.in removed /api/ from its Disallow list, or added an explicit Allow for the documented endpoints, the contradiction disappears and the site becomes the best-instrumented of the four by a wide margin. If the /api/saves 401 turns out to require a header we did not send, that row changes too — we sent a plain GET, and that is all we can speak to.

The structural point survives either way. Four sites built on the same word have produced four unrelated crawler policies, one machine contract, and no agreement between them. The contract that exists is the most carefully built thing any of them has published — verified hashes, an OpenAPI description, an RFC 9727 catalogue — and it points agents at a directory its own robots.txt closes.

FAQ

Do any of the four faymas domains mention AI crawlers in robots.txt?

No. We searched all four files for 28 AI agent names and found zero. The only named agents anywhere are Mediapartners-Google and Google-Display-Ads-Bot, both on faymas.in.

Does faymas.in actually have an llms.txt file?

Yes. It returns 200 with 1,907 bytes of text/plain, and it advertises /api/feed, /api/slug-prompt, /api/health, an OpenAPI description, an RFC 9727 catalogue and an agent-skills index.

Do the documented API endpoints work?

Three of the four do: /api/feed returned 20 prompts, /api/slug-prompt returned a full record, /api/health returned 457 bytes. /api/saves returned 401.

Are any sitemap URLs blocked by robots.txt on faymas.in?

None. We checked all 21,432 sitemap URLs against all seven disallow prefixes and got zero matches.

Does faymas.in return Markdown when you ask for it?

Only on profile pages. Home and prompt pages returned byte-identical HTML with and without the Accept: text/markdown header.