# Sophia Stack X — reference for AI agents and developers
# @author Thomas G. and Sophia, SOPHIA XT LLC (sophiaxt.com)

One Rust binary (`sophia`) is the CMS, web server, build system and package tool.
Sites are content + menus + modules + a template; everything else comes from extensions.
Read this file whole before building. `sophia docs` prints it; `sophia schema` prints the manifest schema.

## Commands
sophia new app|block|component|template <name>   scaffold a working extension in extensions/<name>
sophia check <dir> [--json]                       validate manifest, parse templates, compile server.js
sophia build [dirs...]                            build extensions into <dir>/dist (all when none given)
sophia serve [--data DIR] [--bind HOST:PORT]      run; admin at /admin (first run opens /admin/setup)
sophia mcp                                        MCP server on stdio for agents (tools listed below)
sophia import <file> [--into site | --as slug] [--uploads dir] [--download] [--old-url url] [--dry-run]   from WordPress or Joomla
sophia deploy <user@host> [--domain a.com,www.a.com] [--site slug] [--tls-email you@a.com] [--with-data] [--rollback] [--status]
sophia bundle [--binary file] [--out file] | update <bundle> | update --latest (on the server, as root; signed releases only) | server-install [--domain ...] | site-set <slug> [--hosts a,b] [--public-url url] | ping <url>
sophia backup [--keep N] | restore <file> --into <empty dir>
sophia pack <dir> --key K | keygen | trust | install <file.sxp>   signed packages (Ed25519)
sophia site-create | user-create | grant | publish <site> | routes <site> | import-stack2 <dir>

## Extension folder
extension.json      manifest (below)
templates/*.html    Jinja templates (minijinja + Python string methods); names are "<ext>:<path>"
server.js           optional server code (JavaScript, sandboxed QuickJS, no Node APIs)
css/, js/, img/     sources bundled by "build" (CSS minified, every output fingerprinted)
After editing: `sophia check extensions/<name>` then `sophia build extensions/<name>`, then restart or use Reload extensions.

## extension.json (all keys optional except name)
{
  "name": "my-ext", "title": "My Ext", "version": "0.1.0", "description": "one line",
  "kind": "app|component|template|package",
  "collections": [ { "key": "event", "label": "Events", "label_one": "Event", "fields": [FIELD...],
                     "list_template": "list.html", "item_template": "item.html", "per_page": 10,
                     "sort": "date|title", "builder": false } ],
  "blocks":   [ { "type": "promo", "label": "Promo", "category": "Marketing", "template": "blocks/promo.html", "help": "...", "fields": [FIELD...] } ],
  "modules":  [ { "type": "hours", "label": "Hours", "template": "modules/hours.html", "fields": [FIELD...] } ],
  "plugins":  [ { "key": "seo", "label": "...", "handler": "sitemap|autometa|schema_org|redirects|inject|webhook|ai_alt", "fields": [...] } ],
  "template": { "positions": ["header","sidebar","footer"], "layout": "layout.html", "error": "error.html", "page": "page.html", "fields": [...] },
  "build":    { "bundles": { "site.css": ["css/a.css"], "site.js": ["js/a.js"], "style.css": ["css/ext.css"] }, "copy": ["img"] },
  "server": "server.js",
  "permissions": ["data", "content.read", "content.write", "accounts", "http:api.example.com"],
  "models":   [ { "key": "ticket", "label": "Tickets", "label_one": "Ticket", "help": "...", "title_field": "subject", "fields": [FIELD...] } ],
  "admin":    [ { "key": "queue", "label": "Support queue", "help": "...", "perm": "content.edit" } ],
  "settings": [FIELD...],
  "strings":  { "en": { "read_more": "Read more" }, "es": { ... } },
  "limits":   { "timeout_ms": 3000 }
}

## FIELD
{ "name": "title", "label": "Title", "type": TYPE, "required": false, "help": "shown under the input",
  "default": ..., "max": 200, "options": ["a","b"] | [{"value":"a","label":"A"}],
  "fields": [FIELD...] (for list), "collection": "event" (for item), "translatable": true }
TYPE: text textarea richtext number toggle select image file url date color code email secret list item
Saved values are cleaned against the schema (richtext sanitised, urls checked, unknown keys dropped).
image/file store a media id; templates receive {url, alt, width, height, srcset}. item stores an entry id; templates receive the entry.

## Templates
Context: site {name, slug, settings, languages}, lang, page {title, description, ...}, menus {<key>: [{title,url,active,children}]},
theme (template settings), item (entry: id, title, alias, url, data, refs, publish_at), items + pagination (lists),
block (block data) + entries (when a block/module has a "collection" field), config (module settings), siblings (entry pages).
Functions: position(name) has_position(name) asset("ext:file") link("/path") t("key") render_blocks(blocks) today() ext_styles()
Filters: excerpt(n) date(fmt) time12 tel  (+ all minijinja builtins, |safe for richtext)
Override another extension's view from a template: overrides/<ext>/<path>.
Blocks render inside <section class="sx-block sx-block--TYPE" id="b-ID">.

## server.js (sandboxed; each call is fresh; 64 MB, 3 s default)
sx.route("GET", "/items/:id", (req) => ({ json: {...} }))        // public: /x/<ext>/items/42
sx.route("POST", "/submit", async (req) => ({ redirect: "/x/my-ext/thanks" }))
sx.admin("queue", (req) => ({ html: "<p>...</p>" }))              // admin screen listed in "admin"
sx.on("item.saved" | "published" | "account.created" | "record.saved", (event) => {...})
req: { method, path, params, query, body, json, form, headers (content-type, accept, user-agent, referer), ip, live, account: {id,email,name,roles}|null, site }
Live regions (no build step): include <script src="{{ asset('core:live.js') }}" defer></script>, then
  <div data-sx-live="/x/my-ext/queue" data-sx-every="15"></div>   loads { html } from the route, refreshes every 15 s
  <form data-sx-target="#queue" method="post" action="/x/my-ext/close">  sends with fetch, swaps the answer into #queue
  <a data-sx-target="#detail" href="/x/my-ext/item/4">               loads into #detail; data-sx-swap="outer" replaces the element
  req.live is true for these requests, so one route can answer a full page or just the part.
admin req adds: user {id,name,email}, csrf (put <input type=hidden name=_csrf> in forms), base (screen url), form
Return one of: { json } { html } { text } { page: {title, html} } (wrapped in the site layout) { redirect } plus optional status, headers.
Throw an Error to answer 500; set err.status = 401/403/404 for those.
API (each needs its permission):
  sx.data.list(model, {where:{field:value}, order:"-created_at", limit, offset, account_id})  -> {items:[{id,data,account_id,created_at}], total}
  sx.data.get/create/update/remove(model, ...)            [data]   records are validated against the model fields
  sx.content.list(collection, {limit, search}) / get(collection, alias)   [content.read]
  sx.content.draft(collection, title, data)                [content.write]  (an editor publishes)
  sx.accounts.get(id) / list({role}) / setRoles(id, roles)  [accounts]
  sx.http.fetch(url, {method, headers, body}) -> {status, ok, text, json}   [http:<host>]
  sx.mail.send({to, subject, text, html?, replyTo?}) -> {id, queued, held}   [mail]  one address per message;
      queued in the outbox and delivered by the server; held=true means no mail server is set up yet.
      Limit 300 per site per hour. Wrap it in try/catch so a refused message never fails the request.
  sx.settings()  sx.site() -> {slug, name, languages, default_lang, base_url}  sx.render("tpl.html", vars) -> html  sx.escape(s)  sx.log(...)
  sx.rateLimit(key, max, seconds) -> true while under the limit (counts this use; per site and extension)
  sx.requireRole(req, "staff")  throws 401/403 unless the signed-in site account has the role

## Site accounts
Admin > Site accounts sets who can sign up (off | invite | open), default roles, and email confirmation
(off | ask: send a link, let them in | require: no sign-in until confirmed). account.verified is true once confirmed.
Pages: /account/login /account/signup /account (profile) /account/forgot /account/reset?t= /account/verify?t=;
POST /account/logout, /account/verify/resend. Restyle at overrides/core/account/*.html; emails at
overrides/core/email/account_reset.txt and account_verify.txt (plain text, vars: account, link, site).
Reset links last 1 hour and replace earlier ones; confirmation links last 3 days. Both are single use.

## Importing from WordPress and Joomla
WordPress: the .xml from Tools > Export (pages, posts, media in use, menus, WooCommerce products and
variations). Joomla: a database dump .sql (articles, categories, menus; articles a menu links to become
pages). Admin > Import, `sophia import`, or the MCP tool import_site. --dry-run lists what is in the file. Importing into an existing site keeps its home page and blog list unless --set-home is given.
Pictures come from a folder copy (--uploads) or the old site (--download). Old addresses become 301
redirects and links in the text follow. Running it again updates what it brought in. HTML is cleaned;
plugin shortcodes are removed and listed in the report. Approved comments come over; custom post types do not.

## Installing
`sophia` with no command starts it, keeps data in sophia-data beside the program and opens /admin/setup.
Linux server: curl -fsSL https://sophiastack.sophiaxt.com/download/install.sh | sudo sh -s -- --domain a.com --tls-email you@a.com
The program holds its extensions; a checkout's extensions/ folder takes their place when present.

## Deploying to a Linux server
`sophia deploy root@server --domain example.com --tls-email you@example.com` installs the first time and
updates after that. It needs ssh with a key, systemd, and root or passwordless sudo. The Linux program comes
from dist-linux/sophia (scripts/build-linux.ps1 makes a static one) or --binary. Layout: /opt/<name>/versions/<stamp>,
/opt/<name>/current, /opt/<name>/data. The service binds 127.0.0.1 and runs as its own account in a systemd sandbox.
nginx or Caddy is set up in front only with --domain, and only for names no other site on that server uses;
--install-proxy installs nginx and certbot. --tls-email accepts Let's Encrypt's terms. --behind-https says something
else provides HTTPS. A web server you run yourself must set X-Real-IP, because sign-in and spam limits count by it.
Every update stops the service, copies the database to backups/pre-update-<stamp>.db, switches version and waits for
/healthz. If it does not answer in 30 seconds the previous version and that database come back. --rollback goes to the
earlier program and leaves content alone. --with-data copies this install's content, media and accounts, and
--replace-data overwrites what is there (the old folder is kept). A new install with no content prints a one-time
setup link; /admin/setup refuses anyone without it. Three versions and three database copies are kept.

## What AI readers get from a published site
Every page has index.md beside index.html (linked with rel=alternate type=text/markdown), the release has
llms-full.txt (every page in one Markdown file) and llms.txt (a map, SEO extension). A request with
Accept: text/markdown gets the Markdown copy of the page.

## Lists and dates
A collection's "sort" is "date" (newest first), "title", "updated", "field:<name>" (ascending by a data
field; "field:-<name>" descending) or "upcoming:<date field>" (only entries dated today or later, soonest
first; past entries keep their own pages). A block whose settings name a collection gets `entries`; it
may add "upcoming": "<date field>" or "sort" to override the order, and "count". Static pages change
only when the site is published; Settings > Publish automatically republishes daily or hourly when something changed.
Each release holds _redirects.json (this server answers with 301s) and _redirects.nginx (include it when nginx serves the files).

## Forms (extension "forms")
Forms are entries of the "form" collection: intro, questions (label, type text | email | phone |
long text | number | date | choice | checkbox, choices, required, hint), button, thank-you text, notify
address, redirect. Place one with the "form" block. Answers post to /x/forms/send, are checked against
the form's questions, kept as "entry" records (Admin > Form answers) and emailed with reply-to the
sender. Spam checks: hidden field, at least 3 seconds on the page, per-visitor hourly limit.

## Design (template.style)
A template lists what Admin > Design may change, under "template": {"style": {...}} in extension.json:
  "tokens": [{"var": "--accent", "label": "Accent", "type": "color|font|select|length|number", "default": "#0f766e",
             "group": "Colors", "help": "", "options": [{"label", "value"}], "min": 14, "max": 21, "step": 0.5, "unit": "px",
             "from_theme": "accent"}]   from_theme: the template setting that held this value before
  "contrast": [{"fg": "--text" or "#ffffff", "bg": "--paper", "label": "Text on the page", "large": false}]   WCAG AA 4.5 (3 when large)
  "presets": [{"name": "Night", "values": {"--paper": "#0e141b"}}]
Use the variables in the template's CSS. Saved values live in site.settings.style and render as
<style id="sx-style">:root{...}</style> before </head>. Accepted: #rrggbb colors, option values exactly,
numbers within min..max with the token's unit. style_set checks everything first and saves nothing if one value is refused.

## Comments (extension "comments")
Block "comments" on a page, or module "comments" in content-bottom (only_under: "/blog/" limits it to articles).
Pages stay static: the browser reads /x/comments/list and posts JSON to /x/comments/post, so new ones need no republish.
Settings: open, who (anyone|accounts), mode (hold|accounts|open), notify, per_hour, blocked. Admin screen: Moderate comments.
Text only, inserted as text. Emails stay in the admin. Spam checks: hidden field, minimum time, hourly limit, repeats, blocked words; more than two links always waits.
Model "comment" is reachable through the MCP record tools. WordPress imports bring approved comments and their replies.

## Events (extension "events")
Entries of "event": start date, time, end, place, address, online link, summary, picture, description,
price, tickets link, status (scheduled | cancelled | postponed | sold out). The list shows upcoming
events; the "upcoming_events" block shows the next few; event pages carry an add-to-calendar file and
schema.org Event markup.

## Shop (extension "shop")
Products are entries of the "product" collection: price (major units, e.g. 24.99), compare_at, image,
gallery, summary, description, sku, stock (empty = unlimited), variants (options: label, price, stock,
sku; each option keeps its own stock), category, shipping (toggle, default on). Cart rows are
{id, v: option label, qty}. Discount codes live in the shop settings ("codes": code, percent or amount,
min, ends, limit); the discount comes off the items before tax and free shipping is judged after it.
On Stripe the discount travels as a one-use coupon.
Blocks product_grid and buy_box; module cart_link. Link the product list from a menu item of kind list.
The cart lives in the browser; /x/shop/cart prices it through POST /x/shop/api/quote and checkout posts
to /x/shop/checkout, where the server recomputes every price, tax (one rate on items) and shipping (flat,
free over an amount, or pickup). Pay by card through Stripe Checkout (secret key + webhook secret in the
shop settings; webhook endpoint https://<site>/x/shop/stripe for checkout.session.completed) or pay later.
Orders are numbered per site from 1001, carry a private link /x/shop/order/<token>, email a receipt and
a store alert, and count stock down when paid (pay-later: when placed); cancelling puts stock back.
Admin > Orders lists, filters and exports CSV; the orders.manage permission guards customer details.
Templates see shop settings (no secrets) as ext_config.shop; money is formatted with the money filter:
{{ 24.5|money("usd") }} -> $24.50.

## Email
A super user sets the SMTP server under Admin > This install > Email (server, port, STARTTLS/TLS,
username, password, From). Every message goes through one outbox: retries six times, erases the text
once delivered, keeps subject and address for 30 days. Without a server, messages wait there and a
super user can open them, which is how to follow reset links while testing. Links in emails use the
site's base_url, else the request host only when it is a listed host or localhost.

## MCP tools (sophia mcp)
Read first: docs. Content: sites, collections, blocks, entries, entry_get, entry_save, entry_delete.
Structure: menus, menu_item_save, menu_item_delete, modules, module_save, module_delete.
Media: media_list, media_upload (local path), media_alt. App data: models, records, record_get,
record_save, record_delete. People: accounts, account_update. Extensions: extension_new,
extension_check, extension_enable, ext_settings, ext_settings_save, schema. Design: style_get, style_set.
Check and ship: page_preview (one path, drafts included), publish_preview (dry run: added, changed,
removed, errors), publish, releases, rollback. SEO: seo_audit, seo_changes, seo_change_decide.
Shop: orders, order_get, order_action (paid | fulfil | cancel | refunded | note, email: true to tell the customer).
Import: import_site (local only). Install: backup, backups, audit_log, mail_outbox.
entry_save on an existing id only changes the fields you send; title, alias and state stay unless given.
Remote: POST /api/v1/mcp with Authorization: Bearer <site token with the "mcp" scope> speaks the same
JSON-RPC over HTTP, bound to that token's site (the site argument is filled in). Tools that touch the
server's own files or the whole install (media_upload, extension_new, extension_check, backup, backups,
mail_outbox) are local only. Only a site admin can create an mcp token (Admin > API).
Every write is checked like the admin form and logged as mcp.<tool> in the audit log. Run
publish_preview before publish; rollback undoes a publish.

## Rules that keep sites safe
Never write raw HTML from user input without sx.escape. Declare only the permissions you use.
Secrets go in settings fields of type "secret". Content edits from code are drafts; people publish.
Static publishing: published pages are files; /x/, /account, /admin, /api need the app (proxy them in nginx).

## Content API
/api/v1/site, /api/v1/content/<collection>[/<alias>], /api/v1/menus/<key>, POST /api/v1/content/<collection>
Authorization: Bearer <site token from Admin > API>.