Orchestration functions (cloud.* API)

The eight functions at a glance#

How a function is called (the request shape)#

Each function is registered Public scope (the L2 scope, exactly like the engine's own auth.* governance fns), with the SECURITY BOUNDARY being the role re-check inside each function plus L3 (the planner gates the _cp_* write). The console builds a request record with the operator's role and user injected under ctx — the SAME way the worker funnel injects them — plus the named fields. The role is never at the top level, so a client cannot spoof it.

// The request record the console builds (from actions.rs::cloud_record):
Record::new()
  .with("ctx", Value::Map(vec![
      ("role".into(), Value::Str(caller.role)),   // e.g. "admin"
      ("user".into(), Value::Str(caller.user)),   // e.g. "op"
  ]))
  .with("id",     Value::Str("acme".into()))
  .with("owner",  Value::Str("acme-corp".into()))
  .with("mode",   Value::Str("standalone".into()))
  // ... the action's other fields

The role gate (deny-by-default)#

Cloud governance is a SEPARATE authority domain from any tenant's governance, ranked custodian (3) > admin (2) > user (1) > unknown (0). The gate compares the caller's ctx.role rank against the function's minimum. A missing or unknown role ranks 0, so it can never out-rank a real role. Provision / scale / switch / suspend / resume / invoice require admin+; terminate requires custodian.

fn role_rank(role: &str) -> u8 {
    match role { "custodian" => 3, "admin" => 2, "user" => 1, _ => 0 }
}
// require_role: ok if role_rank(caller) >= role_rank(minimum),
// else AccessDenied: "cloud role '<minimum>' or higher required".

cloud.provision  with ctx.role = "user", fields { id: "t1" }

{ code: AccessDenied, message: "cloud role 'admin' or higher required" }

An unauthenticated call (no ctx) also returns AccessDenied; an admin calling cloud.terminate returns AccessDenied (custodian only).

The success payload + the audit#

On success most functions return the new tenant state via ok_state: the tenant id, the simulated PLANNED note, and the current status/mode/threads/mem_gb/storage_gb/replicas/node_id read back from the row. Every function also writes a best-effort _cp_audit row (operator, action, tenant_id, node_id, result) — an audit failure never blocks the action, whose own write already committed through the funnel.

// ok_state returns (from cloud_fns.rs):
{ tenant_id, simulated,
  status, mode, threads, mem_gb, storage_gb, replicas, node_id }

// audit row written for each action:
INSERT INTO _cp_audit { id: "aud-<ts>", ts_us: <ts>, operator: <user>,
  action: <name>, tenant_id: <id>, node_id: <node>, result: <result> }

cloud.provision (admin+)#

Creates a new managed-DMS tenant. Input fields: id (required), owner, mode(standalone|cluster), storage(dynamic|fixed), threads, mem_gb, storage_gb. Defaults and clamps: threads default 1 (1–256), mem_gb default 2 (1–1024), storage_gb default 10 (1–100000). A cluster starts at 2 replicas (master + 1 follower on distinct nodes); standalone is a single instance. The tenant is bin-packed onto a node with headroom; if none, it is recorded with status = pending_capacity and an empty node_id. The matching _cp_subscriptions row (sub-<id>, plan metered) is created and the node's allocation is bumped by cap × replicas. Provisioning a tenant whose id already exists returns InvalidInput.

cloud.provision  ctx.role = "admin", ctx.user = "op"
fields {
  id: "acme", owner: "acme-corp",
  mode: "standalone", storage: "dynamic",
  threads: 2, mem_gb: 4, storage_gb: 20
}

{
  tenant_id: "acme",
  simulated: "PLANNED: boot tenant DMS on node-eu-1 (own WAL/_sys_*/seed), mint tenant custodian (returned once)",
  status: "active", mode: "standalone",
  threads: 2, mem_gb: 4, storage_gb: 20,
  replicas: 1, node_id: "node-eu-1"
}

If no node has headroom: status = "pending_capacity", node_id = "", and simulated = "PLANNED: no node headroom — placer would request a new bare-metal node; tenant DMS boot deferred".

cloud.scale_vertical (admin+)#

Resizes a tenant's per-instance CPU/memory/storage cap to a new ABSOLUTE value. Input: id, threads, mem_gb, storage_gb (any omitted field keeps the current value; same clamps as provision). It updates the cap on the tenant row and applies the delta × replicas to the placed node's allocation. Rejected with InvalidInput if the tenant is terminated or does not exist. SIMULATED: signaling the running DMS to re-read its cap and expand its qexec budget / storage LIVE — no rebuild, no downtime.

cloud.scale_vertical  ctx.role = "admin"
fields { id: "acme", threads: 8, mem_gb: 16, storage_gb: 50 }

{
  tenant_id: "acme",
  simulated: "PLANNED: signal the running DMS to re-read its cap LIVE (2->8 thr, 4->16 GB mem, 20->50 GB storage) — expand qexec budget / storage, no rebuild or downtime",
  status: "active", mode: "standalone",
  threads: 8, mem_gb: 16, storage_gb: 50, replicas: 1, node_id: "node-eu-1"
}

The node's alloc_threads now tracks the new cap (e.g. 8 for a single-replica tenant).

cloud.scale_horizontal (admin+)#

Changes a CLUSTER tenant's replica count. Input: id, replicas (new count, clamped 1–16). It updates replicas on the tenant row and adjusts the placed node's allocation by the per-instance cap × the replica delta. Only meaningful for a cluster tenant — a standalone tenant must cloud.switch_mode to cluster first, otherwise this returns InvalidInput: "horizontal scaling needs cluster mode — switch mode first". SIMULATED: placing/removing replicas across distinct nodes via the replication path (promote/drain), preserving isolation.

cloud.scale_horizontal  ctx.role = "admin"
fields { id: "solo", replicas: 3 }   (solo must already be mode=cluster)

{
  tenant_id: "solo",
  simulated: "PLANNED: rebalance cluster 2->3 replicas across distinct nodes via the replication path (promote/drain), preserving isolation",
  status: "active", mode: "cluster", replicas: 3, ...
}

On a standalone tenant: { code: InvalidInput, message: "horizontal scaling needs cluster mode — switch mode first" }

cloud.switch_mode (admin+)#

Switches a tenant between standalone and cluster. The on-disk layout is identical across modes, so this is NON-DESTRUCTIVE in BOTH directions (a hard requirement): standalone→cluster adds followers (replicas → at least 2); cluster→standalone drops to one instance (replicas → 1). Input: id, mode(standalone|cluster). Switching to the mode the tenant is already in returns InvalidInput. It flips mode + replicas and adjusts the node allocation by the replica delta.

cloud.switch_mode  ctx.role = "admin"
fields { id: "acme", mode: "cluster" }

{
  tenant_id: "acme",
  simulated: "PLANNED (standalone->cluster: add follower(s) on distinct nodes via replication) — NON-DESTRUCTIVE: on-disk layout is identical across modes",
  status: "active", mode: "cluster", replicas: 2, ...
}

The reverse (mode: "standalone") sets replicas back to 1; simulated note reads "cluster->standalone: drain follower(s), keep one instance".

cloud.suspend / cloud.resume (admin+)#

cloud.suspend sets status = suspended (e.g. for non-payment); SIMULATED: stop the tenant DMS from serving while preserving its data + seed. A terminated tenant cannot be suspended (InvalidInput). cloud.resume is the inverse: it requires a currently suspended tenant (else InvalidInput) and sets status back to active when the tenant has a node, or pending_capacity when it does not; SIMULATED: restart the tenant DMS in place (data + seed were preserved on suspend, so this is non-destructive).

cloud.suspend  ctx.role = "admin"  fields { id: "acme" }
cloud.resume   ctx.role = "admin"  fields { id: "acme" }

-- suspend:
{ tenant_id: "acme", status: "suspended",
  simulated: "PLANNED: stop the tenant DMS from serving (data + seed preserved; resume re-provisions in place)" }
-- resume:
{ tenant_id: "acme", status: "active",
  simulated: "PLANNED: restart the tenant DMS in place (data + seed preserved)" }

A second resume on an already-active tenant: { code: InvalidInput, message: "tenant is not suspended" }

cloud.terminate (custodian only)#

The destructive top-tier action, restricted to a cloud CUSTODIAN. It sets status = terminated, releases the node allocation this tenant held (cap × replicas, subtracted and clamped at 0), and cancels the subscription (status = cancelled). A tenant already terminated returns InvalidInput. SIMULATED: wiping the tenant's data + seed (irreversible).

cloud.terminate  ctx.role = "custodian"  fields { id: "acme" }

{
  tenant_id: "acme", status: "terminated",
  simulated: "PLANNED: wipe the tenant's data + seed (irreversible); node capacity released"
}

The node's alloc_threads drops back to 0 (capacity released); the subscription sub-acme becomes status = cancelled. An admin calling this gets AccessDenied.

cloud.generate_invoice (admin+)#

Generates an invoice for a tenant from its metered usage. REAL: writes one _cp_usage sample and one _cp_invoices row. SIMULATED: the metering itself — usage is synthesized from the tenant's current cap × a nominal 24-hour period, priced at the seeded plan rate (a real meter streams per-tenant counters into _cp_usage). The total is (threads × price_thread_hr × 24 + mem_gb × price_mem_gb_hr × 24 + storage_gb × price_storage_gb_mo / 30) × replicas, formatted to two decimals. The invoice is written with status = "due" and period = "last 24h (simulated)".

cloud.generate_invoice  ctx.role = "admin"
fields { id: "billme" }   (billme: 4 thr, 8 GB, 30 GB, replicas 1)

{
  tenant_id: "billme",
  invoice_id: "inv-<ts>",
  total: "<dollars>",   -- e.g. 4*0.012*24 + 8*0.006*24 + 30*0.10/30 = computed total
  ok: true,
  simulated: "PLANNED meter: usage synthesized from cap × period; a real meter streams per-tenant counters into _cp_usage"
}

A _cp_invoices row (status "due") and a _cp_usage sample are persisted; both are read back on the Billing screen.

Return-state and error summary#

Ok

The function persisted to _cp_* and returns the new state plus a simulated PLANNED note.

AccessDenied

ctx.role ranks below the function's minimum (deny-by-default).

InvalidInput

Missing required field, unknown tenant, illegal state transition (terminated/already-X), or a mode mismatch (horizontal scale without cluster).