Overview

What `qpkgs` is#

qpkgs is a git submodule that holds the genuinely shared packages of Qirava — the de-facto "standard library." It currently contains 13 standalone crates. There is no top-level workspace Cargo.toml; the crates are wired purely by path dependency, and the effective "member list" is whatever a consumer declares. The DMS (qdms) path-depends on all 13, and that is where the tree is built and checked.

The split is deliberate: two substrate crates define the runtime and the value model once, and every other crate is a thin, isolated function package that registers built-ins against that runtime. Because the packages share one value model and one runtime, a query engine, a scheduler, or any product can mix and match exactly the namespaces it needs without dragging in the rest.

Zero-dependency philosophy#

Every crate in this tree is written against the Rust standard library only — no third-party crates. Hashing (SHA-256, SHA-1, HMAC) is implemented from scratch; base64 and hex codecs are from scratch; the regex engine is a from-scratch parser → bytecode → backtracking VM; UUID v4/v7 generation is from scratch; arbitrary-precision integers and decimals are a hand-rolled bignum. Even the host resource scan reaches the C library that std already links (statvfs) rather than adding a crate.

• No supply chain to audit, pin, or trust beyond std itself.
• Deterministic, reproducible builds — nothing to resolve from a registry.
• Exactness where it matters: money and crypto use exact BigDecimal/BigInt, never lossy floats.
• A single value ABI shared by every package and by the DMS, so args and results never need a translation layer.

The one allowed internal edge is qcrypto → qencoding (crypto reuses the hex codec). Everything else is a leaf: each function package depends on exactly qexec + qvalue and nothing more. The substrate itself depends only on std.

The family map#

The tree is the substrate plus one function package per area.* namespace. The substrate defines the runtime and the value model; the function packages register the public built-ins callable by name.

          qvalue        qexec           (substrate; std only)
             ^   ^        ^
             |   +---+----+
        qencoding   |
             ^      |
        qcrypto     +-- qarray  qobject  qstring  qmath
                    |   qnumber qconvert qregex  qtime  quuid
                    v
          (every area.* package -> qexec + qvalue)

The value model: Value and Record#

qvalue defines one dynamically-typed Value enum that every package and the DMS share. It deliberately distinguishes fast and exact numeric kinds so callers never silently lose precision, and it makes Vector first-class for AI/embedding workloads.

Null

The absent value.

Int(i64)

Common fast 64-bit integer.

BigInt

Arbitrary-precision integer of any size.

Decimal

Arbitrary-precision exact decimal of any range/accuracy — money and crypto, with no floating-point error.

Float(f64)

Approximate / scientific double.

Timestamp(i64)

Nanoseconds since the Unix epoch (sub-second).

Bool / Str / Bytes

Boolean, UTF-8 string, raw byte buffer.

Vector(Vec<f32>)

First-class embedding vector for AI/vector workloads.

List(Vec<Value>)

Ordered sequence of values.

Map(Vec<(String, Value)>)

Ordered key→value map; ordering keeps the binary encoding deterministic.

A Record is an ordered set of named fields (Vec<(String, Value)>) — the structured carrier for both args and results. It is built fluently and read by name, including nested dotted paths:

use qvalue::model::{Record, Value};

let r = Record::new()
    .with("id", Value::Int(42))
    .with("name", Value::Str("alice".into()))
    .with("active", Value::Bool(true));

assert_eq!(r.get("id"), Some(&Value::Int(42)));

// `get_path` resolves a flat field first, otherwise navigates nested Maps:
let nested = Record::new().with(
    "addr",
    Value::Map(vec![("city".into(), Value::Str("oslo".into()))]),
);
assert_eq!(nested.get_path("addr.city"), Some(&Value::Str("oslo".into())));

Records cross thread and machine boundaries through a compact, length-prefixed, little-endian binary codec — encode_record(&record) -> Vec<u8> and decode_record(&bytes) -> Option<Record>. The codec is zero-dependency and round-trips every Value kind; decode_record returns None on a truncated or corrupt buffer (and refuses to allocate on a bogus field count), so it is safe to feed untrusted bytes.

The Record convention (how every built-in is called)#

This is the single rule that unifies the whole stdlib. Every public built-in is called the same way: arguments are a Record whose fields are positional string keys — "0", "1", "2", … — and the result is a Record with a single field named "result". Both directions are the binary-encoded Record, so the same convention holds in-process and over the wire.

1. Build a Record of positional args: Record::new().with("0", …).with("1", …).
2. Encode it with encode_record to get the input bytes.
3. Call runtime.execute("area.fn", &bytes, fifo, execution_key) and .wait() for the FunctionResponse.
4. Decode response.data with decode_record and read the "result" field.

Functions read their positional args from that record (array.* takes its list as arg "0"; number.* reads "0", "1", … in order) and always answer with {"result": Value}. The response itself is a typed FunctionResponse with ok, a ResponseCode (Ok, InvalidInput, ResourceLimit, PublicFunctionNotFound, …), a message, the data bytes, and meta. code == Ok (and ok == true) on success.

End-to-end example: calling array.sum#

This is the full convention in one round trip, transcribed from the qarray test that asserts the built-ins "speak the DMS convention." We build a runtime, register the array.* package, then call array.sum over the list [1, 2, 3]. The arg list goes in as field "0"; the answer comes back as field "result" = Int(6).

use qexec::{ExecConfig, ExecutionKey, FunctionRuntime};
use qvalue::model::{decode_record, encode_record, Record, Value};

// 1. Build a runtime and register the array.* package against it.
let mut b = FunctionRuntime::builder(ExecConfig::default_for(2));
qarray::register(&mut b).unwrap();
let rt = b.build().unwrap();

// 2. Args = a Record of positional keys; the list is arg "0".
let args = Record::new()
    .with("0", Value::List(vec![Value::Int(1), Value::Int(2), Value::Int(3)]));

// 3. Execute by name and wait for the typed response.
let resp = rt
    .execute("array.sum", &encode_record(&args), false, Some(ExecutionKey(1)))
    .wait();

// 4. Result is a Record with a single "result" field.
let result = decode_record(&resp.data)
    .and_then(|r| r.get("result").cloned())
    .unwrap();

// resp: FunctionResponse
resp.ok      == true
resp.code    == ResponseCode::Ok
resp.message == "ok"
resp.data    == encode_record(&Record::new().with("result", Value::Int(6)))
resp.meta    == ResponseMeta { root_key: "array.sum", worker_id: Some(0), helper_calls: 0 }

// after decoding resp.data:
result == Value::Int(6)
// (array.sum returns Int when the total is whole, Float otherwise)

The same shape works for every namespace: string.upper with "0": Str("hi") returns "result": Str("HI"); number.add with "0" and "1" returns the exact "result"; uuid.v4 takes no args and returns "result": Str(<uuid>). Learn the convention once and every built-in in the family is callable the same way.

Where to go next#

• Substrate: qexec for the executor, runtime, registry, scopes, and resource budgets; qvalue for the full Value/Record/bignum model and codec.
• Data shaping: qarray (lists), qobject (maps), qstring (text).
• Numbers: qmath (fast f64) vs qnumber (exact arbitrary precision); qconvert for cross-type casts.
• Bytes & ids: qencoding (base64/hex), qcrypto (SHA-256/HMAC/SHA-1), quuid (v4/v7), qregex (matching).
• Time: qtime for now / now_ms / date arithmetic.