Files

26 KiB

Start your domain

Where to add a module, an entity, a migration, a permission, a procedure, an audit event, a usage meter, a job, a page and a message catalogue, with the commands. Everything below was done once, in a copy of this template, as written here: the module is called widgets, it passed pnpm kit gate, and every code block is a file of it.

You write modules. You do not edit apps/: the web shell and the worker find a module from modules/index.ts and its manifest. (Rules the gates enforce: AGENTS.md.)

Before you start

pnpm install
pnpm dev          # Postgres, the fake platform, the demo data, the worker and the web shell

Sign in at the address it prints (http://127.0.0.1:3000/auth/dev-login?email=ada@example.test). The demo data has Acme Review with a general workspace and a project called Team notes; a project of your own is made from the workspace page (/<org>/general) by choosing your module in the Module list.

Where each thing goes

You add Where Command
A module modules/<id>/, listed in modules/index.ts pnpm new:module <id> [--uses files,proposals]
An entity (a table) modules/<id>/contract/entities.ts, defineEntity
A migration modules/<id>/migrations/NNNN_name.sql, forward only pnpm vitest run modules/<id> -u writes the first
A permission permissions of modules/<id>/module.ts
A procedure contract in contract/procedures.ts, handler in adapters/procedures.ts, rules in core/
Its isolation fixture modules/<id>/__tests__/isolation.ts pnpm test:isolation
An audit event .audit("<id>.<event>") on the contract, and audit: [...] in the manifest
A usage meter .usage("<meter>") on the contract, and usage: { meters: [...] } in the manifest
A job jobs of the manifest, enqueued by a write procedure
A page ui/pages.tsx, one line in modulePages of modules/index.ts, a ui.nav entry
A message catalogue modules/<id>/i18n/en.ts and lt.ts, declared as i18n: { en, lt }
The API document openapi.json pnpm openapi

1. The module

pnpm new:module widgets --uses files

--uses files because the job below reads the scope of its run through the files module's token (step 7); use proposals for a module that proposes changes a person approves. The command writes modules/widgets/ (module.ts, index.ts, contract/, core/, i18n/, a test), adds the module to modules/package.json and the catalogue when it needs a kit module, and appends it to domainModules in modules/index.ts. A module with no procedure passes the gate as it is; the steps below fill it in.

2. An entity

// modules/widgets/contract/entities.ts
// The tables of the widgets module. One, project-owned: `tenancy: "project"` gives the table its
// `org_id`, `workspace_id` and `project_id` columns and the row-level security policy on `org_id`.
import { defineEntity, idSchema } from "@conusai-kit/data"
import { z } from "zod"

export const widgets = defineEntity({
  name: "widget",
  table: "widgets_widget",
  tenancy: "project",
  fields: {
    name: z.string().min(1).max(120),
    createdBy: idSchema,
  },
  indexes: [["createdBy"]],
})

/** Every table this module owns, in the order a migration creates them. */
export const WIDGETS_ENTITIES = Object.freeze([widgets])

tenancy: "project" gives the table org_id, workspace_id and project_id and the row-level security policy; use "workspace" or "org" for rows that belong higher up. Fields are JSON types.

3. A migration

pnpm vitest run modules/widgets -u

The test in __tests__/migration.test.ts snapshots schemaStatements() of the module's entities into migrations/0001_widgets.sql, so the SQL is generated from the entity and cannot drift from it:

// modules/widgets/__tests__/migration.test.ts
// The module's migration is its own file and exactly `schemaStatements()` over its entities (decision
// AT-D20: the entity definitions are the one source of a column, so a hand-written column cannot drift
// from the schema the module validates its rows with). `pnpm vitest run modules/widgets -u` writes
// 0001_widgets.sql from it the first time; once that file has run, it is never edited -- a later change to
// the table is a new numbered file written by hand, and this test then compares the first file alone.

import { schemaStatements } from "@conusai-kit/data-kysely"
import { describe, expect, it } from "vitest"
import { WIDGETS_ENTITIES, widgetsModule } from "../index.js"

/** The statements for these tables alone: `schemaStatements` leads with the kernel's own. */
function tablesOf(entities: Parameters<typeof schemaStatements>[0]): string {
  const kernel = schemaStatements([]).length
  return `${schemaStatements(entities)
    .slice(kernel)
    .map((statement) => `${statement};`)
    .join("\n\n")}\n`
}

describe("the migration of the widgets module", () => {
  it("is its own file, exactly its table", async () => {
    await expect(tablesOf(WIDGETS_ENTITIES)).toMatchFileSnapshot("../migrations/0001_widgets.sql")
  })

  it("is declared by the manifest, so the migrate job finds it, and the table is row-level secured", () => {
    expect(widgetsModule.migrations).toEqual({ postgres: "migrations" })
    const sql = tablesOf(WIDGETS_ENTITIES)
    expect(sql).toContain('create table "widgets_widget"')
    expect(sql).toContain('alter table "widgets_widget" force row level security')
  })
})

Review the file and commit it. The manifest says where the folder is (migrations: { postgres: "migrations" }), and the migrate job applies it after the app's own, in file-name order, and refuses a file that changed after it ran. A later change to a table is a new numbered file you write (0002_...sql); /api/ready stays unready until the database has every file this build ships.

4. A permission, and a procedure

Permissions are <module>.<resource>.<action> with the lowest role that holds them, declared in the manifest (step 7 shows it whole). A procedure declares its kind, scope and permission, takes slugs and never an id of a parent, and is a contract first:

// modules/widgets/contract/procedures.ts
// The procedure contracts of the widgets module: schemas and names, no handler and no port. Each
// declares its kind, its scope and its permission; the kernel checks all three before a handler runs.
// The input carries slugs, never an id of a parent: the resolver's own reads are the membership check.

import { procedure } from "@conusai-kit/core"
import { z } from "zod"
import { WIDGET_CREATED, WIDGETS_CREATED_METER } from "./constants.js"

const inProject = { org: z.string(), workspace: z.string(), project: z.string() }

export const widgetViewSchema = z.object({
  widgetId: z.string(),
  name: z.string(),
  createdAt: z.string(),
})

export type WidgetView = z.output<typeof widgetViewSchema>

export const widgetList = procedure("widgets.widget.list")
  .kind("read")
  .scope("project")
  .permission("widgets.widget.read")
  .input(z.object(inProject))
  .output(z.object({ widgets: z.array(widgetViewSchema) }))
  .contract()

export const widgetCreate = procedure("widgets.widget.create")
  .kind("write")
  .scope("project")
  .permission("widgets.widget.write")
  .input(z.object({ ...inProject, name: z.string().min(1).max(120) }))
  .output(widgetViewSchema)
  // The transport records both once the call has succeeded: one audit event and one usage fact.
  .audit(WIDGET_CREATED)
  .usage(WIDGETS_CREATED_METER)
  .contract()

The handler resolves its ports from the container the transport built and calls a use case. A read handler is refused every write port by name -- the list handler below asks for the data source and nothing else:

// modules/widgets/adapters/procedures.ts
// The handlers of the module's procedures and its job: each resolves its ports from the container the
// transport built and calls one use case. No rule lives here.

import { type Container, implement, type JobContext } from "@conusai-kit/core"
import { DATA_SOURCE, JOB_QUEUE } from "@conusai-kit/data"
import { FILES_SCOPE } from "@conusai-kit/files"
import { widgetCreate, widgetList } from "../contract/index.js"
import { createWidget, listWidgets, tidyWidget, type WidgetScope, type WidgetsDeps } from "../core/index.js"

// A `read` procedure cannot resolve a write port: the kernel refuses `JOB_QUEUE` to it by name. The two
// handlers therefore ask for exactly what they use.
const readDeps = (container: Container): Pick<WidgetsDeps, "data"> => ({ data: container.resolve(DATA_SOURCE) })

const writeDeps = (container: Container): WidgetsDeps => ({
  ...readDeps(container),
  jobs: container.resolve(JOB_QUEUE),
})

export const widgetListImpl = implement(widgetList, ({ ctx }) => listWidgets(readDeps(ctx.container), ctx.scope))

export const widgetCreateImpl = implement(widgetCreate, ({ input, ctx }) =>
  createWidget(writeDeps(ctx.container), ctx.scope, { name: input.name }),
)

export const WIDGET_PROCEDURES = [widgetListImpl, widgetCreateImpl]

/**
 * The job. The worker rebuilds the scope the job was enqueued in and binds it as `FILES_SCOPE`, the
 * kernel's name for "the scope of this run": a module needs no binding of its own to read it.
 */
export async function runTidy(payload: { widgetId: string }, context: JobContext): Promise<void> {
  const scope = context.container.resolve(FILES_SCOPE) as WidgetScope
  await tidyWidget({ data: context.container.resolve(DATA_SOURCE) }, scope, payload.widgetId)
}

The use cases are pure TypeScript over those ports, so they run against the in-memory data source with no database:

// modules/widgets/core/widgets.ts
// What the widgets module does, as pure TypeScript over the ports in this file: no driver, no SDK, no
// framework (the boundary gate refuses them), so every rule here is tested against the in-memory
// data source with no database.

import type { TenantScope } from "@conusai-kit/core"
import type { DataSource, JobQueue } from "@conusai-kit/data"
import type { z } from "zod"
import { TIDY_JOB, type WidgetView, widgets } from "../contract/index.js"

/** A project scope: the only level a widget can live at. */
export type WidgetScope = TenantScope & { workspaceId: string; projectId: string }

export interface WidgetsDeps {
  readonly data: DataSource
  readonly jobs: JobQueue
}

type WidgetRow = z.output<typeof widgets.schema>

const view = (row: WidgetRow): WidgetView => ({ widgetId: row.id, name: row.name, createdAt: row.createdAt })

export async function listWidgets(
  deps: Pick<WidgetsDeps, "data">,
  scope: WidgetScope,
): Promise<{ widgets: WidgetView[] }> {
  // `forScope` is the only way to a repository: the tenant predicate is part of every query it runs.
  const page = await deps.data
    .forScope(scope)
    .of(widgets)
    .list({ order: [{ field: "createdAt", direction: "desc" }], limit: 200 })
  return { widgets: page.items.map(view) }
}

export async function createWidget(
  deps: WidgetsDeps,
  scope: WidgetScope,
  input: { name: string },
): Promise<WidgetView> {
  const row = await deps.data.forScope(scope).of(widgets).insert({ name: input.name, createdBy: scope.actor.userId })
  // Long work is a job: the request enqueues it and answers. The key makes a repeated enqueue one job.
  await deps.jobs.enqueue(scope, TIDY_JOB, { widgetId: row.id }, { key: `tidy:${row.id}` })
  return view(row)
}

/** The job: collapses the whitespace in a widget's name. A widget deleted since is nothing to do. */
export async function tidyWidget(deps: Pick<WidgetsDeps, "data">, scope: WidgetScope, widgetId: string): Promise<void> {
  const repo = deps.data.forScope(scope).of(widgets)
  const row = await repo.get(widgetId)
  if (!row) return
  const name = row.name.replace(/\s+/g, " ").trim()
  if (name !== row.name) await repo.update(row.id, { name }, row.version)
}

Its isolation fixture

The generated suite (pnpm test:isolation) calls every tenant-scoped procedure as a member of one organization with another's slugs and ids and expects a refusal. A procedure with no fixture fails it by name:

// modules/widgets/__tests__/isolation.ts
// What the generated tenant-isolation suite (test/isolation.test.ts) needs of this module, from this
// folder: the rows it seeds in each tenant, and one fixture per tenant-scoped procedure. The suite finds
// this file by its place, so there is no line to add outside the module; a procedure with no fixture
// fails the suite by name.
//
// A fixture is built from the *other* tenant's ids: the call is made as a member of one organization
// with another's slugs and ids, and has to be refused -- for being another tenant's, not for naming
// nothing, which is why `seed` leaves a widget in every tenant.

import type { TransportCaller } from "@conusai-kit/core"
import type { IsolationFixture, IsolationTenant } from "@conusai-kit/testing"

/** What a module's isolation file exports: the shape the suite reads, so it needs no import of the suite. */
export interface ModuleIsolation {
  seed(
    call: TransportCaller,
    inProject: { org: string; workspace: string; project: string },
  ): Promise<Record<string, string>>
  fixtures: Record<string, IsolationFixture>
}

const inProjectOf = (t: IsolationTenant) => ({ org: t.orgSlug, workspace: t.workspaceSlug, project: t.projectSlug })

export const isolation: ModuleIsolation = {
  async seed(call, inProject) {
    const widget = (await call.callByName("widgets.widget.create", { ...inProject, name: "Seed widget" })) as {
      widgetId: string
    }
    return { widgetId: widget.widgetId }
  },
  fixtures: {
    "widgets.widget.list": { input: (t) => inProjectOf(t) },
    "widgets.widget.create": { input: (t) => ({ ...inProjectOf(t), name: "Intruder" }) },
  },
}

5. An audit event and a usage meter

Both are named on the procedure's contract (.audit(...), .usage(...) in step 4) and declared in the manifest (audit, usage.meters in step 7). The web transport records them once the call has succeeded: one event in the platform's audit log, which the organization's administrators read at /<org>/audit, and one usage fact against the ConusAI organization that owns the deployment, with the organization's slug as the customer. No content goes in an audit event.

// modules/widgets/contract/constants.ts
// The module's names, in a file with no import: a browser bundle can take them without loading the
// data layer the schemas live beside.

/** An audit event (spec §9.4), declared in the manifest and named by the procedure that causes it. */
export const WIDGET_CREATED = "widgets.widget_created"

/** A usage meter: one fact per widget created, metered against the organization that owns it. */
export const WIDGETS_CREATED_METER = "widgets_created"

/** The job that tidies a widget after it is created. A request enqueues it and answers. */
export const TIDY_JOB = "widgets.tidy"

For an event that must happen exactly once even if the request's process dies -- a note's audit event and usage fact are the example -- publish a domain event in the same transaction and subscribe to it: modules/notes (core/subscribers.ts) is the worked pattern.

6. A job

Long work is a job. A write procedure enqueues it and answers (createWidget in step 4 does); the handler runs in the worker, in a span of its own, with the scope the job was enqueued in:

// the handler, in adapters/procedures.ts
export async function runTidy(payload: { widgetId: string }, context: JobContext): Promise<void> {
  const scope = context.container.resolve(FILES_SCOPE) as WidgetScope
  await tidyWidget({ data: context.container.resolve(DATA_SOURCE) }, scope, payload.widgetId)
}

FILES_SCOPE is the kernel's name for "the scope of this run", bound once for every module. The test runs it as the worker would (app.runJobs() in test/modules/widgets/widgets-flow.test.ts).

7. The manifest

// modules/widgets/module.ts
// The widgets module's manifest (spec §4.1): the one place it registers anything. `modules/index.ts`
// lists the manifest; nothing in apps/ changes.

import { defineModule } from "@conusai-kit/core"
import { repositoryBindings } from "@conusai-kit/data"
import { z } from "zod"
import { runTidy, WIDGET_PROCEDURES } from "./adapters/procedures.js"
import { TIDY_JOB, WIDGET_CREATED, WIDGETS_CREATED_METER, WIDGETS_ENTITIES } from "./contract/index.js"
import { en, lt } from "./i18n/index.js"

const tables = repositoryBindings(...WIDGETS_ENTITIES)

export const widgetsModule = defineModule({
  id: "widgets",
  version: "0.1.0",
  kit: "^0.1.0",
  requires: ["tenancy", "files"],
  // <module>.<resource>.<action>, with the lowest role that holds it.
  permissions: {
    "widgets.widget.read": "viewer",
    "widgets.widget.write": "editor",
  },
  procedures: WIDGET_PROCEDURES,
  repositories: { postgres: tables, mariadb: tables, mongodb: tables, convex: tables },
  // Forward-only SQL, one numbered file per change, run by the migrate job after the app's own.
  migrations: { postgres: "migrations" },
  jobs: { [TIDY_JOB]: { payload: z.object({ widgetId: z.string() }), handler: runTidy } },
  audit: [WIDGET_CREATED],
  usage: { meters: [WIDGETS_CREATED_METER] },
  // The project-level entry the organization home and the project home list; `widgets` is the segment
  // the web shell's module mount routes to `widgetsPage` (ui/pages.tsx), listed in modules/index.ts.
  ui: { nav: [{ scope: "project", label: "widgets.title", href: "widgets", permission: "widgets.widget.read" }] },
  i18n: { en, lt },
})

8. A page

A server component per path under /<org>/<workspace>/<project>/widgets, with no route file:

// modules/widgets/ui/pages.tsx
// The widgets module's pages, as the one server component the web shell mounts under
// `/<org>/<workspace>/<project>/widgets` (apps/web's module mount reads it from `modules/index.ts`).
// Every read is a procedure through the same caller the API uses; every write is a client component.

import type { Container, TransportCaller } from "@conusai-kit/core"
import type { I18n } from "@conusai-kit/i18n"
import type { ProjectPage } from "@conusai-kit/ui"
import type { ReactElement } from "react"
import { widgetList } from "../contract/index.js"
import { WidgetForm } from "./widget-form.js"

export type WidgetsPage = ProjectPage<TransportCaller, I18n, Container>

export const widgetsPage: WidgetsPage = async ({ slugs, caller, i18n }): Promise<ReactElement> => {
  const t = (key: string, values?: Record<string, string | number>) => i18n.message(`widgets.${key}`, values)
  const { widgets } = await caller.call(widgetList, slugs)
  return (
    <>
      <h1>{t("title")}</h1>
      <p>{t("intro")}</p>
      <WidgetForm slugs={slugs} labels={{ name: t("name"), submit: t("add"), failed: i18n.message("app.failed") }} />
      {widgets.length === 0 ? (
        <p>{t("empty")}</p>
      ) : (
        <>
          <p>{t("count", { count: widgets.length })}</p>
          <ul>
            {widgets.map((widget) => (
              <li key={widget.widgetId}>{widget.name}</li>
            ))}
          </ul>
        </>
      )}
    </>
  )
}

Add it to modules/index.ts, beside notes:

export const modulePages: Readonly<Record<string, () => Promise<ModuleProjectPage>>> = {
  notes: async () => (await import("./notes/ui/pages.js")).notesPage,
  widgets: async () => (await import("./widgets/ui/pages.js")).widgetsPage,
}

A client component asks useTransport() of @conusai-kit/ui for call, which carries the session and the CSRF token. The ui.nav entry of the manifest makes the page appear on the project's home page and in the lists of the workspace and the organization.

// modules/widgets/ui/widget-form.tsx
"use client"

// Adds a widget. A client component asks `useTransport()` of @conusai-kit/ui for `call`: the one API
// path every caller uses, with the session, the CSRF token and the problem+json refusals handled by
// the shell. Every label is a message the server passed in.

import { type ProjectSlugs, useTransport } from "@conusai-kit/ui"
import { type FormEvent, type ReactElement, useId, useState } from "react"

export interface WidgetFormLabels {
  readonly name: string
  readonly submit: string
  /** The app's `app.failed` message, with `{detail}` still in it. */
  readonly failed: string
}

export function WidgetForm({ slugs, labels }: { slugs: ProjectSlugs; labels: WidgetFormLabels }): ReactElement {
  const { call, describe } = useTransport()
  const nameId = useId()
  const [problem, setProblem] = useState<string | undefined>(undefined)

  const submit = async (event: FormEvent<HTMLFormElement>): Promise<void> => {
    event.preventDefault()
    const name = String(new FormData(event.currentTarget).get("name") ?? "").trim()
    if (name.length === 0) return
    setProblem(undefined)
    try {
      await call("widgets.widget.create", { ...slugs, name })
      window.location.reload()
    } catch (error) {
      setProblem(labels.failed.replace("{detail}", describe(error)))
    }
  }

  return (
    <form onSubmit={submit}>
      <div className="kit-field">
        <label htmlFor={nameId}>{labels.name}</label>
        <input id={nameId} name="name" type="text" required maxLength={120} />
      </div>
      <button type="submit">{labels.submit}</button>
      {problem ? (
        <p className="app-error" role="alert">
          {problem}
        </p>
      ) : null}
    </form>
  )
}

9. A message catalogue

Every label is a message, in both languages. lt.ts is typed from en.ts, so a missing key is a type error, and the scaffolded test fails when the two differ. Lithuanian has four plural forms (one, few, many, other); the example has the three a count of things needs:

// modules/widgets/i18n/en.ts
// English messages of the widgets module (spec §10.4). Every string a page of this module shows is a key
// here and in lt.ts; `__tests__/module.test.ts` fails when the two differ.

export const en = {
  title: "Widgets",
  intro: "Widgets are the example this guide adds.",
  name: "Name",
  add: "Add widget",
  empty: "No widgets yet.",
  count: "{count, plural, one {# widget} other {# widgets}}",
} as const
// modules/widgets/i18n/lt.ts
// Lietuviški widgets modulio pranešimai. Kiekvienas puslapio tekstas yra raktas čia ir en.ts faile.

import type { en } from "./en.js"

type Messages<T> = { [K in keyof T]: T[K] extends string ? string : Messages<T[K]> }

export const lt: Messages<typeof en> = {
  title: "Valdikliai",
  intro: "Valdikliai yra pavyzdys, kurį prideda šis vadovas.",
  name: "Pavadinimas",
  add: "Pridėti valdiklį",
  empty: "Valdiklių dar nėra.",
  count: "{count, plural, one {# valdiklis} few {# valdikliai} other {# valdiklių}}",
}

10. Test and gate

pnpm vitest run modules/widgets test/modules/widgets   # no database
pnpm format                         # Biome
pnpm openapi                        # a new procedure changes openapi.json; a test fails on drift
pnpm gate                           # typecheck, lint, boundaries, tests (isolation included), build
pnpm kit gate                       # the same, in a clean copy, installed from the registry
pnpm e2e                            # Chromium and axe against the whole app (Docker for Postgres)
// test/modules/widgets/widgets-flow.test.ts
// The widgets module against the whole app on the in-memory data source
// (test/support/notes-app.ts wires the registry, the queue and the outbox exactly as the apps do): a
// widget is created, listed, tidied by its job, audited and metered -- and a viewer may read and not write.

import { describe, expect, it } from "vitest"
import { makeNotesApp } from "../../support/notes-app.js"

describe("the widgets module", () => {
  it("creates a widget, lists it, and tidies its name in the job it enqueued", async () => {
    const app = makeNotesApp()
    const project = await app.seedProject("acme", "owner@acme.test")

    const created = (await project.call.callByName("widgets.widget.create", {
      ...project.slugs,
      name: "  Left   handed   widget ",
    })) as { widgetId: string }
    expect(app.queue.jobs.map((job) => job.job)).toEqual(["widgets.tidy"])

    expect(await app.runJobs()).toBe(1)
    const listed = (await project.call.callByName("widgets.widget.list", project.slugs)) as {
      widgets: Array<{ widgetId: string; name: string }>
    }
    expect(listed.widgets).toEqual([
      expect.objectContaining({ widgetId: created.widgetId, name: "Left handed widget" }),
    ])
  })

  it("declares the audit event and the usage meter the transport records for a create", () => {
    const app = makeNotesApp()
    const route = app.registry.router["widgets.widget.create"]
    expect(route?.audit).toBe("widgets.widget_created")
    expect(route?.usage).toBe("widgets_created")
    expect(app.registry.audit["widgets.widget_created"]).toBe("widgets")
  })

  it("lets a viewer read and refuses them a write", async () => {
    const app = makeNotesApp()
    const project = await app.seedProject("acme", "owner@acme.test")
    const viewer = await app.addMember(project, "viewer@acme.test", "viewer")

    await expect(viewer.call.callByName("widgets.widget.list", project.slugs)).resolves.toEqual({ widgets: [] })
    await expect(
      viewer.call.callByName("widgets.widget.create", { ...project.slugs, name: "Nope" }),
    ).rejects.toMatchObject({ problem: { status: 404 } })
  })
})

Your tests are yours, and live in two places: modules/<id>/__tests__/ for what needs only the module (use cases against createMemoryDataSource, the migration, the isolation fixtures), and test/modules/<id>/ for what needs the whole app, such as the flow below. Both are exempt from the template's licence header and its TEMPLATE-FILES list (node scripts/licence.mjs --check); a file you add anywhere else under test/ or apps/ is a template file and fails that check. A test under modules/ may not import test/support/notes-app.ts, which reaches the worker's system actor, and the boundary gate says so. test/support/notes-app.ts is the whole app on the in-memory data source, the one to build on.

For a screen, copy the shape of e2e/notes.spec.ts: sign in with signIn, make an organization with makeOrganization, open /<org>/<workspace>/<project>/widgets, and call violations(page) on every state.

Demo data for your module

apps/worker/src/seed.ts writes the demo data of pnpm dev with the kit's own procedures. A step added to seedDemo that calls your module's procedures as one of the demo people is the one edit outside modules/ that is meant for you; keep it idempotent (ask whether the row is there first).