Guide 2: HEART-Embedded photo editing software#

This guide takes a photo editor from day 1 to an application that knows its own health, protects people's photos, and serves plain requests through one standard language.

The finished example is in heart-core-rs/examples/photo-editor. Its central flow runs in conformance/wasm.mjs: a plain request, then approval, then the app's own export function.

Before you start: the two rules for a creative app#

1. People's photos never leave the computer through HEART. Pixels, file paths and location data are never-share. If you add cloud AI, it receives summaries such as an exposure histogram, never the image.
2. HEART asks before it changes someone's work. Exporting, enhancing or deleting always shows a confirmation, even when the request came in plain words.

Day 1: Choose how HEART lives in your app#

An app does not need a network port. The Core runs inside the app.

The rest of this guide uses the WebAssembly route.

Step 1: Give the app an identity (the HEART Card)#

{
  "system": { "id": "com.example.lumen.photo-editor", "name": "Lumen Photo Editor",
              "version": "0.1.0", "alias": "lumen" },
  "level": "full",
  "bindings": ["http-push", "in-process"],
  "pulse": { "intervalSec": 60 },
  "modules": ["custom", "core"],
  "shares": { "pulse": "public", "rhythm": "internal", "userData": "never-share" },
  "valve": {
    "llmCeiling": "internal",
    "classifications": [
      { "path": "rhythm:/attributes/filePath", "level": "never-share" },
      { "path": "rhythm:/attributes/gps", "level": "never-share" }
    ]
  },
  "ai": { "modes": ["byok"], "providers": ["any"], "offline": "rules-only" }
}

level must be full, because plain-language serving creates actions that need approval. Testing caught this: a Card set to connected could create such actions but never approve them. The standard now states the rule.

Step 2: Decide what is private#

Even a field you forget to classify is treated as private. HEART fails closed.

Step 3: Define what the app can do#

Each feature that plain language or other tools may use becomes a command definition with protocol: "function", meaning "the app performs this itself".

{
  "name": "com.example.lumen.export_image",
  "kind": "command",
  "title": "Export image",
  "description": "Export the open image as JPEG, PNG or WebP at a chosen width.",
  "inputSchema": { "type": "object", "required": ["format", "width"], "additionalProperties": false,
    "properties": { "format": { "type": "string", "enum": ["jpeg", "png", "webp"] },
                    "width":  { "type": "integer", "minimum": 16, "maximum": 20000 } } },
  "risk": "write",
  "privacy": "internal",
  "callers": ["human", "ai"],
  "native": { "protocol": "function", "target": "app",
              "request": { "call": "exportImage", "format": "{{arguments.format}}", "width": "{{arguments.width}}" } }
}

The example defines export image, auto enhance, clear preview cache, and a property GPU memory used.

Text inputs must be limited by enum or pattern. HEART refuses a definition with a free-text input, because free text could inject a command.

Step 4: Connect the Core to your app#

// heart-worker.js
const { instance } = await WebAssembly.instantiate(wasmBytes, {});
const w = instance.exports;

function send(fn, value, ...rest) {
  const bytes = new TextEncoder().encode(JSON.stringify(value));
  const ptr = w.heart_alloc(bytes.length);
  new Uint8Array(w.memory.buffer, ptr, bytes.length).set(bytes);
  return fn(ptr, bytes.length, ...rest);
}
const result = () => JSON.parse(new TextDecoder().decode(
  new Uint8Array(w.memory.buffer, w.heart_result_ptr(), w.heart_result_len())));

// Start, with a fresh random seed each time.
send(w.heart_start, { card, custom, rules, allowlist: ["com.example.lumen.clear_cache"],
                      instance: "desktop-1", seed: Array.from(crypto.getRandomValues(new Uint8Array(32))) }, Date.now());
setInterval(() => w.heart_tick(Date.now()), 1000);

// Every request goes through one function.
async function api(request) {
  send(w.heart_api, request, Date.now());
  let r = result();
  while (r.effect) {                         // HEART needs the app to do something
    const outcome = await perform(r.effect); // your code
    send(w.heart_resume, { token: r.token, ...outcome }, Date.now());
    r = result();
  }
  return r;                                  // { status, body }
}

// Carry out effects: your app's own functions, or a model call.
async function perform(effect) {
  if (effect.kind === "native" && effect.native.protocol === "function") {
    const { call, ...args } = effect.native.request;
    try { return { ok: await app[call](args) }; } catch (e) { return { error: String(e) }; }
  }
  return { error: "not supported" };
}

HEART itself never touches files or the network. It asks your app through effects, so it can never do anything your code does not allow.

Step 5: Report what happens (the host feed)#

send(w.heart_report, { event: "autosave.failed", severity: "error", summary: "Disk full" }, Date.now());
send(w.heart_report, { vital: "gpuMemoryShare", value: 0.93 }, Date.now());
send(w.heart_report, { event: "plugin.crashed", component: "film-grain-plugin" }, Date.now());

Never put a file path or image data in a report. Use counts, sizes and the names of your own components.

Step 6: Reflexes (Tending rules)#

Testing confirmed that reporting gpuMemoryShare: 0.95 raises the alert straight away. Only clear_cache may run without asking, because it is listed in allowlist and is reversible.

Step 7: Plain-language requests in the app#

Wire your command bar to POST /heart/v1/serve. With no AI model, these are the tested results:

When the result is pending-approval, show a dialog with the exact values. When the person confirms, call POST /heart/v1/actions/{id}/decision with approve. HEART then asks your app, through an effect, to run exportImage.

Step 8: Bring your own key (cloud AI)#

With ai.modes = ["byok"], each person enters their own provider key in your settings. The key stays on their computer, and HEART never sees it.

When HEART needs a model, it hands your app a consult effect holding a provider-neutral request that has already passed the Valve. Your app sends it to the provider with the user's key, and returns the reply. Set a daily budget in your settings screen.

HEART checks every answer: a model can only choose from commands that exist, and every value it claims the person said must actually appear in what they said.

Step 9: Let your support team see app health (optional)#

If you run a HEART Gateway, the app can push its Card and Pulse there. Only the levels you choose leave the computer, and never photos. Your team then sees heart://support-hub/systems/lumen/pulse for each installation that opted in.

Ask the user's permission first, and record it.

Step 10: Test before shipping#

What is not ready yet#

• Calling app functions from a separate heartd process. Today it works only with the Core inside the app.
• A TypeScript package and a C interface. The example calls the WASM exports directly.
• Use beyond testing in Node. It has not run inside a real photo editor yet.

Timeline#