Online shop server: on a real system#

A web shop on a Linux server with a HEART beside it: the server we assume, where everything runs, the code and configuration, how to install it, what to check before going live, and what the shop owner sees.

For illustration only. The server, the shop and its metrics below are assumptions, chosen so the steps can be concrete. Your shop will differ. This system has not been built: no real shop, customer or payment was involved. The Python template and heartd are tested on computers with simulated devices only, and have not run beside a real shop. Check every step on your own staging server.


Placement	A: on the shop's own server, as its own process beside the shop (a sidecar). Nothing here is safety-certified.
Start from	The Python template (`heart/embeddedstandard/python/v1.2611.3`), with the fault rules of the simulator. `heartd` for the Playbooks (Section 3.6).
Claim	HEART Basic, Autonomy Level A1. The three Playbooks (A4) need HEART Full and a rating first (Section 3.6).
Try it first	The simulator and the design case study

1. The system we assume#

Part	Assumed choice	Notes
Server	One Linux virtual machine, Ubuntu 24.04 LTS, 4 vCPU, 8 GB RAM	Any Linux with systemd and Python 3.8 or newer will do
Shop	A Python web shop (Django or Flask), run as 8 worker processes: systemd units `shop-worker@0` to `shop-worker@7`, grouped by `shop.target` (each worker has `PartOf=shop.target`)	The simulator also has 8 workers
Front	nginx on ports 80 and 443, in front of the workers	The only thing open to the internet
Cache	Redis, with the page cache in its own database
Database	PostgreSQL, on another server	The HEART never connects to it. It sees only query time, from the shop's metrics
Logs	`/var/log/shop`, rotated by logrotate	On the same disk as the shop
Outside services	A payment provider and a shipping API, over HTTPS	The shop checks them itself
Metrics	The shop serves a small JSON document on loopback: `http://127.0.0.1:8081/internal/heart-metrics`	You write this. It is not part of any template. Totals only, never a customer
Drain	Two loopback endpoints in the shop: `POST /internal/drain` (take no new orders) and `POST /internal/undrain`	You write these too. The drain flag lives in memory, so a restarted shop takes orders
On-call	A chat or pager webhook	Where `escalate()` sends alerts
Network	The HEART API on `127.0.0.1:43278` only; outbound HTTPS to the HEART Hub site and the webhook	Port 43278 is never opened in the firewall

Example metrics document#

{"workers": ["ok", "ok", "ok", "stuck", "ok", "ok", "ok", "ok"], "stuckSec": [0, 0, 0, 41, 0, 0, 0, 0],
 "requestsPerSec": 31.0, "servedPerSec": 30.8, "errorRatio": 0.002, "p95Ms": 103, "queue": 0,
 "memoryUsedPct": 60, "cacheHitPct": 93, "paymentOk": true, "shippingOk": true, "runningOrders": 2}

The names are ours. runningOrders is the number of checkouts in progress, which a graceful restart waits for. There is no customer, basket, order number or revenue in it, so none can reach the HEART.

2. Where everything runs#

┌──────────────────────────── Linux server (VM) ─────────────────────────────┐
│ nginx :443 ──► shop-worker@0..7 (shop.target) ──► Redis; PostgreSQL elsewhere│
│                   │ loopback: /internal/heart-metrics, /internal/drain       │
│                   ▼ read only                                                │
│ heart.service (user heart): checks every 30 s, API on 127.0.0.1:43278 ───────────► HEART Hub
│                   └─ may restart shop.target, only after a graceful drain ───────► on-call webhook
└──────────────────────────────────────────────────────────────────────────────┘     (outbound only)
/var/lib/heart (mode 0700): heart-monitor.json (device ID, claim code, key) · heart-lifecycle.json (owner key, Actions)

A separate process. A shop that crashes does not take its HEART with it, and a HEART that crashes leaves the shop serving (Part 2, Rule 4). It never sits between a customer and a page.
No customer data. The HEART reads only the metrics document and the free space on the disk. It has no database login, and card data never leaves the payment provider.
One action, through the API: a graceful restart of the shop, with the owner key. Orders, refunds, deploys and the payment provider stay in the shop's own admin tools. The three automatic fixes come later, with Playbooks (Section 3.6).

3. The code#

Copy heart.py from the Python template to /opt/heart/heart.py, and change it as below. Section 3.5 adds the other fault codes of the case study.

What	Where in `heart.py`
Identity	The four lines marked `CHANGE` (3.1)
Main job	`check_main_function()` (3.2)
Escalation	`escalate()` (3.3)
Drain and restart	`before_lifecycle()`, `carry_out()` and `LIFECYCLE_OPS` (3.4)
The other fault codes	A new `check_rules()`, called from `tend()` (3.5)
HEART Hub, lifecycle	Environment variables in the systemd unit (Section 4): `HEART_MONITOR_URL`, `HEART_LIFECYCLE`

Add one import at the top: import subprocess.

3.1 Identity#

SYSTEM_ID = "com.example.shop.web"       # CHANGE: as the case study's Card
NAME = "Example shop web tier"           # CHANGE
VERSION = "4.2.0"                        # CHANGE: the shop's version
INSTANCE = "web-1"                       # CHANGE: must not identify a person

Keep TEMPLATE as it is. It records where the code started.

3.2 The main job#

The case study's main job is: customers who arrive are served. A payment outage is not a lost main job: customers can still browse, and it has its own fault code (3.5).

METRICS_URL = "http://127.0.0.1:8081/internal/heart-metrics"   # assumed: your shop serves it on loopback


def read_metrics():
    with urllib.request.urlopen(METRICS_URL, timeout=5) as r:
        return json.load(r)


def check_main_function():
    """CHANGE: return (True, "") while the shop serves customers, else (False, "what is wrong, in plain words")."""
    try:
        m = read_metrics()
    except Exception:  # noqa: BLE001
        return False, "The shop does not answer its own health check."
    if m["workers"].count("ok") == 0 or (m["requestsPerSec"] > 1 and m["servedPerSec"] < 0.1):
        return False, "The shop is not serving any customer."
    return True, ""

Do not check the main job with a test order. A synthetic checkout would create orders and payments. Read what the shop already measures.

3.3 Escalation#

tend() calls escalate() while it holds the template's lock, so send in the background:

ONCALL_URL = os.environ.get("HEART_ONCALL_WEBHOOK")   # our own variable, not a template option


def escalate(alert):
    """CHANGE: tell the on-call person, in plain words."""
    text = f"Shop: {alert['summary']}. {alert.get('explanation', '')}"
    print(f"HEART escalate: {text}", file=sys.stderr)          # also in the journal
    if ONCALL_URL:
        req = urllib.request.Request(ONCALL_URL, data=json.dumps({"text": text}).encode(),
                                     headers={"Content-Type": "application/json"})
        threading.Thread(target=lambda: urllib.request.urlopen(req, timeout=10), daemon=True).start()

The message holds the fault's summary and explanation, nothing about customers. Check what your chat or pager service keeps.

3.4 Drain and restart#

The template restarts its own program. For a shop, a restart must first drain (take no new orders, let the running ones finish), then restart the shop, as the simulator does (API Standard 13.2 rule 3). If the running orders do not finish by timeoutSec, the restart fails and is never forced.

LIFECYCLE_OPS = ("restart",)   # changed from ("restart", "shutdown"): a shop is not shut down through HEART

SHOP_ADMIN = "http://127.0.0.1:8081/internal"   # assumed: your shop's loopback admin endpoints


def shop_post(path):
    urllib.request.urlopen(urllib.request.Request(SHOP_ADMIN + path, data=b"{}", method="POST"), timeout=5)


def before_lifecycle(operation, deadline):
    """CHANGE: drain. New checkouts get 503 'please try again in a minute'; running ones finish."""
    shop_post("/drain")
    try:
        while time.time() <= deadline:
            if read_metrics()["runningOrders"] == 0:
                return True
            time.sleep(0.5)
    except Exception as e:  # noqa: BLE001
        print(f"HEART: drain check failed: {e}", file=sys.stderr)
    shop_post("/undrain")      # not ready in time: take orders again. The restart fails; it is not forced.
    return False


def carry_out(operation):
    """CHANGE: restart the shop, then this program, as the template does (its Action is then marked done)."""
    sys.stderr.flush()
    if operation == "restart":
        subprocess.run(["/usr/bin/systemctl", "restart", "shop.target"], timeout=120)
        os.execv(sys.executable, [sys.executable] + sys.argv)
    os._exit(0)

systemctl restart shop.target restarts every worker with PartOf=shop.target. The heart user may do this, and nothing else, through one polkit rule (Section 4).

A force restart ("mode": "force") is destructive. Leave HEART_MONITOR_APPROVALS off, and the template refuses it with 403 presence-required: a person who needs one runs systemctl restart shop.target at the server. Section 9 shows the other choice.

3.5 The other fault codes#

The template raises only system.main-function-lost. Copy the case study's other rules from RULES in onlineShopSimulator.py, with your own thresholds:

SHOP_RULES = [   # CHANGE: code, severity, condition, summary. The thresholds are the case study's illustrations.
    ("software.crashed", "warning", lambda m: max(m["stuckSec"]) >= 30, "A web worker has stopped answering"),
    ("performance.memory-exhausted", "error", lambda m: m["memoryUsedPct"] >= 90, "Memory is nearly used up"),
    ("storage.nearly-full", "warning", lambda m: 2 < m["diskFreePct"] <= 10, "The disk is nearly full"),
    ("storage.full", "critical", lambda m: m["diskFreePct"] <= 2, "The disk is full: logs and uploads fail"),
    ("connectivity.dependency-unreachable", "error", lambda m: not (m["paymentOk"] and m["shippingOk"]),
     "An outside service the shop needs cannot be reached"),
    ("performance.backlog", "warning", lambda m: m["queue"] > 500, "Requests are piling up in the queue"),
    ("performance.slow", "warning", lambda m: m["p95Ms"] > 1500, "Pages take much longer than normal"),
]


def check_rules():
    """Ours, not the template's. Called from tend() inside its lock. Returns True if any rule holds."""
    try:
        m = read_metrics()
    except Exception:  # noqa: BLE001  the main-job check reports a shop that does not answer
        return False
    st = os.statvfs("/var/log/shop")
    m["diskFreePct"] = 100 * st.f_bavail / st.f_blocks
    any_open = False
    for code, severity, holds, summary in SHOP_RULES:
        active = holds(m)
        open_alert = next((a for a in state["alerts"] if a["code"] == code and a["status"] != "resolved"), None)
        if active and open_alert is None:
            alert = {"id": str(uuid.uuid4()), "kind": f"{SYSTEM_ID}.{code.split('.')[1]}", "code": code,
                     "severity": severity, "status": "escalated", "detectedAt": now(), "summary": summary, "explanation": ""}
            state["alerts"] = (state["alerts"] + [alert])[-100:]
            escalate(alert)
        if not active and open_alert is not None:
            open_alert.update(status="resolved", resolvedAt=now())
        any_open = any_open or active
    return any_open

In tend(), change the line that sets the status to:

            state["status"] = "failing" if not ok else ("degraded" if check_rules() else "healthy")

For the plain-words explanations, copy EXPLAIN from the simulator as well. performance.overloaded needs the shop's capacity, so add it when your metrics give one. software.crash-loop belongs with the Playbooks (3.6).

The template checks once every 30 seconds, so a condition is raised on the first check that sees it. The case study holds a condition for a few seconds first; to do the same, raise only on two checks in a row.

3.6 The three Playbooks (A4): HEART Full, with `heartd`#

The Python template has no Playbooks, allowlist or Stop. They are HEART Full. The simulator's Heart class has them, but nobody has rated it. The reference implementation that has them is heartd, configured as in its configuration guide. This is a sketch for rotate-full-logs; check it with heartd check before relying on it.

The capability, in custom_dir (custom/rotate-logs.json). It runs a program from an argument list, never a shell:

{"name": "com.example.shop.rotateLogs", "kind": "command", "title": "Rotate and compress the logs",
 "description": "Start new log files and compress the old ones.",
 "inputSchema": {"type": "object", "required": ["keepDays"], "additionalProperties": false,
                 "properties": {"keepDays": {"type": "integer", "minimum": 3, "maximum": 30}}},
 "risk": "write", "privacy": "internal", "physical": false, "callers": ["human", "rule"],
 "native": {"protocol": "process", "timeoutMs": 60000,
            "request": {"argv": ["/usr/bin/systemctl", "start", "shop-logrotate@{{arguments.keepDays}}.service"]}}}

The Playbook, in the rules file. The vendor ships it proposed: it does nothing until the shop owner changes it to adopted (in this file, adoption is by whoever installs it):

[{"id": "rotate-full-logs", "enabled": true, "status": "proposed",
  "description": "When the disk has 10 % or less free, rotate and compress the logs.",
  "when": {"signal": "diskFreePct", "op": "<=", "value": 10},
  "then": {"raise": {"code": "storage.nearly-full", "severity": "warning", "summary": "The disk is nearly full"},
           "capability": "com.example.shop.rotateLogs", "arguments": {"keepDays": 7}}}]

The allowlist entry and its envelope, in heartd.toml. Only the owner adds it, and the HEART never widens it:

[core]
mode = "act"

[[core.allowlist]]
target = "com.example.shop.rotateLogs"
initiators = ["rule"]

[core.allowlist.envelope]
maxPerDay = 4
cooldownSec = 300

[core.allowlist.envelope.arguments.keepDays]
min = 3
max = 30

The shop sends diskFreePct (and its other numbers) to heartd through host_socket, one line of JSON each: {"vital": "diskFreePct", "value": 8.5}. Register the owner's passkey under [presence], so that turning automatic action back on after a Stop needs a person, as the case study's Step 7 asks.

Open points to check before you rely on this:

restart-stuck-worker needs the number of the stuck worker. A rule's arguments are fixed, so one way is one rule per worker, each matching its own component. Test it.
clear-bloated-cache has an interlock, cache hit rate below 50 %. That needs the hit rate as a signal heartd can read immediately before it runs.
heartd takes port 43278 in place of heart.py, and does not report to HEART Hub in this release.
Claim A4 only after HEART-Rating and the case study's Step 11 tests pass: HEART Full, Autonomy Level A1 · A4 for software, storage and performance.

4. Build and install#

There is nothing to build. You need Python 3.8 or newer (Ubuntu 24.04 has 3.12).

sudo useradd --system --home-dir /var/lib/heart --shell /usr/sbin/nologin heart
sudo install -d -m 0755 /opt/heart /etc/heart
sudo install -m 0644 heart.py /opt/heart/heart.py
sudo install -m 0600 heart.env /etc/heart/heart.env          # HEART_ONCALL_WEBHOOK=https://...  (a secret)
sudo install -m 0644 heart.service /etc/systemd/system/heart.service
sudo install -m 0644 50-heart-shop.rules /etc/polkit-1/rules.d/50-heart-shop.rules
echo "net.ipv4.ip_local_reserved_ports = 43278" | sudo tee /etc/sysctl.d/99-heart.conf
sudo sysctl --system
sudo systemctl daemon-reload
sudo systemctl enable --now heart

Reserving the port stops Linux lending 43278 to an outgoing connection (from the heartd configuration guide).

The systemd unit (`/etc/systemd/system/heart.service`)#

[Unit]
Description=HEART for the shop
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/bin/python3 /opt/heart/heart.py
User=heart
Group=heart
StateDirectory=heart
StateDirectoryMode=0700
WorkingDirectory=/var/lib/heart
Environment=HEART_MONITOR_URL=https://heartcortex.ai
Environment=HEART_LIFECYCLE=1
EnvironmentFile=/etc/heart/heart.env
Restart=on-failure
RestartSec=2

# The HEART must never take resources from the shop.
MemoryMax=64M
CPUQuota=10%
Nice=10

NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
PrivateDevices=true

[Install]
WantedBy=multi-user.target

HEART_MONITOR_URL set turns HEART Hub reporting on, as MONITOR = True would. Use your own monitor site's address if you run one.
The template keeps heart-monitor.json and heart-lifecycle.json in its working directory, so they land in /var/lib/heart, mode 0600.
The limits follow packaging/heartd.service in heart-core-rs. 64 MB for Python is our guess: measure it.

The polkit rule (`/etc/polkit-1/rules.d/50-heart-shop.rules`)#

The heart user may restart the shop, and start the log rotation unit that the heartd sketch uses. Nothing else:

polkit.addRule(function (action, subject) {
  if (action.id == "org.freedesktop.systemd1.manage-units" && subject.user == "heart") {
    var unit = action.lookup("unit"), verb = action.lookup("verb");
    if (verb == "restart" && unit == "shop.target") return polkit.Result.YES;
    if (verb == "start" && /^shop-logrotate@[0-9]+\.service$/.test(unit)) return polkit.Result.YES;
  }
});

First start#

journalctl -u heart -b --no-pager

HEART: owner key (shown once, keep it safe): ...
{"heart": "heart/embeddedstandard/python/v1.2611.3", "listening": "127.0.0.1:43278"}
HEART Hub: register this device at https://heartcortex.ai/register?id=5c1e9a07d3b2f468 with claim code AB3CD-EF4GH-JK5LM-NP6QR

Put the owner key in your secret store. Then check it, on the server:

curl -s http://127.0.0.1:43278/heart/v1/pulse
KEY=...   # the owner key
curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:43278/heart/v1/arrhythmias

No reverse proxy. Do not route /heart/ through nginx: the API stays on loopback. An on-call person who needs it from elsewhere uses an SSH tunnel: ssh -L 43278:127.0.0.1:43278 shop-1.

Variant: a sidecar container (Docker Compose)#

If the shop runs in containers, run the HEART as its own container that shares the shop container's network, so it reaches the shop's loopback endpoints:

services:
  web:
    image: registry.example.com/shop:4.2.0
    volumes: ["shop-logs:/var/log/shop"]
  heart:
    image: python:3.12-slim
    command: ["python3", "/opt/heart/heart.py"]
    working_dir: /var/lib/heart
    network_mode: "service:web"          # the shop's 127.0.0.1 is its 127.0.0.1
    environment:
      HEART_MONITOR_URL: https://heartcortex.ai
    volumes:
      - ./heart.py:/opt/heart/heart.py:ro
      - heart-state:/var/lib/heart
      - shop-logs:/var/log/shop:ro
    read_only: true
    restart: unless-stopped
volumes:
  shop-logs:
  heart-state:

Leave HEART_LIFECYCLE off here. The container cannot restart the shop without Docker's socket, and must never be given it. Restarts are your orchestrator's.
When the web container is recreated, restart the heart container too, because it shares web's network.
The disk it sees is the volume's. To watch the containers themselves, see Docker with a HEART (experimental).

5. Before going live#

One identity per server. The device ID, claim code and key are made at first start in /var/lib/heart. If you make a VM image or snapshot after that, every copy shares them. Delete heart-monitor.json and heart-lifecycle.json before making an image.
The journal holds the secrets once. The owner key and the claim code are printed at first start. Read them, store them, and keep the journal readable only by administrators (the adm and systemd-journal groups on Ubuntu).
Keep /var/lib/heart in backups, or a rebuilt server is a new device that must be registered again.
Firewall: 43278 and 7685 closed from outside; outbound HTTPS to the monitor site and the webhook open.
Time: keep the clock synchronised (Ubuntu's systemd-timesyncd does). Signed decisions from the monitor site are refused if more than 300 seconds old.
Staging first. Run every test of Section 7 on a staging copy of the shop, then the HEART-Rating client, and claim only what passed.

6. What the owner sees#

Registration. The owner signs in at heartcortex.ai with an emailed code, opens the register?id=... address from the journal, and types the claim code. The page shows Healthy. It is private until the owner shares it. See the owner's manual.
Every day. Nothing. The HEART checks the shop every 30 seconds.
When something goes wrong:

Real event	The HEART notices	The owner and on-call see
A release makes every worker fail at start	`system.main-function-lost`: "The shop is not serving any customer."	Failing on the page; a message on call
One worker hangs on a request that never finishes	`software.crashed`, after 30 s stuck	A warning, Degraded. With A1, a person restarts the worker
Debug logging left on fills the disk	`storage.nearly-full` at 10 % free; `storage.full` at 2 %	A warning, then critical
A cache eviction bug uses up memory	`performance.memory-exhausted` at 90 %	An error, on call
The payment provider is down	`connectivity.dependency-unreachable`	An error: "customers can browse but cannot pay". Nobody fixes it through HEART
A flash sale	`performance.backlog` above 500 waiting	A warning. Adding servers costs money: the owner decides
A slow database	`performance.slow`, p95 over 1.5 s	A warning
A quiet night, or a big sale that is served	Nothing: business, not a fault	Nothing
The server is down, or the HEART stopped	Nothing can be sent	After 90 seconds the page shows Not heard from, and one email

What the site keeps: the status, the timing and each alert's fault code, severity and times. Never traffic, revenue, an Action, a worker number or anything about a customer. The summary and explanation reach the on-call route and the owner-key API only.
A planned restart. The owner, or a deploy script, asks with the owner key:

curl -s -X POST -H "Authorization: Bearer $KEY" -d '{"timeoutSec": 60}' \
     http://127.0.0.1:43278/heart/v1/lifecycle/restart      # 202, with the Action in Location

Customers who try to pay during the drain see the shop's 503 message ("please try again in a minute"), their baskets kept.

7. Test it on staging#

Cause every fault for real, on a staging copy with no real customers or payments, and check the right fault code appears within a minute on the API and on the HEART Hub page.

Fault	How to cause it on staging	Expect
Shop down	`sudo systemctl stop shop.target`	`system.main-function-lost`, Failing, a message on call
A worker hangs	`sudo kill -STOP <PID of shop-worker@3>` for a minute, then `kill -CONT`	`software.crashed`, then resolved
Disk nearly full	`fallocate -l <size> /var/log/shop/fill.bin` until 10 % or less is free; delete it after	`storage.nearly-full`; at 2 %, `storage.full`
Memory	Use memory with a stress tool, for example `stress-ng --vm 1 --vm-bytes 90%`	`performance.memory-exhausted`
Payment provider down	Point staging's payment setting at an address that does not answer, or block it in the firewall	`connectivity.dependency-unreachable`
Backlog	A load test at six times normal traffic	`performance.backlog`
Slow database	In a staging `psql` session, `BEGIN; LOCK TABLE <a busy table> IN ACCESS EXCLUSIVE MODE;` for 2 minutes	`performance.slow`
HEART stopped	`sudo systemctl stop heart`	The shop keeps serving; Not heard from after 90 s and one email; then Healthy after a start
HEART crashes	`sudo kill -9 <PID of heart>`	systemd starts it again in 2 s, with the same device ID
Graceful restart	`POST /heart/v1/lifecycle/restart` with `{"timeoutSec": 60}` and no slow order	`202`; the shop restarts; the Action ends `succeeded`
Restart that cannot finish	Start a staging checkout slower than 10 s, then ask with `{"timeoutSec": 10}`	The Action ends `failed`, "It was not forced"; the shop takes orders again
Force restart	`{"mode": "force"}`	`403 presence-required`
Shutdown	`POST /heart/v1/lifecycle/shutdown`	`501 not-implemented` (not offered)
Any other change	`curl -X PUT http://127.0.0.1:43278/heart/v1/properties`	`501 not-implemented` from the template. Nothing changes

Then run the HEART-Rating client against http://127.0.0.1:43278 from the server, and claim only what passed: HEART Basic, Autonomy Level A1.

8. Restarts and deploys: what to expect#

Deploys are not HEART's. Your deploy tool rolls out a new version. If it stops all workers at once, the HEART sees the shop stop serving and raises system.main-function-lost for that moment. Deploy one worker at a time, or have the deploy script ask for a graceful restart through the API.
A graceful restart can fail, on purpose. A checkout still running at timeoutSec stops it, and the shop goes on taking orders. Ask again, or restart by hand at the server.
A restart through HEART also restarts HEART. Its API is away for a few seconds, and the Action reads succeeded once it is back. The page does not go to Not heard from, which needs 90 seconds of silence.
With A1, people still do the fixing. The HEART tells on-call within 30 seconds that a worker hangs or the disk fills. A person restarts the worker or rotates the logs, until the Playbooks of Section 3.6 are adopted and rated.

9. Going further#

Approve a force restart from anywhere. Set HEART_MONITOR_APPROVALS=1. A force restart then waits as awaiting-approval, and the owner approves it on the monitor site with a code emailed to them (presence method remote-site). Without it, a force restart is done at the server. See the owner's manual, Section 9.
Explanations from an AI model. Set HEART_BRAIN=1 with the monitor on. The monitor site acts as the Command Center and holds the model keys; an answer is shown as a suggestion and never acted on. See the brain lease.
Several servers. One HEART per server, each with its own device ID. A hosting company can run its own monitor site from the open code in heart-monitor/.
A Node.js shop. The JavaScript template (heart/embeddedstandard/javascript-node/v1.2611.3) has checkMainFunction(), beforeLifecycle(), carryOut() and escalate(), with HEART_LIFECYCLE. It does not report to HEART Hub, and it always refuses a force restart. See restart and sleep.
The Playbooks (Section 3.6), for the narrow A4 of the case study: three reversible fixes, each with a range, a daily limit and a cooldown, suspended when a fix does not help twice in a row.