The Application API

Devices speak MQTT to modbee serve. Your application - a platform
backend, a gateway's own software - can speak to it a different way: over
the Application API, gRPC on a local socket, with no MQTT library at all.

The contract is proto/modbee/app/v1/app.proto.
The server answers all five of its services:

Hello's features lists what a server serves (authority, messages, connections, admin), so an
application asks rather than guesses.

System.Attach is an instance's presence. Open it with the messages to
publish should the instance vanish - its Wills, as an MQTT client has one:

const attach = system.Attach(metadata());
attach.write({ open: { instance: 'backend-1', wills: [
  { topic: 'spBv1.0/STATE/host-1', payload: offline, qos: 1, retain: true } ] } });
// ... and on a clean shutdown:
attach.write({ close: {} }); attach.end();

A stream that ends without close - the process killed, the socket gone -
has its Wills published, in order. One that closes first has none.

Turning it on

api:
  grpc: ["unix:/run/modbee/api.sock"]

modbee serve says where it listens:

modbee serve: Application API (gRPC) on unix:/run/modbee/api.sock

• The socket is the lock. It is made readable and writable by the server's
  own user and group only (0660): put your application in that group, or
  run it as that user. A Unix socket's path is at most 107 bytes.

• From another host, add a TCP address. It must have TLS and a token -
  the server refuses to start otherwise:

  api:
    grpc: ["unix:/run/modbee/api.sock", "tcp://0.0.0.0:7660"]
    tls:
      cert: /etc/modbee/api.pem
      key: /etc/modbee/api.key
      client-ca: /etc/modbee/apps-ca.pem    # optional: callers present a certificate it signed
    token-file: /run/secrets/modbee-api
  

  Every call then carries the token as gRPC metadata,
  authorization: Bearer <token>, on the socket as well; one without it is
  refused UNAUTHENTICATED.

• Any gRPC library reaches it: @grpc/grpc-js for Node.js
  (unix:/run/modbee/api.sock is a valid target), google.golang.org/grpc
  for Go, grpcio for Python. The standard grpc.health.v1.Health service
  answers too, for a supervisor's health check.

• The server needs gRPC to serve it: a build where gRPC was not found says
  so and does not start with an api: section (Ubuntu/Debian:
  libgrpc++-dev protobuf-compiler-grpc libprotobuf-dev). The published
  Docker image does not have it yet: build modbee from source with those
  packages installed until it does.

• An application on the same machine, on Windows as on Linux, can use TCP
  on loopback with the token alone - no TLS, since nothing off the machine
  reaches it:

  api:
    grpc: ["tcp://127.0.0.1:7660"]
    token-file: /run/secrets/modbee-api
  

  That is how it is reached on Windows (windows.md); the
  server refuses a loopback address without a token.

Messages

Reading every device's messages: Messages.Consume

Open a stream, send open with a group name and the topic filters it
wants, and the messages arrive, one Delivery each. Acknowledge each one
(ack, by its id) once you have handled it - stored it, acted on it - not
before.

call.write({ open: { group: 'platform', filters: ['#'], maxInFlight: 100 } });
call.on('data', (down) => {
  if (down.delivery) { handle(down.delivery); call.write({ ack: { ids: [down.delivery.id] } }); }
});

Run as many instances as you like with the same group: they share it.

Each delivery says who published it, which a device cannot fake: its
publisher (client_id, the login the server checked, and the login's
tag from its scope), broker_time_ms - when the server took
it, a clock you can trust - and the MQTT 5 properties it was published with.

• $ topics come only when a filter names them ($modbee/provision/+/request).
• events: true adds client events - connected, disconnected, subscribed,
  unsubscribed - dealt like the messages: to the instance that gets that
  client's messages, in order with them, acknowledged by id. The server
  needs events: enabled (events.md).
• exclude_origins keeps messages your application published itself
  (with that origin) out of the group.
• group: "" is a stream of its own, gone when it ends.
• Messages.ListGroups lists the groups; Messages.DeleteGroup removes one
  nobody is connected to.

examples/app-api/nodejs/consumer.js
is a whole member: start two, kill one mid-stream, and every message still
arrives.

Sending to the devices: Messages.Publish

Publish routes a message - topic, payload, QoS, retain, MQTT 5 properties -
to the devices subscribed to it. PublishStream does the same for a stream
of them, each answered by its request_id. origin names who published it,
for exclude_origins and for the publisher.origin a group sees. When the
answer comes is up to you (confirm):

The wait is bounded by the call's gRPC deadline (ten seconds without one).
When it comes first the answer is PUBLISH_PARTIAL, with how many
acknowledged and how many are pending - a subscriber that is offline
counts as pending, and its message waits in its session. A subscriber that
never could have it (its queue full, the message expired or too large for
it) also makes it PARTIAL. Use it for a command that must have reached its
device before your application goes on; a store-and-forward outbox deletes
only what came back PUBLISH_DELIVERED.

Connections to other brokers

Your application can have the server hold an MQTT connection to another
broker for it - a customer's, the platform's (a gateway's uplink), one a
Sparkplug host or edge node lives on - with no MQTT library of its own. The
server's own client does it, so the connection stays up while your
application restarts.

Connections.Put with a ConnectionSpec: an id, the servers in order of
preference, the client id, login, TLS, keep-alive, session, Will.

Get and List say each one's state, which server it is on, and its counts -
never its password or key. Watch streams every change of state.

TLS: ca_pem (empty: the system's trust store), and certificate_pem and
key_pem for mutual TLS, written beside connections.json. The name checked
is the server's host; server_name is not used yet.

A Sparkplug host on a broker you do not own, built on nothing but this:
examples/app-api/sparkplug-host,
which passes the Eclipse Sparkplug TCK's Host Application profile through the
API - its Will renewed on every connect with RECONNECT_WHEN_ASKED, its
STATE birth answered by the other broker's PUBACK.

Administration, and REST

Admin is what an operator - or the platform's own console - does to a
running server, without restarting it:

REST

The same calls, as JSON, on the HTTP listener /metrics is on:

metrics:
  address: 127.0.0.1
  port: 9464
api:
  grpc: ["unix:/run/modbee/api.sock"]
  rest: true

modbee serve: Application API (REST) on http://127.0.0.1:9464/v1/

curl -X PUT localhost:9464/v1/logins/sensor-7 -d '{"login":{"publish":["plant/sensor-7/#"]}}'
# {"login":{"name":"sensor-7","enabled":true,...},"password":"YT4SDLSozVV1cuLkvKLnKeB6xCC49mBb"}
curl 'localhost:9464/v1/retained?filter=plant/%23&payloads=true'
curl -X POST localhost:9464/v1/sessions/pump-1:kick

• The JSON is protobuf's own mapping: field names in lowerCamelCase,
  64-bit numbers as strings, bytes (a password, a payload) as base64, enums
  by name. A field the message does not have is refused, 400.
• Errors are the gRPC status as HTTP - 400 invalid, 401 no token,
  404 not found, 409 exists, 503 unavailable - with
  {"code": 404, "message": "..."}.
• Who may call. With api.token-file, every request carries
  Authorization: Bearer <token>. Without one, REST must be on a loopback
  address (metrics.address: 127.x); the server refuses to start otherwise.
  The listener has no TLS: reach it from another host through a proxy that
  adds it.
• One request at a time, as /metrics is: it is for an operator and a
  console, not for traffic. Messages in volume go over gRPC.

Logins decided by your application

authentication:
  username: platform              # your application's own login: never asked
  password: ...
  authority: {}                   # every other login is asked of your application
api:
  grpc: ["unix:/run/modbee/api.sock"]

Logins then come from your application instead of a login database
(authentication: database and authority cannot be used together). The
fixed logins in the file - username/password (an admin) and users: -
are still decided by the server itself: that is how your application, and the
modbee command, log in without asking themselves.

What is asked, and when

Your application opens Authority.Serve - a stream - and sends open. From
then on, when a device connects with a login the server has no answer for, it
sends your application a ConnectQuestion: the client id, the user name and
password, where it came from, the MQTT version, and the client certificate
when the TLS handshake verified one. Your application sends back a Decision
with the question's id.

Only CONNECT is asked, and only once per login. The answer says how long
it holds (ttl_seconds); until then the same client id, login and password
are answered from what was kept, without asking. The answer to "may it
connect" also says what the login may do - its scope - and the server
checks every PUBLISH and SUBSCRIBE against that itself, so no message ever
waits on your application.

While a CONNECT is asked, nothing waits: that connection is parked until the
answer comes, and every other connection carries on.

The answer

Say REFUSAL_UNAVAILABLE when you could not check - your database is
down - never "bad credentials". A device told its password is wrong stops
trying; one told the server is unavailable comes back. An unavailable answer
is never kept, and a login the server still holds an earlier answer for is let
in on that instead.

The scope

• ${username} and ${clientid} in a filter are put in for each session. A
  value containing /, + or # is never put into a filter: that rule then
  grants nothing (so a login named acme/pump-07 gets its topics from
  explicit rules, as with the login database).
• $ topics are reached only by naming them; a filter starting with a
  wildcard never reaches them. $modbee/admin/... is for the fixed admin
  login only, whatever a scope says.
• subscribe_wide is for a subscriber that asks for more than it may
  have - a Sparkplug host subscribing spBv1.0/#, which is Ignition's
  default. It is granted, and each message through it is checked: on a
  server shared by several customers the host receives its own groups and no
  one else's, retained messages included.
• deny keeps a login out of levels inside a namespace it otherwise
  owns - acme/+/$ota/#, every device's firmware level, for a shared login.
  A subscription that reaches into one (acme/#) is granted, and what it
  would deliver from there is held back.

A platform's shapes, as an example:

When your application is away

authentication:
  authority:
    timeout-ms: 2000         # how long a CONNECT waits for an answer
    stale-for: 86400         # seconds an expired answer still serves while your application is away
    cache-size: 1000000      # answers kept, whole server
    max-pending: 10000       # CONNECTs waiting for an answer at once; one more is told 0x89 Server busy (3.1.1: 3)

Changing a login while it is connected

Each answers once every CPU core's share of the server has done it, with how
many connections, sessions and answers it touched.

An example

examples/app-api/nodejs/authority.js
is a whole authority in Node.js - Hello, Serve, and coming back after a
restart - with a small table standing in for your database:

cd examples/app-api/nodejs && npm install
MODBEE_API=unix:/run/modbee/api.sock node authority.js

modbee-mqtt 0.1.0 (API 1.0, authority) at unix:/run/modbee/api.sock
asked about "acme/pump-07" (client pump-07): allowed