UT_LINK_SUPERVISOR — Escalating recovery of a remote site's comms link

Gets a remote site back online by itself when its link to SCADA or the IoT platform
dies: first a modem restart, then a modem power cycle, then one controlled reboot of
the controller, then modem retries further and further apart. The reboots are counted
and limited per day, and the count is kept over the reboot itself, so a dead link can
never make the controller reboot in a loop.

Pins

Record UT_LINK_SUPERVISOR_DATA

Status codes

statusCode is a UT_STATUS (an HMI reads its number); statusText is the same in
words. downFor says how long the link has been down, timeLeft the time to the step.

How it works

• The steps are timed from when the link went down (or from power-up, if it is down
  then). Each is optional (its time 0 = off). The outputs are pulses of their full
  length, even if the link comes back meanwhile.
• Retries: once the restart and power cycle steps are past, the modem is tried
  again (a power cycle, or a restart if the power cycle step is off). The first retry
  comes restartAfter after the last step (powerCycleAfter if the restart step is
  off), each wait twice the one before, up to maxBackoff. The back-off starts again
  once the link has stayed up for maxBackoff; a short flap keeps it.
• Reboots: at most maxRebootsPerDay. The reboot is counted and saved first;
  RebootReq follows rebootSaveWait later, so the count survives the reboot. At the
  limit no reboot is asked for, however long the link stays down: the retries go on.
• The day: with a set Clock (2000 or later), the count starts again at each date
  change. Without one (unwired, or the clock not set yet), it starts again
  countWindow of running time after the first counted reboot: a rolling window. Its
  progress is saved in 24 steps (one an hour by default), so a reboot or a power cut
  loses at most one step and the window only gets longer; time switched off does not
  count. countResetIn shows when it ends.
• Kept over a power cut: only the reboot count, its day and the window's progress.

Using it

LinkSup(LinkOk := NOT ScadaWd.CommsFail, RebootAllowed := NOT P1.Running, Clock := Now,
        Data := LinkSupCfg);
ModemRestart := LinkSup.RestartModem;
ModemPowerOff := LinkSup.PowerCycle;      (* relay contact in the modem's supply *)
IF LinkSup.RebootReq THEN
  (* ask the controller to reboot *)
END_IF;