HXMD(8)                                                                HXMD(8)

       hxmd  -  collate the output from macro expanded shell commands executed
       in parallel

       hxmd [-gnsz] [-preload] [-B names] [-c  cmd]  [-C  config]  [-d  flags]
       [-E  compares] [-F literal] [-G guard] [-j m4prep] [-k key] [-K filter]
       [-o attributes] [-Q ward] [-r redo] [-X ex-config] [-Y top]  [-Z  zero-
       config] [xapply-opts] [xclate-opts] [m4-opts] control [files]
       hxmd -h
       hxmd -V

       hxmd  [-Agnsvxz]  [-preload]  [-a  c]  [-B  names] [-c cmd] [-C config]
       [-d flags] [-D name=value] [-e var=dicer] [-E names] [-G guard] [-H hr]
       [-i  input]  [-I  dirname]  [-j m4prep] [-J tasks] [-k key] [-K filter]
       [-L cap] [-M prefix] [-N else] [-o  attributes]  [-O  output]  [-Pjobs]
       [-R  req] [-Q ward] [-r redo] [-S shell] [-t tags] [-T title] [-U name]
       [-W widow] [-Z zero-config] control [files]

       Hmxd is a loop that iterates over a list of items,  and the  attributes
       of  the  each items -- while xapply(1l) just loops over items (names of
       the items).  Hxmd is implemented as a front-end for xapply and m4.   It
       also  uses xclate(1) to collate the output from any tasks run in paral-

       The positional parameter control is a shell command expanded by m4 then
       passed  on  to  xapply(1l)  for  parallel  execution.  When the control
       string is the empty string hxmd substitutes the  macro  HX_CMD  in  the
       hope that it is defined in the config file.

       Each of the files presented on the command line is processed through m4
       to create a customized file for the item selected.  Each file is avail-
       able  via the xapply percent expander, as well as an m4 macro named for
       its position on the command-line (prefixed with "HXMD_").

       Any one of files, zero-config, config, control may be drawn from  stdin
       with  the  common  dash  (-)  name.  Note that only one of these may be
       active in any instance of hxmd.

       If m4 fails to expand any of files or the control the selected item  is
       skipped,  but  processing  continues with the next selected item.  This
       allows the m4 macro m4exit to reject  an  item  at  the  last  possible

       The most common case is that each item represents a physical or virtual
       host.  Other common  items  would  be  devices  (tapes,  disks,  etc.),
       logins,  groups, configuration management directories, or names of data
       centers.  In the description below we shall assume  the  common  "host"

       The  environment variable $HXMD is read for options before any explicit
       arguments, unless the environment variable $HXMD_PASS is set to a  non-
       empty  string,  in  which  case  it  is  read.  Many options are passed
       through to another processor, for those see the  manual  page  for  the
       specified command.

              The  number  of  tasks  held ready in addition to those running.
              This allows hxmd to launch the next  task  immediately  after  a
              task  finishes.  The default of 2 usually works fine, unless the
              preparation time for each host is quite  long  compared  to  the
              update time.  Since task preparation is always done sequentially
              this number doesn't drive the concurrent demand for CPU as  high
              as -P does.

              Passed  on  the  xapply  to  present elements from tags as shell
              parameters.  This has some subtle interactions with shell evalu-
              ation.  Under -A the tokens from ptbw are appended as positional
              parameters which are not evaluated by the shell.  Without -A the
              tokens are appended to the shell command as text (via the xapply
              markup %t*), which forces the shell to interpret them as parame-
              ters.   Both  are  useful  tactics, depending on what the tokens
              represent.  Specification  of  an  explicit  -c  overrides  this
              internal  logic completely.  Use -dX to view the markup provided
              to xapply's cmd positional parameter.

       -a c
              Passed to xapply as given.  If you change this you should  spec-
              ify  a new value for -c.  An internal work-around allows setting
              this to plus (+) or even dollar ($).  Older versions  of  xapply
              and hxmd did not handle this at all.

       -B names
              Each  of  the  name  macros  must be defined (to any value) in a
              specified configuration file (under -C, -X, or -Z) for a host to
              be  selected.   Alternately any of names may be prefixed with an
              exclamation mark ('!') to select hosts which do  not  have  that
              macro name defined.

              When multiple names are specified (separated by a comma (',') or
              as multiple -B options) each name must be satisfied for the host
              to be selected.

       -B number
              When  multiple  config files are presented a count of the number
              of files that define a host may  be  matched  to  synthesize  an
              intersection  (-B  2)  or disjunction (-B 1) relationship.  Each
              file presented to -C or -Z which  defines  the  host  (at  least
              once) adds one to this counter.

       -c cmd
              Specify a custom cmd for xapply, rather than the internally con-
              structed defaults: "%+", with -J, -R, or -t: "%+ %t*", under -A:
              "%+  $@",  with $SHELL set to csh: "%+ $argv[*]", or with $SHELL
              set to perl: "%+ , @ARGV".  There is little reason to change the
              default, unless you use an even stranger shell.

       -C config
              The  list  of  hosts  and  attributes in the same format distrib
              used.  Multiple configuration files may be presented, the  first
              definition  of  any macro attribute for a given hosts remains in
              effect.  Multiple config files  may  be  separated  by  a  colon
              (':').  A leading path component of double-dash (--) is replaced
              by the first absolute path in the search list.

              Any config specification of a directory searches that  directory
              for a file with the name of the program suffixed with ".cf".

              After all config files have been read any macros defined in each
              zero-config file are applied as defaults, with the last  defini-
              tion given as the final value.

       -d flags
              Debug  mask  to allow some visibility into the processing logic.
              Not for general use, also filtered a bit and passed on the m4.

              B   Trace boolean excludes
              C   Debug cache recipe processing (Cache.m4)
              D   Debug defines given to m4
              G   Trace guards and compares
              L   Display host searches
              M   Debug missing values in host definitions
              R   Trace file cleanup
              X   Trace execution of the slave xclate (and xapply) process
              ?   Output a brief on-line flags usage message and exit

              These are reserved by a the layer above us, msrc(8l):

              P   msrc: trace the data collected from the makefile
              S   msrc: trace the m4 files built (distfile and script)

              All tools in this stack, that accept  flags,  also  provide  the
              usage  (?) option.  The mmsrc command supports the same flags as
              msrc.  Any flags that look like letters m4 might accept as flags
              are  passed  on, but only from the last value of flags specified
              on the command-line.

       -D name=value
              Filter defines passed on to m4.  This is likely used to  pass  a
              target  value  to  all  instances  of m4.  See m4 for a complete
              description.  Compare this to -Y, which just  impacts  the  host
              selection stream, or -Q which just impacts the redo logic.

       -e var=dicer
              All  instances  are passed on to xapply as presented.  A strange
              side-effect of the way xapply is called (with %+ as the cmd)  is
              that  the  ordinal associations of the percent parameters is off
              by 1, so %2 is really what one might think should be  %1.   This
              is  because the command-line -e is done before the %+ shifts the
              parameter once to the left.  It is not  so  much  a  bug  as  an
              astonishing feature.

       -E compare
              Select  hosts  based  on  macro values.  Each host must meet the
              expressions given to participate  in  guard  selection  process.
              The  comparison  may  represent  a string or numeric comparison,
              either of which may be prefixed with an exclamation  mark  ("!")
              to negate each host's selection:

                     The macro must have the exact value.

                     The macro must not have this value.


              In  any of the above macro may be a hostname, m4 macro name fol-
              lowed by parameters enclosed in parenthesis, or a  number.   The
              command-line  parser  does  not  count  m4 quotes, so you cannot
              quote an unbalanced parenthesis in the parameter list.

                     The macro must be numerically related to  the  expression
                     in  the  way  rel-op specifies (<, >, <=, >=, ==, !=).  A
                     fixed integer left-hand-side may be  presented,  with  an
                     optional  leading  minus-sign  "-".   The  expression may
                     actually be any m4 expression, usually based on data from
                     the present host's macro attributes.

              When you need a very complex selection use -Y and/or -j to force
              an m4 include directive into the selection stream  to  define  a
              macro  to  make  the expression easier to specify on the command

       -F literal
              Force more (fewer) parameters to be interpreted as literal text,
              rather than as filenames.  This allows macro expanded text to be
              processed by the xapply dicer. An alternate use is to force con-
              trol  to  be read from a file (rather than literal text) with -F
              0, the default value of 1 generates the usage given in the  syn-

              A  negative literal counter forces parameters from the right end
              of the command-line to be taken as literal values.  Assuming the
              literal  doesn't include all the arguments, the control is taken
              as a file containing an m4 marked-up script in that case.

       -G guard
              An expression to select hosts by name.  The m4 expression  guard
              must  reduce to the name of a host (or hosts, separated by white
              space) which should be selected.  The default guard is the macro
              HOST.   Note  that  a  given  host  may  select hosts other that
              itself, for example a client might select her  IMAP  server  for
              some tasks.  The selected hosts must be listed in some config or
              zero-config and must be allowed  by  -B's  inclusion  specifica-
              tions.   (It is still poor form to list hosts in a zero-config.)

              Multiple guards are  permitted,  if  not  encouraged.   Multiple
              selections of the same host do not process the host again.

              Print only a brief help message.

       -H hr
              Passed on to xclate as given.

              Install a gtfw(1l) diversion around this instance, if none exist
              in the current environment.

       -i input
              The xapply input selection option, when given.  This is used  in
              combination with -C - to give the tasks a useful stdin.

       -I dirname
              Include  directory  name passed to m4.  The special name double-
              dash (--) is replaced with the first absolute path in the search

       -j m4prep
              Each  specified  file is included in the m4 command line used to
              filter files after any -D, -U,  or  -I  options.   Use  this  to
              include  a  common macro definition header in every file.  Using
              this option to force other arbitrary options into  the  m4  com-
              mand-line  is  not a good idea, better to replace m4 in a forced
              $PATH with a script that filters options then calls the real m4.
              A  leading  path  component  of  double-dash (--) is replaced as

              The macro HXMD_PHASE may be used as a flag to check the name  of
              the processing step, but is usually not required.  See below.

       -J tasks
              Passed  to  ptbw (via xapply) to scale the total number of tasks
              started by all conspirative descendant xapply's.

       -k key
              Change the name of the key macro from  HOST  to  key.   This  is
              really  not  generally recommended; it adds a configuration item
              bound to each configuration file which is hard to keep track of,
              and  makes  the default join operation impossible when they mis-
              match.  On the other hand, it allows a  reverse-map  abstraction
              that can be worth the complexity.

       -K filter
              This  is  the  command that drives any redo logic, after all the
              tasks have finished and their status has been  processed.   When
              the  redo logic is triggered the default value is ${PAGER:-more}
              HXMD_0.  See REDO below.

       -L cap
              Passed on to xclate as given.

       -M prefix
              Replace the macro prefix  HXMD_  with  prefix,  don't  use  this
              unless you are going to set it for efmd and msrc as well.

              Passed on to xapply as given, however the output comes to stderr
              rather than stdout.  This is because the xclate filter maps xap-
              ply's  stray  output  to stdout to the widow stream, which is by
              default stderr (see xclate's -W option).

       -N else
              Passed on the xapply to trap zero hosts matching the expressions
              provided.   The  exit code from this command is not available to
              REDO logic (under -r), since it doesn't represent any single key
              in  the  configuration files, so it becomes the exit code of the
              hxmd process.

       -o attributes
              Build  a  configuration  file  that  represents   the   selected
              attributes,  in  the order selected.  The empty string is useful
              and valid list.  The  generated  file  includes  the  attributes
              listed  as  columns  after the key macro, all specifications are
              joined with a single tab between them.  The  generated  file  is
              only available after host selection, the file's name is given by
              the m4 macro HXMD_U_MERGED.

              A prefix of exclamation mark ('!') or hash ('#') on a name=value
              specification under -D exempts the specification from the gener-
              ated configuration file, but is not passed on to m4.

              Specification of  the  key  macro  in  the  attributes  list  is
              allowed,  this  moves  that  macro to another column (it used to
              duplicate it).  The markup percent (%) is also allowed as a name
              for the current key macro.

       -O output
              Passed on to xclate as given.

              The  number  of  tasks xapply runs in parallel.  See -V's output
              for the compile-time default.

       -Q ward
              This option may be repeated to build a header block at  the  top
              of  the  redo  logic  m4  stream.   It  is comparable to -Y, but
              impacts the redo command  expansion  rather  than  the  original
              selection  stream.   It  is  likely used to set a divert, define
              macros, or include a file to make redo processing easier.   Many
              wards may be provided, the order of the options is preserved.

       -r redo
              This  m4 expression is processed once for each selected host, to
              form something like a guard list.   It  transforms  the  status,
              unique index, and various host attributes into a stanza given to
              filter to remeditate any failed updates.   In  addition,  rather
              than  any  processed files, the original command line files list
              is presented in the HXMD_N list.  When any redo logic  is  trig-
              gered  the default value is HOST HXMD_STATUS HXMD_U, with -k and
              -M taken into account.  See REDO below.

       -R req
              Passed on to xapply to request the allocation  of  req  elements
              from tags for each worker task.

              Turn off slow-start, see SLOW START below.  The xapply option of
              the same name (-s) is always presented  to  xapply  unless  -P's
              jobs parameter is forced to 1, or either -H or -T are presented.

       -S shell
              The xapply shell option, when presented.

       -t tags
              Passed on to xapply, when presented.

       -T title
              Passed on to xclate as given.

       -U name
              Tell m4 to undefine the macro name.

              Show only version information and exit.

       -W widow
              Passed on to xclate as given.  Often used to divert any -n  out-
              put to a file.

              Passed  on  to  xapply  as  given,  to trace actions as they are

       -X ex-configs
              Provided files are scanned as a config file would be, however no
              additional  new  hosts  are accepted.  Any host specified in any
              config or zero-config is extended by any in-scope macro  defini-
              tion, but never created.  This is quite useful when operating on
              a subset of hosts from a larger political domain,  sometimes  -B
              is not enough when you are not sure every host is defined in the
              "larger" configuration.  Such files never increment the  counter
              used by -B (HXMD_B).

              A directory specification implies a file in that directory named
              for the program name plus ".xf".

       -Y top
              The m4 expression top is inserted at the top of the host  selec-
              tion  m4  input stream, the option may be repeated to install as
              many top directives as necessary.  This is likely used to change
              the  default diversion, include a file, or define a common func-
              tion for all guards and compares.

       -Z zero-config
              Zero configuration is not exactly correct for this  option,  but
              it  is  close.  The option sets default macro bindings from each
              of the zero-config files listed.  In effect each zero-config  is
              read before each of the -C configuration files, then after those
              config files are processed these values are treated as  defaults
              for  any  unset macros, for every host defined in any file.  The
              last value assigned from the list of zero-config  files  is  the
              default,  while  the first explicit reference to a host locks in
              the macro values in-scope at that point in the processing.   Any
              hosts  defined  within  zero-config  are treated as defined once
              (for -B's count), which is not recommended, unless the  file  is
              also listed as a ex-config under -X.

              A directory specification implies a file in that directory named
              for the program name plus ".zf".

              Remove the $HXMD_PASS environment variable's value after  it  is
              interpolated into the argument vector. (That is to say set it to
              the empty string.)  This might  be  considered  a  self-destruct
              switch,  as it is often presented in that same environment vari-

              This environment variable is a colon separated list of  directo-
              ries, these are searched to find zero-config files, config files
              and each of the files provided.

              Use in exactly the same way as  xapply,  specifies  the  default
              value when -P is presented with no adjacent integer value.  This
              can lead to  quadratic  parallelism,  when  multiple  copies  of
              either program are nested without logic to reduce this value.

              Used as the top-level directory for any scratch files (or direc-
              tories) created.

              The shell which executes any filter  command,  else  use  -S  or
              $SHELL.  When none of those are present try /bin/sh.

              Intersected  with  the character flags given to the -d option to
              pass valid debug flags on to m4.  This should  contain  all  the
              debug  flags  the  local version of m4 will accept.  This option
              also may set some debug/trace options for other subsystems.

              Either of these  environment  variables  may  be  consulted  for
              options.   When  HXMD_PASS  is  set  and not empty it is read in
              preference to HXMD.  Otherwise HXMD is read.   In  either  case,
              the proposed options are inserted before the command-line speci-

       The control command and each of the files presented are passed  through
       m4  with  several macros defined to construct custom data for each host

       These custom  files  are  constructed  in  a  subdirectory  of  $TMPDIR
       (default  /tmp/)  created  by mkdtemp(3).  Each generated file is named
       for the basename of the corresponding source file on the command  line,
       and should have about the same modes as the source file (given the lim-
       its of the processes effective uid and gid).

       The m4 macros are drawn from the command line options destined for  m4,
       the  files,  and the macros defined for the target host, and any m4prep
       files included under -j.

       A macro for each custom filename is formed with the prefix "HXMD_"  and
       the  number of the file (0 is the control command, 1 is the first file,
       and so on).  Additionally the macro "HXMD_C" contains the count of  the
       number of files presented..

              The count of the files given on the command line.

              The  name of the Nth generated file, all filenames are available
              for all files (as they are known in advance).

              The count of the config files which define this host.

              The name of a configuration file that defined the host.  Some of
              the attributes may be from another file, of course.  This mostly
              used to flag the source of the auto.cf data.

       The built-in m4 macro m4exit may be used to de-select any host,  simply
       exit non-zero for any of the m4 filters.

              The  accumulated  list  of all the configuration files provided,
              separated by colons.  This combined with the HOST macro and  the
              macro  below should allow recursive calls to hxmd.  These macros
              are defined only once in any m4 stream, always before  any  user
              header lines provided to -Q or -Y.

              The  accumulated  list  of all the zero configuration files con-
              sulted.  This macro is not defined unless some  zero  configura-
              tion files were specified (or implied).

              The  accumulated  list of all the extra configuration files con-
              sulted.  This macro is not defined unless some  ex-config  files
              were specified.

              This  macro  expands  to a unique number for each host selected,
              starting at zero.  This matches the default behavior of xapply's
              %u markup.  Note that this macro is not defined in the selection
              process.  If a host fails to expand any files no  shell  command
              runs, so the unique number is set to negative 1 (-1).

              The count of the number of unique hosts presently defined.  This
              is provided for client metrics.  It is defined to be zero before
              any  definition,  then  assumes  the count of the hosts defined.
              Note that due to hosts skipped due to failed file expansion, the
              number of processed hosts may be less than this total.

              The  count  of  the  number  of unique hosts presently selected.
              This is provided for use in any ''percent done'' metric.  It  is
              undefined  before  any  selection, then assumes the count of the
              hosts selected.

              The file generated by -o.  This is a  valid  hxmd  configuration
              file  and  is often used to make recursive calls, or to chain to
              msrc.  It also includes the macros requested in -o's white-space
              separated  attributes list.  The new file is usually passed down
              under -C or -X.  It is removed before hxmd exits.

              This macro indicates the destination of the current m4  stream's
              output.   It is set to "selection", a counting number, "filter",
              or "redo" as hxmd processes the many instances of  m4  explained
              below.   To avoid pollution of the the macro name-space any file
              may consult this macro,  especially  those  included  under  -j.
              Avoid  depending  on  this  by  number, that breaks when someone
              changes the  command-specification  in  really  unobvious  ways.
              Under  mmsrc the command stream is called the "cmd" phase, under
              efmd "selection" is followed directly by "output".

       An additional m4 filter is created to select the hosts from the  config
       files  presented.   The output from this filter must be a list of hosts
       to process, separated by white-space.  Hostnames that are  not  defined
       in any configuration file are ignored.

       The  input  stream  begins with any top lines provided to -Y, which are
       commonly used to set divert levels, or include some macro  definitions.

       For  each host that -B allows, its macro attributes are pushdef'd, then
       any -E macro checks generated by -E are applied, then  any  guards  are
       applied,  some  cleanup  for checks are also appeneded.  To finish that
       cycle each of the hosts' macro attributes are popdef'd.   This  process
       is repeated for each defined host.

       Think  of  multiple compares as being "and"'d while multiple guards are

       These selection expressions are all processed in a single execution  of
       m4 to allow diversions to order the hosts as needed.

       The  results  from  that filter are read as a sequence of hosts to pro-
       cess.  Each host matching a name from the config (or  zero-config)  and
       allowed by -B are processed in the order they are first presented.

       Any  non-zero  exit code presented via the built-in m4 macro m4exit may
       be used to fail the entire selection process, while an explict success-
       ful exit just truncates the list.

       The  command  line  passed to xclate includes all the options specified
       for xapply and some forced options.

       The -m option is forced on the xclate command to start  a  new  managed
       environment, the -N option is presented to direct notifications back to

       The -fzmu options are forced on the xapply command run  as  the  master
       process  under  the  xclate to specify the input stream is -print0-like
       (see find(1)) and to set the managed output switch.  The -u option pro-
       vides  the  notification  stream as a sequence of the task numbers fin-
       ished, rather than some random value of xapply's %1.

       The parallel factor (-P) defaults to 6 (which is good for dual  proces-
       sor  hosts).   The suggested value for aggressive abuse of an multipro-
       cessor host is between 3 and 5 times the number of CPU threads.  Remem-
       ber  that the tasks started are not all the work being done, as many m4
       processes may be run to filter control commands and files  even  before
       the first task is launched.

       Large  parallel factors specified for the xapply -P option tend to cre-
       ate issues in programs like ssh-agent and the X server.  The slow-start
       logic in hxmd ramps up the number of parallel tasks to avoid this "fork
       bomb" effect.  It does not limit the rate  at  which  individual  tasks
       complete,  and are reused (all required tasks might complete before the
       parallel factor reaches jobs for quick control commands).

       A series of m4 processes filter control and files to fill a "slot"  for
       each  host.  When these were run with no delay between them hxmd tended
       to drive the system load too high, a 1/35 second sleep between  initial
       slot  fills helped a lot.  This feature is always enabled, however hxmd
       actually filters faster as completed tasks  finish:  terminating  tasks
       refill  their slot with no delay, and hxmd fills about 2 slots ahead of
       the parallel factor (tune via preload).

       There is an invariant here that makes this process easier to use: slots
       are  always  filled  sequentially, and the files are always constructed
       left to right.  Factor in that the sequence order of the selected hosts
       is carried into slot processing, as is the command line order of files.
       Thus provisions may made to carry state generated by  a  previous  host
       (or hosts) to any subsequent one(s).

       The  -D  macro  binding  option is intended to document the use of this
       invariant.  Any name like CURRENT_STATE might be used to  document  the
       file built to hold any host-to-host state.  At the file-to-file level a
       file named empty in hxmd's library is usually given  as  the  first  of
       files.   Convention documents that empty file as a state-file, which is
       reconstructed for each host, that is used to carry information from one
       m4  instance  to  another (usually with an m4 include directive).  Once
       built the file is usually moved (appended) to  a  "safe"  place  before
       each host's processing completes.

       These slot fill processes, when combined with the work performed by the
       control command for each task, may still drive the  system  load  quite
       high.  To avoid overwhelming the system with tasks hxmd starts the par-
       allel factor at 1 then increases it by 1 about twice a second until the
       specified  jobs  are  all  enabled.   This description is intentionally
       vague, as the math done has only limited precision, and is  subject  to
       change without notice.

       The  -s option removes this safety feature by starting all jobs as soon
       as they are ready.  Use with care, ptbw, or not at all.

       In the previous section we assume that all host-based  markup  is  pro-
       cessed  with  m4.   It is sometimes quite cumbersome to use m4 to build
       files for each host.  If you need more power you may construct a "cache
       directory"  that  produces  output for each host with any processor you
       can drive from make(1).

       Any of files included on the command-line which are  actually  directo-
       ries  must  contain  a  make  recipe file marked-up in m4 markup called
       "Cache.m4".  This file is expanded with all the common attribute  defi-
       nitions,  then used as a make recipe file to create the contents of the
       cached file.  The name of the processed recipe file is provided as  the
       m4 macro HXMD_CACHE_RECIPE.

       To  drive the make process hxmd needs a target name.  Starting with the
       basename of the directory, any leading dots are removed.   Then  every-
       thing  after  the  right-most  remaining  dot (if any) is removed.  The
       empty string is mapped to the name of the host being  processed.   This
       target is available as HXMD_CACHE_TARGET, and the path to the directory
       itself as HXMD_CACHE_DIR.  (Since HXMD_CACHE_DIR may be relative to the
       current working directory, never use as if it were an absolute path.)

       The  make  process  must output the contents of the file (not build the
       file locally).  This allows hxmd to collect the stream into a  file  it
       manages so it may be removed after processing.

       Thus  a  specification  of "." (the current working directory) in files
       processes a local "Cache.m4" into a recipe file, then updates the  name
       of  the  host.   Building  a symbolic link to "." named ".all" uses the
       common target "all" for every host, of course the "Cache.m4" file might
       use  m4  markup  to customize that target for each host.  This allows a
       common directory to provide a caching service for  multiple  uses.   No
       locking  is  required  for  a single instance of hxmd as the processing
       here is done in sequence  (as  all  markup  processing  is,  as  stated

       Cache  directories are often used to gather information from the target
       host, to update CURRENT_STATE files, or to provide  audit  information.
       Support  in  msrc  (under -m) allows for the setup and teardown of such
       directories.  It is also possible to leverage hxmd's  -K  specification
       (below).   Usually  there  is  a  target in the cache recipe file (or a
       "Control" recipe file) that allows 1-stop cleanup.

       There is no mandate for a make recipe file named Control: rather it  is
       a  local  convention that any Cache.m4 files has a plain (not marked-up
       for m4) recipe file that has targets that  clean  any  cached  data  to
       reset  the  cache after a run, no matter the success of the task.  Usu-
       ally the Control recipe file has a "clean" target  that  links  to  the
       cache  cleanup,  see  msrc(8l).   Another valid local site policy would
       specify an mk(1l) marker within Cache.m4 to provide  any  cleanup  com-

       Under  any  combination  of  the  -r, -K, or -Q options we get a second
       chance to process each selected host.  This time we know the exit  sta-
       tus from the command, see exit(3).

       In  this  mode  hxmd  passes xclate the -r option, which enables status
       code recovery.  As tasks finish hxmd collects the status  from  xclate,
       and  keeps  a  list  of all those values.  After all the processes have
       exited 2 additional m4 process environments are constructed.

       The first begins with definitions to  define  HXMD_N  to  the  original
       files parameters.  This replaces the binding to the customized versions
       from m4 under normal processing.  The count of the files  is  bound  to
       HXMD_C, as usual.  The macro HXMD_0 is bound to the name of the file we
       are building, which sounds odd, but makes recursive  use  of  the  file
       easier.  These defines are not subsequently removed or refreshed.  Each
       of the ward directives provided under -Q are processed once.

       For each host selected, each the directives presented  to  control  are
       defined,  after  2 additional macros: HXMD_STATUS and HXMD_U.  The exit
       status is bound to HXMD_STATUS, and the  value  that  %u  had  for  the
       host's control command is bound to HXMD_U.  Each of the redo directives
       presented  to  -r  are  processed.   Then  all  those  definitions  are
       retracted, before the next host is processed.

       The output from that m4 stream is saved in a temporary file (which will
       be removed immediately before hxmd exits).  The name of  that  file  is
       bound to the macro HXMD_0 for the next step.

       A  script  is  formed  by opening yet another m4 process to convert the
       filter directive into a shell command.  The stream  includes  the  same
       original files and count bindings.  None of the ward directives or host
       specific definitions are included, as they should be reflected  in  the
       output cached in HXMD_0.

       If  the  filter  starts  with  a  pipe (|) character, that character is
       removed and the file created in the previous step is  open'd  read-only
       as  stdin  for  the  new shell.  Otherwise stdin will be left as-is, in
       which case it is expected that  the  shell  command  will  used  HXMD_0

       The stdout of the filter process is directed to the stderr of the orig-
       inal hxmd process.

       What action this shell program takes to close-the-loop  on  the  update
       process is driven by the needs of the implementor.  Some queue an at(1)
       job to retry failed hosts, some send e-mail, or try a  more  aggressive
       intervention.   The  exit  status  of  hxmd  is the exit status of this

       The status bound to HXMD_STATUS is not always the exit  status  of  the
       control  command.  When any of the files are marked-up such that m4exit
       forces a non-zero exit, then the status for that host is 1000 plus  the
       value specified.  If any m4 process exits with a signal then the status
       value is 2000 plus the signal number.  When xapply (pid %p) receives  a
       USR1  signal it short circuits any remaining tasks by never fork(2)'ing
       them (it still consumes the stream of  commands  to  prevent  a  broken
       pipe).   Any short-circuited commands are assigned the synthetic status
       3000. See xapply(1l) and signal(3).  This allows a distinction  between
       failed  commands, signal termination, failed preparation, or short-cir-
       cuit requests.

       hxmd -V
              Output only the standard ksb-tool version information.

       hxmd -C site.cf -n HOST
              Given the HOST macro is the fully qualified domain name of  each
              host, the output is a list of fully qualified domain names.

       hxmd -B FIXME "scp %1 HOST:/tmp/rx$$ && ssh HOST -t \$SHELL \
                  -x /tmp/rx$$ FIXME \&\& rm /tmp/rx$$" myscript
              Run  the  script myscript for each host with FIXME defined, pro-
              cessed for that host, under $SHELL.  Remove the  remote  script,
              after  it  is  run  with the expansion of FIXME as the parameter

       hxmd -B FIXME "scp HXMD_1 HOST:/tmp/rx$$ && ssh HOST -t \$SHELL \
                  -x /tmp/rx$$ FIXME \&\& rm /tmp/rx$$" myscript
              Same as the above, but use an m4 macro, rather than  the  dicer,
              to  find  the  name  of the myscript file after processing.  The
              preferred interface is site policy: heavy xapply use implies  an
              obvious preference.

       hxmd -B FIXME -P1 -i/dev/tty "scp ..." myscript
              Allow  for  hosts  that  require  an interactive password or key
              authentication request.  By default, input  is  redirected  from

       hxmd -Cmy.cf -F2 HX_CMD HOST data.m4
              Place the value of "HOST" where xapply can see it as %1, and the
              file data.m4 filtered by m4 as %2.  We assume HX_CMD is  defined
              in my.cf to put them to good use.

       hxmd -C my.cf -F -3 tfile HOST COLOR $$
              Process  HOST,  COLOR,  and  the process-id of the current shell
              thought m4 as literal strings, process tfile as  a  file.   Then
              run the processed version of tfile with the three literal param-
              eters bound to $1, $2, and $3.

       hxmd -C site.cf -D IMON='hostname' -E HOST=IMON 'echo HOST'
              Long way to find out that the local hostname is defined  in  the
              dmz.cf configuration file.

       hxmd -C site.cf -G 'hostname' 'echo HOST'
              Do  the same check (as above) with a guard.  The output is empty
              when the current hostname is not defined in the  specified  con-
              figuration file.

       hxmd -C dmz.cf -Y "define(\'MATCH',\'SUN5')dnl" -E HOSTTYPE=MATCH 'echo HOST'
              List all the hosts that have a HOSTTYPE of SUN5, the MATCH macro
              option might come from a different data-source than  the  guard.
              Another  source for a definition of MATCH would be a file speci-
              fied under -Z.

       hxmd -P1 -C dmz.cf -E HOSTTYPE=SUN5 -F3 'echo %[1.1] in %[1:$]' 'HOST' 'LOC'
              Use hxmd largely as a linear xapply to report on the location of
              each  SUN5  host  by  short-name and its location after the last

       hxmd -C black.cf -C blue.cf -B 2 'echo COLOR HOST' |sort
              List all the hosts that are in both black.cf and blue.cf  sorted
              by  color.   Note  hosts with no COLOR macro show as the literal

       hxmd -C black.cf -C blue.cf -B 1 'echo HOST'
              List all the hosts that  are  in  exactly  one  of  black.cf  or

       hxmd -C black.cf -X blue.cf 'echo COLOR HOST' |sort
              List  all  the  hosts  that are in black.cf with extra data from
              blue.cf.  Hosts that are given only  in  blue.cf  are  not  pro-

       hxmd -C black.cf -C blue.cf -E "WOUND=BRUISE" -G RAISE 'echo HOST'
              Compare  two  macros (WOUND and BRUISE) for each host to process
              only the hosts select by the RAISE macro.  Note that when  RAISE
              has more than one host, they must be separated with white-space.

       hxmd -C all.cf -E "1!expr(-index(SERVICE,deploy))" 'echo HOST'
              List the hosts for which the macro  SERVICE  contains  the  word
              "deploy".  Note that the m4 macro index returns -1 when the sub-
              string is not in the first argument, which we convert to a posi-
              tive 1, then compare for inequality.

       hxmd -C all.cf -E "-1!=index(SERVICE,deploy)" 'echo HOST'
              The same request as above, given as a relational operation.

       hxmd -C black.cf -C blue.cf -G RAISE -B 2 'echo HOST'
              List  the  hosts that are RAISE'd by hosts in both configuration
              files, and themselves appear in both.

       hxmd -C black.cf -C blue.cf -G "ifelse(HXMD_B,2,\'RAISE')" 'echo HOST'
              List the hosts that are RAISE'd by hosts in  both  configuration
              files, which must appear in at least one file.

       hxmd -C black.cf -C blue.cf -B 2 'echo RAISE' |sort -u
              List  the  hosts that are RAISE'd by hosts in both configuration
              files, even if they don't appear in either  configuration  file.
              When  RAISE  might  hold  more than one hostname some additional
              filers might be applied.

       hxmd -C black.cf:blue.cf -G RAISE -o COLOR '[ %u -eq 0 ] &&
                  cat HXMD_U_MERGED'
              Better than the above, we capture  the  hosts  selected  by  the
              RAISE  attribute  and  remember  their COLOR attribute in a well
              formatted configuration file (which is valid input for -C or -X,
              but  not  so  good  for  -Z).   Each of the hosts must appear in
              either black.cf or blue.cf -- which makes  the  semantics  quite
              different.   The expression on the front ensures we only get one
              copy, not one for each selected host.

       hxmd -D!CHOICE=lv426 -C black.cf:blue.cf -G RAISE -o COLOR \
                  '[ %u -eq 0 ] && cat HXMD_U_MERGED'
              The same as above, but a local definition of "CHOICE" is  avail-
              able  only  to  the  m4  started  by  this instance of hxmd, not
              included as an active attribute in the HXMD_U_MERGED file.

       hxmd -d M -C grizzly.cf -B !HOST ': HOST'
              Just check the configuration file for proper format, since  each
              host  must  at  least have HOST defined the boolean filter never
              allows any execution of the shell comment.

       hxmd -C adams.cf -B TEA -B !TEA -x 'destroy HOST'
              Unfortunately hosts cannot hold both TEA and not TEA at the same
              time, this will never match a host.

       hxmd -C adams.cf -G TEA -x 'destroy HOST'
              This  typing  error (changing the 'b' to a 'g') is usually harm-
              less as the odds that the name of a tea is also the  FQDN  of  a
              host are almost infinitely improbable.

       hxmd -Z common.cf -C host.cl -x 'PREEN HOST'
              Apply  some  macro  command  (PREEN)  to  each  host in the list
              host.cl.  The common.cf file defines the required macros for all
              hosts,  as  by  convention  the  file  host.cl is just a list of

       hxmd -K '|sed -e "/^0/d"' -r "HXMD_STATUS HOST" -C host.cl 'PREEN HOST'
              Run the PREEN command for each host,  and  report  the  list  of
              failed  hosts,  on  stderr.   A better script might direct sed's
              output someplace safer, see the next example.

       hxmd -K 'sed -e "/^0/d" <HXMD_0 |Mail -s "preen failed" charon' \
                  -r "HXMD_STATUS HOST" -C host.cl 'PREEN HOST'
              About the same as above, but we mail the  results  to  "charon".
              The message has 2 columns (the exit status and the host's name).

       hxmd -I $HOME/lib/hxmd -Q "include(Format.m4)dnl" \
                  -K "include(ReQueue.m4)Minutes(HXMD_0,HXMD_C,15)" \
                  -r "FORMAT(HOST)" -C host.cl 'PREEN HOST'
              A more indirect way to specify a very complex configuration.

       SHELL=/bin/ksh hxmd -P1 -Csite.cf 'F=HOST ;
                  [ -f ${F} -o -f ${F%%%%.*}.dom.example.com ] || echo HOST'
              Output the names of hosts that don't have a file named for  them
              in  the current directory.  The file may be the HOST name or the
              FQDN we build by removing all but the first dots from that name.
              Note  that  we  have  to  double the percent-signs in the ksh(1)
              markup because xapply uses them as markup.

       gencmd |hxmd -Cmy.cf -F0 - data.m4
              Assuming gencmd builds an m4 marked-up shell script  we  process
              it with macros from my.cf then run it with $1 set to a processed
              version of data.m4.  I'll grant you this might make you think  a
              lot  about  how  we  got  to this level of abstraction (distrac-

       jot 6 0 |hxmd -Q dnl -P1 -C -  'if [ 3 = HOST ]; then
                  kill -USR1 %p; exit 63; fi; sleep HOST'
              Process the integers from 0 to 5, short-circuit the  loop  at  3
              and exit 63.  After processing display the default redo informa-
              tion, which notes that 4 and 5 were never attempted:
                 0 0 0
                 1 0 1
                 2 0 2
                 3 63 3
                 4 3000 4
                 5 3000 5

       The configuration file format is designed to make it easy to list hosts
       with  common  attribute macros.  A macro may be defined with an assign-

       The value may be given as an m4 string (as above) or a C string:

       Or the macro may be undefined by setting the value to  a  period  ('.')
       with no quotes around it, as:

       The  format  for  a  host declaration is defined by a header line.  The
       header line starts with a percent ('%') and lists the macros needed  to
       complete  a  host definition, separated by spaces or tabs.  For example
       to set the macro columns to "HOST", "IP", and "OWNER":
               %HOST  IP  OWNER

       The declaration of a host is simply a line which provides the  promised
       values separated by white space, in quotes or not:
              imp.npcguild.org 'KS Braunsdorf'

       By  providing  a  literal  period  ('.') for a column the corresponding
       macro is left undefined.  By providing too few  values  the  right-most
       unfulfilled  macros  are left undefined.  Use a the debug option (-d M)
       to sanity check configuration files.  Extra values  on  the  end  of  a
       declaration are always a fatal error.

       A column header may also be specified as a literal period ('.') to dis-
       card an entire column of data.  This is only used to allow a  processor
       (other  than hxmd) to mask data from a file, as editing the header line
       is much less risky than filtering the (quoted and complex) data  lines.

       The  common  hash  ('#')  comments  are  also allowed (to end of line).
       Blank lines are ignored (unlike crazy distrib).

       When it is too difficult to use m4 to generate a needed file  or  value
       directly  from hxmd one might use a shell command or perl(1) program to
       prefabricate the data, while  generating  an  additional  configuration
              while ! mkdir $TDIRX ; do
              xapply -f "make-it %1 >$TDIRX/%u; echo %1 \\\'$TDIRX/%u'" host.cl >host.cf
              hxmd -C host.cf ...
              rm -r $TDIRX
       In  effect  this  is what hxmd is doing for you with m4 for each of the
       files presented.  With the added feature that  the  files  are  removed
       after they are used, and only created slightly in advance of their use.

       Replication of an identical host definition  has  no  effect  on  hxmd.
       This  is an important difference between the way distrib and hxmd treat
       their configuration files.  While  hxmd  reads  all  the  configuration
       files  given then processes the list of unique hosts, distrib only pro-
       cesses hosts from the last config file presented  and  will  process  a
       given host (from that file) more than once.  It is unclear that this is
       a bug in either program, but it creates some confusion, none the  less.

       A  list  of  hosts  is  a valid configuration file for hxmd.  Putting a
       macro assignment at the top, HASSRC="yes" allows an intersection with a
       super-set  configuration  file.   There is no way to do this under dis-
       trib's rules.

       The m4 quote style is not allowed in distrib's config,  before  version

       This  is  a  table of approximate conversions from distrib's options to

              Distrib option                     hxmd option
                -C config.cf   -C config.cf
               -C config.mcf   efmd -G 'hostname' -Csite.cf -F0 \
                               config.mcf | hxmd -C - cmd
                    -G guard   -G guard, but result is a hostname, not yes or 0
                          -I   now the default
                      not -I   '-E !HOST='hostname' -E !HOST="\'localhost'"
                          -a   -B HOST
                          -S   -B HASSRC [-X source.cf]
                     -m host   -E HOST=host
                               -G host
                               -E SHORTHOST=host
                     -t type   -E HOSTTYPE=type
              -t type1,type2   -G "ifelse(HOSTTYPE,\'type1',\'HOST')" \
                               -G "ifelse(HOSTTYPE,\'type2',\'HOST')"

       Another common question I get is "Can we  match  the  SHORTHOST  column
       (from  distrib)  against  a list in a file?"  Configuration files join,
       under hxmd, only on the key column given by -k.  The intersection (dis-
       junction)  set is selected with -B 2 (1).  For more complex expressions
       a temporary file (or pipeline) may be constructed.

       For  this  example  assume  we  have  a  list  of  short  hostnames  in
       /tmp/ksb.cl  and a configuration in npc.cf which is distrib-compatible.
       To "aggravate" the intersection set of hosts I would run:
              hxmd -kSHORTHOST -C/tmp/ksb.cl:npc.cf -B2 'aggravate HOST'

       Note that this game is not limited to the SHORTHOST column, but  it  is
       hardly  fair  to use this as a logical-or operation on other fields: -k
       HOSTOS is strongly discouraged.  In effect that requests the first  (or
       a random) node for each HOSTOS listed.

       The  key  values may not include white-space anymore.  For a while hxmd
       allowed white-space in key value, but that  became  too  confusing  for
       most  applications,  and  makes  guard  parameters  harder to spell and

       Consider that hxmd is used to iterate over items which are  not  hosts,
       so  mapping  case  would be a bug.  However hostnames (the default key)
       are not case sensitive, so there should be a way to allow a case insen-
       sitive  key  comparisons in the selection process.  I suggest that map-
       ping all questionable hostnames to lower-case via tr(1), or  the  like,
       is a fine compromise.

       Guard  logic  sometimes compares attribute macros which are not defined
       for every node.  In this case, m4 outputs complaints  on  stderr  about
       errors from eval (aka expr).  It is not possible for hxmd to completely
       check arbitrary expressions before giving them to m4.

       Xapply is enough power for most applications, this program  is  a  hard
       icon for naive users.  But xapply is not enough power for msrc, so this
       is a step between the two.

       It would be poor-form to install a program named "HX_CMD".

       The entire config stream must be read to complete the m4 selection pro-
       cess.   This  means  sending  the  output of a long running pipeline to
       "hxmd -C - ..."  waits for the entire input pipeline to  finish  before
       it  starts  the  m4 selection filter, let alone any tasks.  This should
       also be viewed as a feature.

       The heavy use of m4 still shows "questionable judgment".

       Failed requests leave junk under $TMPDIR, which is nifty for debugging,
       but needs to be purged once in a while.

       The  code allows for multiple -Z zero-config files specified as a colon
       (:) separated list.  Using this is usually deemed "poor form", you usu-
       ally really need multiple -X files.

       Sometimes  m4  substitutes unintended macro names in the control string
       (viz.  "include", "unix", "dumpdef", or any  other  other  m4  macros).
       Using  shell  single  quotes (') presents some difficulty as well.  The
       author suggests the same tactics used for complex xapply cmd templates,
              + surround control in shell double quotes (")

              +  use  m4  quotes  ('...') around any text that might include a
              common m4 macro, only unquote macros you wish to expand

              + use a backslash to protect backquote ('), quote  ("),  literal
              dollar sign ($), and backslash (\) itself

              +  the  m4 facilities dumpdef and -t are quite useful to explain
              odd results, in combination with -n

              + put long commands in  a  file,  debug  the  file  with  m4(1),
              efmd(8l)  or distrib's -E option, include it on the command line
              with "include(path)dnl", or specify "-F 0 path".

              + don't ever use csh as your login shell

       KS Braunsdorf, NonPlayer Character Guild
       hxmd swirl rmSpam.ksb.npcguild.org

       sh(1),  m4(1),  find(1),  csh(1),   hxmd(5l),   efmd(8l),   xapply(1l),
       xclate(1l),   ptbw(1l),   distrib(8l),   msrc(8l),  mmsrc(8l),  mk(1l),
       dicer(5l), sed(1)

                                     LOCAL                             HXMD(8)