ptbw - parallel token broker wrapper
ptbw -m [-dQz] [-J jobs] [-N path] [-R req] [-t tags] [utility]
ptbw [-AqQ] [-depth] [-R req] [-t tags] [client]
Wrapping an application in ptbw allows multiple cooperating processes
to queue for an available resource (CPU, modem, device, or what have
you). The utility program inherits an environment which allows ptbw to
manage a resource pool for descendent clients.
Parallel processing is a key feature of UNIX shell commands. Common
utilities such as make, batch(1), at(1), pr(1), and xapply, as well as
network and system daemons, and shell pipelines all use the timesharing
features of UNIX. Based on the speed of the underlying hardware, pro-
cess timing, and unrelated resource utilization the results of the
interactions between generations of parallel processing vary widely.
Static allocation of resources makes this issue worse. As resource
demand in the system eases it should be possible to migrate resources
to more intensive tasks. Options like make's -j do not adjust to the
over-all demands on the system. And work-arounds (like uucp's lock
files) only limit through-put, they never dynamically adjust to allo-
cate more resources.
For example a make with a large -j max_jobs might run a recipe that
starts a recursive make with a very large max_jobs. When that task
usually only finds up-to-date dependencies this will go unnoticed but,
in a rare case must run a large update, it may consume massive CPU.
The degradation to overall system though-put, caused by thrashing the
virtual memory system, may unreasonably impact the host. This can be
amplified with xapply's -P option interacting recursively with make and
xapply itself. It is almost a given with large hxmd instances.
With ptbw it is possible for very large process trees to cooperate to
maximize system though-put, without complexity enough to make the sys-
tem too costly to maintain. This is clearly important for systems
which support interactive transactions as well a resource constrained
Processors like ptbw are becoming more relevant as hosts with more CPUs
and threads become available: code back-ported to lesser nodes may sim-
ply not run at all without clever resource management.
If the program is called as ptbw then no options are forced. The envi-
ronment variable $PTBW is consulted for options, see ENVIRONMENT below.
Specify resource control from an outer copy of ptbw. This may
allow some more complex resource management, but is usually poor
form, as it defeats a limit imposed by the directly enclosing
instance of ptbw -m.
Append the allocated resource to the client argument vector.
This may allow some shells direct access to the arguments under
their -c option. See CLIENT below.
Use this option start managers which are detached from the
linked environment. The normal operation of ptbw requires each
master process to publish the name of the controlling socket in
the environment, so descendant processes may connect to allocate
resources. When the name of the manager's socket is passed by
an out-of-band protocol it may be undesirable to include it in
the standard chain.
Print a short help message.
Set the minimum number of clients expected to run in parallel.
This also specifies, in combination with -R, the number of
tokens created when no tags data is provided.
Manage resources for descendant processes. The utility is exe-
cuted as given (not through any shell indirection) as nice(1) or
time(1) would run their utility. See MANAGER below.
Bind the master to a specific name on the filesystem. This is
useful when an unrelated process must connect to the service.
Absolute paths are taken as-is, a relative path is bound in a
This toggle removes ptbw complaints about constrained resources,
for example when an explicit tags file is too short to support
the req or jobs parameters.
When the enclosing diversion is running with no child process
(it was started with a colon (:) as the utility), this lets that
persistent instance know it is time to stop. The enclosing
instance waits for all the current clients to finish before ter-
mination. It might be a bug that new clients are also accepted,
but that's not clear. This option is ignored when a request to
display the tableau is the only client command.
Resources are allocated in groups of req members, at most. When
a tags file is provided, this sanity checks the count of the of
resource tags provided, it may provide more than req requires,
but never less.
Specify the file which contains controlled resource tags, which
is always opened read-only. Each line in this file is taken as
a managed resource, unless it starts with an octothorpe (aka
hash, "#") in which case it is a comment. Blank lines are taken
as empty resources (the empty string). If the file requires any
clean-up filtering before use it is up to the calling process to
pre-process it. The special tags value dash ("-") instructs
ptbw to gather tokens from an enclosing ptbw; which one may be
controlled with -depth.
Without the specification of tags the names of the tokens are
the integers from zero to req*jobs-1.
Show only version information, and a trace of any enclosing
If the token source is a file, this option indicates that the
tokens are terminated by a NUL (\000) character, rather a CR
Use to find the directory any enclosing xclate process has cre-
ated. A manager instance may use this same directory to hold
the client UNIX domain socket.
Parallel to xclate's use. These variable keep track of the full
path name to the enclosing manager's client socket. We can
reuse the tightest enclosing directory, if any.
Also parallel to xclate's use under -d. This variable holds the
name of unix domain socket for the tightest enclosing manager
process which was started with -d in effect.
The list of resources allocated by the tightest enclosing ptbw
for a client, separated by newlines. By default each client
instance overwrites this variable, thus descendant processes
will not see values allocated to outer levels.
$PTBW, $PTBW_1, $PTBW_2, ...
After reading command line options from variable $PTBW, any mas-
ter (-m) process removes it from the environment. It then
installs the variable $PTBW_nesting in its place, if it is
present in the environment. This mocks the behavior of xclate,
for similar reasons. For another way to leaver this see xap-
ply's -e option.
A manager instance releases any open stdin or stdout descriptors, as
not to hold a pipe open, but maintains a descriptor on stderr. The
manager also chdirs out of the current working directory (after start-
ing the inferior process).
An empty utility is a synonym for $SHELL, when set, or the Bourne shell
by default. The special utility name : (a single colon) with no param-
eters is taken as "sleep forever", since sleep(1) doesn't have such an
option. See -Q above.
When -R is specified in master mode and -t is given the special tags
name dash ("-") req is taken as the number of tokens to request from
the enclosing ptbw. If it is not possible to allocate req tokens the
instance fails. If there are more tokens available it may add addi-
tional multiples of req tokens, one for each of jobs.
The default value for -J is a function of the number of CPUs detected
on the system, and the value of req. See -V's version output, or the
default tableau output.
Without -m the default client is similar to who(1) or maybe netstat -i.
Listing the index, locking process, name, and maybe some other details
(which may change in the next release) for each token. The name dash
("-") is an explicit way to ask for this client.
When a client command is specified, the environment variable $ptbw_list
is set before the command is executed to the list of allocated
resources. These resources are released and reallocated to another
client when the child process exits.
When -R is specified, in client mode, ptbw requests req tokens for the
client environment. These are separated by literal newlines in
$ptbw_list. Note that requesting zero resources (-R0) removes
$ptbw_list from the environment, as one might expect (since the empty
string is a valid resource).
As an alternate interface, under -A, each of the req tokens maybe
appended to the argument vector for the client command. These then
become available to the client process as elements of the argv array,
The standard version information.
ptbw -m ptbw -m ptbw -V
The standard version information, plus an example trace of 2
nested wrappers (the last 2 lines).
ptbw -m ptbw
Output only the default tableau for the current host.
ptbw -m -t list.cl ptbw
Output the tableau for the current file list.cl.
xapply -J8 ...
By forcing a -J to xapply this sets a default of "-t -", see
ptbw -m -t list.cl xapply -t - ...
Wrap an xapply in a ptbw master, draw tokens from that master.
xapply -t list.cl ...
A shorthand for the previous example. The default -J value is
the one specified for -P, not the (larger) system default.
ptbw -m -t list.cl ptbw $SHELL -c 'mycmd'
Start a sub-shell wrapped in a ptbw. From this shell the mycmd
has access to an initial resource in "$ptbw_list", and may gar-
ner others via the client interface.
ptbw -m -t list.cl myscript
Start a shell script wrapped in a ptbw. The myscript process
has access to all resources via the client interface.
ptbw -mJ3 -R2 sh -c 'for i in 'jot -c 26 97'; do ptbw -AR2 echo $i; done'
Distribute 2 tokens at a time from a list of 3 pairs, to each of
26 iterations (once for each letter a through z). Each itera-
tion just echos the data from loop (the letter) and from ptbw (2
ptbw -mJ3 -R2 xapply -P3 -R2 'echo %* %t*' 'jot -c 26 97'
Do about the same thing as the "for" example, adding the -P3
after the xapply makes this way more useful (in the real world).
xapply -P3 -J3 -R2 'echo %* %t*' 'jot -c 26 97'
Major Shorthand for the 2 previous examples.
ptbw -AR 3 ksh -c 'echo $0 $1 $2 $3' _
A trivial example of ksh's argument processing applied to the
appended resource tokens. The underscore ("_") is bound to $0,
the first resource is bound to $1, the second to $2 (and so on).
We canonically use the underscore to make the usage more natural
in the shell code.
ptbw -A -R1 env
A brutal abuse of the program, a selected resource is taken as
executable file, then run via env's penchant for indirection.
This is also called "Russian Roulette," for a good reason.
ptbw -m -N /var/run/toker -t $HOME/lib/all.cl ...
Start a resource service attached to /var/run/toker, the target
command (...) must not exit until all the clients of the triple-
dot service are done.
ptbw -md -N /var/run/toker -t $HOME/lib/all.cl ...
Same as above, but do not include the wrapper listening on
/var/run/toker in the global nesting chain.
ptbw -m -N /var/run/toker -t $HOME/lib/all.cl :
As above, but don't really run a utility, just process client
ptbw -t /var/run/toker mytask
Select a token from the /var/run/toker pool to process a task
(mytask), which might read $ptbw_list to access the resource
ptbw -A -t /var/run/toker mytask
As above, but access the selected resource as $1 in mytask.
ptbw -AR3 perl -e '...$ARGV...'
Run an in-line perl fragment with 3 resource values in @ARGV
(since the -A appended them to the client argument vector).
ptbw -R1 ksh -c "echo \$ptbw_list && read sentinel" |&
This starts a co-process that holds a token. The parent shell
might read the token with
read -p MYTOK
then release the token with
print -p quit
Other variants of this co-process template are also useful, but
be warned that deadlock and synchronization issues abound.
ptbw -AR 4 sh -c 'exec make -j $# ...'
This is a prototype interface to allow a make(1) process to wait
for 4 resources before it starts. That doesn't release them as
it is done with them, but we can't solve all the worlds problems
with the first steak-in-the-ground. Note that the "4" count
only appears in the command once: that might be important later.
NR=$(ptbw |sed -n -e '$s/[ ^I]*\([0-9][0-9]*\)[ ^I].*/\1/p')
NR=$(expr $NR + 1)
Compute the maximum number of resources in the current diver-
sion. (Note that the ^I above represents a literal tab charac-
ptbw -R $NR -A xapply 'echo "%q1"'
With the number of record variable above in-scope this command
attempts to output the tableau, of course it must compete for
the whole list with any sibling processes. This may be useful
to sanity check a run-time environment, even more so when echo
is replaced by a more useful check function.
As a bonus the tags file may be the name of a ptbw socket, which obvi-
ates the need for synthesizing a fake environment to attach to an
explicitly path'd resource manager. This is true for xapply's -t
option as well.
If the program is called by another name (other than "ptbw") then the
basename of that name is the name given to any manager or client pro-
cess. This allows a default zero configuration file to be passed
through ptbw from msrc(8l) to hxmd(8l).
In the future it may be possible to add function (other than "iota") to
the synthesizer list.
Other programs may act as resource brokers. This wrapper is the proto-
type for a larger effort to make more effective use of massively paral-
lel UNIX nodes and clusters.
As with xclate, it is unclear to the novice how to use this at all --
let alone how much power the spell controls.
The process id (pid) registered as the owner of a resource is not
always reliable enough to use to signal a process. The race condition
between reading the pid and sending the signal, combined with several
ways to fake the pid that ptbw displays all lead to a bad result. Code
automation to signal a process based on ptbw's recollections only after
some other independent verification process.
The specification of -R in master mode is used as a prediction of the
maximum req provided for each client. When this prediction is wrong
tokens may be allocated and left unused, or too few may be allocated
for any client to actually start.
Be warned, there are dangerous magics in more advanced usages.
We don't quite 'eat our own dog food' as well as we might: we send
resource names in a variable separated with newlines, but we can't
receive such a variable (directly) as a list of resources to manage.
Some process must put them into a file, or a FIFO.
The outer-most environment in ptbw started at zero (viz. $ptbw_link=0),
in previous versions of ptbw. This was not parallel to xclate so we
now start at one. In previous versions a tags file didn't recognize
comments, now it does.
NonPlayer Character Guild
booth swirl ksb dot npcguild dot org
make(1), xapply(1l), hxmd(8l), xclate(1l), wrapw(1l), sh(1), ksh(1),
ls(1), ifconfig(8), perl(1), execve(2), echo(1), env(1), netstat(1)
EASTER EGGS |