mpull - locally update master source products via rsync and mmsrc
mpull [rsync-opts] msrc-dir [mmsrc-opts] [utility]
mpull [rsync-opts] msrc-dir [-lsz] [-B macros] [-C configs]
[-D name[=value]] [-d flags] [-E compares] [-f makefile] [-G guard]
[-I dirname] [-j m4prep] [-k key] [-m prereq] [-o attributes] [-U name]
[-u login] [-X ex-configs] [-y yoke] [-Y top] [-Z zero-config] [util-
At very large scale it is sometimes useful to pull updates to machines
(virtual instances) rather than push them from a central service. This
program uses rsync(1) to fetch an ephemeral copy of the current master
source for a single directory, uses mmsrc(8) to update a local platform
cache, then executes an update utility.
The default configuration file is "auto.cf", since mpull uses mmsrc to
do the local update, but any other may be specified in mmsrc-opts to
force a better choice (under -C).
All the mmsrc rules apply to the local update except one. A local
source cache is updated with the shadow sources, so a -y option is syn-
thesized to set INTO. The value is based on the msrc-dir presented
prefixed with the local cache (see MPULL_SRC below).
The actual compilation or installation of a tool is always done from
that local shadow copy of the master source. So the current working
directory of mpull does not have to contain a recipe file.
The default utility is make(1), because that's mmsrc's default. Use
special utility ":" to just build the shadow copy.
Any leading words that start with a dash (-) are taken to be options to
rsync. The most common use for this is to exclude any revision control
subdirectories (viz. "--exclude=RCS").
When shadow copy must be configured on the master source host, enable
msrcmux(7l) on that host then use muxcat as the client on the local
host. See muxcat(1l).
If the program is called as mpull then no options are forced. See
mmsrc for option descriptions.
As in mmsrc we show the commands we are using to pull data and
launch commands. Not every command is reflected in the output.
Output the standard help list, this is largely cloned from
Show only version information.
Due to limits in the shell's option parser this option is
The other options available under mmsrc(8) are passed as given.
These are inspected for their values: -C, -X, -Z, and -D.
The environment variables HOME, SHELL, TMPDIR, and PATH are consulted
for their traditional purposes. As well as HXMD_LIB, which is used as
hxmd does to set an explicit search list for command-line configuration
These variable are installed in the environment so recursive calls to
mpull may use the same configuration.
The list configuration files provided in a format hxmd accepts.
The default list is "-Cauto.cf", but explicit -C options replace
The name for this host in the above configuration files. The
value is copied from any -DHOST= options. It defaults to
"localhost", as in mmsrc.
The location of the master source repository service. This must
provide either an anonymous rsync service, or rsync over ssh.
This is copied from a host attribute with the same name, if none
is present in the environment. The format for rsync is:
Where port defaults to 873 and both module and host default to
"msrc". The port specification is only required when you can't
start rsync on its canonical privileged port. In that case I
usually pick 18873. For example
For rsync over ssh used 1 colon (:) rather than 2, and replace
the module with the path to the root of the master source:
The path to this hosts shadow copy of the master source. This
is copied from a host attribute with the same name, if none is
present in the environment. Mortal logins usually can't write
in /usr/src, so set this to someplace you can write. For exam-
The name of the rsync program. Defaults to "rsync".
Any extra rsync options required for the rsync command line.
The options we provide (see -d X output) look like:
rsync -arSH --port=port $MPULL_FROM/Pkg/ tempdir/Pkg
The $MPULL_RSOPTS value is inserted after the port specifica-
Be sure to export these so they are visible to mpull, see sh(1).
The standard version information.
mpull local/bin/glob -Clocal.cf make install clean
Construct a platform cache for the glob program on this host
from the attributes in local.cf for "localhost".
mpull local/bin/oue -DHOST=sulaco mk -mInstall man
Install the manual page for oue(1l) as if this host were named
"sulaco", which must be defined in to.cf".
MPULL_SRC=$HOME/src mpull local/bin/oue -Csite.cf :
Force a shadow copy into your home directory under
src/local/bin/oue, and do nothing with it. Note that the host
"localhost" must be defined in that configuration file.
MPULL_FROM=nostromo:/usr/msrc mpull local/bin/mk -dX pwd
Use rsync(1) over ssh(1) to grab mk(1l) from the cache on nos-
tromo, and watch the commands as they are run.
mpull -vv --exclude=RCS local/bin/glob -dX ls \; pwd
Force rsync to skip any RCS directories and show what it is
downloading to fetch the source to glob(1l). Report the actions
taken by mpull and mmsrc to ls(1) the shadow local directory
after it is updated. Also run pwd(1) to display the location of
the shadow directory (before it is removed).
MPULL_FROM=msrc::Pkgs mpull msrc_base -dX ls
Deploy the package directory for msrc_base to the local source
tree. This really doesn't do what you want, because the gather
phase didn't happen from the level3 recipe (it was taken to be a
level2 recipe, which is it not). So what ends up on the local
machine doesn't build. A completely different tcpmux(8l) ser-
vice would be needed to pull package sources to a client.
It would be much better if mpull could parse rsync options on the com-
mand-line before the msrc-dir parameter. But the rsync option parser
is really hard to emulate in a script, since it allows arbitrary
aliases. So we assume the options end with the first word that doesn't
start with a dash (-). Use assignments (--exclude=param) for or quoted
spaces to force option specifications through.
Mpull can't pass -h or -V to rsync as the first option. Which doesn't
really limit the use of rsync at all.
The rule mmsrc uses as a fall-back (the first host defined in the first
-C configuration file) doesn't work for the 2 calls to hxmd mpull uses
to extract MPULL_FROM and MPULL_LOCALROOT. Work around by setting
these in the environment to prevent the calls to hxmd(8).
It would be clever to allow a persistent local copy of the master
source tree: because mpull downloads the configuration management meta-
information with the current files every time. This uses more network
bandwidth than it would if it those files (and subdirs like RCS or CVS)
were persisted locally. If you want to do that you don't need the
script wrapper, just use rsync. If you want to skip the meta informa-
tion put in an exclude option in $MPULL_RSOPTS and you are good to go.
Ironically mpull cannot update mmsrc. Because the source for mmsrc
requires peer directories to execute the master recipe, so it fails to
install the shadow copy in the local source cache. The best way to
update the base tools is to install the level3 package msrc_base.
Download the package source then build it locally.
All the run-time bugs of mmsrc.
NonPlayer Character Guild
mmsrc at no-SPAM here ksb dot-here npcguild.org
mmsrc(8l), msrcmux(7l), rsync(1), hxmd(8l), msrc(8l), muxcat(1l),
m4(1), environ(7), ssh(1), ksh(1)