12.11. Sharing a DVM between users
By default a DVM serves one user: the user it runs as. Tools of any other
user are refused when they connect. The prte_pmix_no_foreign_tools
MCA parameter (default true) controls this; setting it to 0 lets
other users’ tools connect, to the daemon on any node of the DVM:
prte --prtemca prte_pmix_no_foreign_tools 0 ...
What a tool of another user may then do - signal a job, read its output,
query it - is decided by the job’s owner and its access list (see the
users= and groups= runtime options, prterun --help
runtime-options).
12.11.1. Who a job’s processes run as
A job belongs to the user whose tool started it, and its processes run as that user, with that user’s groups:
A DVM running as root starts every job’s processes as the user the job belongs to, on every node. They see themselves as that user, their PMIx server accepts them as that user, and the session directories made for the job (
PMIX_NSDIR,PMIX_PROCDIR) belong to that user and no one else. Jobs of root itself run as root.A DVM running as any other user cannot start processes as another user, so it runs only its own user’s jobs and root’s. A job of any other user is refused, with a message saying why, and the tool that asked is told
PMIX_ERR_NO_PERMISSIONS. It is never run with the DVM’s privileges instead.A tool also refuses, on its own side, to connect to a server another ordinary user runs, unless it is told it may (PMIx’s
pmix_ptl_base_server_access).
So a DVM meant to run several users’ jobs runs as root.
12.11.2. Files the DVM opens for a job
Some directives ask the DVM itself, rather than the job’s processes, to open a file:
a hostfile, or a hostfile of hosts to add to the DVM;
a rankfile or sequential-mapping file;
a file for the process table (
--rtos output-proctable=<file>).
The DVM opens each of these as the job’s own user, with that user’s
groups, so a job may name only files its user could open itself. A file
it may not read is reported as one that cannot be opened, and a process
table is created, or replaced, only where the job’s user may write. An
allocation request that names a hostfile has it read as the requester in
the same way. Output files a job’s --output directive names are
opened as the job’s own user too.
Preloading binaries or files (--preload-files, --preload-binary)
is the exception: the DVM reads them, and its daemons write the copies on
every node, as the user the DVM runs as. Preloading is therefore accepted
only from root and from the DVM’s own user, and a job of any other user
that asks for it is refused, naming the directive.
12.11.3. Who may grow the DVM
Bringing hosts into an elastic DVM - with --add-host,
--add-hostfile or --activate on a job, or with an allocation
request (PMIx_Allocation_request) that names nodes or a hostfile -
changes what the DVM consists of. It is accepted from root, the DVM’s own user, and the users
and groups the DVM was started with (prte --rtos users=...,groups=...),
as creating a session is. When the DVM hands such requests to a PMIx
scheduler, the scheduler decides instead.
A user asks for resources - a new allocation, or more nodes or time for one
of its own - with PMIx_Allocation_request, and the scheduler decides.
PMIx_Session_control with PMIX_SESSION_INSTANTIATE or
PMIX_SESSION_EXTEND is not such a request: it is how the scheduler tells
the DVM what it has decided. With no scheduler attached, those two
directives are accepted only from the same users who may grow the DVM, and
that includes the user the session is for. The user a session is for may
still pause, resume, signal and terminate it.
The DVM reaches every node as the user it runs as, and does not start a
daemon as any other user. A host may therefore not name an account to
reach it as - user@host, or username= in a hostfile - on the
command line, in a hostfile, a rankfile, a hostfile of hosts to add, or
the node list of an allocation or session; such a request is refused,
whoever makes it. A host name may contain only letters, digits, .,
- and _.
12.11.4. Whom an allocation is for
An allocation request (PMIx_Allocation_request) for new or extended
resources is for the user who makes it, and that user’s group, unless it
says otherwise with PMIX_ALLOC_UID and PMIX_ALLOC_GID. A user may
name only itself; root may name any user. Any user may name a group it
belongs to. Anything else is refused with PMIX_ERR_NO_PERMISSIONS.
The allocation belongs to the user it is for, who may use, extend or
release it. Members of its group may too, but only when the request named
the group with PMIX_ALLOC_GID: a group filled in from the requester’s
own is recorded and passed to any scheduler, and grants its members
nothing. A user who wants to share an allocation with a group names it.
With no scheduler attached, the DVM applies the time an allocation request
asks for (PMIX_ALLOC_TIME): the allocation ends that long after the
request is granted. A later PMIX_ALLOC_EXTEND naming a time sets a new
limit, that long from then. The user an allocation is for may change a limit
it set itself; a limit set by root, the DVM’s own user, a user named by
prte --rtos users=...,groups=..., or a scheduler may be changed only by
one of them, and an attempt by the allocation’s user is refused with
PMIX_ERR_NO_PERMISSIONS.