18.2.13. PMIx_Connect

PMIx_Connect, PMIx_Connect_nb — Record a set of processes as “connected” so that the host environment treats them as a single assemblage for fault and event-notification purposes.

18.2.13.1. SYNOPSIS

#include <pmix.h>

pmix_status_t PMIx_Connect(const pmix_proc_t procs[], size_t nprocs,
                           const pmix_info_t info[], size_t ninfo);

pmix_status_t PMIx_Connect_nb(const pmix_proc_t procs[], size_t nprocs,
                              const pmix_info_t info[], size_t ninfo,
                              pmix_op_cbfunc_t cbfunc, void *cbdata);

18.2.13.1.1. Python Syntax

from pmix import *

foo = PMIxClient()
# ... after a successful foo.init() ...
# the peers is a list of Python ``pmix_proc_t`` dictionaries naming all
# participants (including the caller)
peers = [{'nspace': "testnspace", 'rank': PMIX_RANK_WILDCARD},
         {'nspace': "othernspace", 'rank': PMIX_RANK_WILDCARD}]
# the directives is a list of Python ``pmix_info_t`` dictionaries
pydirs = [{'key': PMIX_TIMEOUT,
           'value': {'value': 10, 'val_type': PMIX_INT}}]
rc = foo.connect(peers, pydirs)

18.2.13.2. INPUT PARAMETERS

procs: Pointer to an array of pmix_proc_t structures naming all of the processes that are to be connected. A rank of PMIX_RANK_WILDCARD in any element indicates that all processes in that namespace are participating. Unlike the collective APIs that default to the caller’s own namespace, procs must be non-NULL and nprocs must be greater than zero; otherwise the call returns PMIX_ERR_BAD_PARAM.
nprocs: Number of elements in the procs array.
info: Pointer to an array of pmix_info_t(5) structures conveying directives that qualify the operation (see DIRECTIVES). A NULL value is supported when no directives are desired.
ninfo: Number of elements in the info array.

The non-blocking form takes two additional parameters:

cbfunc: Callback function of type pmix_op_cbfunc_t to be invoked when the connect operation completes.
cbdata: Opaque pointer that is passed, unmodified, to cbfunc.

18.2.13.3. DESCRIPTION

Record the processes specified by the procs array as connected. PMIx_Connect is the blocking form: it does not return until all processes identified in procs have called either PMIx_Connect or PMIx_Connect_nb and the host environment has completed any supporting operations required to meet the terms of the PMIx definition of connected processes (or the operation fails). PMIx_Connect_nb is the non-blocking form: it returns immediately, and the provided cbfunc is invoked with the final status once the operation completes.

The PMIx definition of connected solely implies that the host environment should treat the failure of any process in the assemblage as a reportable event, taking action on the assemblage as if it were a single application. For example, if the environment defaults (in the absence of any application directives) to terminating an application upon failure of any process in that application, then it should terminate all processes in the connected assemblage upon failure of any member. The host environment may choose to assign a new namespace to the connected assemblage and/or assign new ranks to its members for its own internal tracking purposes; whether such namespaces are exposed to clients is implementation defined.

Every element of procs must name a participant, and the calling process must itself be among them; if it is not, the operation returns PMIX_ERR_NOT_A_MEMBER. The ordering of the entries in procs has no significance. However, all processes engaged in a given connect operation must use the same method to identify the participants: callers that describe the target set using PMIX_RANK_WILDCARD are not matched with callers that list the individual processes of a namespace explicitly. A group identifier appearing in procs is first expanded into that group’s member processes.

Upon completion, the server returns any job-level information that participating processes do not already possess. When the connect request spans processes from different namespaces, each participant receives the job-level information for the namespaces other than its own, so that subsequent PMIx_Get(3) queries against the connected peers can be satisfied locally.

A process can only engage in one connect operation involving the identical procs array at a time. However, a process can be simultaneously engaged in multiple connect operations, each involving a different procs array.

PMIx_Connect and PMIx_Connect_nb are collective operations. The PMIx server library aggregates the participation of its local clients, passing a single request to the host environment once all local participants have called the API; the host then executes the collective across all participating nodes.

As with all non-blocking PMIx APIs, callers of PMIx_Connect_nb must keep the procs and info arrays valid until cbfunc is invoked.

18.2.13.4. DIRECTIVES

PMIx libraries are not required to directly support any attributes for this operation, but any provided attributes are passed to the host environment for processing. The following attributes are optional and depend on the implementation and host environment:

PMIX_ALL_CLONES_PARTICIPATE (bool) — all clones of the calling process must participate in the collective operation.
PMIX_TIMEOUT (int) — maximum time, in seconds, for the connect to complete before the host declares an error. This helps avoid “hangs” caused by a programming error that prevents one or more processes from reaching the connect.

18.2.13.5. RETURN VALUE

For the blocking form, PMIX_SUCCESS indicates that all participants have connected and any returned job-level data is available. For the non-blocking form, a return of PMIX_SUCCESS indicates only that the request was accepted for processing and the final status will be delivered to cbfunc.

PMIX_SUCCESS — the connect operation completed successfully.
PMIX_ERR_NOT_A_MEMBER — the calling process is not among the named participants.
PMIX_ERR_BAD_PARAM — an invalid argument was supplied (e.g., a NULL procs array or a zero nprocs).
PMIX_ERR_UNREACH — the local PMIx server could not be reached.
PMIX_ERR_NOT_AVAILABLE — the operation cannot be serviced because the library’s progress engine has been stopped.
PMIX_ERR_INIT — the PMIx library has not been initialized.

Any other negative value indicates an appropriate error condition. PMIx error constants are defined in pmix_common.h.

18.2.13.6. NOTES

Attempting to connect processes solely within the same namespace is essentially a no-op operation. While not explicitly prohibited, a PMIx implementation or host environment may return an error in such cases.

Once processes are connected, the host environment is required to generate a PMIX_ERR_PROC_TERM_WO_SYNC event should any process in the assemblage terminate or call PMIx_Finalize(3) without first disconnecting from the assemblage via PMIx_Disconnect(3). If a job is terminated as a result of that action, the host environment is also required to generate PMIX_ERR_JOB_TERM_WO_SYNC for all jobs that were terminated as a result.

By default, the processes created by PMIx_Spawn(3) are connected to the parent process upon successful launch, following the same semantics described here.