OpenSHMEM 1.6rc2 Draft

content/atomics_intro.tex

@@ -130,10 +130,3 @@
     \label{bitamotypes}
   \end{center}
 \end{table}
-]
-
-
-
-
-
-

content/backmatter.tex

@@ -50,7 +50,7 @@ \section*{Incorporating OpenSHMEM into Programs}\label{sec:writing_programs}
 
 The \KEYWORD{static} keyword makes the \VAR{dest} array symmetric on all \acp{PE}.
 Each \ac{PE} is able to transfer data to a remote \dest{} array by simply
-specifying to an OpenSHMEM routine such as \hyperref[subsec:shmem_put]{\FUNC{shmem\_put}}
+specifying to an \openshmem routine such as \hyperref[subsec:shmem_put]{\FUNC{shmem\_put}}
 the local address of the symmetric data object that will receive the data.
 This local address resolution aids programmability because the address of the
 \dest{} need not be exchanged with the active side (\ac{PE} \CONST{0}) prior to
@@ -151,8 +151,8 @@ \chapter{Undefined Behavior in OpenSHMEM}\label{sec:undefined}
 \tabularnewline
 \hline
 Use of non-symmetric variables & Some routines require remotely accessible
-variables to perform their function.  For example, an \openshmem libray may detect a \PUT{} to a non-symmetric variable
-and choose to abort the program.  
+variables to perform their function.  For example, an \openshmem library may detect a \PUT{} to a non-symmetric variable
+and choose to abort the program.
 However, another implementation may choose to continue execution with or without a warning.
 \tabularnewline
 \hline
@@ -749,6 +749,21 @@ \section{Version 1.6}
     additional arguments. \label{changelog:pcontrol}
 \ChangelogRef{subsec:shmem_pcontrol}
 %
+\item Added a \texttt{const} qualifier to the \VAR{cmp\_values} argument in the
+    following point-to-point synchronization routines:
+    \FUNC{shmem\_wait\_until\_all\_vector},
+    \FUNC{shmem\_wait\_until\_any\_vector},
+    \FUNC{shmem\_wait\_until\_some\_vector},
+    \FUNC{shmem\_test\_all\_vector}, \FUNC{shmem\_test\_any\_vector}, and
+    \FUNC{shmem\_test\_some\_vector}. \label{changelog:p2p_vector_const}
+\ChangelogRef{
+  subsec:shmem_wait_until_all_vector,
+  subsec:shmem_wait_until_any_vector,
+  subsec:shmem_wait_until_some_vector,
+  subsec:shmem_test_all_vector,
+  subsec:shmem_test_any_vector,
+  subsec:shmem_test_some_vector}%
+%
 
 \end{enumerate}
 
@@ -844,11 +859,11 @@ \section{Version 1.5}
 \ChangelogRef{p2psynctypes, dep:p2p_sync_types}%
 %
 \item Clarified that point-to-point synchronization routines preserve the
-  atomicity of OpenSHMEM \acp{AMO}.
+  atomicity of \openshmem \acp{AMO}.
 \ChangelogRef{subsec:amo_guarantees}%
 %
 \item Clarified that symmetric variables used as \VAR{ivar} arguments to
-  point-to-point synchronization routines must be updated using OpenSHMEM
+  point-to-point synchronization routines must be updated using \openshmem
   \acp{AMO}.
 \ChangelogRef{subsec:p2p_intro}%
 %
@@ -1182,7 +1197,7 @@ \section{Version 1.1}
 extensions,
 clarifications to completion semantics and \ac{API} descriptions in agreement with
 the \ac{SGI} SHMEM specification,
-and general readabilty and usability improvements to the document structure.
+and general readability and usability improvements to the document structure.
 
 The following list describes the specific changes in \openshmem[1.1]:
 \begin{enumerate}
@@ -1300,13 +1315,13 @@ \chapter{Errata}\label{sec:errata}
 
 Errors or ambiguities in the \openshmem specification may be discovered after
 publication.
-Errata, or corrections, are included in the the sections below indicating the
-version of the OpenSHMEM specification that required the correction or
+Errata, or corrections, are included in the sections below indicating the
+version of the \openshmem specification that required the correction or
 clarification.
 These corrections have been applied to all subsequent versions of the
 specification and this section serves as a historical record of the changes
-made to assist users and implementers with applying the necessary corrections.
-Errata that result in a change to the specifciation are also included in
+made to assist users and implementors with applying the necessary corrections.
+Errata that result in a change to the specification are also included in
 Annex~\ref{sec:changelog}.
 For an implementation to comply with a particular version of \openshmem, it
 must account for all errata associated with that version as indicated below.
@@ -1328,13 +1343,21 @@ \section{Version 1.5}
      \FUNC{shmem\_test\_all\_vector} routines return 1 when the test set is empty
         (\ref{changelog:v1.6}.\ref{changelog:test_all}).
   \item Clarified that \FUNC{shmem\_team\_split\_strided} and
-     \FUNC{shmem\_team\_split\_2d} return nonzero when the parent team is
+     \FUNC{shmem\_team\_split\_2d} return a nonzero value when the parent team is
      \LibConstRef{SHMEM\_TEAM\_INVALID}
         (\ref{changelog:v1.6}.\ref{changelog:split_strided_2d}).
   \item Corrected the \VAR{level} argument's recommended value in API notes for
      \FUNC{shmem\_pcontrol} to indicate that the value should be greater than 2 to enable
      profiling with profile library defined effects and additional arguments
         (\ref{changelog:v1.6}.\ref{changelog:pcontrol}).
+  \item Added a \texttt{const} qualifier to the \VAR{cmp\_values} argument in the
+     following point-to-point synchronization routines:
+     \FUNC{shmem\_wait\_until\_all\_vector},
+     \FUNC{shmem\_wait\_until\_any\_vector},
+     \FUNC{shmem\_wait\_until\_some\_vector},
+     \FUNC{shmem\_test\_all\_vector}, \FUNC{shmem\_test\_any\_vector}, and
+     \FUNC{shmem\_test\_some\_vector}
+        (\ref{changelog:v1.6}.\ref{changelog:p2p_vector_const}).
 \end{enumerate}
 
 %end of setlength command that was started in frontmatter.tex

content/collective_intro.tex

@@ -58,6 +58,7 @@ \subsubsection*{Team-based collectives}
 \item \FUNC{shmem\_[\FuncParam{TYPENAME}\_]collect[mem]}
 \item \FUNC{shmem\_[\FuncParam{TYPENAME}\_]fcollect[mem]}
 \item \FUNC{shmem\_[\FuncParam{TYPENAME}\_]\{and, or, xor, max, min, sum, prod\}\_reduce}
+\item \FUNC{shmem\_[\FuncParam{TYPENAME}\_]sum\_\{in, ex\}scan}
 \end{itemize}
 
 In addition, all team creation functions are collective operations. In addition to the ordering

content/context_intro.tex

@@ -41,12 +41,12 @@
 explicitly or implicitly, are performed.
 All point-to-point routines that operate on this context will do so with
 respect to the team-relative \ac{PE} numbering of the associated team.
-If the PE number passed to such a routine is invalid, being negative or greater
+If the \ac{PE} number passed to such a routine is invalid, being negative or greater
 than or equal to the size of the \openshmem team, then the behavior is undefined.
 
 By default, contexts are {\em shareable} and, when it is allowed by the
 threading model provided by the \openshmem library, they can be used concurrently by
-multiple threads within the PE where they were created.
+multiple threads within the \ac{PE} where they were created.
 %
 The following options can be supplied during context creation to restrict
 this usage model and enable performance optimizations.  When using a given

content/coverpage.tex

@@ -29,7 +29,7 @@ \section*{Development by}
 \begin{itemize}
 \item For a current list of contributors and collaborators please see\\
   \url{http://www.openshmem.org/site/Contributors/}
-\item For a current list of OpenSHMEM implementations and tools, please see\\
+\item For a current list of \openshmem implementations and tools, please see\\
   \url{http://openshmem.org/site/Links#impl/}
 
 \end{itemize}
@@ -60,7 +60,9 @@ \subsection*{\openshmem 1.6}
 \item Aurelien Bouteiller, \ac{UTK}
 \item Mark Brown, Intel
 \item Bob Cernohous, \ac{HPE}
+\item Matthew Davis, Georgia Tech
 \item James Dinan\footnotemark[1], NVIDIA
+\item Youssef Elmougy, Georgia Tech
 \item Megan Grodowitz, Arm Inc.
 \item Max Grossman, Georgia Tech
 \item Yanfei Guo, \ac{ANL}
@@ -72,6 +74,7 @@ \subsection*{\openshmem 1.6}
 \item Robert Kierski, \ac{HPE}
 \item Bryant Lam, \ac{DoD}
 \item Akhil Langer, NVIDIA
+\item Jens Manser, \ac{DoD}
 \item Tiffany M. Mintz, \ac{ORNL}
 \item Bryan Morgan, Intel
 \item William Okuno\footnotemark[2], \ac{HPE}
@@ -158,8 +161,8 @@ \subsection*{\openshmem 1.5}
 
 \subsection*{\openshmem 1.4}
 
-OpenSHMEM 1.4 is dedicated to the memory of David Charles Knaak. David was a
-highly involved colleague and contributor to the entire OpenSHMEM project. He
+\openshmem 1.4 is dedicated to the memory of David Charles Knaak. David was a
+highly involved colleague and contributor to the entire \openshmem project. He
 will be missed.
 
 \begin{multicols}{2}

content/interoperability.tex

@@ -1,6 +1,6 @@
 \chapter{Interoperability with Other Programming Models}\label{sec:interoperability}
 
-OpenSHMEM routines may be used in conjunction with the routines of other
+\openshmem routines may be used in conjunction with the routines of other
 communication libraries or parallel languages in the same program. This section
 describes the interoperability with other programming models, including
 clarification of undefined behaviors caused by mixed use of different models,
@@ -35,7 +35,7 @@ \subsection{Initialization}
 call to \FUNC{MPI\_Finalize}.
 
 \parimpnotes{
-  Portable implementations of OpenSHMEM and \ac{MPI} must ensure that the initialization
+  Portable implementations of \openshmem and \ac{MPI} must ensure that the initialization
   calls can be made in an arbitrary order within a program; the same rule also
   applies to the finalization calls. A software runtime that utilizes a shared
   communication resource for \openshmem and \ac{MPI} communication may maintain an

content/library_handles.tex

@@ -24,9 +24,9 @@
 \LibHandleDecl{SHMEM\_TEAM\_SHARED} &
 Handle of type \CTYPE{shmem\_team\_t} that corresponds to a team of \acp{PE}
 that share a memory domain.  \LibHandleRef{SHMEM\_TEAM\_SHARED} refers to
-the team of all PEs that would mutually return a non-null address from a
-call to \FUNC{shmem\_ptr} for all symmetric heap objects.  That is,
-\FUNC{shmem\_ptr} must return a non-null pointer to the local PE for all
+the team of all \acp{PE} that would mutually return a non-null address from a
+call to \FUNC{shmem\_ptr} or \FUNC{shmem\_team\_ptr} for all symmetric heap objects.  That is,
+\FUNC{shmem\_ptr} and \FUNC{shmem\_team\_ptr} must return a non-null pointer to the local \ac{PE} for all
 symmetric heap objects on all target \acp{PE} in the team.  This means that
 symmetric heap objects on each \ac{PE} are
 directly load/store accessible by all \acp{PE} in the team.

content/p2p_sync_intro.tex

@@ -16,7 +16,7 @@
 integer types defined in \HEADER{stdint.h} by \Cstd[99]~\S7.18.1.1 and
 \Cstd[11]~\S7.20.1.1. When the \Cstd translation environment
 does not provide exact-width integer types with \HEADER{stdint.h}, an
-\openshmem implemementation is not required to provide support for these types.
+\openshmem implementation is not required to provide support for these types.
 The \FUNC{shmem\_test\_any} and \FUNC{shmem\_wait\_until\_any} routines
 require the \CONST{SIZE\_MAX} macro defined in \HEADER{stdint.h} by
 \Cstd[99]~\S7.18.3 and \Cstd[11]~\S7.20.3.

content/profiling_interface.tex

@@ -6,7 +6,7 @@
 unreasonable to expect that the authors and developers of profiling
 tools for \openshmem will have access to the source code that
 implements \openshmem on any particular machine. It is, therefore,
-necessary to provide a mechanism by which the implementers of such
+necessary to provide a mechanism by which the implementors of such
 tools can collect whatever performance information they wish
 \emph{without} access to the underlying implementation.
 

content/programming_model_overview.tex

@@ -2,7 +2,10 @@
 mechanisms to share information among \openshmem processes, or \acp{PE}, and
 private data objects that are accessible by only the \ac{PE} itself. The \ac{API}
 allows communication and synchronization operations on both private (local to
-the \ac{PE} initiating the operation) and remotely accessible data objects. The key
+the \ac{PE} initiating the operation) and remotely accessible data objects. A
+remote \ac{PE} is defined as any \ac{PE} whose remotely accessible data objects
+are accessible to the \ac{PE} initiating an operation and may include the
+initiating \ac{PE}. The key
 feature of \openshmem is that data transfer operations are
 \emph{one-sided} in nature. This means that a local \ac{PE} executing
 a data transfer routine does not require the participation of the remote \ac{PE}

content/shmem_alltoall.tex

@@ -98,7 +98,7 @@
     If \VAR{team} compares equal to \LibConstRef{SHMEM\_TEAM\_INVALID} or is
     otherwise invalid, the behavior is undefined.
 
-    Before any \ac{PE} calls a \FUNC{shmem\_alltoall} routine, the following
+    Before the local \ac{PE} calls a \FUNC{shmem\_alltoall} routine, the following
     conditions must be ensured, otherwise the behavior is undefined:
     \begin{itemize}
         \item The \dest{} array on all \acp{PE} in the team is ready to
@@ -110,10 +110,10 @@
     array is ready across all \acp{PE} prior to calling this routine.
 
     Upon return from a \FUNC{shmem\_alltoall} routine, the following is true for
-    the local PE:
+    the local \ac{PE}:
     \begin{itemize}
         \item Its \VAR{dest} symmetric data object is completely updated and the 
-	data has been copied out of the source data object.
+	data has been copied out of the \VAR{source} data object.
     \end{itemize}
 
 \begin{DeprecateBlock}
@@ -145,10 +145,10 @@
     Otherwise, the behavior is undefined.
 
     Upon return from a \FUNC{shmem\_alltoall} routine, the following is true for
-    the local PE:
+    the local \ac{PE}:
     \begin{itemize}
     	\item Its \VAR{dest} symmetric data object is completely updated and the 
-	data has been copied out of the source data object.
+	data has been copied out of the \VAR{source} data object.
     	\item For active-set-based routines,
     	the values in the \VAR{pSync} array are restored to the original values.
     \end{itemize}

content/shmem_atomic_fetch_and.tex

@@ -38,7 +38,7 @@
 }
 
 \apireturnvalues{
-  The value pointed to by \VAR{dest} on PE \VAR{pe} immediately before the
+    The value pointed to by \VAR{dest} on \ac{PE} \VAR{pe} immediately before the
   operation is performed.
 }
 

content/shmem_atomic_fetch_or.tex

@@ -38,7 +38,7 @@
 }
 
 \apireturnvalues{
-  The value pointed to by \VAR{dest} on PE \VAR{pe} immediately before the
+  The value pointed to by \VAR{dest} on \ac{PE} \VAR{pe} immediately before the
   operation is performed.
 }
 

content/shmem_atomic_fetch_or_nbi.tex

@@ -37,11 +37,11 @@
 \apidescription{
     The nonblocking \FUNC{shmem\_atomic\_fetch\_or\_nbi} routines perform an
     atomic fetching bitwise OR on the remotely accessible data object pointed
-    by \VAR{dest} at PE \VAR{pe} with the operand \VAR{value}. This routine
+    by \VAR{dest} at \ac{PE} \VAR{pe} with the operand \VAR{value}. This routine
     returns after initiating the operation. The operation is considered complete
     after a subsequent call to \FUNC{shmem\_quiet}. At the completion of
     \FUNC{shmem\_quiet}, these routines have performed a fetching bitwise OR on
-    \VAR{dest} at PE \VAR{pe} with the operand \VAR{value} and fetched the prior
+    \VAR{dest} at \ac{PE} \VAR{pe} with the operand \VAR{value} and fetched the prior
     contents of \VAR{dest} into the \VAR{fetch} local data object.
 }
 

content/shmem_atomic_fetch_xor.tex

@@ -39,7 +39,7 @@
 }
 
 \apireturnvalues{
-  The value pointed to by \VAR{dest} on PE \VAR{pe} immediately before the
+  The value pointed to by \VAR{dest} on \ac{PE} \VAR{pe} immediately before the
   operation is performed.
 }
 

content/shmem_atomic_fetch_xor_nbi.tex

@@ -37,11 +37,11 @@
 \apidescription{
     The nonblocking \FUNC{shmem\_atomic\_fetch\_xor\_nbi} routines perform an
     atomic fetching bitwise XOR on the remotely accessible data object pointed
-    by \VAR{dest} at PE \VAR{pe} with the operand \VAR{value}. This routine
+    by \VAR{dest} at \ac{PE} \VAR{pe} with the operand \VAR{value}. This routine
     returns after initiating the operation. The operation is considered complete
     after a subsequent call to \FUNC{shmem\_quiet}. At the completion of
     \FUNC{shmem\_quiet}, these routines have performed a fetching bitwise XOR on
-    \VAR{dest} at PE \VAR{pe} with the operand \VAR{value} and fetched the prior
+    \VAR{dest} at \ac{PE} \VAR{pe} with the operand \VAR{value} and fetched the prior
     contents of \VAR{dest} into the \VAR{fetch} local data object.
 }
 

content/shmem_broadcast.tex

@@ -45,7 +45,7 @@
   respectively.
 }
 \apiargument{IN}{PE\_root}{Zero-based ordinal of the \ac{PE}, with respect to
-    the calling PEs, from which the data is copied.}
+    the calling \acp{PE}, from which the data is copied.}
 
 \begin{DeprecateBlock}
 
@@ -85,7 +85,7 @@
       the team.
     \end{itemize}
 
-    Before any \ac{PE} calls a broadcast routine, the following conditions
+    Before the local \ac{PE} calls a broadcast routine, the following conditions
     must be ensured, otherwise the behavior is undefined:
     \begin{itemize}
         \item The \dest{} array on all \acp{PE} in the team is ready to
@@ -114,7 +114,7 @@
 
     For active-set-based broadcasts:
     \begin{itemize}
-        \item The \VAR{dest} object is updated on all PEs other than the root PE.
+        \item The \VAR{dest} object is updated on all \acp{PE} other than the root \ac{PE}.
         \item All \acp{PE} in the active set defined by the
         \VAR{PE\_start}, \VAR{logPE\_stride}, \VAR{PE\_size} triplet
         must participate in the operation.
@@ -144,7 +144,7 @@
     Upon return from an active-based broadcast routine, the following are true for the local
     \ac{PE}:
     \begin{itemize}
-      \item If the current PE is not the root PE, the \dest{} data object is updated.
+      \item If the current \ac{PE} is not the root \ac{PE}, the \dest{} data object is updated.
       \item The \source{} data object may be safely reused.
       \item The values in the \VAR{pSync} array are restored to the
         original values.

content/shmem_collect.tex

@@ -66,7 +66,7 @@
     \openshmem \FUNC{collect} and \FUNC{fcollect} routines perform a collective
     operation to concatenate \VAR{nelems}
     data items from the \source{} array into the
-    \dest{} array, over an \openshmem team in processor number order. 
+    \dest{} array, over an \openshmem team in \ac{PE} number order. 
     The resultant \dest{} array contains the contribution from
     \acp{PE} as follows:
 
@@ -88,7 +88,7 @@
     If \VAR{team} compares equal to \LibConstRef{SHMEM\_TEAM\_INVALID} or is
     otherwise invalid, the behavior is undefined.
 
-    Before any \ac{PE} calls a collect routine, the following conditions must
+    Before the local \ac{PE} calls a collect routine, the following conditions must
     be ensured, otherwise the behavior is undefined:
     \begin{itemize}
         \item The \dest{} array on all \acp{PE} in the team is ready to
@@ -104,7 +104,7 @@
     operation to concatenate \VAR{nelems}
     data items from the \source{} array into the
     \dest{} array, over an \openshmem active set
-    in processor number order. The resultant \dest{} array contains the contribution from
+    in \ac{PE} number order. The resultant \dest{} array contains the contribution from
 	\acp{PE} as follows:
     \begin{itemize}
         \item For an active set, the data from \ac{PE} \VAR{PE\_start} is first, then the
@@ -151,7 +151,7 @@
     with no performance degradation when \VAR{nelems} is a non-power-of-two value.
 \end{DeprecateBlock}
     The collective routines that operate on teams containing a
-    non-power-of-two of PEs do so with some performance degradation. They operate
+    non-power-of-two of \acp{PE} do so with some performance degradation. They operate
     with no performance degradation when \VAR{nelems} is a non-power-of-two value.
 
 }