$Id: README,v 1.1.2.1 2007-06-20 17:42:07 vinod Exp $
Introduction:
  A new notification functionality has been introduced in ARMCI primarify for 
  the Co-Array fortran compiler.

Description:
  Notification is a mechanism for synchronization.
    1) process s is the process doing a notify call
    2) process d is the destiniation of this notification call

  Process s makes a armci_notify(d) call. This call is guarenteed to reach
  the process d (the destination here) only after all prior calls made by 
  process s to process d complete and are commited into process d's memory.
  armci_notify is a mechanism by which if s notifies d and d does a 
  corresponding notify_wait, after d returns from notify_wait it knows that 
  all communication calls initiated to d by s, before s issued the 
  notification message, are completed and d's memory will reflect that they 
  are completed.

Function Definitions:

  int armci_notify(int proc)
  parameter:
    proc - the process/task to which the notification message is sent to
  return:
    the current value of the pending operations count for process proc
  blocking: 
    this is a non-blocking call on most networks

  int armci_notify_wait(int proc,int *pval)
  parameter:
    proc - the process/task to which the notification message is sent to
    pval - pointer to the current received count from the initiator of the 
           notify call at the moment. 
  return:
    returns the number of calls that were sent from s (initiator of notify)
    since the invocation of the last notify_wait
  blocking: 
    this is a blocking call


Sample Program:
  an example program is at armci/src/testnotify.c
  here is a sample where process waits for message from left and sends to right
        left=me-1;right=(me==nproc-1)?-1:me+1;
        if(left!=-1){
          rc = armci_notify_wait(left,&wc);
        } 
        if(right!=-1){
          ARMCI_PutS((double*)b[me]+idx, stride,
                     (double*)b[right]+idx, stride, count, strl,right);
          lc=armci_notify(right);
        }
