VirtualFluids 0.2.0
Parallel CFD LBM Solver
Loading...
Searching...
No Matches
Public Member Functions | Protected Member Functions | Static Protected Member Functions | Friends | List of all members
vf::gpu::IndexRearrangementForStreams Class Reference
gpu » core » DataStructureInitializer

class that is used to rearrange the arrays of node indices for communication between gpus. The rearrangement is needed for communication hiding with cuda streams More...

#include <IndexRearrangementForStreams.h>

Inheritance diagram for vf::gpu::IndexRearrangementForStreams:
vf::gpu::grid_generator_test::IndexRearrangementForStreamsDouble

Public Member Functions

 IndexRearrangementForStreams (std::shared_ptr< Parameter > para, std::shared_ptr< GridBuilder > builder, vf::parallel::Communicator &communicator)
 
virtual ~IndexRearrangementForStreams ()=default
 
virtual std::array< ProcessNeighbor27, 4 > initCommunicationArraysForCommAfterFinetoCoarse (const ProcessNeighbor27 &sendNeighborHost, const ProcessNeighbor27 &sendNeighborDevice, const ProcessNeighbor27 &recvNeighborHost, const ProcessNeighbor27 &recvNeighborDevice, int level, int direction) const
 Initialize the arrays for the communication after the interpolation from fine to coarse.
 

Protected Member Functions

std::vector< uint > exchangeIndicesForCommAfterFtoC (const ProcessNeighbor27 &sendNeighbor, const ProcessNeighbor27 &recvNeighbor, const std::vector< uint > &sendIndicesForCommAfterFtoCPositions) const
 send sendIndicesForCommAfterFtoCPositions to receiving process and receive recvIndicesForCommAfterFtoCPositions from neighboring process
 
std::vector< uint > reorderSendIndicesForCommAfterFtoC (uint *indices, int direction, int level) const
 The send indices are reordered for the communication after the interpolation from fine to coarse.
 
std::vector< uint > aggregateCoarseNodesForCtoF (int level) const
 Aggregate all nodes in the coarse cells for the interpolation in coarse to fine.
 
void reorderRecvIndicesForCommAfterFtoC (uint *recvIndices, int direction, int level, const std::vector< uint > &recvIndicesForCommAfterFtoCPositions) const
 Reorder the receive indices in the same way that the send indices were reordered.
 

Static Protected Member Functions

static ProcessNeighbor27 makeProcessNeighborToCommAfterFtoC (const ProcessNeighbor27 &neighbor, uint numberOfNodes)
 Initializes pointers for reduced communication after the interpolation from fine to coarse by copying them from "normal" communication.
 
static std::vector< uint > findIndicesNotInCommAfterFtoC (uint numberOfIndices, const uint *indices, const std::vector< uint > &indicesAfterFtoC)
 Find all indices which are not part of the communication after the interpolation from fine to coarse.
 

Friends

class IndexRearrangementForStreamsTest_reorderSendIndices
 
class IndexRearrangementForStreamsTest_reorderRecvIndices
 
class IndexRearrangementForStreamsTest_exchangeIndicesForCommAfterFtoC
 

Detailed Description

class that is used to rearrange the arrays of node indices for communication between gpus. The rearrangement is needed for communication hiding with cuda streams

This class changes the order of the node indices that are needed for communication between gpus. The indices are reordered so that they can be split into two groups: nodes that are part if the interpolation between grid levels, and nodes that are not. These groups are needed for communication hiding. For details see [master thesis of Anna Wellmann]

Definition at line 60 of file IndexRearrangementForStreams.h.

Constructor & Destructor Documentation

◆ IndexRearrangementForStreams()

vf::gpu::IndexRearrangementForStreams::IndexRearrangementForStreams ( std::shared_ptr< Parameter >  para,
std::shared_ptr< GridBuilder >  builder,
vf::parallel::Communicator &  communicator 
)

Definition at line 62 of file IndexRearrangementForStreams.cpp.

◆ ~IndexRearrangementForStreams()

virtual vf::gpu::IndexRearrangementForStreams::~IndexRearrangementForStreams ( )
virtualdefault

Member Function Documentation

◆ aggregateCoarseNodesForCtoF()

std::vector< uint > vf::gpu::IndexRearrangementForStreams::aggregateCoarseNodesForCtoF ( int  level) const
protected

Aggregate all nodes in the coarse cells for the interpolation in coarse to fine.

For the coarse cells in the interpolation from coarse to fine only one node is stored. This methods looks for the other nodes of each cell and puts them into vector. Duplicate nodes are only stored once.

Definition at line 211 of file IndexRearrangementForStreams.cpp.

◆ exchangeIndicesForCommAfterFtoC()

std::vector< uint > vf::gpu::IndexRearrangementForStreams::exchangeIndicesForCommAfterFtoC ( const ProcessNeighbor27 &  sendNeighbor,
const ProcessNeighbor27 &  recvNeighbor,
const std::vector< uint > &  sendIndicesForCommAfterFtoCPositions 
) const
protected

send sendIndicesForCommAfterFtoCPositions to receiving process and receive recvIndicesForCommAfterFtoCPositions from neighboring process

Returns
recvIndicesForCommAfterFtoCPositions

Definition at line 141 of file IndexRearrangementForStreams.cpp.

◆ findIndicesNotInCommAfterFtoC()

std::vector< uint > vf::gpu::IndexRearrangementForStreams::findIndicesNotInCommAfterFtoC ( uint  numberOfIndices,
const uint *  indices,
const std::vector< uint > &  indicesAfterFtoC 
)
staticprotected

Find all indices which are not part of the communication after the interpolation from fine to coarse.

Definition at line 237 of file IndexRearrangementForStreams.cpp.

◆ initCommunicationArraysForCommAfterFinetoCoarse()

std::array< ProcessNeighbor27, 4 > vf::gpu::IndexRearrangementForStreams::initCommunicationArraysForCommAfterFinetoCoarse ( const ProcessNeighbor27 &  sendNeighborHost,
const ProcessNeighbor27 &  sendNeighborDevice,
const ProcessNeighbor27 &  recvNeighborHost,
const ProcessNeighbor27 &  recvNeighborDevice,
int  level,
int  direction 
) const
virtual

Initialize the arrays for the communication after the interpolation from fine to coarse.

Only the nodes involved in the interpolation need to be exchanged. Therefore in this method all nodes, which are part of the interpolation as well as the communication, are identified. See [master thesis of Anna Wellmann (p. 59-62: "Reduzieren der auszutauschenden Knoten")]

Returns
array of Process Neighbors after FtoC in same order as input parameters

Reimplemented in vf::gpu::grid_generator_test::IndexRearrangementForStreamsDouble.

Definition at line 70 of file IndexRearrangementForStreams.cpp.

◆ makeProcessNeighborToCommAfterFtoC()

ProcessNeighbor27 vf::gpu::IndexRearrangementForStreams::makeProcessNeighborToCommAfterFtoC ( const ProcessNeighbor27 &  neighbor,
uint  numberOfNodes 
)
staticprotected

Initializes pointers for reduced communication after the interpolation from fine to coarse by copying them from "normal" communication.

Definition at line 200 of file IndexRearrangementForStreams.cpp.

◆ reorderRecvIndicesForCommAfterFtoC()

void vf::gpu::IndexRearrangementForStreams::reorderRecvIndicesForCommAfterFtoC ( uint *  recvIndices,
int  direction,
int  level,
const std::vector< uint > &  recvIndicesForCommAfterFtoCPositions 
) const
protected

Reorder the receive indices in the same way that the send indices were reordered.

When the send indices are reordered, the receive indices need to be reordered accordingly.

Precondition
sendIndicesForCommAfterFtoCPositions should not be empty
Parameters
recvIndicesis the pointer to the vector with the receive indices, which will be reordered in this function
recvIndicesForCommAfterFtoCPositionsstores the positions of the neighbors send indices before reordering and is used to reorder the receive indices in the same way

Definition at line 166 of file IndexRearrangementForStreams.cpp.

◆ reorderSendIndicesForCommAfterFtoC()

std::vector< uint > vf::gpu::IndexRearrangementForStreams::reorderSendIndicesForCommAfterFtoC ( uint *  indices,
int  direction,
int  level 
) const
protected

The send indices are reordered for the communication after the interpolation from fine to coarse.

The indices of nodes which are part of the interpolation are moved to the front of vector with the send indices.

Precondition
para->getParH(level)->intCF needs to be initialized
Returns
each sendindex's positions before reordering

Definition at line 95 of file IndexRearrangementForStreams.cpp.

Friends And Related Symbol Documentation

◆ IndexRearrangementForStreamsTest_exchangeIndicesForCommAfterFtoC

friend class IndexRearrangementForStreamsTest_exchangeIndicesForCommAfterFtoC
friend

Definition at line 128 of file IndexRearrangementForStreams.h.

◆ IndexRearrangementForStreamsTest_reorderRecvIndices

friend class IndexRearrangementForStreamsTest_reorderRecvIndices
friend

Definition at line 127 of file IndexRearrangementForStreams.h.

◆ IndexRearrangementForStreamsTest_reorderSendIndices

friend class IndexRearrangementForStreamsTest_reorderSendIndices
friend

Definition at line 126 of file IndexRearrangementForStreams.h.

The documentation for this class was generated from the following files: