[petsc-users] Testing MatGetSubMatrices

Discussion:

Bikash Kanungo

2015-10-20 15:53:44 UTC

Hi,

I was trying to perform a simple test on MatGetSubMatrices. I have an
MPIDENSE matrix X which has been populated and assembled and another matrix
XTemp of the same type and size. Since X is MPIDENSE in order to extract a
sub-matrix from it, the IS must be sorted. This can be done easily through
ISSort. Once the sub-matrix is obtained, I'm trying to set the values in
XTemp by using the sorted IS as the global indices and the sub-matrix as
the values to be set. Once XTemp is set and assembled, I evaluate XTemp - X
and see if the norm of the resultant matrix is zero or not. The norm turned
out to be as high as 1.0e6.

I'm providing a simplified version of the code that performs the above
test. Kindly let me know if I have done any mistake is designing the test.

//
// Given an initialized and assembled MPIDENSE Mat X and
// an uninitialized MPIDENSE Mat XTemp of the same size as that of X
//
IS * ISRowIds, * ISColIds;
PetscMalloc(1*sizeof(IS), &ISRowIds);
PetscMalloc(1*sizeof(IS), &ISColIds);

//
// globalRowIds and globalColIds are STL vectors containing
// the global node ids for which the sub-matrix from X need
// to be extracted
//
const PetscInt * rowIdsPtr = globalRowIds.data();
const PetscInt * colIdsPtr = globalColIds.data();

ISCreateGeneral(PETSC_COMM_WORLD, globalRowIds.size(), rowIdsPtr,
PETSC_COPY_VALUES, &ISRowIds[0]);
ISCreateGeneral(PETSC_COMM_WORLD, globalColIds.size(), colIdsPtr,
PETSC_COPY_VALUES, &ISColIds[0]);
ISSort(ISRowIds[0]);
ISSort(ISColIds[0]);

Mat * subMat;
MatGetSubMatrices(X, 1, ISRowIds, ISColIds, MAT_INITIAL_MATRIX,
&subMat);

PetscScalar * subMatValues;
MatDenseGetArray(*subMat, &subMatValues);

const PetscInt * rowIndicesSorted, * colIndicesSorted;
ISGetIndices(ISRowIds[0], &rowIndicesSorted);
ISGetIndices(ISColIds[0], &colIndicesSorted);

MatSetValues(XTemp,
globalRowIds.size(),
rowIndicesSorted,
globalColIds.size(),
colIndicesSorted,
subMatValues,
INSERT_VALUES);

//
// commencing assembly
//
MatAssemblyBegin(XTemp,
MAT_FINAL_ASSEMBLY);

//
// concluding assembly
//
MatAssemblyEnd(XTemp,
MAT_FINAL_ASSEMBLY);

MatAXPY(XTemp, -1.0, X, DIFFERENT_NONZERO_PATTERN);
double diffNorm;
MatNorm(XTemp, NORM_1, &diffNorm);

Thanks,
Bikash

--
Bikash S. Kanungo
PhD Student
Computational Materials Physics Group
Mechanical Engineering
University of Michigan

Matthew Knepley

2015-10-20 15:58:46 UTC

Permalink

Post by Bikash Kanungo
Hi,
I was trying to perform a simple test on MatGetSubMatrices. I have an
MPIDENSE matrix X which has been populated and assembled and another matrix
XTemp of the same type and size. Since X is MPIDENSE in order to extract a
sub-matrix from it, the IS must be sorted. This can be done easily through
ISSort. Once the sub-matrix is obtained, I'm trying to set the values in
XTemp by using the sorted IS as the global indices and the sub-matrix as
the values to be set. Once XTemp is set and assembled, I evaluate XTemp - X
and see if the norm of the resultant matrix is zero or not. The norm turned
out to be as high as 1.0e6.

Make the problem very small (10 rows maybe) and starting printing things
out.

Matt

Post by Bikash Kanungo
I'm providing a simplified version of the code that performs the above
test. Kindly let me know if I have done any mistake is designing the test.
//
// Given an initialized and assembled MPIDENSE Mat X and
// an uninitialized MPIDENSE Mat XTemp of the same size as that of X
//
IS * ISRowIds, * ISColIds;
PetscMalloc(1*sizeof(IS), &ISRowIds);
PetscMalloc(1*sizeof(IS), &ISColIds);
//
// globalRowIds and globalColIds are STL vectors containing
// the global node ids for which the sub-matrix from X need
// to be extracted
//
const PetscInt * rowIdsPtr = globalRowIds.data();
const PetscInt * colIdsPtr = globalColIds.data();
ISCreateGeneral(PETSC_COMM_WORLD, globalRowIds.size(), rowIdsPtr,
PETSC_COPY_VALUES, &ISRowIds[0]);
ISCreateGeneral(PETSC_COMM_WORLD, globalColIds.size(), colIdsPtr,
PETSC_COPY_VALUES, &ISColIds[0]);
ISSort(ISRowIds[0]);
ISSort(ISColIds[0]);
Mat * subMat;
MatGetSubMatrices(X, 1, ISRowIds, ISColIds, MAT_INITIAL_MATRIX,
&subMat);
PetscScalar * subMatValues;
MatDenseGetArray(*subMat, &subMatValues);
const PetscInt * rowIndicesSorted, * colIndicesSorted;
ISGetIndices(ISRowIds[0], &rowIndicesSorted);
ISGetIndices(ISColIds[0], &colIndicesSorted);
MatSetValues(XTemp,
globalRowIds.size(),
rowIndicesSorted,
globalColIds.size(),
colIndicesSorted,
subMatValues,
INSERT_VALUES);
//
// commencing assembly
//
MatAssemblyBegin(XTemp,
MAT_FINAL_ASSEMBLY);
//
// concluding assembly
//
MatAssemblyEnd(XTemp,
MAT_FINAL_ASSEMBLY);
MatAXPY(XTemp, -1.0, X, DIFFERENT_NONZERO_PATTERN);
double diffNorm;
MatNorm(XTemp, NORM_1, &diffNorm);
Thanks,
Bikash
--
Bikash S. Kanungo
PhD Student
Computational Materials Physics Group
Mechanical Engineering
University of Michigan

--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

Bikash Kanungo

2015-10-20 17:52:42 UTC

Permalink

Thanks for the suggestion Matthew. The issue was that MatDenseGetArray
stores the value in a compressed column format rather than a row format. So
while setting the values into XTemp I must carefully swap between row and
column indices.

Regards,
Bikash

Post by Matthew Knepley

Make the problem very small (10 rows maybe) and starting printing things
out.
Matt

--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

--
Bikash S. Kanungo
PhD Student
Computational Materials Physics Group
Mechanical Engineering
University of Michigan