Tuesday, November 8, 2011

Retrieving all the record from an entity

By default, the RetrieveMultiple method retrieves only 5000 records, this is a limit from the standpoint of performance. In order to retrieve more than 5000 records, please fallow the pattern explained in the below code.

I have written a console application to retrieve all the Account records. The below code is a method to achieve that. The main logic in retrieving all the records is by passing the PageInfo object to a QueryExpression.
private static void GetAllActiveAccounts(OrganizationServiceProxy service)
{
EntityCollection retrieved;
const int servicePageSize = 5000;
int pageNumber = 1;
string pagingCookie = string.Empty;
const int pageSize = servicePageSize;
int totalRecordsCount = 0;

do
{
var cols = new ColumnSet();
cols.AddColumns(new string[] { ACCOUNT_ID, ACCOUNT_NAME });
var filter = new FilterExpression { FilterOperator = LogicalOperator.And };
filter.AddCondition(new ConditionExpression(STATE_CODE, ConditionOperator.Equal, new object[] { 0 }));


var query = new QueryExpression
{
ColumnSet = cols,
Criteria = filter,
EntityName = ENTITY_ACCOUNT,
PageInfo = new PagingInfo()
{
PageNumber = 1,
Count = pageSize
}
};

if (pageNumber != 1)
{
query.PageInfo.PageNumber = pageNumber;
query.PageInfo.PagingCookie = pagingCookie;
}

retrieved = service.RetrieveMultiple(query);
if (retrieved.MoreRecords)
{
pageNumber++;
pagingCookie = retrieved.PagingCookie;
}

try
{
if (retrieved.Entities.Count > 0)
{
totalRecordsCount += retrieved.Entities.Count;

foreach (var accountEntity in retrieved.Entities)
{
if (accountEntity.Attributes.Contains(ACCOUNT_ID))
{
var accountId = (Guid)accountEntity.Attributes[ACCOUNT_ID];
var accountName = (string)accountEntity.Attributes[ACCOUNT_NAME];
Console.WriteLine("Account Name: " + accountName);
}
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.ReadLine();
}

} while (retrieved.MoreRecords);

Console.WriteLine("Total records: " + totalRecordsCount);
}

5 comments:

Anonymous said...

Thank you very much for your code. It has helped me
Alvaro from Spain

Anonymous said...

Hey Chaitanya,

I had no clue you were blogging. I just ran across this entry. Very nice write-up. I hope all is well, it saddens me that we haven't been able to stay in touch, but I do understand.

I hope all is well :).

Sincerely,

Jason Foerch

Farina said...

Thank you much for this code.
I was banging my head for the last two days. Your code really saved me from my predicament.

Anonymous said...

Thank you a lot
Refactor little :

private static HashSet GetAllEntityRecords(string entityName)
{
var retrievedEntity = new HashSet();
const int pageSize = 5000;

var queryEntity = new QueryExpression()
{
ColumnSet = new ColumnSet(true),
EntityName = entityName,
PageInfo = new PagingInfo()
{
PageNumber = 1,
Count = pageSize
}
};

EntityCollection results;

do
{
results = _proxy.RetrieveMultiple(queryEntity);

queryEntity.PageInfo.PageNumber++;
queryEntity.PageInfo.PagingCookie = results.PagingCookie;

if(results.Entities != null && results.Entities.Any())
retrievedEntity.UnionWith(results.Entities);

} while (results.MoreRecords);

return retrievedEntity;
}

Unknown said...

Thank u. this really help me!