[英]C# Out of Memory retrieving entity details from Dynamics 365
Since I have millions of records in Dynamics 365 and I need to retrieve them all, I am using paging to retrieve the entity data:由于我在 Dynamics 365 中有数百万条记录并且我需要全部检索它们,因此我使用分页来检索实体数据:
// Query using the paging cookie.
// Define the paging attributes.
// The number of records per page to retrieve.
int queryCount = 5000;
// Initialize the page number.
int pageNumber = 1;
// Initialize the number of records.
int recordCount = 0;
// Define the condition expression for retrieving records.
ConditionExpression pagecondition = new ConditionExpression();
pagecondition.AttributeName = "parentaccountid";
pagecondition.Operator = ConditionOperator.Equal;
pagecondition.Values.Add(_parentAccountId);
// Define the order expression to retrieve the records.
OrderExpression order = new OrderExpression();
order.AttributeName = "name";
order.OrderType = OrderType.Ascending;
// Create the query expression and add condition.
QueryExpression pagequery = new QueryExpression();
pagequery.EntityName = "account";
pagequery.Criteria.AddCondition(pagecondition);
pagequery.Orders.Add(order);
pagequery.ColumnSet.AddColumns("name", "emailaddress1");
// Assign the pageinfo properties to the query expression.
pagequery.PageInfo = new PagingInfo();
pagequery.PageInfo.Count = queryCount;
pagequery.PageInfo.PageNumber = pageNumber;
// The current paging cookie. When retrieving the first page,
// pagingCookie should be null.
pagequery.PageInfo.PagingCookie = null;
Console.WriteLine("Retrieving sample account records in pages...\n");
Console.WriteLine("#\tAccount Name\t\tEmail Address");
while (true)
{
// Retrieve the page.
EntityCollection results = _serviceProxy.RetrieveMultiple(pagequery);
if (results.Entities != null)
{
// Retrieve all records from the result set.
foreach (Account acct in results.Entities)
{
Console.WriteLine("{0}.\t{1}\t{2}", ++recordCount, acct.Name,
acct.EMailAddress1);
}
}
// Check for more records, if it returns true.
if (results.MoreRecords)
{
Console.WriteLine("\n****************\nPage number {0}\n****************", pagequery.PageInfo.PageNumber);
Console.WriteLine("#\tAccount Name\t\tEmail Address");
// Increment the page number to retrieve the next page.
pagequery.PageInfo.PageNumber++;
// Set the paging cookie to the paging cookie returned from current results.
pagequery.PageInfo.PagingCookie = results.PagingCookie;
}
else
{
// If no more records are in the result nodes, exit the loop.
break;
}
}
After a couple of hours of pulling data, the program ends saying OutofMemory exception.在提取数据几个小时后,程序以 OutofMemory 异常结束。 It would appear that memory usage grows with every new page.
似乎 memory 的使用量随着每个新页面的增加而增加。 How does one clear the memory usage to avoid this problem?
如何清除 memory 用法以避免此问题?
It's good that you are retrieving only a couple columns.最好只检索几列。 Getting all the columns would certainly increase memory usage.
获取所有列肯定会增加 memory 的使用。
They way that you're processing each page it would seem that the runtime should release each results
object after you output it to the console.他们处理每个页面的方式似乎运行时应该在您将 output 发布到控制台之后将每个
results
object 发布。 It seems that something might be causing each results
objects to stay on the heap.似乎某些东西可能导致每个
results
对象都留在堆上。
Maybe it's the reuse of the same pagequery
object.也许是对同一个
pagequery
object的重用。
The only thing that looks like it persists from the results
is the paging cookie. results
中唯一看起来像它的东西是分页cookie。
pagequery.PageInfo.PagingCookie = results.PagingCookie;
Maybe try moving the creation of the QueryExpression
into a method and create a new one for each page.也许尝试将
QueryExpression
的创建移动到一个方法中,并为每个页面创建一个新方法。 For example:例如:
private QueryExpression getQuery(int pageNum, string pagingCookie)
{
//all query creation code
}
What is going on with the condition for the parentaccountid
? parentaccountid
的条件是怎么回事? Does a single parent have millions of child accounts?单亲父母有数百万的孩子账户吗?
You might also want to consider tweaking the while
loop:您可能还需要考虑调整
while
循环:
var continue = true;
while(continue)
{
//process
continue = results.MoreRecords;
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.