In the long run, and this is probably a dumb idea, we may want to take over heap management. There are a lot of reasons for this, performance, error handling, etc., but this is a big bite to take...<br>W<br><br><div class="gmail_quote">
On Fri, Jun 20, 2008 at 11:48 AM, Burlen Loring <<a href="mailto:burlen.loring@kitware.com">burlen.loring@kitware.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Utkarsh,<div class="Ih2E3d"><br>
<br>
Utkarsh Ayachit wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
In the new "new" implementation a resize always needs atleast twice as much memory available. <br>
</blockquote></div>
That is actually how the existing implementation works... On the bright side, I think its also the default for how stl vectors are implemented...and gives the new "new" implementation a realloc like characteristic. In any case, my intuition is that using new will be slower than malloc but easier to achieve portable reliability...<br>
<br>
Can anyone suggest "VTK approved" alternatives?<br>
<br>
Obviously we can't continue to write into memory we don't own, but how can we portably, reliably and efficiently let the caller know that an InsertTuple, GetWritePointer has failed due to failed malloc?<br><font color="#888888">
<br>
Burlen</font><div><div></div><div class="Wj3C7c"><br>
<br>
<br>
Utkarsh Ayachit wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
The nice thing about realloc is that even when the memory grows, it may not result in any memory moves. In the new "new" implementation a resize always needs atleast twice as much memory available. With realloc, if the original memptr is already at the end of the heap (for example), then realloc will simply "grow" the data block allocated, without "moving" the block at all -- hence no copy incurred. Which also makes it possible to grow the array to a larger size than possible with "new|copy|delete" for a given system.<br>
<br>
Utkarsh<br>
<br>
Burlen Loring wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Utkarsh,<br>
<br>
Oh, OK, sorry Dave I misunderstood.<br>
<br>
In the current implementation, if __APPLE__ is defined then realloc is *not* used. Arrays are always moved. From what I understand(I am not a mac user) gcc defines this on mac. & gcc is mac's default compiler. I assume that if not using realloc were a huge issue that mac users would have complained about this? Also we have the choice of doing something like:<br>
<br>
if NewSize<OldSize<br>
update size<br>
update end of array index<br>
else<br>
get new memory<br>
copy<br>
delete old memory<br>
update size<br>
update end of array index<br>
fi<br>
in our Realloc. I didn't bother to do this because, of my point about the mac. If we did this it would have to be up to the user to "squeeze" in order to reclaim the wasted bytes.<br>
<br>
Burlen<br>
<br>
<br>
Utkarsh Ayachit wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Burlen,<br>
<br>
I think what Dave is trying to say is that now since you cannot use realloc, every time the resize happens, it will have to be a new-and-copy operation. With realloc it is possible that the memory block is never moved. I think now I remember why I had reverted the malloc-to-new commit in past -- it was due to same realloc issue.<br>
<br>
Utkarsh<br>
<br>
Burlen Loring wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi David,<br>
<br>
Maybe I should have gone into more detail in my description of the changes. In this class there is a flag to indicate what the "delete method" is as a user can pass in an array to use. They can set either to use free or delete. So what I have done in the line you reference is use realloc if the array has been marked as created with malloc. This is how the class behaved before.<br>
<br>
Burlen<br>
<br>
David Cole wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Well, for one thing, you can't do this:<br>
+ newArray = static_cast<T*>(realloc(this->Array, sz*sizeof(T)));<br>
<br>
if you eliminate all the use of malloc. realloc calls don't make any sense if you are using C++ new/delete.<br>
<br>
<br>
<br>
2008/6/19 Burlen Loring <<a href="mailto:burlen.loring@kitware.com" target="_blank">burlen.loring@kitware.com</a> <mailto:<a href="mailto:burlen.loring@kitware.com" target="_blank">burlen.loring@kitware.com</a>>>:<br>
<br>
Hi all,<br>
<br>
I would like to get some feed back on a change I would like to<br>
make to vtkDataArrayTemplate.txx.<br>
<br>
Here is the problem that prompted the changes:<br>
<br>
Currently during inserts, a data array resizes itself if need be.<br>
Unfortunately, in the case that it runs out of memory during the<br>
resize, it ignores the fact that it didn't get the needed space<br>
and writes into memory it doesn't own. For example take a look at<br>
how GetWritePointer is ignoring the return value from<br>
ResizeAndExtend which, will be 0 to indicate that malloc failed.<br>
And further how InsertNextTuple/InsertTuple has no way of knowing<br>
if the malloc failed. The insert then takes place into memory not<br>
owned by the data array.<br>
<br>
If you are unfortunate enough to use enough ram that malloc fails<br>
during a series of inserts, this can cause some very strange crashes!<br>
<br>
Attached is a patch I'd like to submit to cvs. What I have done is<br>
replace the malloc/free pairs with new/delete pairs. Failure of<br>
new is much easier to deal with (compared to malloc) because it<br>
throws a bad_alloc exception, which clearly indicates the nature<br>
of the problem, and the end user can decide how to handle that.<br>
Additionally the patch makes sure that if new returns 0 rather<br>
than throwing an exception(could happen on older compilers), a<br>
return value indicating an error occurred gets propagated back up<br>
to the caller immediately. For methods that return pointers the<br>
error value is 0 for methods that return vtkIdType the error value<br>
is -1. There are a few methods (eg SetTuple) that are typed as<br>
returning void, which simply return. In the case of new failures<br>
during an insert, this prevents the data array from writing into<br>
memory it doesn't own.<br>
<br>
Any objections/criticism/comments regarding this change?<br>
<br>
Thanks in advance<br>
Burlen<br>
<br>
-- Burlen Loring<br>
Kitware, Inc.<br>
R&D Engineer<br>
28 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: 518-371-3971 x137<br>
<br>
<br>
_______________________________________________<br>
vtk-developers mailing list<br>
<a href="mailto:vtk-developers@vtk.org" target="_blank">vtk-developers@vtk.org</a> <mailto:<a href="mailto:vtk-developers@vtk.org" target="_blank">vtk-developers@vtk.org</a>><br>
<a href="http://www.vtk.org/mailman/listinfo/vtk-developers" target="_blank">http://www.vtk.org/mailman/listinfo/vtk-developers</a><br>
<br>
<br>
</blockquote>
<br>
<br>
</blockquote></blockquote>
<br>
<br>
</blockquote></blockquote>
<br>
<br>
-- <br>
Burlen Loring<br>
Kitware, Inc.<br>
R&D Engineer<br>
28 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: 518-371-3971 x137<br>
<br>
_______________________________________________<br>
vtk-developers mailing list<br>
<a href="mailto:vtk-developers@vtk.org" target="_blank">vtk-developers@vtk.org</a><br>
</div></div><div><div></div><div class="Wj3C7c"><a href="http://www.vtk.org/mailman/listinfo/vtk-developers" target="_blank">http://www.vtk.org/mailman/listinfo/vtk-developers</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>William J. Schroeder, PhD<br>Kitware, Inc.<br>28 Corporate Drive<br>Clifton Park, NY 12065<br><a href="mailto:will.schroeder@kitware.com">will.schroeder@kitware.com</a><br>
<a href="http://www.kitware.com">http://www.kitware.com</a><br>518-371-3971 (phone and fax)