<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>I think you are going to want to allocate a slab of memory the exact nx * ny * nz * sizeof pointer. Than you can encode just the number of pointers and initial memory position in each instance of the collection. </div><div><br></div><div>Lookup of exact elements is constant time, and iteration over the entire set can be done very efficiently as you will have cache coherency on the pointers(slab allocating the actual objects is also recommended). <br><br>Sent from my iPhone</div><div><br>On Nov 20, 2012, at 5:45 PM, Philippe Pébay <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>> wrote:<br><br></div><blockquote type="cite"><div>Hello Dave<br><br>Thanks for the comments. So, this might be the better option: rather than implementing a new concrete array type, using a possibly improved version of the vtkCollection. <br><br>A little more context: no random access is necessary. All that is needed is the ability to iterate over the elements rapidly and in a scalable fashion. The collection size is set once for all at the time of the creation of the hyper tree grid object. The collection is used to collect a serialized list of the nx * ny * nz grid of individual trees. It is, also, necessary that shallow copies be done efficiently, that is, by not only shallow copying all individual trees, but also the collection itself, for we do not want to have to replicate O(10^6) pointers.<br>
<br>Thanks<br>Philippe<br><br><div class="gmail_quote">On Tue, Nov 20, 2012 at 11:34 PM, David Cole <span dir="ltr"><<a href="mailto:david.cole@kitware.com" target="_blank">david.cole@kitware.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Its implementation is just a linked list. It will be as fast as you<br>
can possibly be for:<br>
<br>
- iterating the whole collection and doing something on each element<br>
- adding a new element to the end of the list<br>
<br>
For all else, including finding or removing specific items (by index<br>
or identity), it will be as ridiculously slow as you'd expect a linked<br>
list to be...<br>
<br>
After just a glance at the implementation, I'm not surprised it doesn't "scale".<br>
<br>
If you need scalability, perhaps a better implementation could be<br>
dropped in to the same API.<br>
<br>
<br>
On Tue, Nov 20, 2012 at 5:25 PM, Philippe Pébay<br>
<div class="HOEnZb"><div class="h5"><<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>> wrote:<br>
> Hello David<br>
><br>
> Thanks for the suggestion. I looked into vtkCollection but the code was<br>
> running slow with large collections. But it is possible that I will not get<br>
> any better results with the vtkObjectArray. Are you surprised that the<br>
> vtkCollection did not scale well (my observation on only one platform)?<br>
><br>
> Thanks<br>
> Philippe<br>
><br>
><br>
><br>
> On Tue, Nov 20, 2012 at 6:41 PM, David Thompson <<a href="mailto:david.thompson@kitware.com">david.thompson@kitware.com</a>><br>
> wrote:<br>
>><br>
>> Hi Philippe,<br>
>><br>
>> I think the vtkObjectArray class sounds like a good idea, however be aware<br>
>> that vtkCollection already exists to fulfill a similar purpose.<br>
>><br>
>> David<br>
>><br>
>> On Nov 20, 2012, at 12:28 PM, Philippe Pébay <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>><br>
>> wrote:<br>
>><br>
>> > Hello David<br>
>> ><br>
>> > Yes, I think this is the best option. I am thinking that I could create<br>
>> > this as a new concrete subclass of vtkAbstractArray available to all, and<br>
>> > not only as a helper class internal to my hyper tree implementation. What do<br>
>> > you think? Would this be useful?<br>
>> ><br>
>> > Thanks<br>
>> > Philippe<br>
>> ><br>
>> > On Tue, Nov 20, 2012 at 6:25 PM, David Gobbi <<a href="mailto:david.gobbi@gmail.com">david.gobbi@gmail.com</a>><br>
>> > wrote:<br>
>> > Hi Phillippe,<br>
>> ><br>
>> > Actually I was wrong about vtkVariant not having any size overhead. It<br>
>> > needs<br>
>> > to store the type along with the value. So vtkVariantArray is no good<br>
>> > for this.<br>
>> ><br>
>> > A helper class might be the best option. You could make a<br>
>> > vtkObjectArray<br>
>> > class which would be just like vtkVoidArray except that it would<br>
>> > objects.<br>
>> ><br>
>> > - David<br>
>> ><br>
>> ><br>
>> > On Tue, Nov 20, 2012 at 9:54 AM, Philippe Pébay<br>
>> > <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>> wrote:<br>
>> > > Hello David<br>
>> > ><br>
>> > > Thanks for the summary about the absence of performance hit when using<br>
>> > > SPs<br>
>> > > as opposed to RPs. I will remember it and use them more often as a<br>
>> > > result.<br>
>> > ><br>
>> > > Regarding the variant array, yes, the conversion to vtkObject is a<br>
>> > > concern.<br>
>> > > I think that the most efficient option would be to create a small<br>
>> > > helper<br>
>> > > class that would internally store a raw array of pointers. That helper<br>
>> > > would<br>
>> > > derive from vtkObjectBase. That should avoid any perfomance hit.<br>
>> > ><br>
>> > > Philippe<br>
>> > ><br>
>> > ><br>
>> > > On Tue, Nov 20, 2012 at 5:49 PM, David Gobbi <<a href="mailto:david.gobbi@gmail.com">david.gobbi@gmail.com</a>><br>
>> > > wrote:<br>
>> > >><br>
>> > >> Hi Phillippe,<br>
>> > >><br>
>> > >> A vtkVariant is 64 bits, it should be no larger than a pointer.<br>
>> > >> There<br>
>> > >> is a function call overhead in ToVTKObject(), which might be a<br>
>> > >> concern to<br>
>> > >> you.<br>
>> > >><br>
>> > >> The vtkSmartPointer is exactly the same size as a pointer, so there<br>
>> > >> is no<br>
>> > >> size<br>
>> > >> overhead. It has no computational overhead, either, except for the<br>
>> > >> "operator=",<br>
>> > >> the copy constructor, and the destructor (I wouldn't even count the<br>
>> > >> overhead in<br>
>> > >> the destructor, because you have to call ->Delete() eventually<br>
>> > >> anyway).<br>
>> > >> So do<br>
>> > >> not avoid smart pointers because you are worried about efficiency.<br>
>> > >> The<br>
>> > >> VTK<br>
>> > >> smart pointers are very efficient because vtkObjectBase already has a<br>
>> > >> built-in<br>
>> > >> reference count.<br>
>> > >><br>
>> > >> - David<br>
>> > >><br>
>> > >><br>
>> > >> On Tue, Nov 20, 2012 at 9:35 AM, Philippe Pébay<br>
>> > >> <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>> wrote:<br>
>> > >> > Hello David<br>
>> > >> ><br>
>> > >> > Thanks for the suggestion: I am concerned by the overhead that<br>
>> > >> > comes<br>
>> > >> > with<br>
>> > >> > variants. Wouldn't it be more lightweight to use a vtkVoidArray<br>
>> > >> > instead?<br>
>> > >> > The<br>
>> > >> > array can potentially contain O(10⁶) pointers. By the way this is<br>
>> > >> > for<br>
>> > >> > the<br>
>> > >> > same reason that I have, so far, refrained from using<br>
>> > >> > vtkSmartPointers<br>
>> > >> > as<br>
>> > >> > opposed to raw pointers, by fear of the overhead involved. Is this<br>
>> > >> > extra<br>
>> > >> > caution excessive?<br>
>> > >> ><br>
>> > >> > Thanks<br>
>> > >> > Philippe<br>
>> > >> ><br>
>> > >> > On Tue, Nov 20, 2012 at 5:31 PM, David Gobbi<br>
>> > >> > <<a href="mailto:david.gobbi@gmail.com">david.gobbi@gmail.com</a>><br>
>> > >> > wrote:<br>
>> > >> >><br>
>> > >> >> Hi Philippe,<br>
>> > >> >><br>
>> > >> >> If the pointers are pointers to VTK objects, then you can use<br>
>> > >> >> vtkVariantArray.<br>
>> > >> >><br>
>> > >> >> - David<br>
>> > >> >><br>
>> > >> >><br>
>> > >> >> On Tue, Nov 20, 2012 at 9:29 AM, Philippe Pébay<br>
>> > >> >> <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>> wrote:<br>
>> > >> >> > Just an idea: couldn't a vtkVoidArray be used (or abused) as<br>
>> > >> >> > container<br>
>> > >> >> > of<br>
>> > >> >> > raw pointers, that would be appropriately downcasted internally<br>
>> > >> >> > to<br>
>> > >> >> > the<br>
>> > >> >> > concrete type to which they are supposed to point? For instance,<br>
>> > >> >> > instead<br>
>> > >> >> > of<br>
>> > >> >> > having a<br>
>> > >> >> > vtkMyInternalObject** obj<br>
>> > >> >> > instance variable, couldn't I use a<br>
>> > >> >> > vtkVoidArray* obj<br>
>> > >> >> > instead, where each entry in obj would be the raw pointer to a<br>
>> > >> >> > vtkSmartPointer<vtkMyInternalObject>?<br>
>> > >> >> ><br>
>> > >> >> > Thanks<br>
>> > >> >> > Philippe<br>
>> > >> >> ><br>
>> > >> >> ><br>
>> > >> >> > On Tue, Nov 20, 2012 at 5:14 PM, Philippe Pébay<br>
>> > >> >> > <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>><br>
>> > >> >> > wrote:<br>
>> > >> >> >><br>
>> > >> >> >> Hello Kyle<br>
>> > >> >> >><br>
>> > >> >> >> Yes, I would want to avoid a dependency on Boost. But this is<br>
>> > >> >> >> basically<br>
>> > >> >> >> what I am tried to reproduce (the Boost shared array of<br>
>> > >> >> >> pointers).<br>
>> > >> >> >><br>
>> > >> >> >> Thanks<br>
>> > >> >> >> P<br>
>> > >> >> >><br>
>> > >> >> >><br>
>> > >> >> >> On Tue, Nov 20, 2012 at 5:13 PM, Kyle Lutz<br>
>> > >> >> >> <<a href="mailto:kyle.lutz@kitware.com">kyle.lutz@kitware.com</a>><br>
>> > >> >> >> wrote:<br>
>> > >> >> >>><br>
>> > >> >> >>> Would using boost suffice? Or do you want a solution using<br>
>> > >> >> >>> just<br>
>> > >> >> >>> VTK?<br>
>> > >> >> >>><br>
>> > >> >> >>> -kyle<br>
>> > >> >> >>><br>
>> > >> >> >>> On Tue, Nov 20, 2012 at 7:58 AM, Philippe Pébay<br>
>> > >> >> >>> <<a href="mailto:philippe.pebay@kitware.com">philippe.pebay@kitware.com</a>> wrote:<br>
>> > >> >> >>> > Hello all,<br>
>> > >> >> >>> ><br>
>> > >> >> >>> > I need to keep track of shared ownership of an array of<br>
>> > >> >> >>> > pointers.<br>
>> > >> >> >>> > Can<br>
>> > >> >> >>> > someone point me towards an example of code that already<br>
>> > >> >> >>> > does<br>
>> > >> >> >>> > this?<br>
>> > >> >> >>> > This<br>
>> > >> >> >>> > should amount to a small helper class that contains the<br>
>> > >> >> >>> > array of<br>
>> > >> >> >>> > pointers<br>
>> > >> >> >>> > along with a reference count.<br>
>> > >> >> >>> ><br>
>> > >> >> >>> > Thank you!<br>
>> > >> >> >>> > Philippe<br>
>> > >> >> >>> ><br>
>> > >> >> >>> ><br>
>> > >> >> >>> > --<br>
>> > >> >> >>> > Philippe Pébay, PhD<br>
>> > >> >> >>> > Director of Visualization and High Performance Computing /<br>
>> > >> >> >>> > Directeur de la Visualisation et du Calcul Haute Performance<br>
>> > >> >> >>> > Kitware SAS<br>
>> > >> >> >>> > 26 rue Louis Guérin, 69100 Villeurbanne, France<br>
>> > >> >> >>> > <a href="tel:%2B33%20%280%29%206.83.61.55.70" value="+33683615570">+33 (0) 6.83.61.55.70</a> / 4.37.45.04.15<br>
>> > >> >> >>> > <a href="http://www.kitware.fr" target="_blank">http://www.kitware.fr</a><br>
>> > ><br>
>> > ><br>
>> > ><br>
>> > ><br>
>> > > --<br>
>> > > Philippe Pébay, PhD<br>
>> > > Director of Visualization and High Performance Computing /<br>
>> > > Directeur de la Visualisation et du Calcul Haute Performance<br>
>> > > Kitware SAS<br>
>> > > 26 rue Louis Guérin, 69100 Villeurbanne, France<br>
>> > > <a href="tel:%2B33%20%280%29%206.83.61.55.70" value="+33683615570">+33 (0) 6.83.61.55.70</a> / 4.37.45.04.15<br>
>> > > <a href="http://www.kitware.fr" target="_blank">http://www.kitware.fr</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> > Philippe Pébay, PhD<br>
>> > Director of Visualization and High Performance Computing /<br>
>> > Directeur de la Visualisation et du Calcul Haute Performance<br>
>> > Kitware SAS<br>
>> > 26 rue Louis Guérin, 69100 Villeurbanne, France<br>
>> > <a href="tel:%2B33%20%280%29%206.83.61.55.70" value="+33683615570">+33 (0) 6.83.61.55.70</a> / 4.37.45.04.15<br>
>> > <a href="http://www.kitware.fr" target="_blank">http://www.kitware.fr</a><br>
>> > _______________________________________________<br>
>> > Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
>> ><br>
>> > Visit other Kitware open-source projects at<br>
>> > <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
>> ><br>
>> > Follow this link to subscribe/unsubscribe:<br>
>> > <a href="http://www.vtk.org/mailman/listinfo/vtk-developers" target="_blank">http://www.vtk.org/mailman/listinfo/vtk-developers</a><br>
>> ><br>
>><br>
><br>
><br>
><br>
> --<br>
> Philippe Pébay, PhD<br>
> Director of Visualization and High Performance Computing /<br>
> Directeur de la Visualisation et du Calcul Haute Performance<br>
> Kitware SAS<br>
> 26 rue Louis Guérin, 69100 Villeurbanne, France<br>
> <a href="tel:%2B33%20%280%29%206.83.61.55.70" value="+33683615570">+33 (0) 6.83.61.55.70</a> / 4.37.45.04.15<br>
> <a href="http://www.kitware.fr" target="_blank">http://www.kitware.fr</a><br>
><br>
> _______________________________________________<br>
> Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
><br>
> Visit other Kitware open-source projects at<br>
> <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
><br>
> Follow this link to subscribe/unsubscribe:<br>
> <a href="http://www.vtk.org/mailman/listinfo/vtk-developers" target="_blank">http://www.vtk.org/mailman/listinfo/vtk-developers</a><br>
><br>
><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><font color="#888888">Philippe Pébay, PhD<br></font><font color="#888888">Director of Visualization and High Performance Computing /<br>
</font><font color="#888888">Directeur de la Visualisation et du Calcul Haute Performance<br>
Kitware SAS<br>26 rue Louis Guérin, 69100 Villeurbanne, France</font><br>
<font color="#888888"><a value="+33426685003">+33 (0) 6.83.61.55.70 / 4.37.45.04.15</a></font><font color="#888888"><br><a href="http://www.kitware.fr/" target="_blank">http://www.kitware.fr</a></font><font color="#888888"><a href="http://www.kitware.fr/" target="_blank"></a></font><br>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Powered by <a href="http://www.kitware.com">www.kitware.com</a></span><br><span></span><br><span>Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a></span><br><span></span><br><span>Follow this link to subscribe/unsubscribe:</span><br><span><a href="http://www.vtk.org/mailman/listinfo/vtk-developers">http://www.vtk.org/mailman/listinfo/vtk-developers</a></span><br><span></span><br></div></blockquote></body></html>