[vtkusers] Thread-safety of New and Delete

[David Gobbi]

This should be fine as long as you never have a situation where one of
these internal threads creates an object that is then accessed by one of
the other internal threads. The internal threads have to be isolated from
each other: they can all read the filter's input, but each thread must have
its own independent chunk of memory that it writes to, i.e. no two threads
should ever write to the same memory address.

If each thread creates its own data object, and then these data objects are
merged via a "reduce" operation after the threads are done, then that
should work fine.  Just remember that with a thread pool, the functor will
generally be called more than once per thread, so e.g. if each functor call
resulted in the creation of a new data object, you'd end up with more data
objects than threads.  Thread-local storage object must be designed with
this in mind: each functor call gets the same thread-local storage that
previous functor calls for that thread got.

 - David



On Wed, Jan 13, 2016 at 4:28 PM, eewing <eewing at cs.uoregon.edu> wrote:

> Hi David,
>
> By "Filter-by-filter", I meant that each filter threads its work
> separately, so threads are assigned at the beginning of the filter, and
> reassigned at the beginning of the next filter. Also, we're using a
> threadpool threading model with no direct control of the threads in the
> filters, which could be a problem in the future. The I/O I'm trying to
> thread isn't asynchronous, but rather I'm trying to thread the file readers
> so it can execute faster, and read multiple files simultaneously that make
> up a dataset.
>
> -Elliott
>
>
> On 2016-01-13 15:13, David Gobbi wrote:
>
>> What do you mean by "filter-by-filter" basis? Each filter is in its
>> own thread?
>>
>> If you need asynchronous IO, then a typical approach is to start the
>> thread that does the IO, and then poll that thread until the IO is
>> complete.  When the IO is complete, feed the data into your pipeline.
>> This solution is not specific to VTK in any way, of course.  It's just
>> a general approach to asynchronous IO that works as well with VTK as
>> with anything else.  You can also have the thread read the data
>> piece-by-piece, which would be a bit more complicated.
>>
>> It also works for asynchronous processing.  For long processing tasks,
>> we start a QThread that does the processing and emits a signal when
>> the processing is done.  When our app receives the signal, it knows
>> it's safe to use the result.  In this case Qt is doing the polling for
>> us as part of its mainloop.
>>
>>  - David
>>
>> On Wed, Jan 13, 2016 at 3:39 PM, eewing <eewing at cs.uoregon.edu> wrote:
>>
>> Hi David,
>>>
>>> Thanks for the quick response. That actually answers my issue. I'm
>>> working on threaded I/O as part of a large pipeline, described by
>>> your first example. The pipeline is executed in parts by threads,
>>> but on a filter-by-filter basis, with no way to lock data to a
>>> specific thread.
>>>
>>> What would your recommendation be to mitigate the issue? I've seen
>>> a suggestion to create a "I'm accessing VTK" mutex for whenever you
>>> are accessing VTK, but I'd like to avoid that if possible.
>>>
>>> -Elliott
>>>
>>> On 2016-01-13 14:20, David Gobbi wrote:
>>> Hi Elliott,
>>>
>>> Can you describe your issue in more detail? The VTK New and Delete
>>> calls should be thread safe. The two caveats for VTK and threads
>>> are:
>>>
>>> 1) All classes that do rendering should be in the same thread (all
>>> mappers, actors, renderers, etc), and there shouldn't be multiple
>>> threads hitting the same rendering context.
>>>
>>> 2) Movement of data between threads must be locked or otherwise
>>> done
>>> in a thread-safe manner.
>>>
>>> You would not, for example, create a single pipeline that is
>>> accessed
>>> from multiple threads. Each thread would have its own pipeline
>>> segment, with locks to synchronize a copy (or move) of data from
>>> the
>>> output of one segment to the input of another segment.
>>>
>>> - David
>>>
>>> On Wed, Jan 13, 2016 at 2:27 PM, eewing <eewing at cs.uoregon.edu>
>>> wrote:
>>>
>>> Hello,
>>>
>>> I'm running into some issues with VTK in a threaded environment,
>>> and it seems that the source of the problem is in VTK's New and
>>> Delete calls. So, are the New and Delete calls thread-safe? If not,
>>> is there a recommended alternative that I can use inside my
>>> threads?
>>>
>>> Best,
>>> -Elliott Ewing
>>> University of Oregon
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20160113/1b9cebf5/attachment.html>