[Kwiver-users] Differences in video_input implementations

[Matt Leotta]

Chet,

Thanks for the excellent summary of this issue.  I'm copying the
kwiver-users list to get feedback from others on how this should work.
Either way the video_input should be made to act consistently across
implementations.  All video inputs should start counting at the same value
(either 0 or 1) and all videos should be in the same state when first
opened.

Here is some more background.  Previously all videos (image lists) were
processed with frame numbers starting at zero.  When the video_input API
was added I noticed at least some (I assumed all) video inputs starting
counting at one.  I updated the TeleSculptor GUI to assume that data was
produced with one-based indexing.  I'm not sure it matters too much if we
start with zero or one, as long as it consistent, but one-based counting
might be more intuitive to users.  For example, if you dump frames from a
video using ffmpeg on the command line it numbers the images starting with
one.  Previously our software indexing was off by one from that, which
sometimes led to confusion and bugs.  So I have a slight preference for
starting with one, but I can be persuaded.

The second issue is whether or not to open a video in a valid state or
not.  That is, do you need to call next_frame() before accessing the first
frame of video.  Opening a video in an invalid state (i.e. before the first
frame of video) is somewhat counter intuitive, but I think there is a good
reason for it.  The reason is more to support live video streams than it is
for loading data from disk.  Especially for live video, you want to acquire
the next frame shortly before you process it.  You might open a video (e.g.
connection to a live camera) but not start processing right away.  If you
grab the first frame on open, it might be old by the time you start
processing.  So it makes sense to ask for the next frame right before you
use it.

So in summary, I would vote for videos requiring next_frame() after open()
to get the first frame.  I have a slight preference for one-based frame
counting.  Most importantly.  All video_input algorithms should have the
same behavior for both of these things.

Does anyone else have any other thoughts on this?  This will impact
everyone doing video processing with KWIVER.

Thanks,
Matt


On Mon, Dec 4, 2017 at 3:50 PM, Chet Nieter <chet.nieter at kitware.com> wrote:

> Hey Matt,
>
> I got my new test test_vidl_ffmpeg_video_input working and passing. I had
> to make two adjustments relative to test_video_input_image_list to get it
> to pass The first was vidl_ffmpeg_video_input is valid (good() returns
> true) before the first call to next_frame(). This method just delegates to
> the underlying vidl video stream object. In video_input_image_list the
> good() method checks the frame number. If it is zero then good() returns
> false. The the second thing which appears to be related is the first frame
> for video_input_image_list has a frame number of 1 while the first frame
> (at least for my test video) for vidl_ffmpeg_video_input has a frame number
> of zero. Again this is just delegated to the underlying video stream
> object. It appears that the frame value of zero is being used a placeholder
> for marking the test_video_input_image_list as currently invalid.  Is
> there a reason that video_input_image_list should not be valid until
> next_frame is called. From what I can tell it already has the lists of
> files once open() has been called and it should be able to start pulling
> imagery. This leaves me with questions about my course of action.
>
> 1) Leave everything as is. The means that test_vidl_ffmpeg_video_input
> will, or at least can, start at frame number 0 where the other video_input
> implementations start at 1. Also test_vidl_ffmpeg_video_input is valid
> after open but before next_frame where the others have to have next_frame
> called first.
> 2) Rework the other video_input implementations so that they start with
> frame 0. If there is a reason to continue to require a next_frame() call
> before validity a bool flag could be added to manage that rather than using
> the frame number.
> 3) Allow the other implementations to be valid after open but before
> calling next_frame() making the assumption that they will start on the
> first frame, whether that is indexed as 0 or 1.
> 4) Both 2 & 3
> 5) Kluge vidl_ffmpeg_video_input to have frame number start at 1 and/or be
> invalid before next_frame() to be consistent.
>
> It seems to me that both changes (option 4) make sense to keep the frame
> numbering consistent and to allow behavior that seems to be artificially
> prohibited. Although I understand that would be a change in the behavior of
> the API of existing classes which can be dangerous.
>
> Either way this doesn’t block me from starting on the new seek
> functionality and the testing of that functionality.
>
> Chet
>
> =======================================
> Chet Nieter, Ph.D
> R&D Enigineer, Kitware Inc.
> (919) 869-8876
> http://www.kitware.com/company/team/nieter.html
> =======================================
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/kwiver-users/attachments/20171205/f5316eba/attachment-0001.html>