<div dir="ltr"><div>David,<br><br></div>I missed answering item number 3, but Matt addressed it in his reply.<br><br><p><span style="font-size:11pt;font-family:"calibri","sans-serif";color:rgb(31,73,125)"><span>3)<span style="font:7pt "times new roman"">
</span></span></span><span style="font-size:11pt;font-family:"calibri","sans-serif";color:rgb(31,73,125)">I
assume my code will be used as “plugin”, and VIAME has a sub-folder. I
have used python as APIs to wrap around a bunch of lower level c++
libraries.
</span></p>Our intent is to present your "plugin" as one or more loadable algorithm instances or, less desirably, processes. I'm sure we will need to extend our set of algorithm interfaces to handle new requirements.<br><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 6, 2016 at 7:06 PM, Matt Dawkins <span dir="ltr"><<a href="mailto:matt.dawkins@kitware.com" target="_blank">matt.dawkins@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>For item #1-3 while there is currently no python object_detector base class, you can still write a python sprokit process which takes in an image and processes it (an example is in packages/kwiver/sprokit/<wbr>processes/python). The plugins folder is indeed where we intend for each group to put their modules, with each one having an enable flag in the above cmake to turn the module on and off. External projects (e.g. other gitlab projects) can be added to the packages directory, as a git submodule, or by downloading the package if not using git (there is a DownloadFile cmake macro in the repo).<br><br></div>As of today (5 minutes ago in master) there is a VIAME_ENABLE_PYTHON and VIAME_ENABLE_DOCS flag that can be set in the repo. VIAME_ENABLE_PYTHON will build the Caffe and KWIVER/Sprokit python interfaces.<br><br></div>Also there is now a setup_viame.sh script instead of a setup_KWIVER.sh in latest master in the installation dir, that will set the appropriate paths to run things on linux/mac.<br><br></div>For items #4-5, if you want to disable all GPU support (in OpenCV, Caffe, KWIVER), you can set VIAME_DISABLE_GPU_SUPPORT to ON. If you want to set manual caffe build settings, you need to go down into [build/src/fletch-build/build/<wbr>src/Caffe-build] and run "ccmake ." and then "make install". The current caffe version is from BLVC, just a slightly dated master (a couple months). If the current master supports the latest cuDNN we can update. Alternatively we can add an advanced higher level VIAME build flag that disables cuDNN support.<br><br></div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On Tue, Sep 6, 2016 at 6:10 PM, Linus Sherrill <span dir="ltr"><<a href="mailto:linus.sherrill@kitware.com" target="_blank">linus.sherrill@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div>David,<br><br></div>I will address items 1, 2, 3 and leave the others for Matt.<br><br></div>1) We have Python bindings for some classes, but we have not yet had the time to wrap the image detection related classes.<br><br></div>2) One way to test the system is to see if you can run a pipelined application. There are some environment variables that need to be set up before you can run. Most of these are set up by the install/setup_KWIVER.sh script.<br></div>You will need to manually set SPROKIT_MODULE_PATH to point to the VIAME/build/install/lib/sproki<wbr>t directory. I hope to add this to the setup script soon.<br><br></div>A good initial test is to run the VIAME/build/bin/plugin_explore<wbr>r program. It will generate a prodigious number of log messages and then list all the loadable algorithms. The output should look as follows:<br><br><span style="font-family:monospace,monospace">---- Algorithm search path<br>/disk2/projects/NOAA/VIAME/bui<wbr>ld/install/lib/modules:<br><br>---- Registered module names:<br><span style="background-color:rgb(230,145,56)"> kwiver_algo_matlab_plugin</span><br> kwiver_algo_ocv_plugin<br> kwiver_algo_plugin<br> kwiver_algo_vxl_plugin<br><span style="background-color:rgb(230,145,56)"> viame_scallop_tk_plugin</span><br><br>---- registered algorithms (type_name:impl_name)<br> analyze_tracks:ocv<br> bundle_adjust:hierarchical<br> bundle_adjust:vxl<br> close_loops:bad_frames_only<br> close_loops:exhaustive<br> close_loops:keyframe<br> close_loops:multi_method<br> close_loops:vxl_homography_gui<wbr>ded<br> compute_ref_homography:core<br> convert_image:bypass<br> detect_features:ocv_BRISK<br> detect_features:ocv_FAST<br> detect_features:ocv_GFTT<br> detect_features:ocv_MSER<br> detect_features:ocv_ORB<br> detect_features:ocv_SIFT<br> detect_features:ocv_STAR<br> detect_features:ocv_SURF<br> detect_features:ocv_simple_blo<wbr>b<br> draw_tracks:ocv<br> estimate_canonical_transform:c<wbr>ore_pca<br> estimate_canonical_transform:v<wbr>xl_plane<br> estimate_essential_matrix:vxl<br> estimate_fundamental_matrix:oc<wbr>v<br> estimate_fundamental_matrix:vx<wbr>l<br> estimate_homography:ocv<br> estimate_homography:vxl<br> estimate_similarity_transform:<wbr>vxl<br> extract_descriptors:ocv_BRIEF<br> extract_descriptors:ocv_BRISK<br> extract_descriptors:ocv_FREAK<br> extract_descriptors:ocv_ORB<br> extract_descriptors:ocv_SIFT<br> extract_descriptors:ocv_SURF<br> filter_features:magnitude<br><span style="background-color:rgb(246,178,107)"> image_filter:matlab</span><br> image_io:ocv<br> image_io:vxl<br><span style="background-color:rgb(246,178,107)"> image_object_detector:hough_ci<wbr>rcle_detector<br> image_object_detector:matlab<br> image_object_detector:scallop_</span><wbr>tk_detector<br> initialize_cameras_landmarks:c<wbr>ore<br> match_features:fundamental_mat<wbr>rix_guided<br> match_features:homography_guid<wbr>ed<br> match_features:ocv_brute_force<br> match_features:ocv_flann_based<br> match_features:vxl_constrained<br> optimize_cameras:vxl<br> track_features:core<br> triangulate_landmarks:core<br> triangulate_landmarks:vxl<br> video_input:vxl</span><br><br></div><div>These loadable algorithms are located using the KWIVER_PLUGIN_PATH which is listed in the output. The modules loaded list represent the shared objects that have been detected and loaded. Each shared object can contain multiple algorithms. The algorithm list shows each concrete algorithm that could be loaded. Check the log messages to see if there are any libraries that could not be located.<br></div>The highlighted loadable algorithms are the sample detectors and matlab support. You should see these in your output.<br></div>If you don't see the matlab entries, check to see if the Matlab libraries are in your LD_LIBRARY_PATH.<br><br></div>Each algorithm listed consists of two names. The first name is the type of algorithm and the second is the actual implementation type. For example the entry <span style="font-family:monospace,monospace"><span style="background-color:rgb(246,178,107)">image_object_detector:hough_ci<wbr>rcle_detector<br></span></span></div><span style="font-family:monospace,monospace"><span style="background-color:rgb(246,178,107)"><span style="background-color:rgb(243,243,243)"><span></span></span></span></span>indicates that it implements the image_object_detector interface and it is a hough_circle_detector. <br><br></div>Algorithms can be instantiated in any program and use a configuration based approach to select which concrete implementation to instantiate.<br><br></div>The next thing to check is to verify the process loading environment by running VIAME/install/bin/processopedi<wbr>a. This program will search for and load sprokit processes.<br><br></div>The output should appear as follows (omitting the log messages)<br><br><span style="font-family:monospace,monospace">collate: Collates data from multiple worker processes<br>compute_homography: Compute a frame to frame homography based on tracks<br>detect_features: Detect features in an image that will be used for stabilization<br>distribute: Distributes data to multiple worker processes<br>draw_detected_object_boxes: Draw detected object boxes on images.<br>draw_tracks: Draw feature tracks on image<br>extract_descriptors: Extract descriptors from detected features<br>feature_matcher: Match extracted descriptors and detected features<br>frame_list_input: Reads a list of image file names and generates stream of images and associated time stamps<br>image_filter: Apply selected image filter algorithm to incoming images.<br><span style="background-color:rgb(111,168,220)">image_object_detector: Apply selected image object detector algorithm to incoming images.</span><br>image_viewer: Display input image and delay<br>image_writer: Write image to disk.<br>input_adapter: Source process for pipeline. Pushes data items into pipeline ports.<br>kw_archive_writer: Write kw archives<br><span style="background-color:rgb(111,168,220)">matlab_bridge: Bridge to process written in Matlab</span><br>output_adapter: Sink process for pipeline. Accepts data items from pipeline ports.<br>pass: Pass a data stream through<br>read_d_vector: Read vector of doubles<br>sink: Ignores incoming data<br>stabilize_image: Generate current-to-reference image homographies<br>template: Description of process. Make as long as necessary to fully explain what the process does and how to use it. Explain specific algorithms used, etc.<br>view_image: Display input image and delay. Legacy process and will be removed in a future release. Convert to use "image_viewer" before it is too late.</span><br><br><br></div>The highlighted process types should be in your list. The program has other options, so feel free to experiment.<br><br></div>We will be using the <span style="background-color:rgb(111,168,220)"><span style="font-family:monospace,monospace">image_object_detector</span></span> process type in a pipeline to apply a detector to a stream of images. This process wraps the <span style="font-family:monospace,monospace"><span style="background-color:rgb(246,178,107)">image_object_detector</span></span> algorithm interface in a process. The process can be configured to instantiate any available detector implementation.<br><br></div>For a simple pipeline test, go to -<br><div><div><div><div><div><div><br></div><div><span style="font-family:monospace,monospace">$ cd VIAME/source/packages/kwiver/s<wbr>prokit/pipelines/examples/houg<wbr>h_detector</span><br></div><div><br><span style="font-family:monospace,monospace"><span style="background-color:rgb(246,178,107)"></span></span><div><div><div><div><div><div>Unfortunately there are some minor problems that have to be fixed before this example can be run. This patch is in the process of being included in the master version, but for now, apply the following patch.<br><br><span style="font-family:monospace,monospace">diff --git a/sprokit/pipelines/examples/h<wbr>ough_detector/detect_circles_l<wbr>ist.txt b/sprokit/pipelines/examples/h<wbr>ough_detector/detect_circles_l<wbr>ist.txt<br>index 177599c..3a6005e 100644<br>--- a/sprokit/pipelines/examples/h<wbr>ough_detector/detect_circles_l<wbr>ist.txt<br>+++ b/sprokit/pipelines/examples/h<wbr>ough_detector/detect_circles_l<wbr>ist.txt<br>@@ -1 +1 @@<br>-images/<a href="http://detect_circles_soda.jp">detect_circles_soda.jp</a><wbr>g<br>+detect_circles_soda.jpg<br>diff --git a/sprokit/pipelines/examples/h<wbr>ough_detector/hough_detector.p<wbr>ipe b/sprokit/pipelines/examples/h<wbr>ough_detector/hough_detector.p<wbr>ipe<br>index f757e80..58d6d7b 100644<br>--- a/sprokit/pipelines/examples/h<wbr>ough_detector/hough_detector.p<wbr>ipe<br>+++ b/sprokit/pipelines/examples/h<wbr>ough_detector/hough_detector.p<wbr>ipe<br>@@ -1,8 +1,7 @@<br> # ==============================<wbr>==============================<wbr>====<br> process input<br> :: frame_list_input<br>- #:image_list_file images/image_list_1.txt<br>- :image_list_file images/detect_circles_list.txt<br>+ :image_list_file detect_circles_list.txt<br> :frame_time .3333<br> :image_reader:type ocv</span><br> <br></div><div>This patch just removes the directory name from the image file and image list file.<br><br></div><div>In that directory, run the following command (or look at run_pipe.sh)<br><br><span style="font-family:monospace,monospace">$ pipeline_runner -p hough_detector.pipe</span><br><br></div><div>The results should be an image displayed with a box around each can end.<br><br><div>This is a good checkout of the underlying components.<br><br></div><div>Let me know if you have any problems.<span><font color="#888888"><br><br></font></span></div><span><font color="#888888"><div>-Linus<br></div><div><br></div></font></span></div></div></div></div></div></div></div></div></div></div></div></div><div class="gmail_extra"><span><br><div class="gmail_quote">On Tue, Sep 6, 2016 at 4:14 PM, David Zhang <span dir="ltr"><<a href="mailto:david.zhang@sri.com" target="_blank">david.zhang@sri.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Matt and Linus:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">First of all, thanks for all the emails. I had a successful build today based on the latest version. This is good news. Next are the few questions to move on:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>1)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Is the build so far work for both c++ and python?<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>2)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">What test codes can I run to reflect my build?<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>3)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I assume my code will be used as “plugin”, and VIAME has a sub-folder. I have used python as APIs to wrap around a bunch of lower level c++ libraries.
<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>4)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">How do I change caffe settings from cmake? For example, to disable cuDNN, disable GPU, etc.
<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>5)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">We start to build VIAME on a desktop with Nvidia GPU K20. It has Cuda SDK 7.5 and CuDNN 5.05 installed. So far, the build was broken in caffe that
complains about cuDNN type definitions. (I did not build caffe with cuDNN on my laptop). What is the caffe version you are using in the build? Is it from BLVC /caffe? <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">David
<u></u><u></u></span></p>
</div></div></blockquote></div><br></span><span>-- <br><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><font color="#888888"><b>Linus Sherrill - </b></font><font color="#888888">Staff R&D Engineer<br></font><font color="#888888">Kitware, Inc.<br>28 Corporate Drive<br>Clifton Park, NY 12065-8662<br>E: <a href="mailto:linus.sherrill@kitware.com" target="_blank">linus.sherrill@kitware.com</a><br>P: <a href="tel:518.881.4400" value="+15188814400" target="_blank">518.881.4400</a><br></font></div></div></div></div></div>
</span></div></div>
</blockquote></div><br><br clear="all"><br></div></div><span class="">-- <br><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Matthew Dawkins<br><a href="tel:1-518-881-4416" value="+15188814416" target="_blank">1-518-881-4416</a><br>Senior R&D Engineer<br>Kitware, Inc<br></div></div></div></div></div>
</span></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><font color="#888888"><b>Linus Sherrill - </b></font><font color="#888888">Staff R&D Engineer<br></font><font color="#888888">Kitware, Inc.<br>28 Corporate Drive<br>Clifton Park, NY 12065-8662<br>E: <a href="mailto:linus.sherrill@kitware.com" target="_blank">linus.sherrill@kitware.com</a><br>P: 518.881.4400<br></font></div></div></div></div></div>
</div>