[vtk-developers] [EXTERNAL] Support for logging

Robert Maynard robert.maynard at kitware.com
Wed Aug 8 08:30:38 EDT 2018


What are the plans for controlling logging when running MPI jobs? Will
each rank have a separate log file? I would think for larger MPI runs
we might want logging disabled by default
On Wed, Aug 8, 2018 at 8:28 AM Utkarsh Ayachit
<utkarsh.ayachit at kitware.com> wrote:
>
> @Andras, as Allie mentioned, the vtkDebugMacro is woefully inadequate.
> I'd recommend scanning the logs generated by loguru. They are
> immensely superior is quality than any VTK-based log dump. While we
> could change all of the message processing to go via the logging
> infrastructure, that'd probably be a separate discussion since it'll
> depend greatly on which logging infrastructure we pick.
>
> @Dave , I understand your reluctance for new dependencies, but "use
> something standard" and no external dependency are inherently
> contradictory :). `loguru` is header only. We could "wrap" it under a
> VTK-specific header and then use loguru internally, only if enabled.
> But since logs should always be generatable to be useful, I don't see
> who would want to build without logging enabled.
>
>
> On Tue, Aug 7, 2018 at 5:55 PM David E DeMarle <dave.demarle at kitware.com> wrote:
> >
> > +1, lending moral support for the idea of better log facilities.
> >
> > +0 for loguru. +1 for yeah by all means lets use something standard
> > and good but -1 for any new non critical dependency
> >
> > Although I too have only rarely ever found vtkDebugMacro to be useful
> > in practice I still think it should feed into the new logs rather than
> > be some independent feature. Same goes for timerlogs - they should
> > interoperate sooner rather than later.
> >
> > David E DeMarle
> > Kitware, Inc.
> > Principal Engineer
> > 21 Corporate Drive
> > Clifton Park, NY 12065-8662
> > Phone: 518-881-4909
> >
> > On Tue, Aug 7, 2018 at 1:25 PM Allie Vacanti
> > <allison.vacanti at kitware.com> wrote:
> > >
> > > +1. The loguru outputs I've seen are very nice.
> > >
> > > I tend to find the vtkObject::Debug logging to be largely useless, as it prints very verbose information and it's usually easier to just add a manual print statement when something like this is needed. Source modification is usually needed to set the Debug flag on, and at that point, why not just dump the exact piece of info we care about, when we care about it? I
> > >
> > > Something that would let us categorize the log entries ('rendering' for mappers/renderers, 'pipeline' for executive info, 'filter' for algorithm details, 'dataset' for dataset changes, 'verbose' for every-little-set-and-get-ever) to more easily filter out the stuff we're not interested in would be fantastic.
> > >
> > > On Tue, Aug 7, 2018 at 1:03 PM, Andras Lasso <lasso at queensu.ca> wrote:
> > >>
> > >> VTK has already a logging API that sends messages to stdout/stderr by default but it can be easily configured to send messages to any logging system. Do you mean you would keep the current logging API and just change the default logging backend to something more sophisticated?
> > >>
> > >> Somewhat related issue: For me, it has never been completely clear how VTK debug logs could be used in practice, as most often it is very difficult to set DebugOn for an object (or group of relevant objects). It would be probably better to enable logging by default for all objects and split current debug level to 3 levels (info/debug/trace) for better control of verbosity.
> > >>
> > >> Andras
> > >> ________________________________
> > >> From: "Scott, W Alan via vtk-developers" <vtk-developers at public.kitware.com>
> > >> Sent: Tuesday, August 7, 2018 6:38 PM
> > >> To: Ayachit, Utkarsh (External Contacts); vtk-developers at vtk.org
> > >> Subject: Re: [vtk-developers] [EXTERNAL] Support for logging
> > >>
> > >> Great idea.  For someone that doesn't know where to place breakpoints, but has to try to debug datasets that cannot be sent to Kitware, this would be incredibly valuable.
> > >>
> > >> Alan
> > >>
> > >> ________________________________________
> > >> From: vtk-developers <vtk-developers-bounces at public.kitware.com> on behalf of Utkarsh Ayachit <utkarsh.ayachit at kitware.com>
> > >> Sent: Tuesday, August 7, 2018 8:29 AM
> > >> To: vtk-developers at vtk.org
> > >> Subject: [EXTERNAL] [vtk-developers] Support for logging
> > >>
> > >> Folks,
> > >>
> > >> Of late I've been finding myself needing some sort of logging support
> > >> when debugging complex issues in ParaView and the more I thought about
> > >> it, the more I realized it should really be in VTK too and hence this
> > >> email.
> > >>
> > >> I wonder what folks thought about adding support for generating logs
> > >> to VTK. This goes beyond the vtkTimerLog we currently have.
> > >> vtkTimerLog is great to log filter execution times etc. but it lacks
> > >> features like levels of verbosity, among others that are needed in a
> > >> general purpose logging infrastructure. We could potentially make
> > >> vtkTImerLog use the chosen logging infrastructure under the covers,
> > >> but that's a separate discussion and not entirely relevant to the
> > >> topic at hand.
> > >>
> > >> To illustrate a case where this is useful in VTK: has anyone ever
> > >> tried to debug why a filter is re-executing? One often ends up in the
> > >> weeds of vtkExecutive and subclass and is fraught with frustration.
> > >> Now, imagine that the vtkExecutive subclass just generated a log
> > >> indicate which pipeline pass it's executing, and why -- things would
> > >> be so easy! The log could be added only at higher levels of verbosity,
> > >> thus not pollute application logs with internal details, but would be
> > >> easily generate-able/accessible when needed. Same is true for
> > >> rendering pipelines, determining if the VBOs, for example, are being
> > >> regenerated and why can be easily logged and aid tracking down
> > >> performance issues.
> > >>
> > >> If we are convinced of the need for a logging support, then the
> > >> question becomes which logging library to use? I have been using for
> > >> loguru[1] with great success for a multi-threaded app. I love it's
> > >> output generation style, plus how well it handles logs from multiple
> > >> threads. It's C++ friendly, header only, minimal external
> > >> dependencies.
> > >>
> > >> Thoughts? Suggestions?
> > >>
> > >> Utkarsh
> > >>
> > >>
> > >> [1] https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Femilk%2Floguru&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=CiukFppwivi35wrjPasMsrHDDl6qYr4wEhjsnQDWlww%3D&reserved=0
> > >> _______________________________________________
> > >> Powered by https://na01.safelinks.protection.outlook.com/?url=www.kitware.com&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=2knVTY4AKF5un%2BtsHBC6d9IwxdQU3fcg8k2B0F4y6eE%3D&reserved=0
> > >>
> > >> Visit other Kitware open-source projects at https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kitware.com%2Fopensource%2Fopensource.html&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=FGooxIKSrpR%2F9M%2BqZSmIbUqWzgTWcmQdCIrCzGac0JI%3D&reserved=0
> > >>
> > >> Search the list archives at: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail.org%2Fsearch%2F%3Fq%3Dvtk-developers&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=5yBu79VulvgbW7TSIX5NUsCbR5qUNTliI0U5Ax3yZ%2Bs%3D&reserved=0
> > >>
> > >> Follow this link to subscribe/unsubscribe:
> > >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpublic.kitware.com%2Fmailman%2Flistinfo%2Fvtk-developers&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=VOEka8wGfLzP3j5hAQUVLPFV9XS1CMolWwPCpzSgj3A%3D&reserved=0
> > >>
> > >> _______________________________________________
> > >> Powered by https://na01.safelinks.protection.outlook.com/?url=www.kitware.com&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=2knVTY4AKF5un%2BtsHBC6d9IwxdQU3fcg8k2B0F4y6eE%3D&reserved=0
> > >>
> > >> Visit other Kitware open-source projects at https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kitware.com%2Fopensource%2Fopensource.html&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=FGooxIKSrpR%2F9M%2BqZSmIbUqWzgTWcmQdCIrCzGac0JI%3D&reserved=0
> > >>
> > >> Search the list archives at: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail.org%2Fsearch%2F%3Fq%3Dvtk-developers&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=5yBu79VulvgbW7TSIX5NUsCbR5qUNTliI0U5Ax3yZ%2Bs%3D&reserved=0
> > >>
> > >> Follow this link to subscribe/unsubscribe:
> > >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpublic.kitware.com%2Fmailman%2Flistinfo%2Fvtk-developers&data=02%7C01%7Classo%40queensu.ca%7C0d8cae5a8ec54917f3b408d5fc842b32%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636692567166365790&sdata=VOEka8wGfLzP3j5hAQUVLPFV9XS1CMolWwPCpzSgj3A%3D&reserved=0
> > >>
> > >>
> > >> _______________________________________________
> > >> Powered by www.kitware.com
> > >>
> > >> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
> > >>
> > >> Search the list archives at: http://markmail.org/search/?q=vtk-developers
> > >>
> > >> Follow this link to subscribe/unsubscribe:
> > >> https://public.kitware.com/mailman/listinfo/vtk-developers
> > >>
> > >>
> > >
> > > _______________________________________________
> > > Powered by www.kitware.com
> > >
> > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
> > >
> > > Search the list archives at: http://markmail.org/search/?q=vtk-developers
> > >
> > > Follow this link to subscribe/unsubscribe:
> > > https://public.kitware.com/mailman/listinfo/vtk-developers
> > >
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
> >
> > Search the list archives at: http://markmail.org/search/?q=vtk-developers
> >
> > Follow this link to subscribe/unsubscribe:
> > https://public.kitware.com/mailman/listinfo/vtk-developers
> >
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://public.kitware.com/mailman/listinfo/vtk-developers
>


More information about the vtk-developers mailing list