[CDash] Fwd: filtering and comparing test results on a CDash server

[John Verdicchio]

All
This seems to be the closest thing to a forum for CDash. My team are
investigating migrating our bash based test system to CMake/CTest and
CDash. As a proof of concept we currently have all our tests running under
CMake/CTest and the statistics are gathered and sent to a CDash server. The
server was set up by a member of our DevOps team (they knew nothing of
CDash).

 so here is my problem:
[image: cdash.png]
For each test we run (via ctest) we are sending statistical data about our
program via the <DartMeasurement></DartMeasurement> capability. As you can
see from the attached picture the statistics we gather is a mix of float
and integer data.

Question 1:
Is it possible OOTB to filter this list of statistical data? e.g. only show
the _count_ datasets? This screenshot is a subset of the complete list
around 160 entries. This list is most likely going to grow as we add
additional statistical elements from other parts of the code. If it isn't
possible OOTB, how much coding effort would be involved?

As we change and test our program the statistics for any given test may
change e.g. SurfaceRemesher_cpu_cadProjectionTimer_total_ may change from
21.027 (seconds) to 19.01 seconds.

What we would like to do is be able to select two sets of data from the
same test and "compare" the statistical data. In my example, be able to
difference the SurfaceRemesher_cpu_cadProjectionTimer_total_ = 21.027 -
19.01 = 2.017. Ideally, these comparisons could be more complex that
add/subtract, but relatively arbitary expressions involving +,-,/,*,min,max
and numbers

Question  2:
We could just download the data from the server and do the difference
locally via a python script (this script does exist). However, I'm
wondering if this is possible within the CDash server environment. If not
possible OOTB, how much coding effort would be involved? Could this be
achieved with some .php programming on the CDash side? I can see this as
several problems 1) Presenting and Selecting the two sets of results from
any given test 2) Coding a comparison engine within the CDash server that
takes the two results and compares them 3) presenting the comparison back
to the user.

My job is mesh generation using C++ so server-side programing in a LAMP
stack is very new to me.

What I really want to know is how possible this all is ; how easy it is and
even how much it might cost. By having some idea of the time/effort/cost
this proof-of-concept can be discussed with our management.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cdash/attachments/20210107/5a911f4f/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cdash.png
Type: image/png
Size: 191062 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/cdash/attachments/20210107/5a911f4f/attachment-0002.png>