[Cdash] Duplicate build entries with CDash 2.0

Thu Feb 9 02:30:26 UTC 2012

On Wed, Feb 08, 2012 at 05:16:39PM -0500, David Cole wrote:
> On Wed, Feb 8, 2012 at 4:58 PM, Peter Colberg <peter at colberg.org> wrote:
> 
> > Could you pinpoint me to the code sections which collect build entries
> > from database and display them? Do you have any hints on debugging the
> > code?
> 
> 
> It's the unfortunately exceedingly complex sql query starting on line 705
> of the main index.php file (as of svn trunk right now, revision 3149).
> 
> My guess is that there's some subtlety in there where duplicate build ids
> are possible with PgSql, but do not show up with MySQL... There should be a
> uniqueness enforcement on buildid in that query, but I am uncertain how to
> add that constraint such that it works in all cases, (filtered, subproject,
> ...)
> 
> Those results are further processed by php code that follows the query.
> Perhaps it would be easiest to enforce the buildid uniqueness constraint in
> php code instead...

That SQL query is long indeed.

In the meantime I setup a test environment (with PostgresQL as database),
and tested CDash from SVN trunk r2728 (v1.8) up to r3143 (v2.0).

This was done using a git bisect on a repository created with git svn
clone. For each tested revision, I dropped and recreated the cdash
database, reinstalled cdash, and setup two test projects. Then I
submitted two builds with identical build name to each project,
as described in my original mail.

It turns out revision 2935 causes the duplicated build entries:

daa7f7b89f31b248d172b05237e2b6dc1052b4c3 is the first bad commit
commit daa7f7b89f31b248d172b05237e2b6dc1052b4c3
Author: david.cole <david.cole at 91079597-3006-4846-9bd3-4f3fb3d139a7>
Date:   Tue Aug 2 21:43:38 2011 +0000

    ENH: Include updates from related builds (those with the same stamp, siteid and name) not just the one given buildid. When a subproject-based dashboard submits its cycle of subproject builds, all after a single ctest_start call, also typically followed by a single ctest_update call, and then a loop of subproject configure/build/test sequences, we want the updates gathered at the beginning to be reported for each subproject build in non-collapsed viewing mode... This commit achieves that by introducing the get_updates_buildid_clause function, and then using it everywhere the buildupdate and updatefile tables are queried for update related information.

    git-svn-id: https://www.kitware.com/svn/CDash/trunk@2935 91079597-3006-4846-9bd3-4f3fb3d139a7

:040000 040000 cc7764a4961e71b25cb045eb07f12ca8fe1f3a93 d0b829fa8c4b7be81017f39322042208526e5ed6 M      ajax
:100644 100644 59afa2152893486ff9a6090a2af50c11f6cb0016 dac303bae7ef03784a5f5ba20f55b16dbd9fac1c M      buildSummary.php
:040000 040000 20d4f101f8448595c72ff7928018e5e049a096eb e9cb0cbf739375ff42980e9f931f0bd4fb652420 M      cdash
:100644 100644 95c7538ccad9c930ed9a909682f7a645e4b813e9 4968053cbee02be8bfa54242b9166b0e4b903525 M      filterdataFunctions.php
:100644 100644 053e0da8f269b50f0f84046baa02cb741f45576c e993e830eda4665f3c52951cf2b9afeac011d98a M      index.php
:040000 040000 84173c8b0c1de02287ae8b2c1313db67f072f5bb b2957929717e011dcb12bbe72ebf3b689193cdfa M      models
:100644 100644 3c3632a8739d3a8571fd4e2cd752e77781e58c9b 1b60cb83727762ee04e5007ebc60d855256a7c18 M      user.php
:100644 100644 4e0a945d513e5ce4669f14a5ce9ecc03c8abf2cd 7392f95ba24ed4acb5a39152cd2e71278f8be4b1 M      viewUpdate.php

Does this help in fixing the issue? How is a “related build” defined?
Should that not be limited to the current project?

Thanks,
Peter

# git bisect log
# bad: [5dd54a48f7a00c0ea7ccf47df14eca1121a62ba2] ENH: Added change log for CDash 2.0
# good: [93686da01df58c71b2f2b689a46072df13baadc7] ENH: Added change log for 1.8
git bisect start '5dd54a48f7a00c0ea7ccf47df14eca1121a62ba2' '93686da01df58c71b2f2b689a46072df13baadc7'
# bad: [534a273c1f8a835e0ad020dfb99dcf6a572e0caa] ENH: Fixing total test duration
git bisect bad 534a273c1f8a835e0ad020dfb99dcf6a572e0caa
# good: [d18d21cadbf6eda7c466bcc1f3e110bf44fc003d] ENH: Fix uploadfile test
git bisect good d18d21cadbf6eda7c466bcc1f3e110bf44fc003d
# good: [59effa44a8cfad178fcbdf92a576865c04b8eb8f] ENH: Encoding subproject name
git bisect good 59effa44a8cfad178fcbdf92a576865c04b8eb8f
# bad: [04d1d60f6f9e234194506ed2c9e0b589194837d6] ENH: Add a link to the monitor.php page from the server admin's My CDash page.
git bisect bad 04d1d60f6f9e234194506ed2c9e0b589194837d6
# good: [437025dc8cfef6f987d014c475c3c96f580bb4ac] BUG: Remove 'exit' call and replace it with a trigger_error call.
git bisect good 437025dc8cfef6f987d014c475c3c96f580bb4ac
# good: [ab143d4fdbb409b78c8a017da74be4248b1e03c5] Add new test actualtrilinossubmission that submits 151 xml files that came from a real world Trilinos Experimental dashboard run. Re-arrange order of subproject test since it adds some of the same subproject names that Trilinos does, and subsequent tests depend on the subproject ids of the subprojects added in the subproject test. The subproject test must run before the actualtrilinossubmission test.
git bisect good ab143d4fdbb409b78c8a017da74be4248b1e03c5
# good: [f5d2392cd9a309210cdfdfe105cd0ad7d23451a0] Add logging code to importBackup to measure progress via 'tail -f cdash.log'. Also, log an error if CDash cannot import an individual file.
git bisect good f5d2392cd9a309210cdfdfe105cd0ad7d23451a0
# bad: [daa7f7b89f31b248d172b05237e2b6dc1052b4c3] ENH: Include updates from related builds (those with the same stamp, siteid and name) not just the one given buildid. When a subproject-based dashboard submits its cycle of subproject builds, all after a single ctest_start call, also typically followed by a single ctest_update call, and then a loop of subproject configure/build/test sequences, we want the updates gathered at the beginning to be reported for each subproject build in non-collapsed viewing mode... This commit achieves that by introducing the get_updates_buildid_clause function, and then using it everywhere the buildupdate and updatefile tables are queried for update related information.
git bisect bad daa7f7b89f31b248d172b05237e2b6dc1052b4c3
# good: [d122f842aa343c134f0d0e27aab3de78b22e2d67] Correct arguments to add_log calls in processsubmissions.php - the text and function name args were mixed up. Also, refactor the code that processes an individual file into a ProcessFile function so that it can be called from processsubmissions.php and the newly added processfile.php. A developer may use php on the command line with processfile.php (or call it through a web browser) to process a single xml file for diagnosing problems.
git bisect good d122f842aa343c134f0d0e27aab3de78b22e2d67