[Cdash] Duplicate build entries with CDash 2.0

Thu Feb 9 14:00:02 UTC 2012

On Wed, Feb 8, 2012 at 9:30 PM, Peter Colberg <peter at colberg.org> wrote:

> On Wed, Feb 08, 2012 at 05:16:39PM -0500, David Cole wrote:
> > On Wed, Feb 8, 2012 at 4:58 PM, Peter Colberg <peter at colberg.org> wrote:
> >
> > > Could you pinpoint me to the code sections which collect build entries
> > > from database and display them? Do you have any hints on debugging the
> > > code?
> >
> >
> > It's the unfortunately exceedingly complex sql query starting on line 705
> > of the main index.php file (as of svn trunk right now, revision 3149).
> >
> > My guess is that there's some subtlety in there where duplicate build ids
> > are possible with PgSql, but do not show up with MySQL... There should
> be a
> > uniqueness enforcement on buildid in that query, but I am uncertain how
> to
> > add that constraint such that it works in all cases, (filtered,
> subproject,
> > ...)
> >
> > Those results are further processed by php code that follows the query.
> > Perhaps it would be easiest to enforce the buildid uniqueness constraint
> in
> > php code instead...
>
> That SQL query is long indeed.
>
> In the meantime I setup a test environment (with PostgresQL as database),
> and tested CDash from SVN trunk r2728 (v1.8) up to r3143 (v2.0).
>
> This was done using a git bisect on a repository created with git svn
> clone. For each tested revision, I dropped and recreated the cdash
> database, reinstalled cdash, and setup two test projects. Then I
> submitted two builds with identical build name to each project,
> as described in my original mail.
>
> It turns out revision 2935 causes the duplicated build entries:
>
> daa7f7b89f31b248d172b05237e2b6dc1052b4c3 is the first bad commit
> commit daa7f7b89f31b248d172b05237e2b6dc1052b4c3
> Author: david.cole <david.cole at 91079597-3006-4846-9bd3-4f3fb3d139a7>
> Date:   Tue Aug 2 21:43:38 2011 +0000
>
>    ENH: Include updates from related builds (those with the same stamp,
> siteid and name) not just the one given buildid. When a subproject-based
> dashboard submits its cycle of subproject builds, all after a single
> ctest_start call, also typically followed by a single ctest_update call,
> and then a loop of subproject configure/build/test sequences, we want the
> updates gathered at the beginning to be reported for each subproject build
> in non-collapsed viewing mode... This commit achieves that by introducing
> the get_updates_buildid_clause function, and then using it everywhere the
> buildupdate and updatefile tables are queried for update related
> information.
>
>    git-svn-id: https://www.kitware.com/svn/CDash/trunk@293591079597-3006-4846-9bd3-4f3fb3d139a7
>
> :040000 040000 cc7764a4961e71b25cb045eb07f12ca8fe1f3a93
> d0b829fa8c4b7be81017f39322042208526e5ed6 M      ajax
> :100644 100644 59afa2152893486ff9a6090a2af50c11f6cb0016
> dac303bae7ef03784a5f5ba20f55b16dbd9fac1c M      buildSummary.php
> :040000 040000 20d4f101f8448595c72ff7928018e5e049a096eb
> e9cb0cbf739375ff42980e9f931f0bd4fb652420 M      cdash
> :100644 100644 95c7538ccad9c930ed9a909682f7a645e4b813e9
> 4968053cbee02be8bfa54242b9166b0e4b903525 M      filterdataFunctions.php
> :100644 100644 053e0da8f269b50f0f84046baa02cb741f45576c
> e993e830eda4665f3c52951cf2b9afeac011d98a M      index.php
> :040000 040000 84173c8b0c1de02287ae8b2c1313db67f072f5bb
> b2957929717e011dcb12bbe72ebf3b689193cdfa M      models
> :100644 100644 3c3632a8739d3a8571fd4e2cd752e77781e58c9b
> 1b60cb83727762ee04e5007ebc60d855256a7c18 M      user.php
> :100644 100644 4e0a945d513e5ce4669f14a5ce9ecc03c8abf2cd
> 7392f95ba24ed4acb5a39152cd2e71278f8be4b1 M      viewUpdate.php
>
>
> Does this help in fixing the issue? How is a “related build” defined?
> Should that not be limited to the current project?
>
> Thanks,
> Peter
>
> # git bisect log
> # bad: [5dd54a48f7a00c0ea7ccf47df14eca1121a62ba2] ENH: Added change log
> for CDash 2.0
> # good: [93686da01df58c71b2f2b689a46072df13baadc7] ENH: Added change log
> for 1.8
> git bisect start '5dd54a48f7a00c0ea7ccf47df14eca1121a62ba2'
> '93686da01df58c71b2f2b689a46072df13baadc7'
> # bad: [534a273c1f8a835e0ad020dfb99dcf6a572e0caa] ENH: Fixing total test
> duration
> git bisect bad 534a273c1f8a835e0ad020dfb99dcf6a572e0caa
> # good: [d18d21cadbf6eda7c466bcc1f3e110bf44fc003d] ENH: Fix uploadfile test
> git bisect good d18d21cadbf6eda7c466bcc1f3e110bf44fc003d
> # good: [59effa44a8cfad178fcbdf92a576865c04b8eb8f] ENH: Encoding
> subproject name
> git bisect good 59effa44a8cfad178fcbdf92a576865c04b8eb8f
> # bad: [04d1d60f6f9e234194506ed2c9e0b589194837d6] ENH: Add a link to the
> monitor.php page from the server admin's My CDash page.
> git bisect bad 04d1d60f6f9e234194506ed2c9e0b589194837d6
> # good: [437025dc8cfef6f987d014c475c3c96f580bb4ac] BUG: Remove 'exit' call
> and replace it with a trigger_error call.
> git bisect good 437025dc8cfef6f987d014c475c3c96f580bb4ac
> # good: [ab143d4fdbb409b78c8a017da74be4248b1e03c5] Add new test
> actualtrilinossubmission that submits 151 xml files that came from a real
> world Trilinos Experimental dashboard run. Re-arrange order of subproject
> test since it adds some of the same subproject names that Trilinos does,
> and subsequent tests depend on the subproject ids of the subprojects added
> in the subproject test. The subproject test must run before the
> actualtrilinossubmission test.
> git bisect good ab143d4fdbb409b78c8a017da74be4248b1e03c5
> # good: [f5d2392cd9a309210cdfdfe105cd0ad7d23451a0] Add logging code to
> importBackup to measure progress via 'tail -f cdash.log'. Also, log an
> error if CDash cannot import an individual file.
> git bisect good f5d2392cd9a309210cdfdfe105cd0ad7d23451a0
> # bad: [daa7f7b89f31b248d172b05237e2b6dc1052b4c3] ENH: Include updates
> from related builds (those with the same stamp, siteid and name) not just
> the one given buildid. When a subproject-based dashboard submits its cycle
> of subproject builds, all after a single ctest_start call, also typically
> followed by a single ctest_update call, and then a loop of subproject
> configure/build/test sequences, we want the updates gathered at the
> beginning to be reported for each subproject build in non-collapsed viewing
> mode... This commit achieves that by introducing the
> get_updates_buildid_clause function, and then using it everywhere the
> buildupdate and updatefile tables are queried for update related
> information.
> git bisect bad daa7f7b89f31b248d172b05237e2b6dc1052b4c3
> # good: [d122f842aa343c134f0d0e27aab3de78b22e2d67] Correct arguments to
> add_log calls in processsubmissions.php - the text and function name args
> were mixed up. Also, refactor the code that processes an individual file
> into a ProcessFile function so that it can be called from
> processsubmissions.php and the newly added processfile.php. A developer may
> use php on the command line with processfile.php (or call it through a web
> browser) to process a single xml file for diagnosing problems.
> git bisect good d122f842aa343c134f0d0e27aab3de78b22e2d67
>

Ugh. Looks like it started with one of my commits for better subproject
support.

Related builds are those with the same stamp, site and name, and yes, they
should all be associated with the same project.

You have duplicate rows appearing in your results, right? (Two buildid 1's
on the first project's page, and two buildid 2's on the second project's
page) -- only build id's associated with project 1 actually appear on
project 1's page. Or am I misunderstanding?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cdash/attachments/20120209/28a91f33/attachment-0003.htm>