[CDash] Duplicate rows in build2test

[Zack Galbreath]

On Wed, May 11, 2016 at 7:46 AM, Matějů Miroslav, Ing. <
Mateju.Miroslav at azd.cz> wrote:

> Hi,
>
> I have discovered duplicate rows in the build2test table which is the
> biggest table in my CDash database. The query
>     SELECT `buildid`, `testid`, COUNT(`buildid`) c FROM `build2test` GROUP
> BY `buildid`, `testid` HAVING c > 1 ORDER BY c DESC;
> returned many entries with the highest reported count of 11. Shouldn't the
> pair buildid + testid be unique? I tried to check data of a few reported
> entries and all the rows appeared to have same values in all columns.
>

I think you're right that a unique constraint would be appropriate here.
I'll run some experiments to see what the resulting Test.xml file looks
like when using ctest's --repeat-until-fail command-line option.  That's
the only potential case I can think of where it might make sense for a
single build to contain multiple copies of the same test(s).



> The duplicates may come from a wrongly configured submission processing:
> The default CDASH_SUBMISSION_PROCESSING_TIME_LIMIT was too low and CDash
> tried to asynchronously process big submissions again and again. However,
> the duplication could be prevented most likely if the table had a PRIMARY
> KEY (or UNIQUE constraint) defined. Currently it has none.
>

You're probably right.  If the same Test.xml gets processed more than once,
then you'll likely end up with duplicate rows in build2test (as you are
experiencing).

Another way this could occur is if you're running the same Nightly build
multiple times per day.  CDash used to automatically delete a nightly build
when a new one with the same name and timestamp was received.

This was a headache for parallel submission processing, so instead you now
have to mark such a build as 'done' for it to automatically be removed when
a new copy is submitted.



> Could you recommend any actions I could perform? Is the best to let
> Autoremove take care of it (which is currently disabled in my case)?
>

Auto-remove will certainly help keep your database from growing without
bounds.  If you're feeling adventurous, you could experiment with a unique
constraint on the build2test table.  Otherwise I'll try to look into this
as time permits.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cdash/attachments/20160512/7c2dbaaa/attachment-0001.htm>