[CDash] Deletion of uploaded files

[Zack Galbreath]

Hi Thomas,

Thanks for reaching out. I'll check locally to see if I can reproduce the
issue with ctest_upload() that you're describing.

For paid support options, please see the following link:
https://www.kitware.com/support

On Tue, Aug 9, 2022 at 11:44 AM Thomas Rhiem <T.Rhiem at magmasoft.de> wrote:

> Hello,
>
>
>
> We have some problems submitting results to a CDash dashboard server. We
> assume, that it is caused by a bug in CDash.
>
>
>
> First:
>
> The link to the bug tracker (https://public.kitware.com/Bug/) at the
> CDash FAQ page (
> https://www.paraview.org/Wiki/CDash:FAQ#I_found_a_bug,_what_should_I_do?)
> is broken (404).
>
> It should be updated to the github page.
>
>
>
> the environment:
>
> We had 2 an older CDash  (CDash 3.0.x ) server running and a newer one (CDash
> v3.0.3) migrated from the older one.
>
> Our build is splitted into several build and test stages. For each stage
> we upload the log-file(s)
>
> with ctest_upload().
>
> On the old server we sometimes got http error 500 errors, on the newer
> server we regularly (close to always) get this error for our central build
> dashboard (centralized nightly/daily build), while we see this type of
> failure in our developer dashboard (experimental) in rare cases. Most of
> the time the uploaded file is there (or even 4 times there, cause a retry
> count of  3 is used), but sometimes it is missing.
>
> Exploring the logs, I found, that a file could not be removed:
>
>
>
> 172.18.0.3 - - [09/Aug/2022:14:13:37 +0000] "PUT
> /submit.php?project=CentralBuild&FileName=avdi0de002.corpdir.zz___dev_MAIN_CI_LATEST_FASTLANE-22376%203.%20gen-install___20220808-2300-Nightly___XML___Upload.xml&build=dev_MAIN_CI_LATEST_FASTLANE-22376%203.%20gen-install&site=avdi0de002.corpdir.zz&stamp=20220808-2300-Nightly&MD5=7d9ccdd737a3e06618ad38236dc15bc0
> HTTP/1.1" 500 2673
>
> 172.18.0.3 - - [09/Aug/2022:14:13:41 +0000] "GET
> /ajax/dailyupdatescurl.php?projectid=2 HTTP/1.1" 200 1065
>
> [2022-08-09 16:13:41] development.INFO: Upload quota exceeded, removing
> old files
> {"function":"Project::CullUploadedFiles","project_id":"2","build_id":"67147"}
>
> [2022-08-09 16:13:41] development.ERROR:
> unlink(/home/kitware/cdash/app/cdash/public/upload/ca8a9ec016d2c0d9f47117704b851de592a3a173/bin-install.log):
> No such file or directory {"exception":"[object] (ErrorException(code: 0):
> unlink(/home/kitware/cdash/app/cdash/public/upload/ca8a9ec016d2c0d9f47117704b851de592a3a173/bin-install.log):
> No such file or directory at
> /home/kitware/cdash/app/cdash/include/log.php:30)
>
>
>
> Further exploring the logs,  I found that there are about 15 messages to
> delete this single file (namely
> /home/kitware/cdash/app/cdash/public/upload/ca8a9ec016d2c0d9f47117704b851de592a3a173/bin-install.log).
> I assume that the first try to remove the file is successful, so all
> further tries failed (cause the file is no longer there). But why should
> the same file be deleted so often? Also Note: cause of the RETRY_COUNT
>  value of 3 for the submit the log files are transferred 4 times and each
> file has the same SHA-1 value! (so is the same physical file on disk) (but
> this might only explain 4 tries at most).
>
>
>
> We are interested in solving this issue with your help. Is there a
> possibility to get commercial support for CDash? If so, can you send us
> your support conditions and a quote.
>
>
>
> Regards
>
> T.Rhiem
>
>
>
>
>
>
>
> Thomas Rhiem
> Software Development
>
> *MAGMA Gießereitechnologie GmbH*
>
> P: +49 241 88901 236
> Kackertstrasse 16-18, 52072 Aachen, Germany
> www.magmasoft.de
> T.Rhiem at magmasoft.de
>
> ------------------------------
>  *GERMANY ● USA ● BRAZIL ● SINGAPORE ● SOUTH KOREA ● CHINA ● INDIA ●
> TURKEY ● CZECH REPUBLIC*
>
> International MAGMA User Meeting 2022 - October 19-21 | Holiday Inn Munich
> - City Centre
>
> MAGMA Gießereitechnologie GmbH | Kackertstraße 16-18, 52072 Aachen,
> Germany | Legal form: GmbH, Register court: Aachen HRB 3912, Value added
> tax identification number: DE121745780 | Management: Dr. Marc C. Schneider
> (CEO and President), Dipl.-Ing. Mathieu Weber (Managing Director)
> _______________________________________________
> CDash mailing list
> CDash at public.kitware.com
> https://public.kitware.com/cgi-bin/mailman/listinfo/cdash
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cdash/attachments/20220812/6e46d39f/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: magma_logo_600x600px_2bf73d82-7e11-4b48-882e-15c123d86981.png
Type: image/png
Size: 6445 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/cdash/attachments/20220812/6e46d39f/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IUM2022_6abf80d0-571b-4506-9c2d-7ba42b66a3d0.png
Type: image/png
Size: 84202 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/cdash/attachments/20220812/6e46d39f/attachment-0003.png>