[Cdash] Asynchronous submissions not processed anymore

Tue Jan 7 10:51:59 UTC 2014

Hi Martin,

Did something change in the current configuration of the server (an 
Apache update?). Otherwise can you check that the disk is not full?

Just ideas,
Julien

On 07/01/2014 11:26, Martin Apel wrote:
> Hi all,
>
> we have a CDash server 2.0.2 running for years now, but shortly before
> Christmas it stopped processing
> its asynchronous submissions. The processing can be triggered by hand by
> visiting the URL .../cdash/processsubmissions.php?projectid=1,
> but this used to happen automatically. I have no idea, why this doesn't
> happen automatically anymore, the log file does not reveal
> anything useful.
>
> The monitoring page outputs the relevant statistics:
>
>
>       Currently Processing Submissions as of 2014-01-07 10:18:13 UTC
>
> Nothing is currently processing...
>
>
>       Pending Submissions
>
> project.name, project.id, count of pending queued submissions
>
> SIMPACK NA, 1, 4
>
>
>       Average Wait Times
>
> projectid 1 wait time
> hours_ago, time_local, num_files, avg_hours_delay, mean, shortest, longest
>      0, 5:00 AM, 11, 0.2, 1.5455, 1, 3
>      1, 5:00 AM, 8, 0.1, 1.5000, 1, 3
>      2, 4:00 AM, 6, 0.8, 2.0000, 1, 4
>      3, 2:00 AM, 3, 2.1, 15.3333, 1, 44
>      4, 1:00 AM, 3, 3.6, 1.0000, 1, 1
>      13, 4:00 PM, 3, 12.5, 1.3333, 1, 2
>      14, 4:00 PM, 3, 12.6, 15.0000, 1, 43
>      27, 2:00 AM, 5, 26.3, 8.6000, 1, 38
>      30, 11:00 PM, 3, 29.5, 13.0000, 1, 37
>      38, 4:00 PM, 3, 36.6, 14.6667, 1, 41
>      46, 7:00 AM, 3, 45.2, 13.3333, 1, 38
>      50, 4:00 AM, 3, 48.8, 2.0000, 1, 4
>      51, 3:00 AM, 3, 49.7, 13.3333, 1, 38
>      53, 12:00 AM, 3, 52.4, 18.3333, 1, 53
>      74, 3:00 AM, 3, 73.4, 1.6667, 1, 3
>      75, 2:00 AM, 3, 74.0, 13.3333, 1, 38
>      78, 11:00 PM, 3, 77.1, 13.0000, 1, 37
>      86, 4:00 PM, 3, 84.5, 15.0000, 1, 43
>      98, 3:00 AM, 3, 97.4, 2.0000, 1, 3
>      99, 2:00 AM, 3, 97.9, 13.3333, 1, 38
>      102, 11:00 PM, 3, 101.2, 13.0000, 1, 37
>      110, 4:00 PM, 3, 108.5, 15.6667, 1, 45
>      122, 3:00 AM, 3, 121.4, 1.6667, 1, 3
>      123, 2:00 AM, 3, 122.0, 13.3333, 1, 37
>      126, 11:00 PM, 3, 125.3, 12.3333, 1, 35
>      134, 4:00 PM, 3, 132.4, 15.6667, 1, 45
>      146, 3:00 AM, 3, 145.3, 1.6667, 1, 3
>      147, 2:00 AM, 3, 145.9, 13.0000, 1, 37
>      150, 11:00 PM, 3, 149.2, 13.0000, 1, 37
>      158, 4:00 PM, 3, 156.4, 15.3333, 1, 44
>      170, 3:00 AM, 3, 169.3, 1.6667, 1, 3
>      171, 2:00 AM, 3, 169.9, 13.6667, 1, 38
>      174, 11:00 PM, 3, 173.2, 13.3333, 1, 38
>      182, 4:00 PM, 3, 180.4, 16.3333, 1, 47
>      194, 3:00 AM, 3, 193.2, 2.0000, 1, 4
>      195, 2:00 AM, 3, 193.9, 13.0000, 1, 37
>      198, 11:00 PM, 3, 197.0, 12.6667, 1, 36
>      206, 4:00 PM, 3, 204.3, 15.0000, 1, 42
>      218, 4:00 AM, 3, 216.3, 1.6667, 1, 3
>      219, 3:00 AM, 3, 217.3, 13.6667, 1, 39
>      221, 12:00 AM, 3, 219.7, 18.6667, 1, 54
>      242, 3:00 AM, 3, 241.2, 2.0000, 1, 4
>      243, 2:00 AM, 3, 241.8, 13.0000, 1, 37
>      246, 11:00 PM, 3, 245.0, 12.6667, 1, 36
>      254, 4:00 PM, 3, 252.3, 15.0000, 1, 43
>      266, 3:00 AM, 3, 265.1, 1.6667, 1, 3
>      267, 2:00 AM, 3, 265.7, 14.0000, 1, 40
>      270, 11:00 PM, 3, 269.1, 13.6667, 1, 39
>
>
>       Table `submissionprocessor` (one row per project)
>
> project.name, projectid, pid, lastupdated, locked
>
> SIMPACK NA, 1, 0, 2014-01-07 10:09:04, 1980-01-01 00:00:00
>
>
>       Table `submission` (most recently queued 25)
>
> id, filename, projectid, status, attempts, filesize, filemd5sum, lastupdated, created, started, finished
>
> 226639, /var/www/CDash-2-0-2/backup/502859723.xml, 1, 0, 0, 3528, 54e89efcd9908c4db29d64a912a2c2ff, 1980-01-01 00:00:00, 2014-01-07 10:14:27, 1980-01-01 00:00:00, 1980-01-01 00:00:00
> 226638, /var/www/CDash-2-0-2/backup/1420765816.xml, 1, 0, 0, 5192294, 4a6794bd2953a2cc255ddb1fa1dcdf40, 1980-01-01 00:00:00, 2014-01-07 10:14:26, 1980-01-01 00:00:00, 1980-01-01 00:00:00
> 226637, /var/www/CDash-2-0-2/backup/555091309.xml, 1, 0, 0, 1699, 7abedb2437a6c6b963ece3f977304cd9, 1980-01-01 00:00:00, 2014-01-07 10:14:25, 1980-01-01 00:00:00, 1980-01-01 00:00:00
> 226636, /var/www/CDash-2-0-2/backup/1075704639.xml, 1, 0, 0, 1063, 1d45170765e7380653bcc3f8f9a38846, 1980-01-01 00:00:00, 2014-01-07 10:14:24, 1980-01-01 00:00:00, 1980-01-01 00:00:00
> 226635, /var/www/CDash-2-0-2/backup/1722771811.xml, 1, 2, 1, 1584, 3f1768c96932eb14c69404d69f0d55d0, 2014-01-07 10:09:04, 2014-01-07 09:54:51, 2014-01-07 10:09:03, 2014-01-07 10:09:04
> 226634, /var/www/CDash-2-0-2/backup/758486451.xml, 1, 2, 1, 5162754, feb09272338e599884934f0937921084, 2014-01-07 10:09:03, 2014-01-07 09:54:50, 2014-01-07 10:09:00, 2014-01-07 10:09:03
> 226633, /var/www/CDash-2-0-2/backup/242524917.xml, 1, 2, 1, 1643, df12695150ba0a9b6326069eca3617a9, 2014-01-07 10:09:00, 2014-01-07 09:54:49, 2014-01-07 10:08:59, 2014-01-07 10:09:00
> 226632, /var/www/CDash-2-0-2/backup/328287145.xml, 1, 2, 1, 1000, 9e46d671ef9b1c17658f1235e14d9203, 2014-01-07 10:08:59, 2014-01-07 09:54:48, 2014-01-07 10:08:58, 2014-01-07 10:08:59
> 226631, /var/www/CDash-2-0-2/backup/1413398585.xml, 1, 2, 1, 1584, 0a2471cf0471b5a9f0c9f9fafe4c5e6a, 2014-01-07 10:08:58, 2014-01-07 09:53:53, 2014-01-07 10:08:57, 2014-01-07 10:08:58
> 226630, /var/www/CDash-2-0-2/backup/1643613376.xml, 1, 2, 1, 5165054, 701d41e1b8a2e0756b1a30138c86f88c, 2014-01-07 10:08:57, 2014-01-07 09:53:52, 2014-01-07 10:08:54, 2014-01-07 10:08:57
> 226629, /var/www/CDash-2-0-2/backup/1580744380.xml, 1, 2, 1, 1643, afd6ecbf0bca78d9700f7052f514bb6e, 2014-01-07 10:08:54, 2014-01-07 09:53:51, 2014-01-07 10:08:53, 2014-01-07 10:08:54
> 226628, /var/www/CDash-2-0-2/backup/516354354.xml, 1, 2, 1, 1000, f2f3ce4da201305b25304acb45281c73, 2014-01-07 10:08:53, 2014-01-07 09:53:50, 2014-01-07 10:08:52, 2014-01-07 10:08:53
> 226627, /var/www/CDash-2-0-2/backup/347351203.xml, 1, 2, 1, 5190173, b96b79efcb356f7cb699ff382542aaa8, 2014-01-07 09:52:50, 2014-01-07 09:48:11, 2014-01-07 09:52:47, 2014-01-07 09:52:50
> 226626, /var/www/CDash-2-0-2/backup/196135980.xml, 1, 2, 1, 7980, b9d8fb37323efdd946042f7b9ea5d645, 2014-01-07 09:52:47, 2014-01-07 09:48:10, 2014-01-07 09:52:46, 2014-01-07 09:52:47
> 226625, /var/www/CDash-2-0-2/backup/540120385.xml, 1, 2, 1, 1064, 856656e9f9f09a1ea80ad2265b692823, 2014-01-07 09:52:46, 2014-01-07 09:48:09, 2014-01-07 09:52:45, 2014-01-07 09:52:46
> 226624, /var/www/CDash-2-0-2/backup/822820666.xml, 1, 2, 1, 2495, 19107c45a119eb4b32cad6f3f8819908, 2014-01-07 09:14:30, 2014-01-07 09:10:30, 2014-01-07 09:14:29, 2014-01-07 09:14:30
> 226623, /var/www/CDash-2-0-2/backup/1436949172.xml, 1, 2, 1, 5161118, d809e82fb32f1f6e1b88b19821b5e5f0, 2014-01-07 09:14:29, 2014-01-07 09:10:29, 2014-01-07 09:14:26, 2014-01-07 09:14:29
> 226622, /var/www/CDash-2-0-2/backup/868711020.xml, 1, 2, 1, 1643, 85d1592d4082faf2963727e6b4ed2bc2, 2014-01-07 09:14:26, 2014-01-07 09:10:28, 2014-01-07 09:14:25, 2014-01-07 09:14:26
> 226621, /var/www/CDash-2-0-2/backup/1531771759.xml, 1, 2, 1, 1000, 096731f87d4d4f826a199410c0f280f8, 2014-01-07 09:14:25, 2014-01-07 09:10:27, 2014-01-07 09:14:24, 2014-01-07 09:14:25
> 226620, /var/www/CDash-2-0-2/backup/1002205360.xml, 1, 2, 1, 2495, 3d9698df2114a13199e2947cf2b91366, 2014-01-07 09:14:24, 2014-01-07 09:07:10, 2014-01-07 09:14:23, 2014-01-07 09:14:24
> 226619, /var/www/CDash-2-0-2/backup/1082455045.xml, 1, 2, 1, 5164845, c96cfcc43fb580ca7c099117c370d57f, 2014-01-07 09:14:23, 2014-01-07 09:07:09, 2014-01-07 09:14:20, 2014-01-07 09:14:23
> 226618, /var/www/CDash-2-0-2/backup/1147021750.xml, 1, 2, 1, 1643, 371946b5a5e948744d64966d8da5ef69, 2014-01-07 09:14:20, 2014-01-07 09:07:08, 2014-01-07 09:14:19, 2014-01-07 09:14:20
> 226617, /var/www/CDash-2-0-2/backup/1238403471.xml, 1, 2, 1, 1000, 522474f1cbfc4259b29d090f96bc84e1, 2014-01-07 09:14:19, 2014-01-07 09:07:07, 2014-01-07 09:14:18, 2014-01-07 09:14:19
> 226616, /var/www/CDash-2-0-2/backup/620749966.xml, 1, 2, 1, 5163405, 1dae92592f807e250f0dc9fa5a7ec793, 2014-01-07 08:54:50, 2014-01-07 08:08:30, 2014-01-07 08:54:46, 2014-01-07 08:54:50
> 226615, /var/www/CDash-2-0-2/backup/849392602.xml, 1, 2, 1, 8002, 960e2e3f8fd7a79144c316c8b7e2d897, 2014-01-07 08:54:46, 2014-01-07 08:08:29, 2014-01-07 08:54:45, 2014-01-07 08:54:46
>
> Any ideas what goes wrong here would be very helpful.
>
> Regards,
>
> Martin
>
>
>
>
> _______________________________________________
> Cdash mailing list
> Cdash at public.kitware.com
> http://public.kitware.com/cgi-bin/mailman/listinfo/cdash
>