From zack.galbreath at kitware.com Mon Jan 2 11:01:04 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 2 Jan 2017 11:01:04 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen wrote: > The calendar link no longer works > Looking into this. Thanks for the heads up. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Fri Jan 6 13:36:29 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 6 Jan 2017 13:36:29 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: I just deployed the fix for this issue to open.cdash.org. On Mon, Jan 2, 2017 at 11:01 AM, Zack Galbreath wrote: > On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen > wrote: > >> The calendar link no longer works >> > > Looking into this. Thanks for the heads up. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bill.lorensen at gmail.com Fri Jan 6 19:25:37 2017 From: bill.lorensen at gmail.com (Bill Lorensen) Date: Fri, 6 Jan 2017 19:25:37 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: Wirks again. Thanks! On Jan 6, 2017 1:36 PM, "Zack Galbreath" wrote: > I just deployed the fix for this issue to open.cdash.org. > > On Mon, Jan 2, 2017 at 11:01 AM, Zack Galbreath < > zack.galbreath at kitware.com> wrote: > >> On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen >> wrote: >> >>> The calendar link no longer works >>> >> >> Looking into this. Thanks for the heads up. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From julien.jomier at kitware.com Mon Jan 16 03:33:02 2017 From: julien.jomier at kitware.com (Julien Jomier) Date: Mon, 16 Jan 2017 09:33:02 +0100 Subject: [CDash] ANN: CMake Course - March 13 in Lyon, France Message-ID: Kitware will be holding a CMake training course on March 13, 2017 at Kitware's office in Lyon, France. This one-day course will cover CMake, CTest, CPack and CDash. Visit our website for more information and registration details (early registration and student discounts available): http://training.kitware.fr/browse/129 Note that the course will be taught in English. If you have any questions, please contact me directly or training at kitware.fr. We are looking forward to seeing you in Lyon, Julien From sean at rogue-research.com Mon Jan 16 12:18:46 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 12:18:46 -0500 Subject: [CDash] reducing cdash database size Message-ID: <20170116171846.2088731986@mail.rogue-research.com> Hi all, My cdash database is getting quite big (43 GB)... presumably because of old submissions staying around forever? I tried 'Maintenance > Cleanup Database', but that didn't save anything. In the project settings my 'AutoRemove Timeframe' is 60 days. What can I do to prune things? Thanks, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From zack.galbreath at kitware.com Mon Jan 16 12:36:59 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 16 Jan 2017 12:36:59 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116171846.2088731986@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: On Mon, Jan 16, 2017 at 12:18 PM, Sean McBride wrote: > Hi all, > > My cdash database is getting quite big (43 GB)... presumably because of > old submissions staying around forever? I tried 'Maintenance > Cleanup > Database', but that didn't save anything. In the project settings my > 'AutoRemove Timeframe' is 60 days. > > What can I do to prune things? > Hi Sean, Do you know if old builds are being deleted or not? You can check with the following SQL query: select starttime from build order by starttime limit 1; If not, you can run this deletion process manually from a shell on your server. cd /path/to/CDash php scripts/autoRemoveBuilds.php (this will probably take a long time to run if you have old builds to prune) The removal of old builds should be happening automatically when the first build of the day is submitted. CDash uses cURL to connect to itself and start this process. The relevant code is here: https://github.com/Kitware/CDash/blob/master/include/do_submit.php#L115-L120 One possible solution is setting $CDASH_BASE_URL in your config.local.php. HTH! -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at rogue-research.com Mon Jan 16 15:42:20 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 15:42:20 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: <20170116204220.1519428266@mail.rogue-research.com> On Mon, 16 Jan 2017 12:36:59 -0500, Zack Galbreath said: >Do you know if old builds are being deleted or not? I do not know. >You can check with the >following SQL query: > >select starttime from build order by starttime limit 1; It gives: mysql> select starttime from build order by starttime limit 1; +---------------------+ | starttime | +---------------------+ | 2014-11-21 22:06:47 | +---------------------+ 1 row in set (0.00 sec) So I guess there are indeed old builds in there...? >If not, you can run this deletion process manually from a shell on your >server. > >cd /path/to/CDash >php scripts/autoRemoveBuilds.php >(this will probably take a long time to run if you have old builds to prune) Just done that, for most projects it went like: $ php autoRemoveBuilds.php Foo removing builds for Foo -- removing old buildids for projectid: 8 -- removing old buildids for projectid: 8 But for one: $ php autoRemoveBuilds.php Brainsight removing builds for Brainsight -- removing old buildids for projectid: 1 PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 102320960 bytes) in /Library/Server/Web/Data/Sites/Default/cdash/cdash/pdocore.php on line 289 And my mysql data folder is still 43 GB and your 'select' command still has that 2014 date. :( The machine has 48 GB RAM, can I hack something to allow that 100 MB allocation to proceed? Cheers, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From zack.galbreath at kitware.com Mon Jan 16 15:52:55 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 16 Jan 2017 15:52:55 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116204220.1519428266@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> Message-ID: On Mon, Jan 16, 2017 at 3:42 PM, Sean McBride wrote: > can I hack something to allow that 100 MB allocation to proceed? To increase the amount of memory allowed for old build removal, add the following line to the top of CDash/include/autoremove.php: ini_set('memory_limit','1024M'); To make this a more general, site-wide change, see here: http://php.net/manual/en/ini.core.php#ini.memory-limit -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at rogue-research.com Mon Jan 16 17:07:55 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 17:07:55 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> Message-ID: <20170116220755.255415709@mail.rogue-research.com> On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: >> can I hack something to allow that 100 MB allocation to proceed? > >To increase the amount of memory allowed for old build removal, add the >following line to the top of CDash/include/autoremove.php: >ini_set('memory_limit','1024M'); That worked, thanks. Now the sql command gives: +---------------------+ | starttime | +---------------------+ | 2016-11-17 15:13:52 | +---------------------+ which matches my 'AutoRemove Timeframe' of 60 days. Alas, 'du' still shows 43 GB in my mysql/data folder.... Cheers, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From bloring at lbl.gov Tue Jan 17 15:09:54 2017 From: bloring at lbl.gov (Burlen Loring) Date: Tue, 17 Jan 2017 12:09:54 -0800 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116171846.2088731986@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: <034015bd-7dce-fa4f-9d1a-7840f8773d34@lbl.gov> as a data point, I've also had problems with this, in fact I got a notice from the hosting company that MySQL would be disabled for my site unless I reduced the size of the database. it may have been my fault initially as I had not set CDASH_AUTOREMOVE_BUILDS='1'; in the local php config? Unfortunately there is some problem with the CDash autoRemoveBuilds.php scripts, it seems that they take too long to run(ERROR 2013 (HY000): Lost connection to MySQL server during query/ERROR 2006 (HY000): MySQL server has gone away), but in any case the auto remove scripts left the database in a corrupt state (specifically the test table) and now nothing works. I'm planning at some point to nuke the whole thing and start over. Burlen On 01/16/2017 09:18 AM, Sean McBride wrote: > Hi all, > > My cdash database is getting quite big (43 GB)... presumably because of old submissions staying around forever? I tried 'Maintenance > Cleanup Database', but that didn't save anything. In the project settings my 'AutoRemove Timeframe' is 60 days. > > What can I do to prune things? > > Thanks, > From j.bedouet at infonie.fr Tue Jan 17 18:14:21 2017 From: j.bedouet at infonie.fr (=?UTF-8?Q?Judica=C3=ABl_Bedouet?=) Date: Wed, 18 Jan 2017 00:14:21 +0100 Subject: [CDash] CDash Digest, Vol 104, Issue 4 In-Reply-To: References: Message-ID: Hi, Having removed builds from the database does not mean that you recover space on your hard disk. You should now optimize your tables. See http://dev.mysql.com/doc/en/optimize-table.html With show table status (http://dev.mysql.com/doc/en/show-table-status.html), you can see the size of the tables and the disk space that can be recovered (called overhead under phpMyAdmin). Regards, J. Bedouet Date: Mon, 16 Jan 2017 17:07:55 -0500 > From: "Sean McBride" > To: "Zack Galbreath" > Cc: cdash > Subject: Re: [CDash] reducing cdash database size > Message-ID: <20170116220755.255415709 at mail.rogue-research.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: > > >> can I hack something to allow that 100 MB allocation to proceed? > > > >To increase the amount of memory allowed for old build removal, add the > >following line to the top of CDash/include/autoremove.php: > >ini_set('memory_limit','1024M'); > > That worked, thanks. Now the sql command gives: > > +---------------------+ > | starttime | > +---------------------+ > | 2016-11-17 15:13:52 | > +---------------------+ > > which matches my 'AutoRemove Timeframe' of 60 days. > > Alas, 'du' still shows 43 GB in my mysql/data folder.... > > Cheers, > > -- > ____________________________________________________________ > Sean McBride, B. Eng sean at rogue-research.com > Rogue Research www.rogue-research.com > Mac Software Developer Montr?al, Qu?bec, Canada > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > http://public.kitware.com/mailman/listinfo/cdash > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.bedouet at infonie.fr Tue Jan 17 18:16:50 2017 From: j.bedouet at infonie.fr (=?UTF-8?Q?Judica=C3=ABl_Bedouet?=) Date: Wed, 18 Jan 2017 00:16:50 +0100 Subject: [CDash] reducing cdash database size Message-ID: Hi, Having removed builds from the database does not mean that you recover space on your hard disk. You should now optimize your tables. See http://dev.mysql.com/doc/en/ optimize-table.html With show table status (http://dev.mysql.com/doc/en/show-table-status.html), you can see the size of the tables and the disk space that can be recovered (called overhead under phpMyAdmin). Regards, J. Bedouet Date: Mon, 16 Jan 2017 17:07:55 -0500 > From: "Sean McBride" > To: "Zack Galbreath" > Cc: cdash > Subject: Re: [CDash] reducing cdash database size > Message-ID: <20170116220755.255415709 at mail.rogue-research.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: > > >> can I hack something to allow that 100 MB allocation to proceed? > > > >To increase the amount of memory allowed for old build removal, add the > >following line to the top of CDash/include/autoremove.php: > >ini_set('memory_limit','1024M'); > > That worked, thanks. Now the sql command gives: > > +---------------------+ > | starttime | > +---------------------+ > | 2016-11-17 15:13:52 | > +---------------------+ > > which matches my 'AutoRemove Timeframe' of 60 days. > > Alas, 'du' still shows 43 GB in my mysql/data folder.... > > Cheers, > > -- > ____________________________________________________________ > Sean McBride, B. Eng sean at rogue-research.com > Rogue Research www.rogue-research.com > Mac Software Developer Montr?al, Qu?bec, Canada > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > http://public.kitware.com/mailman/listinfo/cdash > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mateju.Miroslav at azd.cz Wed Jan 18 02:36:48 2017 From: Mateju.Miroslav at azd.cz (=?iso-8859-2?Q?Mat=ECj=F9_Miroslav=2C_Ing=2E?=) Date: Wed, 18 Jan 2017 07:36:48 +0000 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116220755.255415709@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> <20170116220755.255415709@mail.rogue-research.com> Message-ID: <00c635b0d3af49fbb159323d79ddd0be@azdexchstore3.AZD.LOCAL> -----Original Message----- From: CDash [mailto:cdash-bounces at public.kitware.com] On Behalf Of Sean McBride Sent: Monday, January 16, 2017 11:08 PM > Alas, 'du' still shows 43 GB in my mysql/data folder.... Hi Sean, The storage space is not reclaimed by default with MySQL InnoDB engine. If you are running low disk space, consider looking at https://dba.stackexchange.com/questions/64134/deleting-reclaiming-space-from-innodb-table. Hope this helps, Ing.?Miroslav Mat?j? Programmer Analyst A?D Praha s.r.o. Technology Division Research and Development ?irovnick? 2/3146, 106 17 Prague Czech Republic Phone: +420?267?287?476 Web: www.azd.cz From zack.galbreath at kitware.com Mon Jan 2 16:01:04 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 2 Jan 2017 11:01:04 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen wrote: > The calendar link no longer works > Looking into this. Thanks for the heads up. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Fri Jan 6 18:36:29 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 6 Jan 2017 13:36:29 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: I just deployed the fix for this issue to open.cdash.org. On Mon, Jan 2, 2017 at 11:01 AM, Zack Galbreath wrote: > On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen > wrote: > >> The calendar link no longer works >> > > Looking into this. Thanks for the heads up. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bill.lorensen at gmail.com Sat Jan 7 00:25:37 2017 From: bill.lorensen at gmail.com (Bill Lorensen) Date: Fri, 6 Jan 2017 19:25:37 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: Wirks again. Thanks! On Jan 6, 2017 1:36 PM, "Zack Galbreath" wrote: > I just deployed the fix for this issue to open.cdash.org. > > On Mon, Jan 2, 2017 at 11:01 AM, Zack Galbreath < > zack.galbreath at kitware.com> wrote: > >> On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen >> wrote: >> >>> The calendar link no longer works >>> >> >> Looking into this. Thanks for the heads up. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From julien.jomier at kitware.com Mon Jan 16 08:33:02 2017 From: julien.jomier at kitware.com (Julien Jomier) Date: Mon, 16 Jan 2017 09:33:02 +0100 Subject: [CDash] ANN: CMake Course - March 13 in Lyon, France Message-ID: Kitware will be holding a CMake training course on March 13, 2017 at Kitware's office in Lyon, France. This one-day course will cover CMake, CTest, CPack and CDash. Visit our website for more information and registration details (early registration and student discounts available): http://training.kitware.fr/browse/129 Note that the course will be taught in English. If you have any questions, please contact me directly or training at kitware.fr. We are looking forward to seeing you in Lyon, Julien From sean at rogue-research.com Mon Jan 16 17:18:46 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 12:18:46 -0500 Subject: [CDash] reducing cdash database size Message-ID: <20170116171846.2088731986@mail.rogue-research.com> Hi all, My cdash database is getting quite big (43 GB)... presumably because of old submissions staying around forever? I tried 'Maintenance > Cleanup Database', but that didn't save anything. In the project settings my 'AutoRemove Timeframe' is 60 days. What can I do to prune things? Thanks, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From zack.galbreath at kitware.com Mon Jan 16 17:36:59 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 16 Jan 2017 12:36:59 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116171846.2088731986@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: On Mon, Jan 16, 2017 at 12:18 PM, Sean McBride wrote: > Hi all, > > My cdash database is getting quite big (43 GB)... presumably because of > old submissions staying around forever? I tried 'Maintenance > Cleanup > Database', but that didn't save anything. In the project settings my > 'AutoRemove Timeframe' is 60 days. > > What can I do to prune things? > Hi Sean, Do you know if old builds are being deleted or not? You can check with the following SQL query: select starttime from build order by starttime limit 1; If not, you can run this deletion process manually from a shell on your server. cd /path/to/CDash php scripts/autoRemoveBuilds.php (this will probably take a long time to run if you have old builds to prune) The removal of old builds should be happening automatically when the first build of the day is submitted. CDash uses cURL to connect to itself and start this process. The relevant code is here: https://github.com/Kitware/CDash/blob/master/include/do_submit.php#L115-L120 One possible solution is setting $CDASH_BASE_URL in your config.local.php. HTH! -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at rogue-research.com Mon Jan 16 20:42:20 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 15:42:20 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: <20170116204220.1519428266@mail.rogue-research.com> On Mon, 16 Jan 2017 12:36:59 -0500, Zack Galbreath said: >Do you know if old builds are being deleted or not? I do not know. >You can check with the >following SQL query: > >select starttime from build order by starttime limit 1; It gives: mysql> select starttime from build order by starttime limit 1; +---------------------+ | starttime | +---------------------+ | 2014-11-21 22:06:47 | +---------------------+ 1 row in set (0.00 sec) So I guess there are indeed old builds in there...? >If not, you can run this deletion process manually from a shell on your >server. > >cd /path/to/CDash >php scripts/autoRemoveBuilds.php >(this will probably take a long time to run if you have old builds to prune) Just done that, for most projects it went like: $ php autoRemoveBuilds.php Foo removing builds for Foo -- removing old buildids for projectid: 8 -- removing old buildids for projectid: 8 But for one: $ php autoRemoveBuilds.php Brainsight removing builds for Brainsight -- removing old buildids for projectid: 1 PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 102320960 bytes) in /Library/Server/Web/Data/Sites/Default/cdash/cdash/pdocore.php on line 289 And my mysql data folder is still 43 GB and your 'select' command still has that 2014 date. :( The machine has 48 GB RAM, can I hack something to allow that 100 MB allocation to proceed? Cheers, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From zack.galbreath at kitware.com Mon Jan 16 20:52:55 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 16 Jan 2017 15:52:55 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116204220.1519428266@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> Message-ID: On Mon, Jan 16, 2017 at 3:42 PM, Sean McBride wrote: > can I hack something to allow that 100 MB allocation to proceed? To increase the amount of memory allowed for old build removal, add the following line to the top of CDash/include/autoremove.php: ini_set('memory_limit','1024M'); To make this a more general, site-wide change, see here: http://php.net/manual/en/ini.core.php#ini.memory-limit -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at rogue-research.com Mon Jan 16 22:07:55 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 17:07:55 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> Message-ID: <20170116220755.255415709@mail.rogue-research.com> On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: >> can I hack something to allow that 100 MB allocation to proceed? > >To increase the amount of memory allowed for old build removal, add the >following line to the top of CDash/include/autoremove.php: >ini_set('memory_limit','1024M'); That worked, thanks. Now the sql command gives: +---------------------+ | starttime | +---------------------+ | 2016-11-17 15:13:52 | +---------------------+ which matches my 'AutoRemove Timeframe' of 60 days. Alas, 'du' still shows 43 GB in my mysql/data folder.... Cheers, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From bloring at lbl.gov Tue Jan 17 20:09:54 2017 From: bloring at lbl.gov (Burlen Loring) Date: Tue, 17 Jan 2017 12:09:54 -0800 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116171846.2088731986@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: <034015bd-7dce-fa4f-9d1a-7840f8773d34@lbl.gov> as a data point, I've also had problems with this, in fact I got a notice from the hosting company that MySQL would be disabled for my site unless I reduced the size of the database. it may have been my fault initially as I had not set CDASH_AUTOREMOVE_BUILDS='1'; in the local php config? Unfortunately there is some problem with the CDash autoRemoveBuilds.php scripts, it seems that they take too long to run(ERROR 2013 (HY000): Lost connection to MySQL server during query/ERROR 2006 (HY000): MySQL server has gone away), but in any case the auto remove scripts left the database in a corrupt state (specifically the test table) and now nothing works. I'm planning at some point to nuke the whole thing and start over. Burlen On 01/16/2017 09:18 AM, Sean McBride wrote: > Hi all, > > My cdash database is getting quite big (43 GB)... presumably because of old submissions staying around forever? I tried 'Maintenance > Cleanup Database', but that didn't save anything. In the project settings my 'AutoRemove Timeframe' is 60 days. > > What can I do to prune things? > > Thanks, > From j.bedouet at infonie.fr Tue Jan 17 23:14:21 2017 From: j.bedouet at infonie.fr (=?UTF-8?Q?Judica=C3=ABl_Bedouet?=) Date: Wed, 18 Jan 2017 00:14:21 +0100 Subject: [CDash] CDash Digest, Vol 104, Issue 4 In-Reply-To: References: Message-ID: Hi, Having removed builds from the database does not mean that you recover space on your hard disk. You should now optimize your tables. See http://dev.mysql.com/doc/en/optimize-table.html With show table status (http://dev.mysql.com/doc/en/show-table-status.html), you can see the size of the tables and the disk space that can be recovered (called overhead under phpMyAdmin). Regards, J. Bedouet Date: Mon, 16 Jan 2017 17:07:55 -0500 > From: "Sean McBride" > To: "Zack Galbreath" > Cc: cdash > Subject: Re: [CDash] reducing cdash database size > Message-ID: <20170116220755.255415709 at mail.rogue-research.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: > > >> can I hack something to allow that 100 MB allocation to proceed? > > > >To increase the amount of memory allowed for old build removal, add the > >following line to the top of CDash/include/autoremove.php: > >ini_set('memory_limit','1024M'); > > That worked, thanks. Now the sql command gives: > > +---------------------+ > | starttime | > +---------------------+ > | 2016-11-17 15:13:52 | > +---------------------+ > > which matches my 'AutoRemove Timeframe' of 60 days. > > Alas, 'du' still shows 43 GB in my mysql/data folder.... > > Cheers, > > -- > ____________________________________________________________ > Sean McBride, B. Eng sean at rogue-research.com > Rogue Research www.rogue-research.com > Mac Software Developer Montr?al, Qu?bec, Canada > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > http://public.kitware.com/mailman/listinfo/cdash > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.bedouet at infonie.fr Tue Jan 17 23:16:50 2017 From: j.bedouet at infonie.fr (=?UTF-8?Q?Judica=C3=ABl_Bedouet?=) Date: Wed, 18 Jan 2017 00:16:50 +0100 Subject: [CDash] reducing cdash database size Message-ID: Hi, Having removed builds from the database does not mean that you recover space on your hard disk. You should now optimize your tables. See http://dev.mysql.com/doc/en/ optimize-table.html With show table status (http://dev.mysql.com/doc/en/show-table-status.html), you can see the size of the tables and the disk space that can be recovered (called overhead under phpMyAdmin). Regards, J. Bedouet Date: Mon, 16 Jan 2017 17:07:55 -0500 > From: "Sean McBride" > To: "Zack Galbreath" > Cc: cdash > Subject: Re: [CDash] reducing cdash database size > Message-ID: <20170116220755.255415709 at mail.rogue-research.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: > > >> can I hack something to allow that 100 MB allocation to proceed? > > > >To increase the amount of memory allowed for old build removal, add the > >following line to the top of CDash/include/autoremove.php: > >ini_set('memory_limit','1024M'); > > That worked, thanks. Now the sql command gives: > > +---------------------+ > | starttime | > +---------------------+ > | 2016-11-17 15:13:52 | > +---------------------+ > > which matches my 'AutoRemove Timeframe' of 60 days. > > Alas, 'du' still shows 43 GB in my mysql/data folder.... > > Cheers, > > -- > ____________________________________________________________ > Sean McBride, B. Eng sean at rogue-research.com > Rogue Research www.rogue-research.com > Mac Software Developer Montr?al, Qu?bec, Canada > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > http://public.kitware.com/mailman/listinfo/cdash > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mateju.Miroslav at azd.cz Wed Jan 18 07:36:48 2017 From: Mateju.Miroslav at azd.cz (=?iso-8859-2?Q?Mat=ECj=F9_Miroslav=2C_Ing=2E?=) Date: Wed, 18 Jan 2017 07:36:48 +0000 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116220755.255415709@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> <20170116220755.255415709@mail.rogue-research.com> Message-ID: <00c635b0d3af49fbb159323d79ddd0be@azdexchstore3.AZD.LOCAL> -----Original Message----- From: CDash [mailto:cdash-bounces at public.kitware.com] On Behalf Of Sean McBride Sent: Monday, January 16, 2017 11:08 PM > Alas, 'du' still shows 43 GB in my mysql/data folder.... Hi Sean, The storage space is not reclaimed by default with MySQL InnoDB engine. If you are running low disk space, consider looking at https://dba.stackexchange.com/questions/64134/deleting-reclaiming-space-from-innodb-table. Hope this helps, Ing.?Miroslav Mat?j? Programmer Analyst A?D Praha s.r.o. Technology Division Research and Development ?irovnick? 2/3146, 106 17 Prague Czech Republic Phone: +420?267?287?476 Web: www.azd.cz From zack.galbreath at kitware.com Mon Jan 2 16:01:04 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 2 Jan 2017 11:01:04 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen wrote: > The calendar link no longer works > Looking into this. Thanks for the heads up. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zack.galbreath at kitware.com Fri Jan 6 18:36:29 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Fri, 6 Jan 2017 13:36:29 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: I just deployed the fix for this issue to open.cdash.org. On Mon, Jan 2, 2017 at 11:01 AM, Zack Galbreath wrote: > On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen > wrote: > >> The calendar link no longer works >> > > Looking into this. Thanks for the heads up. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bill.lorensen at gmail.com Sat Jan 7 00:25:37 2017 From: bill.lorensen at gmail.com (Bill Lorensen) Date: Fri, 6 Jan 2017 19:25:37 -0500 Subject: [CDash] Calendar not workng In-Reply-To: References: Message-ID: Wirks again. Thanks! On Jan 6, 2017 1:36 PM, "Zack Galbreath" wrote: > I just deployed the fix for this issue to open.cdash.org. > > On Mon, Jan 2, 2017 at 11:01 AM, Zack Galbreath < > zack.galbreath at kitware.com> wrote: > >> On Fri, Dec 30, 2016 at 12:33 PM, Bill Lorensen >> wrote: >> >>> The calendar link no longer works >>> >> >> Looking into this. Thanks for the heads up. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From julien.jomier at kitware.com Mon Jan 16 08:33:02 2017 From: julien.jomier at kitware.com (Julien Jomier) Date: Mon, 16 Jan 2017 09:33:02 +0100 Subject: [CDash] ANN: CMake Course - March 13 in Lyon, France Message-ID: Kitware will be holding a CMake training course on March 13, 2017 at Kitware's office in Lyon, France. This one-day course will cover CMake, CTest, CPack and CDash. Visit our website for more information and registration details (early registration and student discounts available): http://training.kitware.fr/browse/129 Note that the course will be taught in English. If you have any questions, please contact me directly or training at kitware.fr. We are looking forward to seeing you in Lyon, Julien From sean at rogue-research.com Mon Jan 16 17:18:46 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 12:18:46 -0500 Subject: [CDash] reducing cdash database size Message-ID: <20170116171846.2088731986@mail.rogue-research.com> Hi all, My cdash database is getting quite big (43 GB)... presumably because of old submissions staying around forever? I tried 'Maintenance > Cleanup Database', but that didn't save anything. In the project settings my 'AutoRemove Timeframe' is 60 days. What can I do to prune things? Thanks, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From zack.galbreath at kitware.com Mon Jan 16 17:36:59 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 16 Jan 2017 12:36:59 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116171846.2088731986@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: On Mon, Jan 16, 2017 at 12:18 PM, Sean McBride wrote: > Hi all, > > My cdash database is getting quite big (43 GB)... presumably because of > old submissions staying around forever? I tried 'Maintenance > Cleanup > Database', but that didn't save anything. In the project settings my > 'AutoRemove Timeframe' is 60 days. > > What can I do to prune things? > Hi Sean, Do you know if old builds are being deleted or not? You can check with the following SQL query: select starttime from build order by starttime limit 1; If not, you can run this deletion process manually from a shell on your server. cd /path/to/CDash php scripts/autoRemoveBuilds.php (this will probably take a long time to run if you have old builds to prune) The removal of old builds should be happening automatically when the first build of the day is submitted. CDash uses cURL to connect to itself and start this process. The relevant code is here: https://github.com/Kitware/CDash/blob/master/include/do_submit.php#L115-L120 One possible solution is setting $CDASH_BASE_URL in your config.local.php. HTH! -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at rogue-research.com Mon Jan 16 20:42:20 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 15:42:20 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: <20170116204220.1519428266@mail.rogue-research.com> On Mon, 16 Jan 2017 12:36:59 -0500, Zack Galbreath said: >Do you know if old builds are being deleted or not? I do not know. >You can check with the >following SQL query: > >select starttime from build order by starttime limit 1; It gives: mysql> select starttime from build order by starttime limit 1; +---------------------+ | starttime | +---------------------+ | 2014-11-21 22:06:47 | +---------------------+ 1 row in set (0.00 sec) So I guess there are indeed old builds in there...? >If not, you can run this deletion process manually from a shell on your >server. > >cd /path/to/CDash >php scripts/autoRemoveBuilds.php >(this will probably take a long time to run if you have old builds to prune) Just done that, for most projects it went like: $ php autoRemoveBuilds.php Foo removing builds for Foo -- removing old buildids for projectid: 8 -- removing old buildids for projectid: 8 But for one: $ php autoRemoveBuilds.php Brainsight removing builds for Brainsight -- removing old buildids for projectid: 1 PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 102320960 bytes) in /Library/Server/Web/Data/Sites/Default/cdash/cdash/pdocore.php on line 289 And my mysql data folder is still 43 GB and your 'select' command still has that 2014 date. :( The machine has 48 GB RAM, can I hack something to allow that 100 MB allocation to proceed? Cheers, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From zack.galbreath at kitware.com Mon Jan 16 20:52:55 2017 From: zack.galbreath at kitware.com (Zack Galbreath) Date: Mon, 16 Jan 2017 15:52:55 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116204220.1519428266@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> Message-ID: On Mon, Jan 16, 2017 at 3:42 PM, Sean McBride wrote: > can I hack something to allow that 100 MB allocation to proceed? To increase the amount of memory allowed for old build removal, add the following line to the top of CDash/include/autoremove.php: ini_set('memory_limit','1024M'); To make this a more general, site-wide change, see here: http://php.net/manual/en/ini.core.php#ini.memory-limit -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at rogue-research.com Mon Jan 16 22:07:55 2017 From: sean at rogue-research.com (Sean McBride) Date: Mon, 16 Jan 2017 17:07:55 -0500 Subject: [CDash] reducing cdash database size In-Reply-To: References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> Message-ID: <20170116220755.255415709@mail.rogue-research.com> On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: >> can I hack something to allow that 100 MB allocation to proceed? > >To increase the amount of memory allowed for old build removal, add the >following line to the top of CDash/include/autoremove.php: >ini_set('memory_limit','1024M'); That worked, thanks. Now the sql command gives: +---------------------+ | starttime | +---------------------+ | 2016-11-17 15:13:52 | +---------------------+ which matches my 'AutoRemove Timeframe' of 60 days. Alas, 'du' still shows 43 GB in my mysql/data folder.... Cheers, -- ____________________________________________________________ Sean McBride, B. Eng sean at rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montr?al, Qu?bec, Canada From bloring at lbl.gov Tue Jan 17 20:09:54 2017 From: bloring at lbl.gov (Burlen Loring) Date: Tue, 17 Jan 2017 12:09:54 -0800 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116171846.2088731986@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> Message-ID: <034015bd-7dce-fa4f-9d1a-7840f8773d34@lbl.gov> as a data point, I've also had problems with this, in fact I got a notice from the hosting company that MySQL would be disabled for my site unless I reduced the size of the database. it may have been my fault initially as I had not set CDASH_AUTOREMOVE_BUILDS='1'; in the local php config? Unfortunately there is some problem with the CDash autoRemoveBuilds.php scripts, it seems that they take too long to run(ERROR 2013 (HY000): Lost connection to MySQL server during query/ERROR 2006 (HY000): MySQL server has gone away), but in any case the auto remove scripts left the database in a corrupt state (specifically the test table) and now nothing works. I'm planning at some point to nuke the whole thing and start over. Burlen On 01/16/2017 09:18 AM, Sean McBride wrote: > Hi all, > > My cdash database is getting quite big (43 GB)... presumably because of old submissions staying around forever? I tried 'Maintenance > Cleanup Database', but that didn't save anything. In the project settings my 'AutoRemove Timeframe' is 60 days. > > What can I do to prune things? > > Thanks, > From j.bedouet at infonie.fr Tue Jan 17 23:14:21 2017 From: j.bedouet at infonie.fr (=?UTF-8?Q?Judica=C3=ABl_Bedouet?=) Date: Wed, 18 Jan 2017 00:14:21 +0100 Subject: [CDash] CDash Digest, Vol 104, Issue 4 In-Reply-To: References: Message-ID: Hi, Having removed builds from the database does not mean that you recover space on your hard disk. You should now optimize your tables. See http://dev.mysql.com/doc/en/optimize-table.html With show table status (http://dev.mysql.com/doc/en/show-table-status.html), you can see the size of the tables and the disk space that can be recovered (called overhead under phpMyAdmin). Regards, J. Bedouet Date: Mon, 16 Jan 2017 17:07:55 -0500 > From: "Sean McBride" > To: "Zack Galbreath" > Cc: cdash > Subject: Re: [CDash] reducing cdash database size > Message-ID: <20170116220755.255415709 at mail.rogue-research.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: > > >> can I hack something to allow that 100 MB allocation to proceed? > > > >To increase the amount of memory allowed for old build removal, add the > >following line to the top of CDash/include/autoremove.php: > >ini_set('memory_limit','1024M'); > > That worked, thanks. Now the sql command gives: > > +---------------------+ > | starttime | > +---------------------+ > | 2016-11-17 15:13:52 | > +---------------------+ > > which matches my 'AutoRemove Timeframe' of 60 days. > > Alas, 'du' still shows 43 GB in my mysql/data folder.... > > Cheers, > > -- > ____________________________________________________________ > Sean McBride, B. Eng sean at rogue-research.com > Rogue Research www.rogue-research.com > Mac Software Developer Montr?al, Qu?bec, Canada > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > http://public.kitware.com/mailman/listinfo/cdash > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.bedouet at infonie.fr Tue Jan 17 23:16:50 2017 From: j.bedouet at infonie.fr (=?UTF-8?Q?Judica=C3=ABl_Bedouet?=) Date: Wed, 18 Jan 2017 00:16:50 +0100 Subject: [CDash] reducing cdash database size Message-ID: Hi, Having removed builds from the database does not mean that you recover space on your hard disk. You should now optimize your tables. See http://dev.mysql.com/doc/en/ optimize-table.html With show table status (http://dev.mysql.com/doc/en/show-table-status.html), you can see the size of the tables and the disk space that can be recovered (called overhead under phpMyAdmin). Regards, J. Bedouet Date: Mon, 16 Jan 2017 17:07:55 -0500 > From: "Sean McBride" > To: "Zack Galbreath" > Cc: cdash > Subject: Re: [CDash] reducing cdash database size > Message-ID: <20170116220755.255415709 at mail.rogue-research.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Mon, 16 Jan 2017 15:52:55 -0500, Zack Galbreath said: > > >> can I hack something to allow that 100 MB allocation to proceed? > > > >To increase the amount of memory allowed for old build removal, add the > >following line to the top of CDash/include/autoremove.php: > >ini_set('memory_limit','1024M'); > > That worked, thanks. Now the sql command gives: > > +---------------------+ > | starttime | > +---------------------+ > | 2016-11-17 15:13:52 | > +---------------------+ > > which matches my 'AutoRemove Timeframe' of 60 days. > > Alas, 'du' still shows 43 GB in my mysql/data folder.... > > Cheers, > > -- > ____________________________________________________________ > Sean McBride, B. Eng sean at rogue-research.com > Rogue Research www.rogue-research.com > Mac Software Developer Montr?al, Qu?bec, Canada > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > CDash mailing list > CDash at public.kitware.com > http://public.kitware.com/mailman/listinfo/cdash > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mateju.Miroslav at azd.cz Wed Jan 18 07:36:48 2017 From: Mateju.Miroslav at azd.cz (=?iso-8859-2?Q?Mat=ECj=F9_Miroslav=2C_Ing=2E?=) Date: Wed, 18 Jan 2017 07:36:48 +0000 Subject: [CDash] reducing cdash database size In-Reply-To: <20170116220755.255415709@mail.rogue-research.com> References: <20170116171846.2088731986@mail.rogue-research.com> <20170116204220.1519428266@mail.rogue-research.com> <20170116220755.255415709@mail.rogue-research.com> Message-ID: <00c635b0d3af49fbb159323d79ddd0be@azdexchstore3.AZD.LOCAL> -----Original Message----- From: CDash [mailto:cdash-bounces at public.kitware.com] On Behalf Of Sean McBride Sent: Monday, January 16, 2017 11:08 PM > Alas, 'du' still shows 43 GB in my mysql/data folder.... Hi Sean, The storage space is not reclaimed by default with MySQL InnoDB engine. If you are running low disk space, consider looking at https://dba.stackexchange.com/questions/64134/deleting-reclaiming-space-from-innodb-table. Hope this helps, Ing.?Miroslav Mat?j? Programmer Analyst A?D Praha s.r.o. Technology Division Research and Development ?irovnick? 2/3146, 106 17 Prague Czech Republic Phone: +420?267?287?476 Web: www.azd.cz