Task 15589475

Name	hadcm3n_4k6z_1940_40_008303805_0
Workunit	8454940
Created	7 Feb 2013, 0:12:30 UTC
Sent	7 Feb 2013, 0:12:40 UTC
Report deadline	9 May 2013, 7:39:51 UTC
Received	28 Mar 2013, 0:02:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1218615
Run time	13 days 12 hours 16 min 41 sec
CPU time	12 days 18 hours 47 min 34 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6108, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6108, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5416, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4768, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5372, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2128, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2128, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4792, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6212, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1124, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1124, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5632, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5512, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5716, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4520, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4k6zko.pje7c10 Error converting file to netcdf: dataout/4k6zko.pie7c10 Error converting file to netcdf: dataout/4k6zko.pfe7c10 Error converting file to netcdf: dataout/4k6zka.phe7c10 Error converting file to netcdf: dataout/4k6zka.pge7c10 Error converting file to netcdf: dataout/4k6zka.pee7c10 Error converting file to netcdf: dataout/4k6zka.pde7c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=376, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5180, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5876, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5384, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4524, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4940, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3796, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1884, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5588, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4544, iMonCtr=1 Model crash detected, will try to restart... 20:32:38 (6072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4172, iMonCtr=1 Model crash detected, will try to restart... 19:10:54 (3092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Mar 2013 02:50:09	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	518,400	1,104,447	2.1305
25 Mar 2013 00:10:00	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	492,480	1,045,892	2.1237
22 Mar 2013 13:20:25	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	466,560	990,301	2.1226
17 Mar 2013 22:30:39	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	440,640	933,669	2.1189
16 Mar 2013 21:25:38	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	414,720	879,410	2.1205
13 Mar 2013 00:22:46	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	388,800	826,560	2.1259
10 Mar 2013 13:07:38	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	362,880	777,637	2.1430
09 Mar 2013 12:43:09	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	336,960	725,492	2.1531
06 Mar 2013 04:34:52	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	311,040	674,846	2.1696
03 Mar 2013 04:39:57	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	285,120	623,676	2.1874
28 Feb 2013 22:49:55	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	259,200	569,436	2.1969
27 Feb 2013 18:14:20	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	233,280	513,645	2.2018
24 Feb 2013 22:45:28	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	207,360	453,592	2.1875
23 Feb 2013 18:54:03	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	181,440	397,870	2.1928
21 Feb 2013 16:35:56	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	155,520	340,838	2.1916
18 Feb 2013 00:36:25	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	129,600	283,207	2.1852
16 Feb 2013 23:45:07	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	103,680	226,680	2.1863
14 Feb 2013 14:25:25	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	77,760	169,006	2.1734
11 Feb 2013 05:00:22	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	51,840	109,569	2.1136
09 Feb 2013 23:54:47	1218615	15589475	hadcm3n_4k6z_1940_40_008303805_0	25,920	54,699	2.1103