Task 13650687

Name	hadcm3n_yeih_1900_40_007517977_2
Workunit	7715452
Created	21 Nov 2011, 6:58:05 UTC
Sent	21 Nov 2011, 7:00:38 UTC
Report deadline	20 Feb 2012, 14:27:49 UTC
Received	29 Nov 2011, 22:12:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1165115
Run time	5 days 11 hours 11 min 5 sec
CPU time	4 days 20 hours 0 min 14 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	3.30 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.12.35</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:25 (6575): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:07:43 (8473): No heartbeat from core client for 30 sec - exiting 22:07:44 (8473): No heartbeat from core client for 30 sec - exiting 22:07:45 (8473): No heartbeat from core client for 30 sec - exiting 22:07:46 (8473): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:51:28 (10918): No heartbeat from core client for 30 sec - exiting 01:51:29 (10918): No heartbeat from core client for 30 sec - exiting 01:51:30 (10918): No heartbeat from core client for 30 sec - exiting 01:51:31 (10918): No heartbeat from core client for 30 sec - exiting 01:51:32 (10918): No heartbeat from core client for 30 sec - exiting 01:51:33 (10918): No heartbeat from core client for 30 sec - exiting 01:51:34 (10918): No heartbeat from core client for 30 sec - exiting 01:51:36 (10918): No heartbeat from core client for 30 sec - exiting 01:51:37 (10918): No heartbeat from core client for 30 sec - exiting 01:51:38 (10918): No heartbeat from core client for 30 sec - exiting 01:51:39 (10918): No heartbeat from core client for 30 sec - exiting 01:51:40 (10918): No heartbeat from core client for 30 sec - exiting 01:51:41 (10918): No heartbeat from core client for 30 sec - exiting 01:51:42 (10918): No heartbeat from core client for 30 sec - exiting 01:51:43 (10918): No heartbeat from core client for 30 sec - exiting 01:51:44 (10918): No heartbeat from core client for 30 sec - exiting 01:51:45 (10918): No heartbeat from core client for 30 sec - exiting 01:51:46 (10918): No heartbeat from core client for 30 sec - exiting 01:51:47 (10918): No heartbeat from core client for 30 sec - exiting 01:51:48 (10918): No heartbeat from core client for 30 sec - exiting 01:51:49 (10918): No heartbeat from core client for 30 sec - exiting 01:51:50 (10918): No heartbeat from core client for 30 sec - exiting 01:51:51 (10918): No heartbeat from core client for 30 sec - exiting 01:51:52 (10918): No heartbeat from core client for 30 sec - exiting 01:51:53 (10918): No heartbeat from core client for 30 sec - exiting 01:51:54 (10918): No heartbeat from core client for 30 sec - exiting 01:51:55 (10918): No heartbeat from core client for 30 sec - exiting 01:51:56 (10918): No heartbeat from core client for 30 sec - exiting 01:51:57 (10918): No heartbeat from core client for 30 sec - exiting 01:51:58 (10918): No heartbeat from core client for 30 sec - exiting 01:51:59 (10918): No heartbeat from core client for 30 sec - exiting 01:52:00 (10918): No heartbeat from core client for 30 sec - exiting 01:52:01 (10918): No heartbeat from core client for 30 sec - exiting 01:52:02 (10918): No heartbeat from core client for 30 sec - exiting 01:52:03 (10918): No heartbeat from core client for 30 sec - exiting 01:52:04 (10918): No heartbeat from core client for 30 sec - exiting 01:52:05 (10918): No heartbeat from core client for 30 sec - exiting 01:52:06 (10918): No heartbeat from core client for 30 sec - exiting 01:52:07 (10918): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:32 (11018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:01:08 (19972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:51:54 (62903): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:51:55 (62903): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:14:28 (63534): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:02:11 (65341): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:12 (65341): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:57:04 (88244): No heartbeat from core client for 30 sec - exiting 22:57:05 (88244): No heartbeat from core client for 30 sec - exiting 22:57:06 (88244): No heartbeat from core client for 30 sec - exiting 22:57:07 (88244): No heartbeat from core client for 30 sec - exiting 22:57:08 (88244): No heartbeat from core client for 30 sec - exiting 22:57:09 (88244): No heartbeat from core client for 30 sec - exiting 22:57:10 (88244): No heartbeat from core client for 30 sec - exiting 22:57:11 (88244): No heartbeat from core client for 30 sec - exiting 22:57:12 (88244): No heartbeat from core client for 30 sec - exiting 22:57:13 (88244): No heartbeat from core client for 30 sec - exiting 22:57:14 (88244): No heartbeat from core client for 30 sec - exiting 22:57:15 (88244): No heartbeat from core client for 30 sec - exiting 22:57:16 (88244): No heartbeat from core client for 30 sec - exiting 22:57:17 (88244): No heartbeat from core client for 30 sec - exiting 22:57:18 (88244): No heartbeat from core client for 30 sec - exiting 22:57:19 (88244): No heartbeat from core client for 30 sec - exiting 22:57:20 (88244): No heartbeat from core client for 30 sec - exiting 22:57:21 (88244): No heartbeat from core client for 30 sec - exiting 22:57:23 (88244): No heartbeat from core client for 30 sec - exiting 22:57:24 (88244): No heartbeat from core client for 30 sec - exiting 22:57:25 (88244): No heartbeat from core client for 30 sec - exiting 22:57:26 (88244): No heartbeat from core client for 30 sec - exiting 22:57:27 (88244): No heartbeat from core client for 30 sec - exiting 22:57:28 (88244): No heartbeat from core client for 30 sec - exiting 22:57:29 (88244): No heartbeat from core client for 30 sec - exiting 22:57:30 (88244): No heartbeat from core client for 30 sec - exiting 22:57:31 (88244): No heartbeat from core client for 30 sec - exiting 22:57:32 (88244): No heartbeat from core client for 30 sec - exiting 22:57:33 (88244): No heartbeat from core client for 30 sec - exiting 22:57:34 (88244): No heartbeat from core client for 30 sec - exiting 22:57:35 (88244): No heartbeat from core client for 30 sec - exiting 22:57:36 (88244): No heartbeat from core client for 30 sec - exiting 22:57:37 (88244): No heartbeat from core client for 30 sec - exiting 22:57:38 (88244): No heartbeat from core client for 30 sec - exiting 22:57:39 (88244): No heartbeat from core client for 30 sec - exiting 22:57:40 (88244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 140185) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=850, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 140185) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=850, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 140185) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=850, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 140185) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=850, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 140185) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=850, iMonCtr=1 Model crash detected, will try to restart... execl(/Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin, 140185) failed! Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=850, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Nov 2011 07:46:00	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	207,360	379,230	1.8288
28 Nov 2011 08:15:24	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	181,440	329,902	1.8182
26 Nov 2011 15:26:11	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	155,520	282,562	1.8169
25 Nov 2011 09:06:04	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	129,600	232,780	1.7961
24 Nov 2011 16:08:32	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	103,680	181,201	1.7477
23 Nov 2011 16:24:08	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	77,760	133,502	1.7168
23 Nov 2011 02:31:07	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	51,840	89,771	1.7317
22 Nov 2011 05:53:03	1165115	13650687	hadcm3n_yeih_1900_40_007517977_2	25,920	44,688	1.7241