Task 12990055

Name	hadcm3n_s49b_1940_40_007299755_0
Workunit	7497179
Created	20 Jun 2011, 20:14:32 UTC
Sent	20 Jun 2011, 20:14:38 UTC
Report deadline	20 Sep 2011, 3:41:49 UTC
Received	2 Jul 2011, 17:17:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	775427
Run time	7 days 19 hours 14 min 11 sec
CPU time	6 days 10 hours 5 min 51 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.11 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6908, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4608, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6044, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5712, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4536, iMonCtr=1 Model crash detected, will try to restart... 16:27:02 (4644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:53:16 (6476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:08:17 (5092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:10:31 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:54 (7156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:56 (7156): No heartbeat from core client for 30 sec - exiting 14:29:57 (7156): No heartbeat from core client for 30 sec - exiting 21:44:40 (532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/s49bko.pje7c10 Error converting file to netcdf: dataout/s49bko.pie7c10 Error converting file to netcdf: dataout/s49bko.pfe7c10 Error converting file to netcdf: dataout/s49bka.phe7c10 Error converting file to netcdf: dataout/s49bka.pge7c10 Error converting file to netcdf: dataout/s49bka.pee7c10 Error converting file to netcdf: dataout/s49bka.pde7c10 Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Jul 2011 12:12:55	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	233,280	623,815	2.6741
01 Jul 2011 17:25:51	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	207,360	554,824	2.6757
30 Jun 2011 03:47:41	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	181,440	485,207	2.6742
28 Jun 2011 15:58:33	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	155,520	416,696	2.6794
26 Jun 2011 20:55:10	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	129,600	335,331	2.5874
26 Jun 2011 02:14:08	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	103,680	269,926	2.6035
24 Jun 2011 21:17:26	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	77,760	203,440	2.6163
23 Jun 2011 17:17:00	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	51,840	134,586	2.5962
22 Jun 2011 01:37:54	775427	12990055	hadcm3n_s49b_1940_40_007299755_0	25,920	66,873	2.5800