Task 15514280

Name	hadcm3n_3kur_1940_40_008259935_1
Workunit	8415059
Created	27 Dec 2012, 23:15:26 UTC
Sent	28 Dec 2012, 0:41:57 UTC
Report deadline	29 Mar 2013, 8:09:08 UTC
Received	9 Jan 2013, 8:00:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1260592
Run time	9 days 20 hours 6 min 33 sec
CPU time	9 days 17 hours 38 min 6 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	4.13 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:40 (2748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2912, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:48:38 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3636, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Jan 2013 08:05:05	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	959,040	839,451	0.8753
09 Jan 2013 08:05:05	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	933,120	816,933	0.8755
08 Jan 2013 14:10:29	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	907,200	794,188	0.8754
08 Jan 2013 08:01:49	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	881,280	771,294	0.8752
08 Jan 2013 08:01:49	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	855,360	748,559	0.8751
08 Jan 2013 08:01:49	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	829,440	726,151	0.8755
07 Jan 2013 08:00:48	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	803,520	703,434	0.8754
07 Jan 2013 08:00:48	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	777,600	680,988	0.8758
06 Jan 2013 15:57:42	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	751,680	657,762	0.8751
06 Jan 2013 09:41:22	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	725,760	635,154	0.8752
06 Jan 2013 08:00:57	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	699,840	612,812	0.8756
06 Jan 2013 08:00:57	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	673,920	590,874	0.8768
05 Jan 2013 13:10:17	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	648,000	568,653	0.8776
05 Jan 2013 08:05:35	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	622,080	546,246	0.8781
05 Jan 2013 08:05:35	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	596,160	524,232	0.8793
05 Jan 2013 08:05:35	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	570,240	502,014	0.8804
04 Jan 2013 09:05:25	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	544,320	480,421	0.8826
04 Jan 2013 08:05:12	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	518,400	458,555	0.8846
04 Jan 2013 08:05:12	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	492,480	437,250	0.8879
03 Jan 2013 13:05:18	1260592	15514280	hadcm3n_3kur_1940_40_008259935_1	466,560	415,930	0.8915