Task 12012919

Name	hadcm3igeo_w23l_2000_80_06760336_1
Workunit	6963652
Created	12 Nov 2010, 15:30:26 UTC
Sent	16 Nov 2010, 0:28:28 UTC
Report deadline	16 May 2012, 21:11:37 UTC
Received	20 Dec 2010, 21:54:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	938547
Run time
CPU time	15 days 0 hours 55 min 29 sec
Validate state	Invalid
Credit	6,205.25
Device peak FLOPS	1.85 GFLOPS
Application version	HadCM3L Coupled Model Experiment Optimised File I/O v6.04 windows_intelx86
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN Monitor - Quit request from BOINC... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4072, iMonCtr=1 Model crash detected, will try to restart... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2172, iMonCtr=1 Model crash detected, will try to restart... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN Monitor - Quit request from BOINC... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3500, iMonCtr=1 Model crash detected, will try to restart... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN Monitor - Quit request from BOINC... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN Monitor - Quit request from BOINC... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5740, iMonCtr=1 Model crash detected, will try to restart... scan: 0g87_01_2000_atmos.gz scan: 0g87_01_2000_ocean.gz scan: 971_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fobthdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w23l_2000_80_06760336.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2096, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2096, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2096, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2096, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2096, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2096, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Dec 2010 21:58:31	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	492,480	1,238,866	2.5156
20 Dec 2010 21:58:31	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	466,560	1,174,176	2.5167
20 Dec 2010 21:58:31	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	440,640	1,109,814	2.5186
07 Dec 2010 09:50:29	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	414,720	1,045,576	2.5212
06 Dec 2010 08:59:02	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	388,800	981,443	2.5243
05 Dec 2010 08:55:50	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	362,880	916,352	2.5252
04 Dec 2010 07:40:30	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	336,960	850,506	2.5241
03 Dec 2010 00:42:36	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	311,040	784,169	2.5211
01 Dec 2010 22:39:06	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	285,120	718,448	2.5198
30 Nov 2010 21:00:41	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	259,200	652,566	2.5176
29 Nov 2010 20:16:49	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	233,280	586,585	2.5145
28 Nov 2010 19:52:41	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	207,360	521,698	2.5159
27 Nov 2010 18:04:01	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	181,440	456,839	2.5179
26 Nov 2010 16:05:02	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	155,520	392,050	2.5209
25 Nov 2010 13:00:22	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	129,600	326,941	2.5227
23 Nov 2010 00:15:23	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	103,680	262,356	2.5304
21 Nov 2010 21:22:04	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	77,760	197,489	2.5397
20 Nov 2010 10:38:56	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	51,840	132,722	2.5602
19 Nov 2010 06:31:14	938547	12012919	hadcm3igeo_w23l_2000_80_06760336_1	25,920	66,292	2.5576