Task 12011071

Name	hadcm3igeo_w2dv_2000_80_06759966_3
Workunit	6963282
Created	12 Nov 2010, 15:29:42 UTC
Sent	17 Nov 2010, 7:06:07 UTC
Report deadline	18 May 2012, 3:49:16 UTC
Received	9 Jan 2011, 21:45:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1111779
Run time	40 days 7 hours 12 min 59 sec
CPU time	39 days 2 hours 3 min 11 sec
Validate state	Invalid
Credit	22,861.44
Device peak FLOPS	2.79 GFLOPS
Application version	HadCM3L Coupled Model Experiment Optimised File I/O v6.04 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz CPDN Monitor - Quit request from BOINC... scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz Model crashed: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6284, iMonCtr=1 Model crash detected, will try to restart... scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz CPDN Monitor - Quit request from BOINC... scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: boinc_ufs_cpdnout5.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3476, iMonCtr=1 Model crash detected, will try to restart... scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: boinc_ufs_cpdnout5.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz CPDN Monitor - Quit request from BOINC... scan: 0g58_07_2000_atmos.gz scan: 0g58_07_2000_ocean.gz scan: 1040_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: boinc_ufs_cpdnout5.zip scan: boinc_ufs_cpdnout6.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fo8phdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w2dv_2000_80_06759966.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_sca12.gz Model crashed: Model crashed: Model crashed: Model crashed: Model crashed: Model crashed: Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Jan 2011 12:10:19	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,814,400	3,352,989	1.8480
08 Jan 2011 14:59:29	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,788,480	3,303,680	1.8472
08 Jan 2011 11:56:10	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,762,560	3,255,151	1.8468
07 Jan 2011 11:10:22	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,736,640	3,206,455	1.8464
06 Jan 2011 21:27:43	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,710,720	3,158,289	1.8462
02 Jan 2011 17:01:51	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,684,800	3,109,670	1.8457
01 Jan 2011 20:49:50	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,658,880	3,061,052	1.8453
01 Jan 2011 06:07:21	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,632,960	3,012,757	1.8450
31 Dec 2010 16:25:33	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,607,040	2,964,828	1.8449
31 Dec 2010 02:43:40	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,581,120	2,916,554	1.8446
30 Dec 2010 12:59:16	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,555,200	2,868,981	1.8448
29 Dec 2010 15:04:18	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,529,280	2,819,288	1.8435
27 Dec 2010 18:00:50	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,503,360	2,770,752	1.8430
27 Dec 2010 13:47:39	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,477,440	2,722,299	1.8426
26 Dec 2010 14:10:41	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,451,520	2,673,869	1.8421
26 Dec 2010 00:30:28	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,425,600	2,625,667	1.8418
25 Dec 2010 10:49:21	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,399,680	2,577,021	1.8412
24 Dec 2010 19:04:22	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,373,760	2,529,184	1.8411
24 Dec 2010 11:24:52	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,347,840	2,481,139	1.8408
23 Dec 2010 13:13:28	1111779	12011071	hadcm3igeo_w2dv_2000_80_06759966_3	1,321,920	2,432,736	1.8403