Task 11137014

Name	hadsm3dhet2_jyd1_006607527_2
Workunit	6810900
Created	15 Mar 2010, 12:16:57 UTC
Sent	22 May 2010, 3:15:21 UTC
Report deadline	4 May 2011, 8:35:21 UTC
Received	9 Jul 2010, 13:19:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1039268
Run time	31 days 3 hours 30 min 4 sec
CPU time	30 days 4 hours 51 min 4 sec
Validate state	Invalid
Credit	4,068.98
Device peak FLOPS	0.00 GFLOPS
Application version	---
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. MainError: 05:36:56 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. MainError: 06:34:53 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.spin error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.cont error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/climate.doub error: cannot delete old hadsm3dhet2_jyd1_006607527/jobs/ncatts.cpdc CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... start_timer_thread(): CreateThread() failed, errno 9 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x00424D80 read attempt to address 0x00000044 Engaging BOINC Windows Runtime Debugger... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.3.8 Dump Timestamp : 07/09/10 09:18:40 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 1455 LoadLibraryA( dbghelp.dll ): GetLastError = 1455 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 3, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 37, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 43492, QuotaPeakPagedPoolUsage: 43492 QuotaNonPagedPoolUsage: 1904, QuotaPeakNonPagedPoolUsage: 1904 - Virtual Memory Usage - VirtualSize: 214310912, PeakVirtualSize: 214310912 - Pagefile Usage - PagefileUsage: 7094272, PeakPagefileUsage: 8974336 - Working Set Size - WorkingSetSize: 2084864, PeakWorkingSetSize: 2084864, PageFaultCount: 505 * Dump of thread ID 4716 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 312500.000000, User Time: 0.000000, Wait Time: 38532504.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB * Dump of thread ID 784 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 38532504.000000 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.3.8 Dump Timestamp : 07/09/10 09:18:43 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 1455 LoadLibraryA( dbghelp.dll ): GetLastError = 1455 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 3, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 37, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 43492, QuotaPeakPagedPoolUsage: 43492 QuotaNonPagedPoolUsage: 1904, QuotaPeakNonPagedPoolUsage: 1904 - Virtual Memory Usage - VirtualSize: 214310912, PeakVirtualSize: 214310912 - Pagefile Usage - PagefileUsage: 7094272, PeakPagefileUsage: 8974336 - Working Set Size - WorkingSetSize: 2088960, PeakWorkingSetSize: 2088960, PageFaultCount: 506 * Dump of thread ID 5692 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 38532704.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB * Dump of thread ID 3300 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 38532704.000000 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.3.8 Dump Timestamp : 07/09/10 09:18:46 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 1455 LoadLibraryA( dbghelp.dll ): GetLastError = 1455 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 3, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 37, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 43492, QuotaPeakPagedPoolUsage: 43492 QuotaNonPagedPoolUsage: 1944, QuotaPeakNonPagedPoolUsage: 1944 - Virtual Memory Usage - VirtualSize: 215359488, PeakVirtualSize: 215359488 - Pagefile Usage - PagefileUsage: 7102464, PeakPagefileUsage: 8974336 - Working Set Size - WorkingSetSize: 2121728, PeakWorkingSetSize: 2121728, PageFaultCount: 514 * Dump of thread ID 3156 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 38532888.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812AFB * Dump of thread ID 1856 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 38532884.000000 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5736, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Jul 2010 06:21:18	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	183,634	2,595,679	5.8609
08 Jul 2010 20:41:56	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	172,832	2,561,277	5.9278
08 Jul 2010 11:23:05	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	162,030	2,528,173	6.0012
08 Jul 2010 02:21:57	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	151,228	2,495,317	6.0791
07 Jul 2010 16:47:45	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	140,426	2,461,877	6.1597
07 Jul 2010 07:15:17	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	129,624	2,428,167	6.2441
06 Jul 2010 21:46:10	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	118,822	2,394,561	6.3336
06 Jul 2010 12:33:09	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	108,020	2,362,081	6.4315
15 Jun 2010 21:29:54	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	97,218	1,157,379	3.2468
15 Jun 2010 00:14:28	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	86,416	1,122,394	3.2471
13 Jun 2010 18:35:00	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	75,614	1,085,373	3.2413
13 Jun 2010 08:53:59	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	64,812	1,050,441	3.2415
12 Jun 2010 22:44:47	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	54,010	1,015,291	3.2411
12 Jun 2010 12:32:27	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	43,208	979,994	3.2401
12 Jun 2010 02:39:19	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	32,406	944,989	3.2401
10 Jun 2010 21:21:51	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	21,604	909,633	3.2388
10 Jun 2010 04:22:23	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	10,802	874,950	3.2400
09 Jun 2010 18:38:40	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	259,248	839,805	3.2394
08 Jun 2010 22:32:13	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	248,446	804,896	3.2397
07 Jun 2010 19:37:50	1039268	11137014	hadsm3dhet2_jyd1_006607527_2	237,644	770,146	3.2408