Task 15853094

Name	hadcm3n_3m6i_1980_40_008391949_0
Workunit	8542808
Created	21 Jun 2013, 16:39:46 UTC
Sent	1 Jul 2013, 10:34:24 UTC
Report deadline	30 Sep 2013, 18:01:35 UTC
Received	5 Jul 2013, 14:57:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1259513
Run time	1 days 0 hours 12 min 7 sec
CPU time	20 hours 27 min 42 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	3.06 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Unhandled Exception Detected... - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/03/13 10:42:47 Install Directory : Data Directory : D:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( D:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_3m6i_1980_40_008391949;D:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : * Dump of the Process Statistics: * - I/O Operations Counters - Read: 49269, Write: 0, Other 112118 - I/O Transfers Counters - Read: 0, Write: 3296, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 124432, QuotaPeakPagedPoolUsage: 124432 QuotaNonPagedPoolUsage: 5552, QuotaPeakNonPagedPoolUsage: 5632 - Virtual Memory Usage - VirtualSize: 1051308032, PeakVirtualSize: 1051308032 - Pagefile Usage - PagefileUsage: 231616512, PeakPagefileUsage: 231616512 - Working Set Size - WorkingSetSize: 226689024, PeakWorkingSetSize: 226689024, PageFaultCount: 57866 * Dump of thread ID 3340 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 52343752.000000, User Time: 36479373312.000000, Wait Time: 982612.000000 - Unhandled Exception Record - Reason: Single Instruction Executed (0x80000004) at address 0x006C6D27 - Registers - eax=00000007 ebx=0f270908 ecx=00000000 edx=10473da4 esi=0f52baa0 edi=0f253c20 eip=006c6d27 esp=0f104900 ebp=0f2708f8 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0f2708f8 0064a92b 0f251ae0 00000488 0f2e2dcc 0064a92b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006c6d27' 0f2e2db8 00455431 0f52b618 00000122 0f525b78 0f4e2690 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0064a92b' 0f4b915c 0044f0de 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00455431' 0f6d5e78 0065501d 40cd0a6f 40cd7912 100cddf8 0065501d !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0044f0de' 100cdde8 005df71b 00000002 00000002 10332864 005df71b !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '0065501d' 10332858 005da7e1 10332d44 3de20020 005da7e1 1132f8dc !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005df71b' 10332d84 005dc121 1132f8dc 1132f8e0 1132f8e4 1132f8e8 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005da7e1' 10333188 007061a7 1031e8d0 10317b50 1132f7d8 007061a7 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005dc121' 1132f7c8 006f3667 1132f8d8 1132f8fc 1132fab8 006f3667 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '007061a7' 1132faa8 004083f3 04600000 01000000 00000001 004083f3 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '006f3667' 1132fb10 00408130 1132fb24 001224ef 1132fb24 00122451 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004083f3' 1132ffc0 7d4e7c9a 00000000 00000000 7efde000 80000004 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00408130' 1132fff0 00000000 007bde7f 00000000 00000000 00000000 !BaseProcessInitPostImport+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7c9a' * Dump of thread ID 3344 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 982610.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=3404ff74 eip=7d61ccc6 esp=3404ff38 ebp=3404ff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 3404ff9c 7d4d14ef 00000064 00000000 3404ffb8 00749924 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 3404ffac 00749924 00000064 3404ffec 7d4dfb97 00000000 !Sleep+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 3404ffb8 7d4dfb97 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '00749924' 3404ffec 00000000 00749910 00000000 00000000 34560000 !FlsSetValue+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfb97' * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=204, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Signal 4 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2708, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Jul 2013 04:50:15	1259513	15853094	hadcm3n_3m6i_1980_40_008391949_0	51,840	61,364	1.1837
04 Jul 2013 14:28:20	1259513	15853094	hadcm3n_3m6i_1980_40_008391949_0	25,920	29,077	1.1218