climateprediction.net home page
Task 19144374

Task 19144374

Name hadcm3n_xcds_198012_480_010221098_1
Workunit 10221098
Created 10 Dec 2015, 11:15:39 UTC
Sent 10 Dec 2015, 11:16:54 UTC
Report deadline 21 Nov 2016, 16:36:54 UTC
Received 24 Dec 2015, 21:57:35 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1382994
Run time 3 days 4 hours 31 min 10 sec
CPU time 2 days 8 hours 45 min 24 sec
Validate state Invalid
Credit 8,709.12
Device peak FLOPS 2.62 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>7.6.9</core_client_version>
<![CDATA[
<message>
The device does not recognize the command.
 (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
10:02:14 (12532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:17:05 (29728): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:26:21 (30008): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:41:20 (29900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:02:37 (30260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:20:05 (50608): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:46:43 (69900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:54:48 (94560): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:57:05 (25524): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:47:12 (116112): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:13:18 (102504): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:40:04 (83344): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:56:07 (39628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:16:20 (6232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:57:08 (78940): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:45:36 (88704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
11:54:32 (6264): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
18:25:07 (5808): start_timer_thread(): CreateThread() failed, errno 0
04:21:56 (6848): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:23:13 (26828): start_timer_thread(): CreateThread() failed, errno 0
04:24:40 (31412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:26:00 (31952): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
22:11:29 (34108): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
13:38:57 (15720): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
17:27:45 (25932): start_timer_thread(): CreateThread() failed, errno 0
18:23:34 (26564): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:23:39 (14896): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:52:26 (17356): start_timer_thread(): CreateThread() failed, errno 0
21:53:19 (46288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:53:37 (13168): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
13:37:15 (34700): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - Quit request from BOINC...
18:28:05 (11352): start_timer_thread(): CreateThread() failed, errno 0
19:17:10 (9700): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:17:24 (30576): start_timer_thread(): CreateThread() failed, errno 0
22:53:04 (15508): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:53:22 (22140): start_timer_thread(): CreateThread() failed, errno 0
23:11:09 (20172): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:11:33 (20976): start_timer_thread(): CreateThread() failed, errno 0
00:52:19 (40936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:53:18 (41376): start_timer_thread(): CreateThread() failed, errno 0
04:04:56 (40300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:05:52 (2052): start_timer_thread(): CreateThread() failed, errno 0
10:43:42 (6600): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:43:56 (34828): start_timer_thread(): CreateThread() failed, errno 0
13:16:37 (168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:16:53 (14708): start_timer_thread(): CreateThread() failed, errno 0
13:50:37 (44544): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:50:47 (30088): start_timer_thread(): CreateThread() failed, errno 0
14:00:38 (17156): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:00:43 (37860): start_timer_thread(): CreateThread() failed, errno 0
14:43:33 (39708): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:44:11 (37064): start_timer_thread(): CreateThread() failed, errno 0
14:53:34 (25556): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:55:01 (13428): start_timer_thread(): CreateThread() failed, errno 0


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 12/24/15 14:55:01
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_xcds_198012_480_010221098;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 77180000 00179000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 76070000 000f0000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742d0000 00176000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 744a0000 00140000 C:\WINDOWS\SYSTEM32\USER32.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 765c0000 0014d000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.2.10240.16390) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.10240.16390 (th1_st1.150714-1601)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16390

ModLoad: 77100000 0007b000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 746a0000 000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.10240.16384

ModLoad: 76ba0000 00043000 C:\WINDOWS\SYSTEM32\sechost.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 761f0000 000ac000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.2.10240.16412) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74290000 0001e000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74280000 0000a000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74220000 00059000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742b0000 00019000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 67250000 000e0000 C:\WINDOWS\SYSTEM32\OPENGL32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 68810000 00025000 C:\WINDOWS\SYSTEM32\GLU32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 66bf0000 000eb000 C:\WINDOWS\SYSTEM32\DDRAW.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 739e0000 00007000 C:\WINDOWS\SYSTEM32\DCIMAN32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 732c0000 0013f000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 73800000 00008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 15244 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

- Registers -
eax=1132f9b8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=74383e28 esp=1132f9b8 ebp=1132fa14
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad0 00408130 1132fae4 0029340c 1132fae4 00293391 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff80 76083744 7ffde000 76083720 a89d5702 1132ffdc hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 771d9cd4 7ffde000 9d2736b3 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
1132ffdc 771d9c9f ffffffff 771fd4b2 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
1132ffec 00000000 007bde7f 7ffde000 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 19544 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00297360 ecx=00000000 edx=00000000 esi=002971a0 edi=00293c48
eip=771ea45c esp=2172fdcc ebp=2172ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
2172ff80 76083744 00293c48 76083720 98dd5702 2172ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
2172ff94 771d9cd4 00293c48 ad6736b3 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
2172ffdc 771d9c9f ffffffff 771fd4b2 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
2172ffec 00000000 771b43e0 00293c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 17772 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00298160 ecx=00000000 edx=00000000 esi=00297fa0 edi=00293c48
eip=771ea45c esp=31b2fdcc ebp=31b2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
31b2ff80 76083744 00293c48 76083720 881d5702 31b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
31b2ff94 771d9cd4 00293c48 bda736b3 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
31b2ffdc 771d9c9f ffffffff 771fd4b2 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
31b2ffec 00000000 771b43e0 00293c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 27896 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=0029aa90 ecx=00000000 edx=00000000 esi=0029a8d0 edi=00293c48
eip=771ea45c esp=41f2fdcc ebp=41f2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
41f2ff80 76083744 00293c48 76083720 f85d5702 41f2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
41f2ff94 771d9cd4 00293c48 cde736b3 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
41f2ffdc 771d9c9f ffffffff 771fd4b2 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
41f2ffec 00000000 771b43e0 00293c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1
Model crash detected, will try to restart...
14:55:11 (33052): start_timer_thread(): CreateThread() failed, errno 0


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 12/24/15 14:55:11
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_xcds_198012_480_010221098;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 77180000 00179000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 76070000 000f0000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742d0000 00176000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 744a0000 00140000 C:\WINDOWS\SYSTEM32\USER32.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 765c0000 0014d000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.2.10240.16390) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.10240.16390 (th1_st1.150714-1601)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16390

ModLoad: 77100000 0007b000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 746a0000 000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.10240.16384

ModLoad: 76ba0000 00043000 C:\WINDOWS\SYSTEM32\sechost.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 761f0000 000ac000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.2.10240.16412) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74290000 0001e000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 13716 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

- Registers -
eax=1132f9c0 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=74383e28 esp=1132f9c0 ebp=1132fa18
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!+0x0 
1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad0 00408130 1132fae4 000e340c 1132fae4 000e3391 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff80 76083744 7ffde000 76083720 187464b4 1132ffdc hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 771d9cd4 7ffde000 01afddb2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
1132ffdc 771d9c9f ffffffff 771fd4c0 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
1132ffec 00000000 007bde7f 7ffde000 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 22100 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=000e7368 ecx=00000000 edx=00000000 esi=000e71a8 edi=000e3c48
eip=771ea45c esp=2172fdcc ebp=2172ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
2172ff80 76083744 000e3c48 76083720 283464b4 2172ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
2172ff94 771d9cd4 000e3c48 31efddb2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
2172ffdc 771d9c9f ffffffff 771fd4c0 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
2172ffec 00000000 771b43e0 000e3c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 11944 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=000e7db0 ecx=00000000 edx=00000000 esi=000e7bf0 edi=000e3c48
eip=771ea45c esp=31b2fdcc ebp=31b2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
31b2ff80 76083744 000e3c48 76083720 38f464b4 31b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
31b2ff94 771d9cd4 000e3c48 212fddb2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
31b2ffdc 771d9c9f ffffffff 771fd4c0 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
31b2ffec 00000000 771b43e0 000e3c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 14732 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=000e98a0 ecx=00000000 edx=00000000 esi=000e96e0 edi=000e3c48
eip=771ea45c esp=41f2fdcc ebp=41f2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
41f2ff80 76083744 000e3c48 76083720 48b464b4 41f2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
41f2ff94 771d9cd4 000e3c48 516fddb2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
41f2ffdc 771d9c9f ffffffff 771fd4c0 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
41f2ffec 00000000 771b43e0 000e3c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1
Model crash detected, will try to restart...
14:55:21 (17900): start_timer_thread(): CreateThread() failed, errno 0


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 12/24/15 14:55:21
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_xcds_198012_480_010221098;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 77180000 00179000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 76070000 000f0000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742d0000 00176000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 744a0000 00140000 C:\WINDOWS\SYSTEM32\USER32.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 765c0000 0014d000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.2.10240.16390) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.10240.16390 (th1_st1.150714-1601)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16390

ModLoad: 77100000 0007b000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 746a0000 000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.10240.16384

ModLoad: 76ba0000 00043000 C:\WINDOWS\SYSTEM32\sechost.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 761f0000 000ac000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.2.10240.16412) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74290000 0001e000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74280000 0000a000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74220000 00059000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742b0000 00019000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 67250000 000e0000 C:\WINDOWS\SYSTEM32\OPENGL32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 68810000 00025000 C:\WINDOWS\SYSTEM32\GLU32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 66bf0000 000eb000 C:\WINDOWS\SYSTEM32\DDRAW.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 739e0000 00007000 C:\WINDOWS\SYSTEM32\DCIMAN32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 732c0000 0013f000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 73800000 00008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 40772 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

- Registers -
eax=1132f9b8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=74383e28 esp=1132f9b8 ebp=1132fa14
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad0 00408130 1132fae4 002f340c 1132fae4 002f3391 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff80 76083744 7ffde000 76083720 8f6b1ea2 1132ffdc hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 771d9cd4 7ffde000 23db23b2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
1132ffdc 771d9c9f ffffffff 771fd4d7 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
1132ffec 00000000 007bde7f 7ffde000 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 616 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=002f4358 ecx=00000000 edx=00000000 esi=002f4198 edi=002f3c48
eip=771ea45c esp=2172fdcc ebp=2172ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
2172ff80 76083744 002f3c48 76083720 bf2b1ea2 2172ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
2172ff94 771d9cd4 002f3c48 139b23b2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
2172ffdc 771d9c9f ffffffff 771fd4d7 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
2172ffec 00000000 771b43e0 002f3c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 18520 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=002f77d0 ecx=00000000 edx=00000000 esi=002f7610 edi=002f3c48
eip=771ea45c esp=31b2fdcc ebp=31b2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
31b2ff80 76083744 002f3c48 76083720 afeb1ea2 31b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
31b2ff94 771d9cd4 002f3c48 035b23b2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
31b2ffdc 771d9c9f ffffffff 771fd4d7 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
31b2ffec 00000000 771b43e0 002f3c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 17168 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=002fa608 ecx=00000000 edx=00000000 esi=002fa448 edi=002f3c48
eip=771ea45c esp=41f2fdcc ebp=41f2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
41f2ff80 76083744 002f3c48 76083720 dfab1ea2 41f2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
41f2ff94 771d9cd4 002f3c48 731b23b2 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
41f2ffdc 771d9c9f ffffffff 771fd4d7 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
41f2ffec 00000000 771b43e0 002f3c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1
Model crash detected, will try to restart...
14:55:30 (28084): start_timer_thread(): CreateThread() failed, errno 0


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 12/24/15 14:55:30
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_xcds_198012_480_010221098;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 77180000 00179000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 76070000 000f0000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742d0000 00176000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 744a0000 00140000 C:\WINDOWS\SYSTEM32\USER32.dll (6.2.10240.16603) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 765c0000 0014d000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.2.10240.16390) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 10.0.10240.16390 (th1_st1.150714-1601)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16390

ModLoad: 77100000 0007b000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 746a0000 000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.10240.16384

ModLoad: 76ba0000 00043000 C:\WINDOWS\SYSTEM32\sechost.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 761f0000 000ac000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.2.10240.16412) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74290000 0001e000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74280000 0000a000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 74220000 00059000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : bcryptprimitives.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 742b0000 00019000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.2.10240.16384) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 68810000 00025000 C:\WINDOWS\SYSTEM32\GLU32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 67250000 000e0000 C:\WINDOWS\SYSTEM32\OPENGL32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384

ModLoad: 66bf0000 000eb000 C:\WINDOWS\SYSTEM32\DDRAW.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 10.0.10240.16384 (th1.150709-1700)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 10.0.10240.16384



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 15780 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x74383E28

- Registers -
eax=1132f9c0 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=74383e28 esp=1132f9c0 ebp=1132fa1c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000216

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa1c 007bfa90 e06d7363 00000001 00000003 1132fa48 KERNELBASE!RaiseException+0x0 
1132fa54 007bbbcb 1132fa64 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa70 00406e6e 00500000 004083ee 00000000 42430578 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad0 00408130 1132fae4 0018340c 1132fae4 00183391 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff80 76083744 7ffde000 76083720 4dc55a7e 1132ffdc hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 771d9cd4 7ffde000 7109e1e0 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
1132ffdc 771d9c9f ffffffff 771fd4bc 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
1132ffec 00000000 007bde7f 7ffde000 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 40548 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00187368 ecx=00000000 edx=00000000 esi=001871a8 edi=00183c48
eip=771ea45c esp=2172fdcc ebp=2172ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
2172ff80 76083744 00183c48 76083720 7d855a7e 2172ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
2172ff94 771d9cd4 00183c48 4149e1e0 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
2172ffdc 771d9c9f ffffffff 771fd4bc 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
2172ffec 00000000 771b43e0 00183c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 35816 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00187db0 ecx=00000000 edx=00000000 esi=00187bf0 edi=00183c48
eip=771ea45c esp=31b2fdcc ebp=31b2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
31b2ff80 76083744 00183c48 76083720 6d455a7e 31b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
31b2ff94 771d9cd4 00183c48 5189e1e0 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
31b2ffdc 771d9c9f ffffffff 771fd4bc 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
31b2ffec 00000000 771b43e0 00183c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 

*** Dump of thread ID 22576 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00189778 ecx=00000000 edx=00000000 esi=001895b8 edi=00183c48
eip=771ea45c esp=41f2fdcc ebp=41f2ff80
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206

- Callstack -
ChildEBP RetAddr  Args to Child
41f2ff80 76083744 00183c48 76083720 1d055a7e 41f2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 
41f2ff94 771d9cd4 00183c48 21c9e1e0 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 
41f2ffdc 771d9c9f ffffffff 771fd4bc 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 
41f2ffec 00000000 771b43e0 00183c48 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1
Model crash detected, will try to restart...
14:55:41 (5664): No heartbeat from core client for 30 sec - exiting
14:55:42 (5664): No heartbeat from core client for 30 sec - exiting
14:55:43 (5664): No heartbeat from core client for 30 sec - exiting
14:55:44 (5664): No heartbeat from core client for 30 sec - exiting
14:55:45 (5664): No heartbeat from core client for 30 sec - exiting
14:55:46 (5664): No heartbeat from core client for 30 sec - exiting
14:55:47 (5664): No heartbeat from core client for 30 sec - exiting
14:55:48 (5664): No heartbeat from core client for 30 sec - exiting
14:55:49 (5664): No heartbeat from core client for 30 sec - exiting
14:55:50 (5664): No heartbeat from core client for 30 sec - exiting
14:55:51 (5664): No heartbeat from core client for 30 sec - exiting
14:55:52 (5664): No heartbeat from core client for 30 sec - exiting
14:55:53 (5664): No heartbeat from core client for 30 sec - exiting
14:55:54 (5664): No heartbeat from core client for 30 sec - exiting
14:55:56 (5664): No heartbeat from core client for 30 sec - exiting
14:55:57 (5664): No heartbeat from core client for 30 sec - exiting
14:55:58 (5664): No heartbeat from core client for 30 sec - exiting
14:55:59 (5664): No heartbeat from core client for 30 sec - exiting
14:56:00 (5664): No heartbeat from core client for 30 sec - exiting
14:56:01 (5664): No heartbeat from core client for 30 sec - exiting
14:56:02 (5664): No heartbeat from core client for 30 sec - exiting
14:56:03 (5664): No heartbeat from core client for 30 sec - exiting
14:56:04 (5664): No heartbeat from core client for 30 sec - exiting
14:56:05 (5664): No heartbeat from core client for 30 sec - exiting
14:56:06 (5664): No heartbeat from core client for 30 sec - exiting
14:56:07 (5664): No heartbeat from core client for 30 sec - exiting
14:56:08 (5664): No heartbeat from core client for 30 sec - exiting
14:56:09 (5664): No heartbeat from core client for 30 sec - exiting
14:56:10 (5664): No heartbeat from core client for 30 sec - exiting
14:56:11 (5664): No heartbeat from core client for 30 sec - exiting
14:56:12 (5664): No heartbeat from core client for 30 sec - exiting
14:56:13 (5664): No heartbeat from core client for 30 sec - exiting
14:56:14 (5664): No heartbeat from core client for 30 sec - exiting
14:56:15 (5664): No heartbeat from core client for 30 sec - exiting
14:56:16 (5664): No heartbeat from core client for 30 sec - exiting
14:56:16 (39840): start_timer_thread(): CreateThread() failed, errno 0
CPDN Monitor - No 'heartbeat' from BOINC...
14:57:08 (42488): start_timer_thread(): CreateThread() failed, errno 0
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26392, iMonCtr=1
Model crash detected, will try to restart...
14:57:18 (19336): start_timer_thread(): CreateThread() failed, errno 0
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26392, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
24 Dec 2015 11:51:28 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 725,760 812,779 1.1199
24 Dec 2015 01:58:47 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 699,840 777,231 1.1106
23 Dec 2015 15:57:05 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 673,920 741,423 1.1002
23 Dec 2015 05:59:09 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 648,000 705,552 1.0888
22 Dec 2015 19:50:36 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 622,080 669,347 1.0760
22 Dec 2015 09:48:44 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 596,160 633,295 1.0623
21 Dec 2015 23:56:46 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 570,240 598,160 1.0490
21 Dec 2015 13:54:36 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 544,320 562,083 1.0326
21 Dec 2015 03:49:44 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 518,400 526,046 1.0147
20 Dec 2015 17:50:03 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 492,480 490,033 0.9950
20 Dec 2015 07:47:17 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 466,560 454,017 0.9731
19 Dec 2015 21:44:29 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 440,640 418,092 0.9488
19 Dec 2015 11:42:40 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 414,720 382,116 0.9214
19 Dec 2015 01:41:06 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 388,800 346,219 0.8905
18 Dec 2015 15:44:19 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 362,880 310,302 0.8551
18 Dec 2015 05:41:36 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 336,960 274,313 0.8141
17 Dec 2015 19:37:38 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 311,040 238,204 0.7658
17 Dec 2015 09:25:01 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 285,120 213,011 0.7471
16 Dec 2015 20:07:20 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 259,200 206,015 0.7948
16 Dec 2015 09:34:09 1382994 19144374 hadcm3n_xcds_198012_480_010221098_1 233,280 222,633 0.9544


©2024 climateprediction.net