Task 13534855

Name	hadcm3n_yk0w_1900_40_007514595_0
Workunit	7712070
Created	28 Oct 2011, 12:37:37 UTC
Sent	24 Nov 2011, 20:31:44 UTC
Report deadline	24 Feb 2012, 3:58:55 UTC
Received	16 Apr 2012, 12:10:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1053862
Run time	5 days 7 hours 48 min 44 sec
CPU time	4 days 20 hours 43 min 57 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> ols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 763b0000 000aa000 C:\Windows\system32\msvcrt.dll (7.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 7.0.6002.18005 ModLoad: 71ab0000 00023000 C:\Windows\system32\GLU32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 600c0000 000e5000 C:\Windows\system32\DDRAW.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 6f170000 00006000 C:\Windows\system32\DCIMAN32.dll (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18051 ModLoad: 77430000 0018a000 C:\Windows\system32\SETUPAPI.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 Get Product Name Failed. ModLoad: 761a0000 0008d000 C:\Windows\system32\OLEAUT32.dll (6.0.6002.18508) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.0.6002.18508 Company Name : Microsoft Corporation Product Name : Product Version : 6.0.6002.18508 ModLoad: 76460000 00145000 C:\Windows\system32\ole32.dll (6.0.6002.18277) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 750a0000 0000c000 C:\Windows\system32\dwmapi.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 75ec0000 00029000 C:\Windows\system32\imagehlp.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 76240000 0001e000 C:\Windows\system32\IMM32.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 75df0000 000c8000 C:\Windows\system32\MSCTF.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 75db0000 00009000 C:\Windows\system32\LPK.DLL (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18051 ModLoad: 773b0000 0007d000 C:\Windows\system32\USP10.dll (1.626.6002.18244) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.6002.18244 (vistasp2_gdr.100416-0345) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.6002.18244 ModLoad: 62970000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 65b90000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 65fc0000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 75090000 00008000 C:\Windows\system32\version.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 263, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 127792, QuotaPeakPagedPoolUsage: 127792 QuotaNonPagedPoolUsage: 2920, QuotaPeakNonPagedPoolUsage: 2920 - Virtual Memory Usage - VirtualSize: 889987072, PeakVirtualSize: 889987072 - Pagefile Usage - PagefileUsage: 16941056, PeakPagefileUsage: 33525760 - Working Set Size - WorkingSetSize: 4096000, PeakWorkingSetSize: 4096000, PageFaultCount: 1003 * Dump of thread ID 6728 (state: Ready): * - Information - Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 156001.000000, User Time: 0.000000, Wait Time: 1279209.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7719FC56 - Registers - eax=1132f9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=7719fc56 esp=1132f9c8 ebp=1132fa18 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 kernel32!RaiseException+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 000b2539 1132faec 000b24c1 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 771ad309 7ffdf000 1132ffd4 777216c3 7ffdf000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 777216c3 7ffdf000 6778d1ea 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 77721696 007bde7f 7ffdf000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7ffdf000 00000000 00000030 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 7776 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1279208.000000 - Registers - eax=0000003c ebx=00000000 ecx=00000069 edx=00000001 esi=340fff48 edi=00000000 eip=77745ca4 esp=340fff04 ebp=340fff6c cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 340fff6c 77161c6c 00000064 00000000 340fff88 00749924 ntdll!KiFastSystemCallRet+0x0 340fff7c 00749924 00000064 340fff94 771ad309 00000000 kernel32!Sleep+0x0 340fff88 771ad309 00000000 340fffd4 777216c3 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 340fff94 777216c3 00000000 4245d1ea 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 340fffd4 77721696 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 340fffec 00000000 00749910 00000000 00000000 126d00a0 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7120, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5500, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5500, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5500, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5500, iMonCtr=1 Model crash detected, will try to restart... 05:59:48 (5500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:38:07 (5612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x772FFC56 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 12/20/11 23:29:56 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_yk0w_1900_40_007514595;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 773a0000 00128000 C:\Windows\system32\ntdll.dll (6.0.6002.18327) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 772c0000 000dc000 C:\Windows\system32\kernel32.dll (6.0.6002.18449) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 75e40000 0009d000 C:\Windows\system32\USER32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 75d70000 0004b000 C:\Windows\system32\GDI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 766c0000 000c6000 C:\Windows\system32\ADVAPI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 76170000 000c3000 C:\Windows\system32\RPCRT4.dll (6.0.6002.18024) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 6aab0000 000cb000 C:\Windows\system32\OPENGL32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 75ee0000 000aa000 C:\Windows\system32\msvcrt.dll (7.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 7.0.6002.18005 ModLoad: 6b1d0000 00023000 C:\Windows\system32\GLU32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 6a8c0000 000e5000 C:\Windows\system32\DDRAW.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 70840000 00006000 C:\Windows\system32\DCIMAN32.dll (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18051 ModLoad: 763b0000 0018a000 C:\Windows\system32\SETUPAPI.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 Get Product Name Failed. ModLoad: 75a70000 0008d000 C:\Windows\system32\OLEAUT32.dll (6.0.6002.18508) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.0.6002.18508 Company Name : Microsoft Corporation Product Name : Product Version : 6.0.6002.18508 ModLoad: 76560000 00145000 C:\Windows\system32\ole32.dll (6.0.6002.18277) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 74d50000 0000c000 C:\Windows\system32\dwmapi.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 75d40000 00029000 C:\Windows\system32\imagehlp.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 76540000 0001e000 C:\Windows\system32\IMM32.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 774d0000 000c8000 C:\Windows\system32\MSCTF.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 75a60000 00009000 C:\Windows\system32\LPK.DLL (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18051 ModLoad: 75cc0000 0007d000 C:\Windows\system32\USP10.dll (1.626.6002.18244) (-exported- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.6002.18244 (vistasp2_gdr.100416-0345) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.6002.18244 ModLoad: 6a680000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 6af70000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 70790000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 74d40000 00008000 C:\Windows\system32\version.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 337, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 127792, QuotaPeakPagedPoolUsage: 127792 QuotaNonPagedPoolUsage: 2976, QuotaPeakNonPagedPoolUsage: 2976 - Virtual Memory Usage - VirtualSize: 906768384, PeakVirtualSize: 906768384 - Pagefile Usage - PagefileUsage: 33755136, PeakPagefileUsage: 107069440 - Working Set Size - WorkingSetSize: 4112384, PeakWorkingSetSize: 4112384, PageFaultCount: 1007 * Dump of thread ID 7780 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 468003.000000, User Time: 0.000000, Wait Time: 67886.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x772FFC56 - Registers - eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=772ffc56 esp=1132f9c4 ebp=1132fa14 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 kernel32!RaiseException+0x0 1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 000c2539 1132faec 000c24c1 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 7730d309 7ffdf000 1132ffd4 773e16c3 7ffdf000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 773e16c3 7ffdf000 66648662 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 773e1696 007bde7f 7ffdf000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7ffdf000 00000000 00000030 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 7788 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 67884.000000 - Registers - eax=00072423 ebx=00000000 ecx=00000000 edx=00000000 esi=3421ff48 edi=00000000 eip=77405ca4 esp=3421ff04 ebp=3421ff6c cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3421ff6c 772c1c6c 00000064 00000000 3421ff88 00749924 ntdll!KiFastSystemCallRet+0x0 3421ff7c 00749924 00000064 3421ff94 7730d309 00000000 kernel32!Sleep+0x0 3421ff88 7730d309 00000000 3421ffd4 773e16c3 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3421ff94 773e16c3 00000000 43778662 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 3421ffd4 773e1696 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3421ffec 00000000 00749910 00000000 00000000 34730000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7620, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x772FFC56 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 12/20/11 23:30:36 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_yk0w_1900_40_007514595;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 773a0000 00128000 C:\Windows\system32\ntdll.dll (6.0.6002.18327) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 772c0000 000dc000 C:\Windows\system32\kernel32.dll (6.0.6002.18449) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 6.0.6002.18449 (vistasp2_gdr.110412-0338) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18449 ModLoad: 75e40000 0009d000 C:\Windows\system32\USER32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 75d70000 0004b000 C:\Windows\system32\GDI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 766c0000 000c6000 C:\Windows\system32\ADVAPI32.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 76170000 000c3000 C:\Windows\system32\RPCRT4.dll (6.0.6002.18024) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 6aab0000 000cb000 C:\Windows\system32\OPENGL32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 75ee0000 000aa000 C:\Windows\system32\msvcrt.dll (7.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 7.0.6002.18005 ModLoad: 6b1d0000 00023000 C:\Windows\system32\GLU32.dll (6.0.6000.16386) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 6a8c0000 000e5000 C:\Windows\system32\DDRAW.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 70840000 00006000 C:\Windows\system32\DCIMAN32.dll (6.0.6002.18051) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18051 ModLoad: 763b0000 0018a000 C:\Windows\system32\SETUPAPI.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : setupapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 Get Product Name Failed. ModLoad: 75a70000 0008d000 C:\Windows\system32\OLEAUT32.dll (6.0.6002.18508) (-exported- Symbols Loaded) Linked PDB Filename : oleaut32.pdb File Version : 6.0.6002.18508 Company Name : Microsoft Corporation Product Name : Product Version : 6.0.6002.18508 ModLoad: 76560000 00145000 C:\Windows\system32\ole32.dll (6.0.6002.18277) (-exported- Symbols Loaded) Linked PDB Filename : ole32.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 74d50000 0000c000 C:\Windows\system32\dwmapi.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : dwmapi.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 75d40000 00029000 C:\Windows\system32\imagehlp.dll (6.0.6001.18000) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 6.0.6001.18000 (longhorn_rtm.080118-1840) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6001.18000 ModLoad: 76540000 0001e000 C:\Windows\system32\IMM32.DLL (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 ModLoad: 774d0000 000c8000 C:\Windows\system32\MSCTF.dll (6.0.6002.18005) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.0.6000.16386 (vista_rtm.061101-2205) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6000.16386 ModLoad: 75a60000 00009000 C:\Windows\system32\LPK.DLL (6.0.6002.18051) (-nosymbols- Symbols Loaded) Linked PDB Filename : lpk.pdb File Version : 6.0.6002.18051 (vistasp2_gdr.090615-0258) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18051 ModLoad: 75cc0000 0007d000 C:\Windows\system32\USP10.dll (1.626.6002.18244) (-nosymbols- Symbols Loaded) Linked PDB Filename : usp10.pdb File Version : 1.0626.6002.18244 (vistasp2_gdr.100416-0345) Company Name : Microsoft Corporation Product Name : Microsoft(R) Uniscribe Unicode script processor Product Version : 1.0626.6002.18244 ModLoad: 6a680000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 6af70000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 73f00000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 74d40000 00008000 C:\Windows\system32\version.dll (6.0.6002.18005) (-nosymbols- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.0.6002.18005 (lh_sp2rtm.090410-1830) Company Name : Microsoft Corporation Product Name : MicrosoftR WindowsR Operating System Product Version : 6.0.6002.18005 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 119, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 127792, QuotaPeakPagedPoolUsage: 127792 QuotaNonPagedPoolUsage: 2976, QuotaPeakNonPagedPoolUsage: 2976 - Virtual Memory Usage - VirtualSize: 906768384, PeakVirtualSize: 906768384 - Pagefile Usage - PagefileUsage: 33763328, PeakPagefileUsage: 107077632 - Working Set Size - WorkingSetSize: 4079616, PeakWorkingSetSize: 4079616, PageFaultCount: 1000 * Dump of thread ID 8172 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156001.000000, User Time: 0.000000, Wait Time: 70438.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x772FFC56 - Registers - eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=772ffc56 esp=1132f9c4 ebp=1132fa14 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 kernel32!RaiseException+0x0 1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad8 00408130 1132faec 00232539 1132faec 002324c1 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff88 7730d309 7ffdf000 1132ffd4 773e16c3 7ffdf000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 773e16c3 7ffdf000 66646e84 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 1132ffd4 773e1696 007bde7f 7ffdf000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 1132ffec 00000000 007bde7f 7ffdf000 00000000 00000030 ntdll!RtlInitializeExceptionChain+0x0 * Dump of thread ID 8180 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 70438.000000 - Registers - eax=00026161 ebx=00000000 ecx=00000000 edx=00000000 esi=3436ff48 edi=00000000 eip=77405ca4 esp=3436ff04 ebp=3436ff6c cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 3436ff6c 772c1c6c 00000064 00000000 3436ff88 00749924 ntdll!KiFastSystemCallRet+0x0 3436ff7c 00749924 00000064 3436ff94 7730d309 00000000 kernel32!Sleep+0x0 3436ff88 7730d309 00000000 3436ffd4 773e16c3 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3436ff94 773e16c3 00000000 43606e84 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 3436ffd4 773e1696 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 3436ffec 00000000 00749910 00000000 00000000 34880000 ntdll!RtlInitializeExceptionChain+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7620, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x772FFC56 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 12/20/11 23:31:15 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 1114 LoadLibraryA( dbghelp.dll ): GetLastError = 1114 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 5, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 94, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 127792, QuotaPeakPagedPoolUsage: 127792 QuotaNonPagedPoolUsage: 2976, QuotaPeakNonPagedPoolUsage: 2976 - Virtual Memory Usage - VirtualSize: 906768384, PeakVirtualSize: 906768384 - Pagefile Usage - PagefileUsage: 33755136, PeakPagefileUsage: 107057152 - Working Set Size - WorkingSetSize: 4083712, PeakWorkingSetSize: 4083712, PageFaultCount: 1000 * Dump of thread ID 6088 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 156001.000000, Wait Time: 72965.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x772FFC56 * Dump of thread ID 4884 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 72965.000000 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7620, iMonCtr=1 Model crash detected, will try to restart... 23:31:31 (7620): No heartbeat from core client for 30 sec - exiting 23:31:32 (7620): No heartbeat from core client for 30 sec - exiting 23:31:33 (7620): No heartbeat from core client for 30 sec - exiting 23:31:34 (7620): No heartbeat from core client for 30 sec - exiting 23:31:35 (7620): No heartbeat from core client for 30 sec - exiting 23:31:36 (7620): No heartbeat from core client for 30 sec - exiting 23:31:37 (7620): No heartbeat from core client for 30 sec - exiting 23:31:38 (7620): No heartbeat from core client for 30 sec - exiting 23:31:39 (7620): No heartbeat from core client for 30 sec - exiting 23:31:40 (7620): No heartbeat from core client for 30 sec - exiting 23:31:41 (7620): No heartbeat from core client for 30 sec - exiting 23:31:42 (7620): No heartbeat from core client for 30 sec - exiting 23:31:43 (7620): No heartbeat from core client for 30 sec - exiting 23:31:44 (7620): No heartbeat from core client for 30 sec - exiting 23:31:45 (7620): No heartbeat from core client for 30 sec - exiting 23:31:46 (7620): No heartbeat from core client for 30 sec - exiting 23:31:47 (7620): No heartbeat from core client for 30 sec - exiting 23:31:48 (7620): No heartbeat from core client for 30 sec - exiting 23:31:49 (7620): No heartbeat from core client for 30 sec - exiting 23:31:50 (7620): No heartbeat from core client for 30 sec - exiting 23:31:51 (7620): No heartbeat from core client for 30 sec - exiting 23:31:52 (7620): No heartbeat from core client for 30 sec - exiting 23:31:53 (7620): No heartbeat from core client for 30 sec - exiting 23:31:54 (7620): No heartbeat from core client for 30 sec - exiting 23:31:55 (7620): No heartbeat from core client for 30 sec - exiting 23:31:56 (7620): No heartbeat from core client for 30 sec - exiting 23:31:57 (7620): No heartbeat from core client for 30 sec - exiting 23:31:58 (7620): No heartbeat from core client for 30 sec - exiting 23:31:59 (7620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:55:36 (6064): No heartbeat from core client for 30 sec - exiting 21:55:37 (6064): No heartbeat from core client for 30 sec - exiting 21:55:38 (6064): No heartbeat from core client for 30 sec - exiting 21:55:39 (6064): No heartbeat from core client for 30 sec - exiting 21:55:40 (6064): No heartbeat from core client for 30 sec - exiting 21:55:41 (6064): No heartbeat from core client for 30 sec - exiting 21:55:42 (6064): No heartbeat from core client for 30 sec - exiting 21:55:43 (6064): No heartbeat from core client for 30 sec - exiting 21:55:44 (6064): No heartbeat from core client for 30 sec - exiting 21:55:45 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:58:59 (4748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:47:48 (5796): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:18:32 (5508): Can't acquire lockfile (32) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: ANZXªÛ³êÜµ½B CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7740, selfPID=7740, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:13:32 (5112): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:59:06 (3392): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:26:19 (4636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:53:13 (5524): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:51:22 (5204): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:50:05 (4800): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:18:22 (5028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:02:35 (5660): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... 22:43:07 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:14:14 (6048): No heartbeat from core client for 30 sec - exiting 22:14:15 (6048): No heartbeat from core client for 30 sec - exiting 22:14:16 (6048): No heartbeat from core client for 30 sec - exiting 22:14:17 (6048): No heartbeat from core client for 30 sec - exiting 22:14:18 (6048): No heartbeat from core client for 30 sec - exiting 22:14:19 (6048): No heartbeat from core client for 30 sec - exiting 22:14:20 (6048): No heartbeat from core client for 30 sec - exiting 22:14:21 (6048): No heartbeat from core client for 30 sec - exiting 22:14:22 (6048): No heartbeat from core client for 30 sec - exiting 22:14:23 (6048): No heartbeat from core client for 30 sec - exiting 22:14:24 (6048): No heartbeat from core client for 30 sec - exiting 22:14:25 (6048): No heartbeat from core client for 30 sec - exiting 22:14:26 (6048): No heartbeat from core client for 30 sec - exiting 22:14:27 (6048): No heartbeat from core client for 30 sec - exiting 22:14:28 (6048): No heartbeat from core client for 30 sec - exiting 22:14:29 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:05:27 (528): No heartbeat from core client for 30 sec - exiting 23:05:28 (528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:56:09 (5204): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:24:01 (5076): No heartbeat from core client for 30 sec - exiting 01:24:02 (5076): No heartbeat from core client for 30 sec - exiting 01:24:03 (5076): No heartbeat from core client for 30 sec - exiting 01:24:04 (5076): No heartbeat from core client for 30 sec - exiting 01:24:06 (5076): No heartbeat from core client for 30 sec - exiting 01:24:07 (5076): No heartbeat from core client for 30 sec - exiting 01:24:08 (5076): No heartbeat from core client for 30 sec - exiting 01:24:09 (5076): No heartbeat from core client for 30 sec - exiting 01:24:10 (5076): No heartbeat from core client for 30 sec - exiting 01:24:11 (5076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:46:02 (3864): No heartbeat from core client for 30 sec - exiting 23:46:03 (3864): No heartbeat from core client for 30 sec - exiting 23:46:04 (3864): No heartbeat from core client for 30 sec - exiting 23:46:05 (3864): No heartbeat from core client for 30 sec - exiting 23:46:06 (3864): No heartbeat from core client for 30 sec - exiting 23:46:07 (3864): No heartbeat from core client for 30 sec - exiting 23:46:08 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:18:00 (4652): No heartbeat from core client for 30 sec - exiting 21:18:01 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:28:23 (708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:24 (708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 01:39:48 (5064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:37:04 (2748): No heartbeat from core client for 30 sec - exiting 13:37:05 (2748): No heartbeat from core client for 30 sec - exiting 13:37:06 (2748): No heartbeat from core client for 30 sec - exiting 13:37:07 (2748): No heartbeat from core client for 30 sec - exiting 13:37:08 (2748): No heartbeat from core client for 30 sec - exiting 13:37:09 (2748): No heartbeat from core client for 30 sec - exiting 13:37:10 (2748): No heartbeat from core client for 30 sec - exiting 13:37:11 (2748): No heartbeat from core client for 30 sec - exiting 13:37:12 (2748): No heartbeat from core client for 30 sec - exiting 13:37:13 (2748): No heartbeat from core client for 30 sec - exiting 13:37:15 (2748): No heartbeat from core client for 30 sec - exiting 13:37:16 (2748): No heartbeat from core client for 30 sec - exiting 13:37:17 (2748): No heartbeat from core client for 30 sec - exiting 13:37:18 (2748): No heartbeat from core client for 30 sec - exiting 13:37:19 (2748): No heartbeat from core client for 30 sec - exiting 13:37:20 (2748): No heartbeat from core client for 30 sec - exiting 13:37:21 (2748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:40:08 (4936): No heartbeat from core client for 30 sec - exiting 02:40:09 (4936): No heartbeat from core client for 30 sec - exiting 02:40:10 (4936): No heartbeat from core client for 30 sec - exiting 02:40:11 (4936): No heartbeat from core client for 30 sec - exiting forrtl: ANZXªÛ³êÜµ½B CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4632, selfPID=4632, iMonCtr=1 11:51:03 (5580): No heartbeat from core client for 30 sec - exiting 11:51:04 (5580): No heartbeat from core client for 30 sec - exiting 11:51:05 (5580): No heartbeat from core client for 30 sec - exiting 11:51:06 (5580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:44:59 (3884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:01:41 (5892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:48:05 (6124): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... 01:18:49 (6048): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... 02:51:30 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:37:19 (5972): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:10:36 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:26 (620): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Apr 2012 12:13:12	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	259,200	420,232	1.6213
06 Mar 2012 20:24:19	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	233,280	378,494	1.6225
25 Feb 2012 21:23:28	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	207,360	337,072	1.6255
05 Feb 2012 00:51:32	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	181,440	293,524	1.6177
20 Jan 2012 15:56:31	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	155,520	247,021	1.5884
18 Jan 2012 22:24:17	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	129,600	203,821	1.5727
09 Jan 2012 19:19:08	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	103,680	162,725	1.5695
07 Jan 2012 19:21:09	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	77,760	123,640	1.5900
23 Dec 2011 20:06:52	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	51,840	80,474	1.5524
17 Dec 2011 15:05:18	1053862	13534855	hadcm3n_yk0w_1900_40_007514595_0	25,920	39,409	1.5204