Name | hadcm3n_xcyt_1940_40_009153099_0 |
Workunit | 9283435 |
Created | 6 Nov 2014, 15:50:48 UTC |
Sent | 6 Nov 2014, 19:16:15 UTC |
Report deadline | 6 Feb 2015, 2:43:26 UTC |
Received | 2 Dec 2014, 0:14:10 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | -529697949 (0xE06D7363) Unknown error code |
Computer ID | 1411061 |
Run time | 9 days 18 hours 11 min 37 sec |
CPU time | 7 days 22 hours 5 min 13 sec |
Validate state | Invalid |
Credit | 5,287.68 |
Device peak FLOPS | 2.55 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
Stderr | <core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code -529697949 (0xe06d7363) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:03:53 (21388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:10:42 (6624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:15:59 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5980, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5980, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5980, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:57:26 (448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:29:57 (1284): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:11:20 (20848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:51:01 (6188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:56:39 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:37 (6752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:32:28 (7260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:54:22 (6812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x778C2F71 Engaging BOINC Windows Runtime Debugger... ******************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 12/02/14 10:44:16 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_xcyt_1940_40_009153099;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00111000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77aa0000 00167000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 75840000 00140000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.3.9600.17056) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 778b0000 000d0000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.3.9600.17278) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 764e0000 0014c000 C:\WINDOWS\SYSTEM32\USER32.dll (6.3.9600.17344) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 75bb0000 00078000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 75730000 00107000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.3.9600.17246) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 6.3.9600.17246 (winblue_gdr.140801-1518) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17246 ModLoad: 76230000 000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.9600.16384 ModLoad: 762f0000 0003e000 C:\WINDOWS\SYSTEM32\sechost.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 77800000 000b0000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.3.9600.17216) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 754b0000 0001d000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.3.9600.16408) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 6.3.9600.16408 (winblue_gdr.130920-1803) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16408 ModLoad: 754a0000 00009000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 75440000 00052000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.3.9600.17120) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 6.3.9600.17120 (winblue_gdr.140509-1516) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17120 ModLoad: 76330000 00025000 C:\WINDOWS\system32\IMM32.DLL (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 6.3.9600.17031 (winblue_gdr.140221-1952) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.17031 ModLoad: 759d0000 000f7000 C:\WINDOWS\SYSTEM32\MSCTF.dll (6.3.9600.17031) (-exported- Symbols Loaded) Linked PDB Filename : msctf.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 75380000 00025000 C:\WINDOWS\SYSTEM32\ntmarta.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : ntmarta.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 ModLoad: 71420000 000a4000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_se_6.07_windows_intelx86.dll (-exported- Symbols Loaded) Linked PDB Filename : d:\cpdn\cpdnboinc\hadcm3\test\projects\reqs.comlab.ox.ac.uk_cpdn\hadcm3n_se_6.07_windows_intelx86.pdb ModLoad: 70730000 00148000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.3.9600.16520) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.3.9600.16520 (winblue_gdr.140127-0329) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16520 ModLoad: 75430000 00008000 C:\WINDOWS\SYSTEM32\version.dll (6.3.9600.16384) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 6.3.9600.16384 (winblue_rtm.130821-1623) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 6.3.9600.16384 *** Dump of the Process Statistics: *** - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 *** Dump of thread ID 7748 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x778C2F71 - Registers - eax=0018f220 ebx=02b80048 ecx=00000003 edx=00000000 esi=714ba484 edi=013afa00 eip=778c2f71 esp=0018f220 ebp=0018f27c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000212 - Callstack - ChildEBP RetAddr Args to Child 0018f27c 7147e730 e06d7363 00000001 00000003 0018f2a8 KERNELBASE!RaiseException+0x0 0018f2b4 7147bcdc 0018f2c4 714b1fe0 714a13f4 714ab230 hadcm3n_se_6.07_windows_intelx8!umtonc+0x0 0018f2d0 7142fa97 04ebe800 00000010 02d73c28 00000000 hadcm3n_se_6.07_windows_intelx8!umtonc+0x0 0018f2e8 714236b8 4bd7fba5 004bb848 004bb710 0018f6e8 hadcm3n_se_6.07_windows_intelx8!+0x0 00000000 00000000 00000000 00000000 00000000 00000000 hadcm3n_se_6.07_windows_intelx8!+0x0 *** Dump of thread ID 7140 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=c000000f ebx=01ffff4c ecx=00000000 edx=00000000 esi=00000000 edi=01ffff4c eip=77ada83c esp=01ffff0c ebp=01ffff70 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 01ffff70 778b103a 00000064 00000000 01ffff8c 00413da4 ntdll!ZwDelayExecution+0x0 01ffff80 00413da4 00000064 01ffff98 7585919f 00000000 KERNELBASE!Sleep+0x0 01ffff8c 7585919f 00000000 01ffffdc 77af0bbb 00000000 hadcm3n_6.07_windows_intelx86!+0x0 01ffff98 77af0bbb 00000000 5adf4a2c 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 01ffffdc 77af0b91 ffffffff 77adc9ce 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 01ffffec 00000000 00413d90 00000000 00000000 63646168 ntdll!RtlInitializeExceptionChain+0x0 *** Debug Message Dump **** *** Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
01 Dec 2014 22:17:08 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 440,640 | 645,928 | 1.4659 |
30 Nov 2014 23:47:43 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 414,720 | 609,408 | 1.4694 |
30 Nov 2014 21:48:50 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 388,800 | 575,172 | 1.4794 |
29 Nov 2014 01:01:45 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 362,880 | 536,244 | 1.4777 |
27 Nov 2014 10:55:17 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 336,960 | 494,740 | 1.4682 |
25 Nov 2014 21:37:11 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 311,040 | 454,429 | 1.4610 |
24 Nov 2014 10:35:33 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 285,120 | 414,989 | 1.4555 |
23 Nov 2014 07:29:31 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 259,200 | 374,645 | 1.4454 |
17 Nov 2014 17:54:49 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 233,280 | 332,970 | 1.4273 |
16 Nov 2014 23:46:18 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 207,360 | 290,238 | 1.3997 |
16 Nov 2014 01:54:49 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 181,440 | 283,953 | 1.5650 |
14 Nov 2014 12:03:15 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 155,520 | 243,803 | 1.5677 |
13 Nov 2014 05:09:30 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 129,600 | 202,834 | 1.5651 |
11 Nov 2014 07:14:40 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 103,680 | 158,505 | 1.5288 |
10 Nov 2014 06:16:13 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 77,760 | 116,439 | 1.4974 |
09 Nov 2014 08:15:26 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 51,840 | 75,825 | 1.4627 |
08 Nov 2014 07:40:22 | 1336781 | 17357738 | hadcm3n_xcyt_1940_40_009153099_0 | 25,920 | 38,454 | 1.4836 |
©2024 climateprediction.net