Name | hadcm3n_ldg0_1940_40_010020778_1 |
Workunit | 10018849 |
Created | 13 Oct 2015, 23:00:16 UTC |
Sent | 13 Oct 2015, 23:00:20 UTC |
Report deadline | 13 Jan 2016, 6:27:31 UTC |
Received | 28 Oct 2015, 19:05:15 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 1255152 |
Run time | 4 days 14 hours 2 min 22 sec |
CPU time | 4 days 13 hours 36 min 55 sec |
Validate state | Invalid |
Credit | 7,153.92 |
Device peak FLOPS | 4.24 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86 |
Stderr | <core_client_version>7.6.9</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:15:57 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:56 (8152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:27:06 (236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 12:56:32 (8492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:32:49 (10852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 16:52:34 (7996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:53:21 (2500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:26:45 (12416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:41:48 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:29:49 (3088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:34:50 (7888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:39:50 (2456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:44:50 (9356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:51:15 (9760): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 18:51:16 (9760): No heartbeat from core client for 30 sec - exiting 19:20:31 (9852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:21:44 (13972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11496, iMonCtr=1 Model crash detected, will try to restart... 19:22:58 (11496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10484, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x766C3E28 Engaging BOINC Windows Runtime Debugger... ******************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 10/27/15 19:24:43 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_ldg0_1940_40_010020778;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 77aa0000 00179000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.10240.16430) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 77310000 000f0000 C:\WINDOWS\SYSTEM32\KERNEL32.DLL (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 76610000 00176000 C:\WINDOWS\SYSTEM32\KERNELBASE.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 774b0000 00140000 C:\WINDOWS\SYSTEM32\USER32.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 76ec0000 0014d000 C:\WINDOWS\SYSTEM32\GDI32.dll (6.2.10240.16390) (-nosymbols- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.10240.16390 (th1_st1.150714-1601) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16390 ModLoad: 771d0000 0007b000 C:\WINDOWS\SYSTEM32\ADVAPI32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 77110000 000be000 C:\WINDOWS\SYSTEM32\msvcrt.dll (7.0.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.10240.16384 ModLoad: 77890000 00043000 C:\WINDOWS\SYSTEM32\sechost.dll (6.2.10240.16384) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 760f0000 000ac000 C:\WINDOWS\SYSTEM32\RPCRT4.dll (6.2.10240.16412) (-nosymbols- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 74bb0000 0001e000 C:\WINDOWS\SYSTEM32\SspiCli.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 74ba0000 0000a000 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 74b40000 00059000 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 74d10000 00019000 C:\WINDOWS\SYSTEM32\imagehlp.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 6fac0000 000e0000 C:\WINDOWS\SYSTEM32\OPENGL32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 6a980000 00025000 C:\WINDOWS\SYSTEM32\GLU32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 72c50000 000eb000 C:\WINDOWS\SYSTEM32\DDRAW.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 72b70000 00007000 C:\WINDOWS\SYSTEM32\DCIMAN32.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 73e90000 0013f000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 ModLoad: 74b30000 00008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.10240.16384) (-nosymbols- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.10240.16384 (th1.150709-1700) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.10240.16384 *** Dump of the Process Statistics: *** - I/O Operations Counters - Read: 0, Write: 0, Other 0 - I/O Transfers Counters - Read: 0, Write: 0, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0 QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0 - Virtual Memory Usage - VirtualSize: 0, PeakVirtualSize: 0 - Pagefile Usage - PagefileUsage: 0, PeakPagefileUsage: 0 - Working Set Size - WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0 *** Dump of thread ID 11484 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x766C3E28 - Registers - eax=1132f9c0 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe eip=766c3e28 esp=1132f9c0 ebp=1132fa18 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 1132fa18 007bfa90 e06d7363 00000001 00000003 1132fa44 KERNELBASE!RaiseException+0x0 1132fa50 007bbbcb 1132fa60 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fa6c 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fad0 00408130 1132fae4 001a34d9 1132fae4 001a3461 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff80 77323744 7ffde000 77323720 58872391 1132ffdc hadcm3n_um_6.07_windows_intelx8!+0x0 1132ff94 77afa064 7ffde000 9f7cc993 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 1132ffdc 77afa02f ffffffff 77b1d7cb 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 1132ffec 00000000 007bde7f 7ffde000 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 *** Dump of thread ID 6468 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=001a7438 ecx=00000000 edx=00000000 esi=001a7278 edi=001a3d18 eip=77b0a74c esp=2172fdcc ebp=2172ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 2172ff80 77323744 001a3d18 77323720 68c72391 2172ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 2172ff94 77afa064 001a3d18 af3cc993 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 2172ffdc 77afa02f ffffffff 77b1d7cb 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 2172ffec 00000000 77ad45b0 001a3d18 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 *** Dump of thread ID 12576 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=001a82e8 ecx=00000000 edx=00000000 esi=001a8128 edi=001a3d18 eip=77b0a74c esp=31b2fdcc ebp=31b2ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 31b2ff80 77323744 001a3d18 77323720 78072391 31b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 31b2ff94 77afa064 001a3d18 bffcc993 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 31b2ffdc 77afa02f ffffffff 77b1d7cb 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 31b2ffec 00000000 77ad45b0 001a3d18 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 *** Dump of thread ID 5928 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=001a8930 ecx=00000000 edx=00000000 esi=001a8770 edi=001a3d18 eip=77b0a74c esp=41f2fdcc ebp=41f2ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 41f2ff80 77323744 001a3d18 77323720 08472391 41f2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 41f2ff94 77afa064 001a3d18 cfbcc993 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 41f2ffdc 77afa02f ffffffff 77b1d7cb 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 41f2ffec 00000000 77ad45b0 001a3d18 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 *** Dump of thread ID 6248 (state: Initialized): *** - Information - Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000 - Registers - eax=00000000 ebx=630fff40 ecx=00000000 edx=00000000 esi=00000000 edi=00749910 eip=77b08f2c esp=630fff00 ebp=630fff64 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000216 - Callstack - ChildEBP RetAddr Args to Child 630fff64 766c131f 00000064 00000000 630fff80 00749924 ntdll!NtDelayExecution+0x0 630fff74 00749924 00000064 630fff94 77323744 00000000 KERNELBASE!Sleep+0x0 630fff80 77323744 00000000 77323720 2aba2391 630fffdc hadcm3n_um_6.07_windows_intelx8!+0x0 630fff94 77afa064 00000000 ed41c993 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 630fffdc 77afa02f ffffffff 77b1d7cb 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 630fffec 00000000 00749910 00000000 00000000 00000000 ntdll!RtlSetCurrentTransaction+0x0 *** Debug Message Dump **** *** Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10484, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10484, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10484, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10484, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 596,160 | 391,125 | 0.6561 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 570,240 | 374,345 | 0.6565 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 544,320 | 357,024 | 0.6559 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 518,400 | 338,981 | 0.6539 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 492,480 | 322,728 | 0.6553 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 466,560 | 306,711 | 0.6574 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 440,640 | 290,557 | 0.6594 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 414,720 | 274,334 | 0.6615 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 388,800 | 258,531 | 0.6649 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 362,880 | 242,541 | 0.6684 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 336,960 | 225,693 | 0.6698 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 311,040 | 208,715 | 0.6710 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 285,120 | 190,879 | 0.6695 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 259,200 | 173,258 | 0.6684 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 233,280 | 155,517 | 0.6667 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 207,360 | 138,947 | 0.6701 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 181,440 | 121,250 | 0.6683 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 155,520 | 103,695 | 0.6668 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 129,600 | 86,778 | 0.6696 |
27 Oct 2015 18:02:28 | 1255152 | 19038324 | hadcm3n_ldg0_1940_40_010020778_1 | 103,680 | 69,345 | 0.6688 |
©2024 climateprediction.net