climateprediction.net home page
Task 11918596

Task 11918596

Name famous_vntr_1399_200_006716645_6
Workunit 6919898
Created 26 Sep 2010, 11:24:58 UTC
Sent 26 Sep 2010, 11:48:12 UTC
Report deadline 26 Dec 2010, 19:15:23 UTC
Received 25 Oct 2010, 23:43:38 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1062556
Run time 3 days 18 hours 56 min 50 sec
CPU time 3 days 4 hours 51 min 16 sec
Validate state Workunit error - check skipped
Credit 6,176.41
Device peak FLOPS 3.76 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-apple-darwin
Stderr
<core_client_version>6.10.21</core_client_version>
<![CDATA[
<stderr_txt>
 (6534): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (7885): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (7993): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (8697): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (9115): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (10295): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (11060): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (11475): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (11987): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (15337): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (21434): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (21614): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (23848): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (26092): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (27526): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (29582): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (30398): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (31926): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (1941): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (2225): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (2757): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (783): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (1004): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (1469): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (6605): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6732): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
 (9284): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (977): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (2094): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (3012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (3114): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (3516): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Tue Oct  5 18:21:02 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd   11  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd37c edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd3b8 esp: 0xbfffd37c
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3871, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Tue Oct  5 18:21:04 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd   11  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd37c edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd3b8 esp: 0xbfffd37c
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3871, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
 (5041): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6514): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (10071): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (925): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (4102): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (6007): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6029): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (2171): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (2972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (3332): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Tue Oct 12 14:37:41 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd3fc edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd438 esp: 0xbfffd3fc
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=216, iMonCtr=1
Model crash detected, will try to restart...
 (216): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (710): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (1030): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (1190): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (5433): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (5611): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6678): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (401): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Thu Oct 14 06:25:50 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd3fc edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd438 esp: 0xbfffd3fc
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1034, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (673): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Fri Oct 15 07:12:19 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd   11  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd38c edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd3c8 esp: 0xbfffd38c
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=585, iMonCtr=1
Model crash detected, will try to restart...
 (585): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (1584): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (4976): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Sun Oct 17 07:45:38 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd   11  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd37c edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd3b8 esp: 0xbfffd37c
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6811, iMonCtr=1
Model crash detected, will try to restart...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
 (9435): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
 (10933): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (13559): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Mon Oct 18 07:13:46 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd   12  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc2ec edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc328 esp: 0xbfffc2ec
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14727, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Mon Oct 18 07:13:49 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd   11  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd37c edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd3b8 esp: 0xbfffd37c
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14727, iMonCtr=1
Model crash detected, will try to restart...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (14727): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Mon Oct 18 21:13:38 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x003f5826    7  0x00450bd8    8  0x004698c0    9  0x000026b6   10  0x000025dd   11  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffd37c edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffd3b8 esp: 0xbfffd37c
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=19419, iMonCtr=1
Model crash detected, will try to restart...
 (19419): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (21058): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (21613): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
 (24653): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (27037): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (34101): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (35971): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: famous_um_6.11_i686-apple-darwin
built using BOINC library version 6.11.1
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.4 build 10F569
Thu Oct 21 16:31:41 2010

sh: /usr/bin/atos: No such file or directory
  0  0x0048fdcf    1  0x0049054a    2  0x92cbd1fb    3  0xffffffff    4  0x0134e6b3    5  0x0134d44f    6  0x002eeac6    7  0x003f55ea    8  0x00452abd    9  0x004698c0   10  0x000026b6   11  0x000025dd   12  0x00000003  
Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc2ec edx: 0x92c570fa
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc328 esp: 0xbfffc2ec
   ss: 0x0000001f efl: 0x00000206 eip: 0x92c570fa  cs: 0x00000007
   ds: 0x0000001f  es: 0x0000001f  fs: 0x00000000  gs: 0x00000037

Binary Images Description:
    0x1000 -   0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin
 0x12a9000 -  0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libsvml.dylib
 0x131c000 -  0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libifcoremt.dylib
 0x13fc000 -  0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libimf.dylib
 0x1669000 -  0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vntr_1399_200_006716645/lib/libirc.dylib
0x90bb2000 - 0x90bc0fff /usr/lib/libz.1.dylib
0x92c56000 - 0x92dfcfff /usr/lib/libSystem.B.dylib
0x93a39000 - 0x93a3cfff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=36384, iMonCtr=1
Model crash detected, will try to restart...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
 (36384): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (36966): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
 (1827): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
22 Oct 2010 10:57:55 1062556 11918596 famous_vntr_1399_200_006716645_6 1,872,026 276,673 0.1478
22 Oct 2010 10:42:40 1062556 11918596 famous_vntr_1399_200_006716645_6 1,862,666 275,288 0.1478
22 Oct 2010 00:01:06 1062556 11918596 famous_vntr_1399_200_006716645_6 1,853,306 273,892 0.1478
21 Oct 2010 23:30:36 1062556 11918596 famous_vntr_1399_200_006716645_6 1,843,946 272,560 0.1478
21 Oct 2010 23:05:03 1062556 11918596 famous_vntr_1399_200_006716645_6 1,834,586 271,220 0.1478
21 Oct 2010 22:39:32 1062556 11918596 famous_vntr_1399_200_006716645_6 1,825,226 269,895 0.1479
21 Oct 2010 20:33:19 1062556 11918596 famous_vntr_1399_200_006716645_6 1,815,866 268,560 0.1479
21 Oct 2010 19:17:15 1062556 11918596 famous_vntr_1399_200_006716645_6 1,806,506 267,218 0.1479
21 Oct 2010 11:54:04 1062556 11918596 famous_vntr_1399_200_006716645_6 1,797,146 265,942 0.1480
21 Oct 2010 11:23:19 1062556 11918596 famous_vntr_1399_200_006716645_6 1,787,786 264,585 0.1480
21 Oct 2010 10:52:36 1062556 11918596 famous_vntr_1399_200_006716645_6 1,778,426 263,199 0.1480
21 Oct 2010 02:48:36 1062556 11918596 famous_vntr_1399_200_006716645_6 1,769,066 261,790 0.1480
20 Oct 2010 23:32:40 1062556 11918596 famous_vntr_1399_200_006716645_6 1,759,706 260,439 0.1480
20 Oct 2010 23:06:07 1062556 11918596 famous_vntr_1399_200_006716645_6 1,750,346 259,112 0.1480
20 Oct 2010 21:16:55 1062556 11918596 famous_vntr_1399_200_006716645_6 1,740,986 257,760 0.1481
20 Oct 2010 20:51:27 1062556 11918596 famous_vntr_1399_200_006716645_6 1,731,626 256,426 0.1481
20 Oct 2010 20:26:00 1062556 11918596 famous_vntr_1399_200_006716645_6 1,722,266 255,107 0.1481
20 Oct 2010 20:26:00 1062556 11918596 famous_vntr_1399_200_006716645_6 1,712,906 253,776 0.1482
20 Oct 2010 20:00:29 1062556 11918596 famous_vntr_1399_200_006716645_6 1,703,546 252,420 0.1482
20 Oct 2010 19:14:04 1062556 11918596 famous_vntr_1399_200_006716645_6 1,694,186 251,110 0.1482


©2024 climateprediction.net