climateprediction.net home page
Task 12423894

Task 12423894

Name famous_xng1_699_200_007092216_0
Workunit 7295516
Created 18 Dec 2010, 16:42:55 UTC
Sent 23 Dec 2010, 14:53:11 UTC
Report deadline 24 Mar 2011, 22:20:22 UTC
Received 28 Dec 2010, 15:51:19 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1014037
Run time 4 days 13 hours 44 min 55 sec
CPU time 4 days 10 hours 16 min 43 sec
Validate state Invalid
Credit 2,563.26
Device peak FLOPS 2.87 GFLOPS
Application version UK Met Office FAMOUS v6.11
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
 (1330): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (1330): No heartbeat from core client for 30 sec - exiting
 (11464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
 (3229): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
 (3333): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
 (4810): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4810): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (4667): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
 (4667): No heartbeat from core client for 30 sec - exiting
 (32267): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (32267): No heartbeat from core client for 30 sec - exiting
 (32267): No heartbeat from core client for 30 sec - exiting
 (25108): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (25108): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
 (27506): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
 (28654): No heartbeat from core client for 30 sec - exiting
 (28654): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
 (23619): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (24545): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (24545): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (25033): No heartbeat from core client for 30 sec - exiting
 (22413): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (22413): No heartbeat from core client for 30 sec - exiting
 (19854): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (19854): No heartbeat from core client for 30 sec - exiting
 (19854): No heartbeat from core client for 30 sec - exiting
 (19854): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
 (16232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (17403): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (17403): No heartbeat from core client for 30 sec - exiting
 (18649): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (18957): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (15317): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (10326): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (11626): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (14090): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (14090): No heartbeat from core client for 30 sec - exiting
 (14090): No heartbeat from core client for 30 sec - exiting
 (14090): No heartbeat from core client for 30 sec - exiting
 (14090): No heartbeat from core client for 30 sec - exiting
 (12954): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (14092): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (14092): No heartbeat from core client for 30 sec - exiting
 (14092): No heartbeat from core client for 30 sec - exiting
 (14092): No heartbeat from core client for 30 sec - exiting
 (14092): No heartbeat from core client for 30 sec - exiting
 (14092): No heartbeat from core client for 30 sec - exiting
 (14092): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (10458): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
 (6470): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6470): No heartbeat from core client for 30 sec - exiting
 (6561): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6561): No heartbeat from core client for 30 sec - exiting
 (6561): No heartbeat from core client for 30 sec - exiting
 (8990): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (6511): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (665): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (665): No heartbeat from core client for 30 sec - exiting
 (665): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
 (1876): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
 (2607): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (2607): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
 (491): No heartbeat from core client for 30 sec - exiting
 (491): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 60 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 61 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  
Sorry, too many model crashes! :-(
 (32455): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
28 Dec 2010 15:55:26 1014037 12423894 famous_xng1_699_200_007092216_0 776,906 376,669 0.4848
28 Dec 2010 15:55:26 1014037 12423894 famous_xng1_699_200_007092216_0 767,546 371,816 0.4844
28 Dec 2010 15:55:26 1014037 12423894 famous_xng1_699_200_007092216_0 758,186 367,260 0.4844
28 Dec 2010 15:55:26 1014037 12423894 famous_xng1_699_200_007092216_0 748,826 362,575 0.4842
28 Dec 2010 01:01:41 1014037 12423894 famous_xng1_699_200_007092216_0 739,466 357,622 0.4836
27 Dec 2010 23:27:44 1014037 12423894 famous_xng1_699_200_007092216_0 730,106 352,996 0.4835
27 Dec 2010 22:08:34 1014037 12423894 famous_xng1_699_200_007092216_0 720,746 348,221 0.4831
27 Dec 2010 20:46:14 1014037 12423894 famous_xng1_699_200_007092216_0 711,386 343,469 0.4828
27 Dec 2010 19:09:56 1014037 12423894 famous_xng1_699_200_007092216_0 702,026 338,689 0.4824
27 Dec 2010 18:01:33 1014037 12423894 famous_xng1_699_200_007092216_0 692,666 333,850 0.4820
27 Dec 2010 16:24:07 1014037 12423894 famous_xng1_699_200_007092216_0 683,306 329,139 0.4817
27 Dec 2010 14:45:38 1014037 12423894 famous_xng1_699_200_007092216_0 673,946 324,398 0.4813
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 664,586 319,568 0.4809
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 655,226 314,843 0.4805
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 645,866 310,044 0.4800
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 636,506 305,356 0.4797
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 627,146 300,654 0.4794
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 617,786 295,819 0.4788
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 608,426 291,115 0.4785
27 Dec 2010 14:24:43 1014037 12423894 famous_xng1_699_200_007092216_0 599,066 286,365 0.4780


©2024 climateprediction.net