climateprediction.net home page
Task 16146760

Task 16146760

Name hadcm3n_7w8j_1980_40_008453062_3
Workunit 8603918
Created 18 Dec 2013, 14:10:27 UTC
Sent 18 Dec 2013, 14:10:35 UTC
Report deadline 19 Mar 2014, 21:37:46 UTC
Received 21 Aug 2014, 14:57:41 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 945967
Run time 29 days 5 hours 57 min 51 sec
CPU time 29 days 5 hours 57 min 51 sec
Validate state Invalid
Credit 9,642.24
Device peak FLOPS 1.95 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>6.4.5</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
 not running, exiting, bRetVal = 1, checkPID=0, selfPID=6816, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6492, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7680, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7680, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7976, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8144, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7232, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7232, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7232, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7292, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7808, iMonCtr=1
Model crash detected, will try to restart...
17:11:05 (5068): No heartbeat from core client for 30 sec - exiting
17:11:06 (5068): No heartbeat from core client for 30 sec - exiting
17:11:07 (5068): No heartbeat from core client for 30 sec - exiting
17:11:08 (5068): No heartbeat from core client for 30 sec - exiting
17:11:09 (5068): No heartbeat from core client for 30 sec - exiting
17:11:10 (5068): No heartbeat from core client for 30 sec - exiting
17:11:11 (5068): No heartbeat from core client for 30 sec - exiting
17:11:12 (5068): No heartbeat from core client for 30 sec - exiting
17:11:13 (5068): No heartbeat from core client for 30 sec - exiting
17:11:14 (5068): No heartbeat from core client for 30 sec - exiting
17:11:16 (5068): No heartbeat from core client for 30 sec - exiting
17:11:17 (5068): No heartbeat from core client for 30 sec - exiting
17:11:18 (5068): No heartbeat from core client for 30 sec - exiting
17:11:19 (5068): No heartbeat from core client for 30 sec - exiting
17:11:20 (5068): No heartbeat from core client for 30 sec - exiting
17:11:21 (5068): No heartbeat from core client for 30 sec - exiting
17:11:22 (5068): No heartbeat from core client for 30 sec - exiting
17:11:23 (5068): No heartbeat from core client for 30 sec - exiting
17:11:24 (5068): No heartbeat from core client for 30 sec - exiting
17:11:25 (5068): No heartbeat from core client for 30 sec - exiting
17:11:26 (5068): No heartbeat from core client for 30 sec - exiting
17:11:27 (5068): No heartbeat from core client for 30 sec - exiting
17:11:28 (5068): No heartbeat from core client for 30 sec - exiting
17:11:29 (5068): No heartbeat from core client for 30 sec - exiting
17:11:30 (5068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:11:31 (5068): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5876, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2552, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7848, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3216, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7744, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7744, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6660, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5628, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5628, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1312, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1312, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1312, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6612, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7708, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7000, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7000, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6892, iMonCtr=1
Model crash detected, will try to restart...
CPDN Monitor - Quit request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9480, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5284, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5284, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6580, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7716, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7716, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7716, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7204, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7204, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7204, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7048, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8184, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5808, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7828, iMonCtr=1
Model crash detected, will try to restart...
10:59:44 (7216): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:57:06 (8008): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6968, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7652, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7652, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7652, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7652, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7652, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1996, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7732, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7940, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7840, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7840, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7840, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7840, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6612, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8068, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8068, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6648, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6648, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6648, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6648, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6648, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7964, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6992, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6992, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7072, iMonCtr=1
Model crash detected, will try to restart...
10:26:37 (5484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6800, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7796, iMonCtr=1
Model crash detected, will try to restart...
11:07:58 (7348): No heartbeat from core client for 30 sec - exiting
11:07:59 (7348): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:08:00 (7348): No heartbeat from core client for 30 sec - exiting
11:45:23 (6464): No heartbeat from core client for 30 sec - exiting
11:45:24 (6464): No heartbeat from core client for 30 sec - exiting
11:45:25 (6464): No heartbeat from core client for 30 sec - exiting
11:45:27 (6464): No heartbeat from core client for 30 sec - exiting
11:45:28 (6464): No heartbeat from core client for 30 sec - exiting
11:45:29 (6464): No heartbeat from core client for 30 sec - exiting
11:45:30 (6464): No heartbeat from core client for 30 sec - exiting
11:45:31 (6464): No heartbeat from core client for 30 sec - exiting
11:45:32 (6464): No heartbeat from core client for 30 sec - exiting
11:45:33 (6464): No heartbeat from core client for 30 sec - exiting
11:45:34 (6464): No heartbeat from core client for 30 sec - exiting
11:45:35 (6464): No heartbeat from core client for 30 sec - exiting
11:45:36 (6464): No heartbeat from core client for 30 sec - exiting
11:45:37 (6464): No heartbeat from core client for 30 sec - exiting
11:45:39 (6464): No heartbeat from core client for 30 sec - exiting
11:45:40 (6464): No heartbeat from core client for 30 sec - exiting
11:45:42 (6464): No heartbeat from core client for 30 sec - exiting
11:45:43 (6464): No heartbeat from core client for 30 sec - exiting
11:45:44 (6464): No heartbeat from core client for 30 sec - exiting
11:45:47 (6464): No heartbeat from core client for 30 sec - exiting
11:45:49 (6464): No heartbeat from core client for 30 sec - exiting
11:45:50 (6464): No heartbeat from core client for 30 sec - exiting
11:45:51 (6464): No heartbeat from core client for 30 sec - exiting
11:45:53 (6464): No heartbeat from core client for 30 sec - exiting
11:45:54 (6464): No heartbeat from core client for 30 sec - exiting
11:45:55 (6464): No heartbeat from core client for 30 sec - exiting
11:45:56 (6464): No heartbeat from core client for 30 sec - exiting
11:45:57 (6464): No heartbeat from core client for 30 sec - exiting
11:45:58 (6464): No heartbeat from core client for 30 sec - exiting
11:45:59 (6464): No heartbeat from core client for 30 sec - exiting
11:46:00 (6464): No heartbeat from core client for 30 sec - exiting
11:46:01 (6464): No heartbeat from core client for 30 sec - exiting
11:46:02 (6464): No heartbeat from core client for 30 sec - exiting
11:46:03 (6464): No heartbeat from core client for 30 sec - exiting
11:46:04 (6464): No heartbeat from core client for 30 sec - exiting
11:46:05 (6464): No heartbeat from core client for 30 sec - exiting
11:46:06 (6464): No heartbeat from core client for 30 sec - exiting
11:46:07 (6464): No heartbeat from core client for 30 sec - exiting
11:46:08 (6464): No heartbeat from core client for 30 sec - exiting
11:46:09 (6464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2772, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7272, iMonCtr=1
Model crash detected, will try to restart...
11:40:13 (7272): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:14:59 (6360): No heartbeat from core client for 30 sec - exiting
12:15:00 (6360): No heartbeat from core client for 30 sec - exiting
12:15:01 (6360): No heartbeat from core client for 30 sec - exiting
12:15:02 (6360): No heartbeat from core client for 30 sec - exiting
12:15:03 (6360): No heartbeat from core client for 30 sec - exiting
12:15:04 (6360): No heartbeat from core client for 30 sec - exiting
12:15:05 (6360): No heartbeat from core client for 30 sec - exiting
12:15:06 (6360): No heartbeat from core client for 30 sec - exiting
12:15:07 (6360): No heartbeat from core client for 30 sec - exiting
12:15:08 (6360): No heartbeat from core client for 30 sec - exiting
12:15:09 (6360): No heartbeat from core client for 30 sec - exiting
12:15:10 (6360): No heartbeat from core client for 30 sec - exiting
12:15:11 (6360): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/7w8jko.pjj6c10
Error converting file to netcdf: dataout/7w8jko.pij6c10
Error converting file to netcdf: dataout/7w8jko.pfj6c10
Error converting file to netcdf: dataout/7w8jka.phj6c10
Error converting file to netcdf: dataout/7w8jka.pgj6c10
Error converting file to netcdf: dataout/7w8jka.pej6c10
Error converting file to netcdf: dataout/7w8jka.pdj6c10
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6720, iMonCtr=1
Model crash detected, will try to restart...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/7w8jko.pjj6c10
Error converting file to netcdf: dataout/7w8jko.pij6c10
Error converting file to netcdf: dataout/7w8jko.pfj6c10
Error converting file to netcdf: dataout/7w8jka.phj6c10
Error converting file to netcdf: dataout/7w8jka.pgj6c10
Error converting file to netcdf: dataout/7w8jka.pej6c10
Error converting file to netcdf: dataout/7w8jka.pdj6c10
14:19:46 (6588): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/7w8jko.pjj6c10
Error converting file to netcdf: dataout/7w8jko.pij6c10
Error converting file to netcdf: dataout/7w8jko.pfj6c10
Error converting file to netcdf: dataout/7w8jka.phj6c10
Error converting file to netcdf: dataout/7w8jka.pgj6c10
Error converting file to netcdf: dataout/7w8jka.pej6c10
Error converting file to netcdf: dataout/7w8jka.pdj6c10
14:24:54 (7220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/7w8jko.pjj6c10
Error converting file to netcdf: dataout/7w8jko.pij6c10
Error converting file to netcdf: dataout/7w8jko.pfj6c10
Error converting file to netcdf: dataout/7w8jka.phj6c10
Error converting file to netcdf: dataout/7w8jka.pgj6c10
Error converting file to netcdf: dataout/7w8jka.pej6c10
Error converting file to netcdf: dataout/7w8jka.pdj6c10
10:26:47 (3840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:29:51 (7128): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:33:13 (7892): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:36:33 (4156): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:38:40 (1752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:41:34 (5972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:46:03 (5184): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:49:35 (4568): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:52:36 (6780): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:55:42 (9028): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:09:18 (9300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15240, iMonCtr=1
Model crash detected, will try to restart...
09:50:45 (7104): No heartbeat from core client for 30 sec - exiting
09:50:46 (7104): No heartbeat from core client for 30 sec - exiting
09:50:47 (7104): No heartbeat from core client for 30 sec - exiting
09:50:48 (7104): No heartbeat from core client for 30 sec - exiting
09:50:49 (7104): No heartbeat from core client for 30 sec - exiting
09:50:50 (7104): No heartbeat from core client for 30 sec - exiting
09:50:51 (7104): No heartbeat from core client for 30 sec - exiting
09:50:52 (7104): No heartbeat from core client for 30 sec - exiting
09:50:53 (7104): No heartbeat from core client for 30 sec - exiting
09:50:54 (7104): No heartbeat from core client for 30 sec - exiting
09:50:55 (7104): No heartbeat from core client for 30 sec - exiting
09:50:56 (7104): No heartbeat from core client for 30 sec - exiting
09:50:57 (7104): No heartbeat from core client for 30 sec - exiting
09:50:58 (7104): No heartbeat from core client for 30 sec - exiting
09:50:59 (7104): No heartbeat from core client for 30 sec - exiting
09:51:00 (7104): No heartbeat from core client for 30 sec - exiting
09:51:01 (7104): No heartbeat from core client for 30 sec - exiting
09:51:02 (7104): No heartbeat from core client for 30 sec - exiting
09:51:03 (7104): No heartbeat from core client for 30 sec - exiting
09:51:04 (7104): No heartbeat from core client for 30 sec - exiting
09:51:05 (7104): No heartbeat from core client for 30 sec - exiting
09:51:06 (7104): No heartbeat from core client for 30 sec - exiting
09:51:07 (7104): No heartbeat from core client for 30 sec - exiting
09:51:08 (7104): No heartbeat from core client for 30 sec - exiting
09:51:09 (7104): No heartbeat from core client for 30 sec - exiting
09:51:10 (7104): No heartbeat from core client for 30 sec - exiting
09:51:11 (7104): No heartbeat from core client for 30 sec - exiting
09:51:12 (7104): No heartbeat from core client for 30 sec - exiting
09:51:13 (7104): No heartbeat from core client for 30 sec - exiting
09:51:14 (7104): No heartbeat from core client for 30 sec - exiting
09:51:15 (7104): No heartbeat from core client for 30 sec - exiting
09:51:16 (7104): No heartbeat from core client for 30 sec - exiting
09:51:17 (7104): No heartbeat from core client for 30 sec - exiting
09:51:18 (7104): No heartbeat from core client for 30 sec - exiting
09:51:19 (7104): No heartbeat from core client for 30 sec - exiting
09:51:20 (7104): No heartbeat from core client for 30 sec - exiting
09:51:21 (7104): No heartbeat from core client for 30 sec - exiting
09:51:22 (7104): No heartbeat from core client for 30 sec - exiting
09:51:24 (7104): No heartbeat from core client for 30 sec - exiting
09:51:25 (7104): No heartbeat from core client for 30 sec - exiting
09:51:26 (7104): No heartbeat from core client for 30 sec - exiting
09:51:27 (7104): No heartbeat from core client for 30 sec - exiting
09:51:28 (7104): No heartbeat from core client for 30 sec - exiting
09:51:29 (7104): No heartbeat from core client for 30 sec - exiting
09:51:30 (7104): No heartbeat from core client for 30 sec - exiting
09:51:31 (7104): No heartbeat from core client for 30 sec - exiting
09:51:32 (7104): No heartbeat from core client for 30 sec - exiting
09:51:33 (7104): No heartbeat from core client for 30 sec - exiting
09:51:34 (7104): No heartbeat from core client for 30 sec - exiting
09:51:35 (7104): No heartbeat from core client for 30 sec - exiting
09:51:36 (7104): No heartbeat from core client for 30 sec - exiting
09:51:37 (7104): No heartbeat from core client for 30 sec - exiting
09:51:38 (7104): No heartbeat from core client for 30 sec - exiting
09:51:39 (7104): No heartbeat from core client for 30 sec - exiting
09:51:40 (7104): No heartbeat from core client for 30 sec - exiting
09:51:41 (7104): No heartbeat from core client for 30 sec - exiting
09:51:42 (7104): No heartbeat from core client for 30 sec - exiting
09:51:43 (7104): No heartbeat from core client for 30 sec - exiting
09:51:44 (7104): No heartbeat from core client for 30 sec - exiting
09:51:45 (7104): No heartbeat from core client for 30 sec - exiting
09:51:46 (7104): No heartbeat from core client for 30 sec - exiting
09:51:47 (7104): No heartbeat from core client for 30 sec - exiting
09:51:48 (7104): No heartbeat from core client for 30 sec - exiting
09:51:49 (7104): No heartbeat from core client for 30 sec - exiting
09:51:50 (7104): No heartbeat from core client for 30 sec - exiting
09:51:51 (7104): No heartbeat from core client for 30 sec - exiting
09:51:52 (7104): No heartbeat from core client for 30 sec - exiting
09:51:53 (7104): No heartbeat from core client for 30 sec - exiting
09:51:54 (7104): No heartbeat from core client for 30 sec - exiting
09:51:55 (7104): No heartbeat from core client for 30 sec - exiting
09:51:56 (7104): No heartbeat from core client for 30 sec - exiting
09:51:57 (7104): No heartbeat from core client for 30 sec - exiting
09:51:58 (7104): No heartbeat from core client for 30 sec - exiting
09:51:59 (7104): No heartbeat from core client for 30 sec - exiting
09:52:00 (7104): No heartbeat from core client for 30 sec - exiting
09:52:01 (7104): No heartbeat from core client for 30 sec - exiting
09:52:02 (7104): No heartbeat from core client for 30 sec - exiting
09:52:03 (7104): No heartbeat from core client for 30 sec - exiting
09:52:04 (7104): No heartbeat from core client for 30 sec - exiting
09:52:05 (7104): No heartbeat from core client for 30 sec - exiting
09:52:06 (7104): No heartbeat from core client for 30 sec - exiting
09:52:07 (7104): No heartbeat from core client for 30 sec - exiting
09:52:08 (7104): No heartbeat from core client for 30 sec - exiting
09:52:09 (7104): No heartbeat from core client for 30 sec - exiting
09:52:10 (7104): No heartbeat from core client for 30 sec - exiting
09:52:11 (7104): No heartbeat from core client for 30 sec - exiting
09:52:12 (7104): No heartbeat from core client for 30 sec - exiting
09:52:13 (7104): No heartbeat from core client for 30 sec - exiting
09:52:14 (7104): No heartbeat from core client for 30 sec - exiting
09:52:15 (7104): No heartbeat from core client for 30 sec - exiting
09:52:16 (7104): No heartbeat from core client for 30 sec - exiting
09:52:17 (7104): No heartbeat from core client for 30 sec - exiting
09:52:18 (7104): No heartbeat from core client for 30 sec - exiting
09:52:19 (7104): No heartbeat from core client for 30 sec - exiting
09:52:20 (7104): No heartbeat from core client for 30 sec - exiting
09:52:21 (7104): No heartbeat from core client for 30 sec - exiting
09:52:22 (7104): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:52:23 (7104): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1664, iMonCtr=1
Model crash detected, will try to restart...
09:43:22 (12): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:48:09 (3168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:52:44 (4740): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:55:43 (7328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:59:06 (588): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:12:32 (7656): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:14:35 (4924): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:15:44 (8536): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:19:26 (6688): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:59:36 (8256): No heartbeat from core client for 30 sec - exiting
09:59:37 (8256): No heartbeat from core client for 30 sec - exiting
09:59:38 (8256): No heartbeat from core client for 30 sec - exiting
09:59:39 (8256): No heartbeat from core client for 30 sec - exiting
09:59:40 (8256): No heartbeat from core client for 30 sec - exiting
09:59:41 (8256): No heartbeat from core client for 30 sec - exiting
09:59:42 (8256): No heartbeat from core client for 30 sec - exiting
09:59:43 (8256): No heartbeat from core client for 30 sec - exiting
09:59:44 (8256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9852, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9852, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5184, iMonCtr=1
Model crash detected, will try to restart...
10:08:47 (9060): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:09:01 (5772): No heartbeat from core client for 30 sec - exiting
10:09:02 (5772): No heartbeat from core client for 30 sec - exiting
10:09:03 (5772): No heartbeat from core client for 30 sec - exiting
10:09:04 (5772): No heartbeat from core client for 30 sec - exiting
10:09:05 (5772): No heartbeat from core client for 30 sec - exiting
10:09:06 (5772): No heartbeat from core client for 30 sec - exiting
10:09:07 (5772): No heartbeat from core client for 30 sec - exiting
10:09:08 (5772): No heartbeat from core client for 30 sec - exiting
10:09:09 (5772): No heartbeat from core client for 30 sec - exiting
10:09:10 (5772): No heartbeat from core client for 30 sec - exiting
10:09:11 (5772): No heartbeat from core client for 30 sec - exiting
10:09:12 (5772): No heartbeat from core client for 30 sec - exiting
10:09:13 (5772): No heartbeat from core client for 30 sec - exiting
10:09:14 (5772): No heartbeat from core client for 30 sec - exiting
10:09:15 (5772): No heartbeat from core client for 30 sec - exiting
10:09:16 (5772): No heartbeat from core client for 30 sec - exiting
10:09:17 (5772): No heartbeat from core client for 30 sec - exiting
10:09:18 (5772): No heartbeat from core client for 30 sec - exiting
10:09:19 (5772): No heartbeat from core client for 30 sec - exiting
10:09:20 (5772): No heartbeat from core client for 30 sec - exiting
10:09:21 (5772): No heartbeat from core client for 30 sec - exiting
10:09:22 (5772): No heartbeat from core client for 30 sec - exiting
10:09:23 (5772): No heartbeat from core client for 30 sec - exiting
10:09:24 (5772): No heartbeat from core client for 30 sec - exiting
10:09:25 (5772): No heartbeat from core client for 30 sec - exiting
10:09:26 (5772): No heartbeat from core client for 30 sec - exiting
10:09:27 (5772): No heartbeat from core client for 30 sec - exiting
10:09:28 (5772): No heartbeat from core client for 30 sec - exiting
10:09:29 (5772): No heartbeat from core client for 30 sec - exiting
10:09:30 (5772): No heartbeat from core client for 30 sec - exiting
10:09:31 (5772): No heartbeat from core client for 30 sec - exiting
10:09:32 (5772): No heartbeat from core client for 30 sec - exiting
10:09:33 (5772): No heartbeat from core client for 30 sec - exiting
10:09:34 (5772): No heartbeat from core client for 30 sec - exiting
10:09:35 (5772): No heartbeat from core client for 30 sec - exiting
10:09:36 (5772): No heartbeat from core client for 30 sec - exiting
10:09:37 (5772): No heartbeat from core client for 30 sec - exiting
10:09:38 (5772): No heartbeat from core client for 30 sec - exiting
10:09:39 (5772): No heartbeat from core client for 30 sec - exiting
10:09:40 (5772): No heartbeat from core client for 30 sec - exiting
10:09:41 (5772): No heartbeat from core client for 30 sec - exiting
10:09:42 (5772): No heartbeat from core client for 30 sec - exiting
10:09:43 (5772): No heartbeat from core client for 30 sec - exiting
10:09:45 (5772): No heartbeat from core client for 30 sec - exiting
10:09:46 (5772): No heartbeat from core client for 30 sec - exiting
10:09:47 (5772): No heartbeat from core client for 30 sec - exiting
10:09:48 (5772): No heartbeat from core client for 30 sec - exiting
10:09:49 (5772): No heartbeat from core client for 30 sec - exiting
10:09:50 (5772): No heartbeat from core client for 30 sec - exiting
10:09:51 (5772): No heartbeat from core client for 30 sec - exiting
10:09:52 (5772): No heartbeat from core client for 30 sec - exiting
10:09:53 (5772): No heartbeat from core client for 30 sec - exiting
10:09:54 (5772): No heartbeat from core client for 30 sec - exiting
10:09:55 (5772): No heartbeat from core client for 30 sec - exiting
10:09:56 (5772): No heartbeat from core client for 30 sec - exiting
10:09:57 (5772): No heartbeat from core client for 30 sec - exiting
10:09:58 (5772): No heartbeat from core client for 30 sec - exiting
10:09:59 (5772): No heartbeat from core client for 30 sec - exiting
10:10:00 (5772): No heartbeat from core client for 30 sec - exiting
10:10:01 (5772): No heartbeat from core client for 30 sec - exiting
10:10:02 (5772): No heartbeat from core client for 30 sec - exiting
10:10:03 (5772): No heartbeat from core client for 30 sec - exiting
10:10:04 (5772): No heartbeat from core client for 30 sec - exiting
10:10:05 (5772): No heartbeat from core client for 30 sec - exiting
10:10:06 (5772): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:14:41 (6872): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:14:42 (6872): No heartbeat from core client for 30 sec - exiting
10:14:43 (6872): No heartbeat from core client for 30 sec - exiting
10:14:44 (6872): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4684, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6668, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6080, iMonCtr=1
Model crash detected, will try to restart...
10:05:45 (7424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9636, iMonCtr=1
Model crash detected, will try to restart...
09:56:20 (5620): No heartbeat from core client for 30 sec - exiting
09:56:21 (5620): No heartbeat from core client for 30 sec - exiting
09:56:22 (5620): No heartbeat from core client for 30 sec - exiting
09:56:23 (5620): No heartbeat from core client for 30 sec - exiting
09:56:24 (5620): No heartbeat from core client for 30 sec - exiting
09:56:25 (5620): No heartbeat from core client for 30 sec - exiting
09:56:26 (5620): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8676, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7936, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6312, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7620, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8488, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5768, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6204, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6204, iMonCtr=1
Model crash detected, will try to restart...
C09:57:52 (5712): No heartbeat from core client for 30 sec - exiting
09:57:53 (5712): No heartbeat from core client for 30 sec - exiting
09:57:54 (5712): No heartbeat from core client for 30 sec - exiting
09:57:55 (5712): No heartbeat from core client for 30 sec - exiting
09:57:56 (5712): No heartbeat from core client for 30 sec - exiting
09:57:57 (5712): No heartbeat from core client for 30 sec - exiting
09:57:58 (5712): No heartbeat from core client for 30 sec - exiting
09:57:59 (5712): No heartbeat from core client for 30 sec - exiting
09:58:01 (5712): No heartbeat from core client for 30 sec - exiting
09:58:02 (5712): No heartbeat from core client for 30 sec - exiting
09:58:03 (5712): No heartbeat from core client for 30 sec - exiting
09:58:04 (5712): No heartbeat from core client for 30 sec - exiting
09:58:05 (5712): No heartbeat from core client for 30 sec - exiting
09:58:06 (5712): No heartbeat from core client for 30 sec - exiting
09:58:07 (5712): No heartbeat from core client for 30 sec - exiting
09:58:08 (5712): No heartbeat from core client for 30 sec - exiting
09:58:09 (5712): No heartbeat from core client for 30 sec - exiting
09:58:10 (5712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:58:11 (5712): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9592, iMonCtr=1
Model crash detected, will try to restart...
09:56:39 (9000): No heartbeat from core client for 30 sec - exiting
09:56:40 (9000): No heartbeat from core client for 30 sec - exiting
09:56:41 (9000): No heartbeat from core client for 30 sec - exiting
09:56:42 (9000): No heartbeat from core client for 30 sec - exiting
09:56:43 (9000): No heartbeat from core client for 30 sec - exiting
09:56:44 (9000): No heartbeat from core client for 30 sec - exiting
09:56:45 (9000): No heartbeat from core client for 30 sec - exiting
09:56:46 (9000): No heartbeat from core client for 30 sec - exiting
09:56:47 (9000): No heartbeat from core client for 30 sec - exiting
09:56:48 (9000): No heartbeat from core client for 30 sec - exiting
09:56:49 (9000): No heartbeat from core client for 30 sec - exiting
09:56:50 (9000): No heartbeat from core client for 30 sec - exiting
09:56:51 (9000): No heartbeat from core client for 30 sec - exiting
09:56:52 (9000): No heartbeat from core client for 30 sec - exiting
09:56:53 (9000): No heartbeat from core client for 30 sec - exiting
09:56:54 (9000): No heartbeat from core client for 30 sec - exiting
09:56:55 (9000): No heartbeat from core client for 30 sec - exiting
09:56:56 (9000): No heartbeat from core client for 30 sec - exiting
09:56:57 (9000): No heartbeat from core client for 30 sec - exiting
09:56:58 (9000): No heartbeat from core client for 30 sec - exiting
09:56:59 (9000): No heartbeat from core client for 30 sec - exiting
09:57:00 (9000): No heartbeat from core client for 30 sec - exiting
09:57:01 (9000): No heartbeat from core client for 30 sec - exiting
09:57:02 (9000): No heartbeat from core client for 30 sec - exiting
09:57:06 (9000): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:57:08 (9000): No heartbeat from core client for 30 sec - exiting
09:57:10 (9000): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8588, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6688, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9484, iMonCtr=1
Model crash detected, will try to restart...
10:34:10 (7608): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/7w8jko.pjk3c10
Error converting file to netcdf: dataout/7w8jko.pik3c10
Error converting file to netcdf: dataout/7w8jko.pfk3c10
Error converting file to netcdf: dataout/7w8jka.phk3c10
Error converting file to netcdf: dataout/7w8jka.pgk3c10
Error converting file to netcdf: dataout/7w8jka.pek3c10
Error converting file to netcdf: dataout/7w8jka.pdk3c10
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11260, iMonCtr=1
Model crash detected, will try to restart...
10:02:06 (6772): No heartbeat from core client for 30 sec - exiting
10:02:07 (6772): No heartbeat from core client for 30 sec - exiting
10:02:09 (6772): No heartbeat from core client for 30 sec - exiting
10:02:10 (6772): No heartbeat from core client for 30 sec - exiting
10:02:11 (6772): No heartbeat from core client for 30 sec - exiting
10:02:12 (6772): No heartbeat from core client for 30 sec - exiting
10:02:13 (6772): No heartbeat from core client for 30 sec - exiting
10:02:14 (6772): No heartbeat from core client for 30 sec - exiting
10:02:15 (6772): No heartbeat from core client for 30 sec - exiting
10:02:16 (6772): No heartbeat from core client for 30 sec - exiting
10:02:17 (6772): No heartbeat from core client for 30 sec - exiting
10:02:18 (6772): No heartbeat from core client for 30 sec - exiting
10:02:19 (6772): No heartbeat from core client for 30 sec - exiting
10:02:20 (6772): No heartbeat from core client for 30 sec - exiting
10:02:21 (6772): No heartbeat from core client for 30 sec - exiting
10:02:23 (6772): No heartbeat from core client for 30 sec - exiting
10:02:24 (6772): No heartbeat from core client for 30 sec - exiting
10:02:25 (6772): No heartbeat from core client for 30 sec - exiting
10:02:26 (6772): No heartbeat from core client for 30 sec - exiting
10:02:28 (6772): No heartbeat from core client for 30 sec - exiting
10:02:29 (6772): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:05:42 (2080): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:26:42 (8300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:19:00 (484): No heartbeat from core client for 30 sec - exiting
10:19:01 (484): No heartbeat from core client for 30 sec - exiting
10:19:02 (484): No heartbeat from core client for 30 sec - exiting
10:19:03 (484): No heartbeat from core client for 30 sec - exiting
10:19:04 (484): No heartbeat from core client for 30 sec - exiting
10:19:05 (484): No heartbeat from core client for 30 sec - exiting
10:19:06 (484): No heartbeat from core client for 30 sec - exiting
10:19:07 (484): No heartbeat from core client for 30 sec - exiting
10:19:08 (484): No heartbeat from core client for 30 sec - exiting
10:19:09 (484): No heartbeat from core client for 30 sec - exiting
10:19:10 (484): No heartbeat from core client for 30 sec - exiting
10:19:11 (484): No heartbeat from core client for 30 sec - exiting
10:19:12 (484): No heartbeat from core client for 30 sec - exiting
10:19:13 (484): No heartbeat from core client for 30 sec - exiting
10:19:14 (484): No heartbeat from core client for 30 sec - exiting
10:19:15 (484): No heartbeat from core client for 30 sec - exiting
10:19:16 (484): No heartbeat from core client for 30 sec - exiting
10:19:17 (484): No heartbeat from core client for 30 sec - exiting
10:19:18 (484): No heartbeat from core client for 30 sec - exiting
10:19:19 (484): No heartbeat from core client for 30 sec - exiting
10:19:20 (484): No heartbeat from core client for 30 sec - exiting
10:19:21 (484): No heartbeat from core client for 30 sec - exiting
10:19:22 (484): No heartbeat from core client for 30 sec - exiting
10:19:23 (484): No heartbeat from core client for 30 sec - exiting
10:19:24 (484): No heartbeat from core client for 30 sec - exiting
10:19:25 (484): No heartbeat from core client for 30 sec - exiting
10:19:26 (484): No heartbeat from core client for 30 sec - exiting
10:19:27 (484): No heartbeat from core client for 30 sec - exiting
10:19:28 (484): No heartbeat from core client for 30 sec - exiting
10:19:29 (484): No heartbeat from core client for 30 sec - exiting
10:19:30 (484): No heartbeat from core client for 30 sec - exiting
10:19:31 (484): No heartbeat from core client for 30 sec - exiting
10:19:32 (484): No heartbeat from core client for 30 sec - exiting
10:19:33 (484): No heartbeat from core client for 30 sec - exiting
10:19:34 (484): No heartbeat from core client for 30 sec - exiting
10:19:35 (484): No heartbeat from core client for 30 sec - exiting
10:19:36 (484): No heartbeat from core client for 30 sec - exiting
10:19:37 (484): No heartbeat from core client for 30 sec - exiting
10:19:38 (484): No heartbeat from core client for 30 sec - exiting
10:19:39 (484): No heartbeat from core client for 30 sec - exiting
10:19:41 (484): No heartbeat from core client for 30 sec - exiting
10:19:42 (484): No heartbeat from core client for 30 sec - exiting
10:19:43 (484): No heartbeat from core client for 30 sec - exiting
10:19:44 (484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:19:45 (484): No heartbeat from core client for 30 sec - exiting
10:19:46 (484): No heartbeat from core client for 30 sec - exiting
10:19:47 (484): No heartbeat from core client for 30 sec - exiting
10:19:48 (484): No heartbeat from core client for 30 sec - exiting
10:19:49 (484): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6104, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6132, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6132, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9792, iMonCtr=1
Model crash detected, will try to restart...
10:31:10 (7616): No heartbeat from core client for 30 sec - exiting
10:31:11 (7616): No heartbeat from core client for 30 sec - exiting
10:31:12 (7616): No heartbeat from core client for 30 sec - exiting
10:31:20 (7616): No heartbeat from core client for 30 sec - exiting
10:31:21 (7616): No heartbeat from core client for 30 sec - exiting
10:31:22 (7616): No heartbeat from core client for 30 sec - exiting
10:31:23 (7616): No heartbeat from core client for 30 sec - exiting
10:31:24 (7616): No heartbeat from core client for 30 sec - exiting
10:31:25 (7616): No heartbeat from core client for 30 sec - exiting
10:31:26 (7616): No heartbeat from core client for 30 sec - exiting
10:31:27 (7616): No heartbeat from core client for 30 sec - exiting
10:31:28 (7616): No heartbeat from core client for 30 sec - exiting
10:31:29 (7616): No heartbeat from core client for 30 sec - exiting
10:31:30 (7616): No heartbeat from core client for 30 sec - exiting
10:31:31 (7616): No heartbeat from core client for 30 sec - exiting
10:31:32 (7616): No heartbeat from core client for 30 sec - exiting
10:31:33 (7616): No heartbeat from core client for 30 sec - exiting
10:31:34 (7616): No heartbeat from core client for 30 sec - exiting
10:31:35 (7616): No heartbeat from core client for 30 sec - exiting
10:31:36 (7616): No heartbeat from core client for 30 sec - exiting
10:31:37 (7616): No heartbeat from core client for 30 sec - exiting
10:31:38 (7616): No heartbeat from core client for 30 sec - exiting
10:31:39 (7616): No heartbeat from core client for 30 sec - exiting
10:31:40 (7616): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:31:41 (7616): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11684, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8528, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5240, iMonCtr=1
Model crash detected, will try to restart...
10:24:21 (6168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:04:13 (9484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:04:14 (9484): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8936, iMonCtr=1
Model crash detected, will try to restart...
10:10:00 (900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2288, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7756, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4260, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4260, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2516, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7452, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10092, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10076, iMonCtr=1
Model crash detected, will try to restart...
10:57:10 (9212): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7744, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4756, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1
Model crash detected, will try to restart...
10:59:10 (9844): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:59:11 (9844): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6376, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4212, iMonCtr=1
Model crash detected, will try to restart...
11:17:41 (9580): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:17:42 (9580): No heartbeat from core client for 30 sec - exiting
11:17:43 (9580): No heartbeat from core client for 30 sec - exiting
11:17:44 (9580): No heartbeat from core client for 30 sec - exiting
11:17:45 (9580): No heartbeat from core client for 30 sec - exiting
11:17:46 (9580): No heartbeat from core client for 30 sec - exiting
11:17:47 (9580): No heartbeat from core client for 30 sec - exiting
11:17:48 (9580): No heartbeat from core client for 30 sec - exiting
11:17:49 (9580): No heartbeat from core client for 30 sec - exiting
11:17:50 (9580): No heartbeat from core client for 30 sec - exiting
11:17:51 (9580): No heartbeat from core client for 30 sec - exiting
11:19:39 (8776): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7428, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8760, iMonCtr=1
Model crash detected, will try to restart...
10:54:04 (13940): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29408, iMonCtr=1
Model crash detected, will try to restart...
11:05:05 (16356): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:05:06 (16356): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22724, iMonCtr=1
Model crash detected, will try to restart...
11:43:44 (14072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:43:46 (14072): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14452, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14452, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14452, iMonCtr=1
Model crash detected, will try to restart...
11:04:33 (12560): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:04:34 (12560): No heartbeat from core client for 30 sec - exiting
11:04:35 (12560): No heartbeat from core client for 30 sec - exiting
11:04:36 (12560): No heartbeat from core client for 30 sec - exiting
11:04:37 (12560): No heartbeat from core client for 30 sec - exiting
11:04:38 (12560): No heartbeat from core client for 30 sec - exiting
11:04:39 (12560): No heartbeat from core client for 30 sec - exiting
11:04:40 (12560): No heartbeat from core client for 30 sec - exiting
11:04:41 (12560): No heartbeat from core client for 30 sec - exiting
11:04:42 (12560): No heartbeat from core client for 30 sec - exiting
11:04:43 (12560): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=23904, iMonCtr=1
Model crash detected, will try to restart...
11:14:34 (14036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:31 (42300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:15:24 (13240): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:15:26 (13240): No heartbeat from core client for 30 sec - exiting
11:18:26 (31264): No heartbeat from core client for 30 sec - exiting
11:18:28 (31264): No heartbeat from core client for 30 sec - exiting
11:18:29 (31264): No heartbeat from core client for 30 sec - exiting
11:18:30 (31264): No heartbeat from core client for 30 sec - exiting
11:18:31 (31264): No heartbeat from core client for 30 sec - exiting
11:18:32 (31264): No heartbeat from core client for 30 sec - exiting
11:18:33 (31264): No heartbeat from core client for 30 sec - exiting
11:18:34 (31264): No heartbeat from core client for 30 sec - exiting
11:18:35 (31264): No heartbeat from core client for 30 sec - exiting
11:18:36 (31264): No heartbeat from core client for 30 sec - exiting
11:18:37 (31264): No heartbeat from core client for 30 sec - exiting
11:18:39 (31264): No heartbeat from core client for 30 sec - exiting
11:18:40 (31264): No heartbeat from core client for 30 sec - exiting
11:18:41 (31264): No heartbeat from core client for 30 sec - exiting
11:18:42 (31264): No heartbeat from core client for 30 sec - exiting
11:18:43 (31264): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12584, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15516, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15516, iMonCtr=1
Model crash detected, will try to restart...
11:04:47 (13768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17484, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17484, iMonCtr=1
Model crash detected, will try to restart...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
20 Aug 2014 15:09:35 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 803,520 2,521,649 3.1383
14 Aug 2014 14:54:40 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 777,600 2,435,460 3.1320
14 Aug 2014 14:54:40 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 751,680 2,349,868 3.1262
24 Jul 2014 12:31:13 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 725,760 2,264,500 3.1202
17 Jul 2014 16:40:13 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 699,840 2,182,171 3.1181
10 Jul 2014 12:17:39 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 673,920 2,100,746 3.1172
03 Jul 2014 16:47:40 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 648,000 2,020,075 3.1174
24 Jun 2014 17:03:44 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 622,080 1,939,445 3.1177
17 Jun 2014 12:56:29 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 596,160 1,858,733 3.1178
10 Jun 2014 09:03:16 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 570,240 1,777,883 3.1178
30 May 2014 13:03:22 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 544,320 1,697,316 3.1182
22 May 2014 12:05:46 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 518,400 1,614,963 3.1153
14 May 2014 10:19:55 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 492,480 1,533,911 3.1147
06 May 2014 13:09:36 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 466,560 1,452,589 3.1134
24 Apr 2014 15:26:15 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 440,640 1,371,482 3.1125
16 Apr 2014 13:32:21 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 414,720 1,289,594 3.1096
09 Apr 2014 10:09:58 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 388,800 1,211,375 3.1157
02 Apr 2014 11:53:48 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 362,880 1,130,755 3.1161
25 Mar 2014 17:37:22 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 336,960 1,049,735 3.1153
19 Mar 2014 14:52:33 945967 16146760 hadcm3n_7w8j_1980_40_008453062_3 311,040 968,973 3.1153


©2024 climateprediction.net