climateprediction.net home page
Task 16274599

Task 16274599

Name hadcm3n_7yyh_1980_40_008456588_3
Workunit 8607444
Created 21 Jan 2014, 9:13:23 UTC
Sent 21 Jan 2014, 9:13:55 UTC
Report deadline 22 Apr 2014, 16:41:06 UTC
Received 7 Mar 2014, 0:38:03 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1214491
Run time 21 days 21 hours 1 min 4 sec
CPU time 16 days 14 hours 47 min 17 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.10 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>7.2.33</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 193 (0xc1)
</message>
<stderr_txt>
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4216, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3732, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3732, iMonCtr=1
Model crash detected, will try to restart...
13:07:46 (4068): No heartbeat from core client for 30 sec - exiting
13:07:47 (4068): No heartbeat from core client for 30 sec - exiting
13:07:48 (4068): No heartbeat from core client for 30 sec - exiting
13:07:49 (4068): No heartbeat from core client for 30 sec - exiting
13:07:50 (4068): No heartbeat from core client for 30 sec - exiting
13:07:51 (4068): No heartbeat from core client for 30 sec - exiting
13:07:52 (4068): No heartbeat from core client for 30 sec - exiting
13:07:53 (4068): No heartbeat from core client for 30 sec - exiting
13:07:54 (4068): No heartbeat from core client for 30 sec - exiting
13:07:55 (4068): No heartbeat from core client for 30 sec - exiting
13:07:56 (4068): No heartbeat from core client for 30 sec - exiting
13:07:57 (4068): No heartbeat from core client for 30 sec - exiting
13:07:58 (4068): No heartbeat from core client for 30 sec - exiting
13:07:59 (4068): No heartbeat from core client for 30 sec - exiting
13:08:00 (4068): No heartbeat from core client for 30 sec - exiting
13:08:01 (4068): No heartbeat from core client for 30 sec - exiting
13:08:02 (4068): No heartbeat from core client for 30 sec - exiting
13:08:03 (4068): No heartbeat from core client for 30 sec - exiting
13:08:04 (4068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:08:19 (4468): No heartbeat from core client for 30 sec - exiting
21:08:20 (4468): No heartbeat from core client for 30 sec - exiting
21:08:21 (4468): No heartbeat from core client for 30 sec - exiting
21:08:22 (4468): No heartbeat from core client for 30 sec - exiting
21:08:23 (4468): No heartbeat from core client for 30 sec - exiting
21:08:24 (4468): No heartbeat from core client for 30 sec - exiting
21:08:26 (4468): No heartbeat from core client for 30 sec - exiting
21:08:27 (4468): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4828, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5224, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5084, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5616, iMonCtr=1
Model crash detected, will try to restart...
20:20:56 (5764): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:20:57 (5764): No heartbeat from core client for 30 sec - exiting
20:20:58 (5764): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5848, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Ocean Restart file copy failed on 7yyhko.dai69r0
Atmos Hold Restart file rename failed on atmos_restart.hold
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5100, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7032, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3532, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3956, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3956, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1
Model crash detected, will try to restart...
11:26:41 (5288): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:30:23 (4572): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5492, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5220, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5284, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5708, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5524, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5524, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5524, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5524, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4232, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5420, iMonCtr=1
Model crash detected, will try to restart...
Atmos Hold Restart file rename failed on atmos_restart.hold
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4336, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4336, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5228, iMonCtr=1
Model crash detected, will try to restart...
22:48:19 (5612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:48:20 (5612): No heartbeat from core client for 30 sec - exiting
22:48:22 (5612): No heartbeat from core client for 30 sec - exiting
22:48:23 (5612): No heartbeat from core client for 30 sec - exiting
22:48:24 (5612): No heartbeat from core client for 30 sec - exiting
22:48:25 (5612): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4492, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5896, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5768, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4088, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5168, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5168, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1608, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1608, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5776, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5776, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3440, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3312, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3776, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5824, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5908, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5148, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5148, iMonCtr=1
Model crash detected, will try to restart...
08:28:02 (5608): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4184, iMonCtr=1
Model crash detected, will try to restart...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x77847383 read attempt to address 0xFFFFFFF8

Engaging BOINC Windows Runtime Debugger...

Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_7yyh_1980_40_008456588/dataout/shmem_restart.day
Signal 11 received, exiting...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
06 Mar 2014 17:47:12 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 518,400 1,435,636 2.7694
05 Mar 2014 09:23:24 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 492,480 1,365,381 2.7725
03 Mar 2014 14:25:31 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 466,560 1,289,978 2.7649
28 Feb 2014 10:31:31 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 440,640 1,216,926 2.7617
25 Feb 2014 13:14:53 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 414,720 1,146,126 2.7636
22 Feb 2014 11:02:47 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 388,800 1,072,218 2.7578
20 Feb 2014 15:12:50 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 362,880 1,001,714 2.7605
18 Feb 2014 20:05:48 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 336,960 929,572 2.7587
18 Feb 2014 12:00:44 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 311,040 856,907 2.7550
14 Feb 2014 20:49:51 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 285,120 786,623 2.7589
11 Feb 2014 15:56:47 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 259,200 712,528 2.7490
10 Feb 2014 07:31:58 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 233,280 642,340 2.7535
07 Feb 2014 10:30:33 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 207,360 570,292 2.7503
04 Feb 2014 20:53:39 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 181,440 497,994 2.7447
02 Feb 2014 21:39:04 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 155,520 427,119 2.7464
31 Jan 2014 00:46:14 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 129,600 355,401 2.7423
31 Jan 2014 00:46:14 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 103,680 283,768 2.7370
27 Jan 2014 11:03:01 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 77,760 213,009 2.7393
24 Jan 2014 11:00:50 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 51,840 141,921 2.7377
22 Jan 2014 14:29:48 1214491 16274599 hadcm3n_7yyh_1980_40_008456588_3 25,920 70,369 2.7149


©2024 climateprediction.net