climateprediction.net home page
Task 15643304

Task 15643304

Name hadcm3n_3fvq_1980_40_008318434_1
Workunit 8469569
Created 28 Feb 2013, 6:22:25 UTC
Sent 28 Feb 2013, 6:22:51 UTC
Report deadline 30 May 2013, 13:50:02 UTC
Received 14 Mar 2013, 8:53:13 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1120510
Run time 13 days 18 hours 15 min 20 sec
CPU time 12 days 3 hours 45 min 28 sec
Validate state Invalid
Credit 8,709.12
Device peak FLOPS 3.21 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr 
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
02:57:24 (45821): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
15:30:55 (32187): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:30:58 (32294): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:30:31 (35468): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:20:45 (72330): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
11:26:59 (60651): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:27:00 (60651): No heartbeat from core client for 30 sec - exiting
11:27:01 (60651): No heartbeat from core client for 30 sec - exiting
11:27:02 (60651): No heartbeat from core client for 30 sec - exiting
11:27:03 (60651): No heartbeat from core client for 30 sec - exiting
11:27:04 (60651): No heartbeat from core client for 30 sec - exiting
11:27:05 (60651): No heartbeat from core client for 30 sec - exiting
11:27:06 (60651): No heartbeat from core client for 30 sec - exiting
11:27:07 (60651): No heartbeat from core client for 30 sec - exiting
11:27:08 (60651): No heartbeat from core client for 30 sec - exiting
11:27:09 (60651): No heartbeat from core client for 30 sec - exiting
11:27:10 (60651): No heartbeat from core client for 30 sec - exiting
11:27:11 (60651): No heartbeat from core client for 30 sec - exiting
11:27:12 (60651): No heartbeat from core client for 30 sec - exiting
11:27:13 (60651): No heartbeat from core client for 30 sec - exiting
11:27:14 (60651): No heartbeat from core client for 30 sec - exiting
11:27:15 (60651): No heartbeat from core client for 30 sec - exiting
11:27:16 (60651): No heartbeat from core client for 30 sec - exiting
11:27:17 (60651): No heartbeat from core client for 30 sec - exiting
11:27:18 (60651): No heartbeat from core client for 30 sec - exiting
11:27:19 (60651): No heartbeat from core client for 30 sec - exiting
11:27:20 (60651): No heartbeat from core client for 30 sec - exiting
11:27:21 (60651): No heartbeat from core client for 30 sec - exiting
11:27:22 (60651): No heartbeat from core client for 30 sec - exiting
11:27:23 (60651): No heartbeat from core client for 30 sec - exiting
11:27:24 (60651): No heartbeat from core client for 30 sec - exiting
11:27:25 (60651): No heartbeat from core client for 30 sec - exiting
11:27:26 (60651): No heartbeat from core client for 30 sec - exiting
11:27:27 (60651): No heartbeat from core client for 30 sec - exiting
11:27:28 (60651): No heartbeat from core client for 30 sec - exiting
11:27:29 (60651): No heartbeat from core client for 30 sec - exiting
11:27:30 (60651): No heartbeat from core client for 30 sec - exiting
11:27:31 (60651): No heartbeat from core client for 30 sec - exiting
11:27:32 (60651): No heartbeat from core client for 30 sec - exiting
11:27:33 (60651): No heartbeat from core client for 30 sec - exiting
11:27:34 (60651): No heartbeat from core client for 30 sec - exiting
11:27:35 (60651): No heartbeat from core client for 30 sec - exiting
11:27:36 (60651): No heartbeat from core client for 30 sec - exiting
11:27:37 (60651): No heartbeat from core client for 30 sec - exiting
11:27:38 (60651): No heartbeat from core client for 30 sec - exiting
11:27:39 (60651): No heartbeat from core client for 30 sec - exiting
11:27:40 (60651): No heartbeat from core client for 30 sec - exiting
11:27:41 (60651): No heartbeat from core client for 30 sec - exiting
11:27:42 (60651): No heartbeat from core client for 30 sec - exiting
11:27:43 (60651): No heartbeat from core client for 30 sec - exiting
11:27:44 (60651): No heartbeat from core client for 30 sec - exiting
11:27:45 (60651): No heartbeat from core client for 30 sec - exiting
11:27:46 (60651): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
13:03:27 (271): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
OPEN:  File Creation Failed: No space left on device
OPEN:  Unable to Open File dataout/3fvqko.dak86r0 for Read/Write

Model crashed: DUMPCTL : Fail to open output dump - may already exist                                                                                                                                                                                                          tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fvq_1980_40_008318434/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E1E32  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003DF4CC  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003651C5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  002DE95F  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00389F90  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000048E1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000027FB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00002729  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2384, iMonCtr=1
Model crash detected, will try to restart...
16:08:14 (2384): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
OPEN:  File Creation Failed: No space left on device
OPEN:  Unable to Open File dataout/3fvqka.dak8c30 for Read/Write

Model crashed: DUMPCTL : Fail to open output dump - may already exist                                                                                                                                                                                                          tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_3fvq_1980_40_008318434/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E1E32  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003DF4CC  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003651C5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  002DE95F  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00389F90  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000048E1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000027FB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00002729  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10199, iMonCtr=1
Model crash detected, will try to restart...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O 02:12:00 (10199): No heartbeat from core client for 30 sec - exiting

BUFFOUT: Write Failed: Result too large
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: Result too large
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITHEAD: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Mar 2013 01:07:35 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 725,760 1,063,213 1.4650
13 Mar 2013 13:00:57 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 699,840 1,025,729 1.4657
13 Mar 2013 01:53:14 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 673,920 988,830 1.4673
12 Mar 2013 12:37:37 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 648,000 948,927 1.4644
12 Mar 2013 00:39:08 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 622,080 909,118 1.4614
11 Mar 2013 11:39:10 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 596,160 869,040 1.4577
10 Mar 2013 23:44:54 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 570,240 829,908 1.4554
10 Mar 2013 12:37:28 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 544,320 792,880 1.4566
10 Mar 2013 01:24:48 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 518,400 755,728 1.4578
09 Mar 2013 14:25:15 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 492,480 719,195 1.4604
09 Mar 2013 03:11:03 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 466,560 681,955 1.4617
08 Mar 2013 15:20:34 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 440,640 642,784 1.4588
08 Mar 2013 03:34:50 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 414,720 603,900 1.4562
07 Mar 2013 15:35:36 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 388,800 565,747 1.4551
07 Mar 2013 03:43:47 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 362,880 526,913 1.4520
06 Mar 2013 16:00:40 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 336,960 489,251 1.4520
06 Mar 2013 03:39:40 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 311,040 452,145 1.4537
05 Mar 2013 15:05:27 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 285,120 414,768 1.4547
05 Mar 2013 03:39:26 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 259,200 377,484 1.4563
04 Mar 2013 15:25:37 1120510 15643304 hadcm3n_3fvq_1980_40_008318434_1 233,280 338,254 1.4500


©2024 climateprediction.net