climateprediction.net home page
Task 22257734

Task 22257734

Name oifs_43r3_ps_0541_1987050100_123_956_12173185_0
Workunit 12173185
Created 21 Dec 2022, 11:32:50 UTC
Sent 3 Jan 2023, 17:25:38 UTC
Report deadline 2 Feb 2023, 17:25:38 UTC
Received 5 Jan 2023, 7:33:20 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1519251
Run time 13 hours 57 min 12 sec
CPU time 11 hours 56 min 47 sec
Validate state Invalid
Credit 2,419.20
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.97 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,763.23 MB
Stderr 
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
nn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1188374:1188374] [20230105:112247:1672885367:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  11:23:29 STEP 1296 H=1296:00 +CPU= 37.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:51 STEP 1297 H=1297:00 +CPU= 20.868
  11:25:12 STEP 1298 H=1298:00 +CPU= 20.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:26:36 STEP 1299 H=1299:00 +CPU= 32.543
  11:26:57 STEP 1300 H=1300:00 +CPU= 21.150
  11:27:17 STEP 1301 H=1301:00 +CPU= 20.152
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:52 STEP 1302 H=1302:00 +CPU= 33.418
  11:29:12 STEP 1303 H=1303:00 +CPU= 20.414
  11:29:33 STEP 1304 H=1304:00 +CPU= 20.827
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:31:07 STEP 1305 H=1305:00 +CPU= 33.015
  11:31:28 STEP 1306 H=1306:00 +CPU= 20.635
  11:31:48 STEP 1307 H=1307:00 +CPU= 19.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:36 STEP 1308 H=1308:00 +CPU= 35.742
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001308
  11:33:57 STEP 1309 H=1309:00 +CPU= 20.454
  11:34:17 STEP 1310 H=1310:00 +CPU= 20.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:36:02 STEP 1311 H=1311:00 +CPU= 32.892
  11:36:22 STEP 1312 H=1312:00 +CPU= 20.602
  11:36:42 STEP 1313 H=1313:00 +CPU= 19.973
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:38:25 STEP 1314 H=1314:00 +CPU= 32.161
  11:38:45 STEP 1315 H=1315:00 +CPU= 20.007
  11:39:06 STEP 1316 H=1316:00 +CPU= 20.886
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:28 STEP 1317 H=1317:00 +CPU= 32.455
  11:40:49 STEP 1318 H=1318:00 +CPU= 20.843
  11:41:10 STEP 1319 H=1319:00 +CPU= 21.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:59 STEP 1320 H=1320:00 +CPU= 38.655
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_54.zip
Uploading the intermediate file: upload_file_54.zip
  11:43:20 STEP 1321 H=1321:00 +CPU= 20.163
  11:43:41 STEP 1322 H=1322:00 +CPU= 21.081
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:45:14 STEP 1323 H=1323:00 +CPU= 32.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:46:26 STEP 1324 H=1324:00 +CPU= 21.056
  11:46:47 STEP 1325 H=1325:00 +CPU= 21.352
  11:47:20 STEP 1326 H=1326:00 +CPU= 32.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:48:52 STEP 1327 H=1327:00 +CPU= 20.404
  11:49:15 STEP 1328 H=1328:00 +CPU= 22.338
  11:49:46 STEP 1329 H=1329:00 +CPU= 31.804
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:50:58 STEP 1330 H=1330:00 +CPU= 20.846
  11:51:19 STEP 1331 H=1331:00 +CPU= 20.857
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:52:47 STEP 1332 H=1332:00 +CPU= 36.129
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001332
  11:53:07 STEP 1333 H=1333:00 +CPU= 20.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:54:20 STEP 1334 H=1334:00 +CPU= 21.037
  11:54:53 STEP 1335 H=1335:00 +CPU= 33.176
  11:55:14 STEP 1336 H=1336:00 +CPU= 20.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:56:36 STEP 1337 H=1337:00 +CPU= 20.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:57:51 STEP 1338 H=1338:00 +CPU= 33.746
  11:58:13 STEP 1339 H=1339:00 +CPU= 21.403
  11:58:34 STEP 1340 H=1340:00 +CPU= 20.768
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:06 STEP 1341 H=1341:00 +CPU= 31.989
  12:00:27 STEP 1342 H=1342:00 +CPU= 21.073
  12:00:48 STEP 1343 H=1343:00 +CPU= 21.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:02:38 STEP 1344 H=1344:00 +CPU= 37.503
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_55.zip
Uploading the intermediate file: upload_file_55.zip
  12:02:58 STEP 1345 H=1345:00 +CPU= 19.972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:04:01 STEP 1346 H=1346:00 +CPU= 21.113
  12:04:34 STEP 1347 H=1347:00 +CPU= 32.794
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:05:57 STEP 1348 H=1348:00 +CPU= 21.499
  12:06:18 STEP 1349 H=1349:00 +CPU= 21.095
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:42 STEP 1350 H=1350:00 +CPU= 33.140
  12:08:02 STEP 1351 H=1351:00 +CPU= 20.375
  12:08:23 STEP 1352 H=1352:00 +CPU= 20.740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:09:57 STEP 1353 H=1353:00 +CPU= 32.585
  12:10:18 STEP 1354 H=1354:00 +CPU= 21.216
  12:10:39 STEP 1355 H=1355:00 +CPU= 20.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:12:16 STEP 1356 H=1356:00 +CPU= 35.834
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001356
  12:12:36 STEP 1357 H=1357:00 +CPU= 20.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:59 STEP 1358 H=1358:00 +CPU= 21.430
  12:14:31 STEP 1359 H=1359:00 +CPU= 32.739
  12:14:53 STEP 1360 H=1360:00 +CPU= 21.133
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:16:15 STEP 1361 H=1361:00 +CPU= 21.307
  12:16:49 STEP 1362 H=1362:00 +CPU= 33.297
  12:17:09 STEP 1363 H=1363:00 +CPU= 20.013
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:41 STEP 1364 H=1364:00 +CPU= 21.111
  12:19:14 STEP 1365 H=1365:00 +CPU= 32.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:20:27 STEP 1366 H=1366:00 +CPU= 21.203
  12:20:49 STEP 1367 H=1367:00 +CPU= 21.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:22:00 STEP 1368 H=1368:00 +CPU= 38.705
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_56.zip
Uploading the intermediate file: upload_file_56.zip
  12:22:19 STEP 1369 H=1369:00 +CPU= 19.252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:23:27 STEP 1370 H=1370:00 +CPU= 21.066
  12:24:00 STEP 1371 H=1371:00 +CPU= 32.391
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:25:13 STEP 1372 H=1372:00 +CPU= 21.082
  12:25:33 STEP 1373 H=1373:00 +CPU= 20.120
  12:26:06 STEP 1374 H=1374:00 +CPU= 32.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:27:28 STEP 1375 H=1375:00 +CPU= 21.244
  12:27:48 STEP 1376 H=1376:00 +CPU= 20.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:29:13 STEP 1377 H=1377:00 +CPU= 33.550
  12:29:34 STEP 1378 H=1378:00 +CPU= 20.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:30:46 STEP 1379 H=1379:00 +CPU= 20.971
  12:31:21 STEP 1380 H=1380:00 +CPU= 35.362
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:32:43 STEP 1381 H=1381:00 +CPU= 20.335
  12:33:05 STEP 1382 H=1382:00 +CPU= 21.459
  12:33:37 STEP 1383 H=1383:00 +CPU= 32.264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:34:59 STEP 1384 H=1384:00 +CPU= 21.264
  12:35:20 STEP 1385 H=1385:00 +CPU= 20.951
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:36:55 STEP 1386 H=1386:00 +CPU= 33.489
  12:37:15 STEP 1387 H=1387:00 +CPU= 19.911
  12:37:36 STEP 1388 H=1388:00 +CPU= 20.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:39:21 STEP 1389 H=1389:00 +CPU= 33.155
  12:39:42 STEP 1390 H=1390:00 +CPU= 20.818
  12:40:04 STEP 1391 H=1391:00 +CPU= 21.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:41:54 STEP 1392 H=1392:00 +CPU= 39.163
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_57.zip
Uploading the intermediate file: upload_file_57.zip
  12:42:15 STEP 1393 H=1393:00 +CPU= 20.664
  12:42:36 STEP 1394 H=1394:00 +CPU= 21.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:44:21 STEP 1395 H=1395:00 +CPU= 32.418
  12:44:42 STEP 1396 H=1396:00 +CPU= 21.466
  12:45:03 STEP 1397 H=1397:00 +CPU= 20.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:46:29 STEP 1398 H=1398:00 +CPU= 33.787
  12:46:49 STEP 1399 H=1399:00 +CPU= 20.425
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:48:13 STEP 1400 H=1400:00 +CPU= 22.214
  12:48:46 STEP 1401 H=1401:00 +CPU= 32.435
  12:49:07 STEP 1402 H=1402:00 +CPU= 20.827
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:50:28 STEP 1403 H=1403:00 +CPU= 21.216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:52:04 STEP 1404 H=1404:00 +CPU= 35.813
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001404
  12:52:25 STEP 1405 H=1405:00 +CPU= 20.912
  12:52:46 STEP 1406 H=1406:00 +CPU= 20.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:54:11 STEP 1407 H=1407:00 +CPU= 32.581
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:55:13 STEP 1408 H=1408:00 +CPU= 21.319
  12:55:35 STEP 1409 H=1409:00 +CPU= 21.010
  12:56:08 STEP 1410 H=1410:00 +CPU= 33.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:57:19 STEP 1411 H=1411:00 +CPU= 20.419
  12:57:40 STEP 1412 H=1412:00 +CPU= 21.691
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:00:09 STEP 1413 H=1413:00 +CPU= 36.676
  13:00:35 STEP 1414 H=1414:00 +CPU= 24.765
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:02:30 STEP 1415 H=1415:00 +CPU= 24.205
  13:03:14 STEP 1416 H=1416:00 +CPU= 42.525
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_58.zip
Uploading the intermediate file: upload_file_58.zip
  13:03:35 STEP 1417 H=1417:00 +CPU= 20.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:05:27 STEP 1418 H=1418:00 +CPU= 21.990
  13:06:05 STEP 1419 H=1419:00 +CPU= 36.609
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:18:10 STEP 1420 H=1420:00 +CPU= 23.355
  13:18:34 STEP 1421 H=1421:00 +CPU= 22.996
  13:19:12 STEP 1422 H=1422:00 +CPU= 37.330
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:21:06 STEP 1423 H=1423:00 +CPU= 22.644
  13:21:30 STEP 1424 H=1424:00 +CPU= 23.752
  13:22:07 STEP 1425 H=1425:00 +CPU= 36.237
  13:22:31 STEP 1426 H=1426:00 +CPU= 23.497
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1987050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0541
(argv4) batchid: 956
(argv5) wuid: 12173185
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8c3a4d34663911c5e558ff7ed3433ca0 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0541_1987050100_123_956_12173185.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0541_1987050100_123_956_12173185.zip
ic_ancil_file: ic_ancil_12173185
ifsdata_file: ifsdata_12173185
climate_data_file: clim_data_12173185
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_3f85b90dd5fe601264d8dc7b07dfb78b to: /var/lib/boinc-client/slots/13/ic_ancil_12173185.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12173185.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_fa503f36fc3ddbf56347f1cc2d70f74c to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12173185.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12173185.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12173185.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12173185.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12173185.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12173185.xml
-- Model is restarting --
Adjusting last_iter, 1427, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12173185.xml
last_cpu_time: 39157
upload_file_number: 59
last_iter: 1415
last_upload: 5097600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016
The child process has been launched with process id: 1207767
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1207767:1207767] [20230105:132358:1672892638:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:27:05 STEP 1416 H=1416:00 +CPU= 41.921
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:39:20 STEP 1417 H=1417:00 +CPU= 22.893
  13:39:44 STEP 1418 H=1418:00 +CPU= 23.912
  13:40:21 STEP 1419 H=1419:00 +CPU= 36.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:43:37 STEP 1420 H=1420:00 +CPU= 23.197
  13:44:00 STEP 1421 H=1421:00 +CPU= 22.796
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:49:51 STEP 1422 H=1422:00 +CPU= 38.035
  13:50:15 STEP 1423 H=1423:00 +CPU= 22.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:52:50 STEP 1424 H=1424:00 +CPU= 23.458
  13:53:26 STEP 1425 H=1425:00 +CPU= 35.754
  13:53:46 STEP 1426 H=1426:00 +CPU= 20.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:09 STEP 1427 H=1427:00 +CPU= 21.824
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:56:48 STEP 1428 H=1428:00 +CPU= 36.491
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001428
  13:57:08 STEP 1429 H=1429:00 +CPU= 20.239
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:51 STEP 1430 H=1430:00 +CPU= 21.341
  13:59:23 STEP 1431 H=1431:00 +CPU= 32.355
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:00:36 STEP 1432 H=1432:00 +CPU= 20.900
  14:00:58 STEP 1433 H=1433:00 +CPU= 21.313
  14:01:30 STEP 1434 H=1434:00 +CPU= 32.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:02:54 STEP 1435 H=1435:00 +CPU= 20.997
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:03:55 STEP 1436 H=1436:00 +CPU= 20.692
  14:04:27 STEP 1437 H=1437:00 +CPU= 31.665
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:05:39 STEP 1438 H=1438:00 +CPU= 20.355
  14:06:00 STEP 1439 H=1439:00 +CPU= 21.286
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:07:49 STEP 1440 H=1440:00 +CPU= 37.645
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_59.zip
Uploading the intermediate file: upload_file_59.zip
  14:08:10 STEP 1441 H=1441:00 +CPU= 21.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:02 STEP 1442 H=1442:00 +CPU= 21.058
  14:10:35 STEP 1443 H=1443:00 +CPU= 32.672
  14:10:56 STEP 1444 H=1444:00 +CPU= 20.242
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:18 STEP 1445 H=1445:00 +CPU= 20.551
  14:12:51 STEP 1446 H=1446:00 +CPU= 32.897
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:14:22 STEP 1447 H=1447:00 +CPU= 20.043
  14:14:45 STEP 1448 H=1448:00 +CPU= 22.266
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:16:29 STEP 1449 H=1449:00 +CPU= 32.662
  14:16:50 STEP 1450 H=1450:00 +CPU= 21.323
  14:17:10 STEP 1451 H=1451:00 +CPU= 20.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:19:07 STEP 1452 H=1452:00 +CPU= 35.972
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001452
  14:19:28 STEP 1453 H=1453:00 +CPU= 20.182
  14:19:48 STEP 1454 H=1454:00 +CPU= 20.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:21:33 STEP 1455 H=1455:00 +CPU= 32.829
  14:21:53 STEP 1456 H=1456:00 +CPU= 20.781
  14:22:14 STEP 1457 H=1457:00 +CPU= 20.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:23:57 STEP 1458 H=1458:00 +CPU= 32.952
  14:24:17 STEP 1459 H=1459:00 +CPU= 20.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:51 STEP 1460 H=1460:00 +CPU= 21.189
  14:26:23 STEP 1461 H=1461:00 +CPU= 32.544
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:06 STEP 1462 H=1462:00 +CPU= 20.797
  14:28:28 STEP 1463 H=1463:00 +CPU= 21.696
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:30:07 STEP 1464 H=1464:00 +CPU= 37.594
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_60.zip
Uploading the intermediate file: upload_file_60.zip
  14:30:27 STEP 1465 H=1465:00 +CPU= 20.461
  14:30:48 STEP 1466 H=1466:00 +CPU= 20.411
  14:31:20 STEP 1467 H=1467:00 +CPU= 31.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:32:52 STEP 1468 H=1468:00 +CPU= 21.301
  14:33:12 STEP 1469 H=1469:00 +CPU= 19.757
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:34:56 STEP 1470 H=1470:00 +CPU= 33.296
  14:35:17 STEP 1471 H=1471:00 +CPU= 20.397
  14:35:37 STEP 1472 H=1472:00 +CPU= 20.531
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:36:53 STEP 1473 H=1473:00 +CPU= 32.976
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:38:14 STEP 1474 H=1474:00 +CPU= 20.693
  14:38:35 STEP 1475 H=1475:00 +CPU= 20.473
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:40:12 STEP 1476 H=1476:00 +CPU= 36.249
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001476
  14:40:33 STEP 1477 H=1477:00 +CPU= 20.524
  14:40:54 STEP 1478 H=1478:00 +CPU= 20.532
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:42:37 STEP 1479 H=1479:00 +CPU= 32.804
  14:42:58 STEP 1480 H=1480:00 +CPU= 20.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:44:10 STEP 1481 H=1481:00 +CPU= 20.399
  14:44:42 STEP 1482 H=1482:00 +CPU= 31.956
  14:45:02 STEP 1483 H=1483:00 +CPU= 20.024
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:46:25 STEP 1484 H=1484:00 +CPU= 20.809
  14:46:57 STEP 1485 H=1485:00 +CPU= 31.768
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:48:10 STEP 1486 H=1486:00 +CPU= 21.370
  14:48:30 STEP 1487 H=1487:00 +CPU= 20.829
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:50:00 STEP 1488 H=1488:00 +CPU= 37.854
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_61.zip
Uploading the intermediate file: upload_file_61.zip
  14:50:20 STEP 1489 H=1489:00 +CPU= 20.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:51:35 STEP 1490 H=1490:00 +CPU= 19.748
  14:52:07 STEP 1491 H=1491:00 +CPU= 32.272
  14:52:28 STEP 1492 H=1492:00 +CPU= 20.726
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:53:49 STEP 1493 H=1493:00 +CPU= 20.452
  14:54:22 STEP 1494 H=1494:00 +CPU= 32.837
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:55:44 STEP 1495 H=1495:00 +CPU= 20.430
  14:56:05 STEP 1496 H=1496:00 +CPU= 21.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:57:39 STEP 1497 H=1497:00 +CPU= 32.049
  14:58:01 STEP 1498 H=1498:00 +CPU= 21.211
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:59:23 STEP 1499 H=1499:00 +CPU= 21.288
  14:59:59 STEP 1500 H=1500:00 +CPU= 35.252
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001500
  15:00:19 STEP 1501 H=1501:00 +CPU= 20.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:02:01 STEP 1502 H=1502:00 +CPU= 20.939
  15:02:33 STEP 1503 H=1503:00 +CPU= 32.154
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:03:47 STEP 1504 H=1504:00 +CPU= 21.268
  15:04:07 STEP 1505 H=1505:00 +CPU= 20.384
  15:04:40 STEP 1506 H=1506:00 +CPU= 32.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:06:10 STEP 1507 H=1507:00 +CPU= 20.422
  15:06:31 STEP 1508 H=1508:00 +CPU= 20.857
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:08:13 STEP 1509 H=1509:00 +CPU= 32.004
  15:08:34 STEP 1510 H=1510:00 +CPU= 20.825
  15:08:55 STEP 1511 H=1511:00 +CPU= 20.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:10:34 STEP 1512 H=1512:00 +CPU= 38.197
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_62.zip
Uploading the intermediate file: upload_file_62.zip
  15:10:55 STEP 1513 H=1513:00 +CPU= 20.166
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:12:17 STEP 1514 H=1514:00 +CPU= 21.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:13:52 STEP 1515 H=1515:00 +CPU= 32.246
  15:14:13 STEP 1516 H=1516:00 +CPU= 20.685
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:15:25 STEP 1517 H=1517:00 +CPU= 20.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:17:08 STEP 1518 H=1518:00 +CPU= 32.594
  15:17:29 STEP 1519 H=1519:00 +CPU= 20.804
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:18:32 STEP 1520 H=1520:00 +CPU= 21.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:19:46 STEP 1521 H=1521:00 +CPU= 31.875
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:20:59 STEP 1522 H=1522:00 +CPU= 20.729
  15:21:19 STEP 1523 H=1523:00 +CPU= 20.629
  15:21:55 STEP 1524 H=1524:00 +CPU= 35.364
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001524
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:23:26 STEP 1525 H=1525:00 +CPU= 21.214
  15:23:47 STEP 1526 H=1526:00 +CPU= 20.776
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1987050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0541
(argv4) batchid: 956
(argv5) wuid: 12173185
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8c3a4d34663911c5e558ff7ed3433ca0 to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0541_1987050100_123_956_12173185.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0541_1987050100_123_956_12173185.zip
ic_ancil_file: ic_ancil_12173185
ifsdata_file: ifsdata_12173185
climate_data_file: clim_data_12173185
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_3f85b90dd5fe601264d8dc7b07dfb78b to: /var/lib/boinc-client/slots/13/ic_ancil_12173185.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12173185.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_fa503f36fc3ddbf56347f1cc2d70f74c to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12173185.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12173185.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12173185.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12173185.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12173185.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12173185.xml
-- Model is restarting --
Adjusting last_iter, 1527, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12173185.xml
last_cpu_time: 41988
upload_file_number: 63
last_iter: 1511
last_upload: 5443200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016
The child process has been launched with process id: 1226186
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1226186:1226186] [20230105:152701:1672900021:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:28:43 STEP 1512 H=1512:00 +CPU= 36.515
  15:29:04 STEP 1513 H=1513:00 +CPU= 21.000
  15:29:24 STEP 1514 H=1514:00 +CPU= 20.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:31:06 STEP 1515 H=1515:00 +CPU= 31.374
  15:31:27 STEP 1516 H=1516:00 +CPU= 21.226
  15:31:48 STEP 1517 H=1517:00 +CPU= 20.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:33:23 STEP 1518 H=1518:00 +CPU= 33.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:34:45 STEP 1519 H=1519:00 +CPU= 20.393
  15:35:06 STEP 1520 H=1520:00 +CPU= 21.216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:36:49 STEP 1521 H=1521:00 +CPU= 32.203
  15:37:10 STEP 1522 H=1522:00 +CPU= 20.864
  15:37:30 STEP 1523 H=1523:00 +CPU= 19.971
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:38:58 STEP 1524 H=1524:00 +CPU= 35.482
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001524
  15:39:18 STEP 1525 H=1525:00 +CPU= 19.641
  15:39:38 STEP 1526 H=1526:00 +CPU= 20.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:41:11 STEP 1527 H=1527:00 +CPU= 32.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:42:53 STEP 1528 H=1528:00 +CPU= 21.194
  15:43:14 STEP 1529 H=1529:00 +CPU= 21.085
  15:43:47 STEP 1530 H=1530:00 +CPU= 32.359
  15:44:08 STEP 1531 H=1531:00 +CPU= 20.464
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:45:39 STEP 1532 H=1532:00 +CPU= 20.645
  15:46:11 STEP 1533 H=1533:00 +CPU= 31.925
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:53 STEP 1534 H=1534:00 +CPU= 21.399
  15:48:14 STEP 1535 H=1535:00 +CPU= 20.947
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:50:04 STEP 1536 H=1536:00 +CPU= 37.733
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12173185/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0541_1987050100_123_956_12173185_0_r1579072016_63.zip
Uploading the intermediate file: upload_file_63.zip
  15:50:25 STEP 1537 H=1537:00 +CPU= 21.111
  15:50:46 STEP 1538 H=1538:00 +CPU= 20.813
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:40 STEP 1539 H=1539:00 +CPU= 32.113
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:52 STEP 1540 H=1540:00 +CPU= 20.811
  15:54:13 STEP 1541 H=1541:00 +CPU= 20.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:46 STEP 1542 H=1542:00 +CPU= 33.162
  15:56:07 STEP 1543 H=1543:00 +CPU= 20.163
  15:56:28 STEP 1544 H=1544:00 +CPU= 20.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:22 STEP 1545 H=1545:00 +CPU= 32.770
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:03 STEP 1546 H=1546:00 +CPU= 20.267
  16:00:24 STEP 1547 H=1547:00 +CPU= 21.004
  16:00:59 STEP 1548 H=1548:00 +CPU= 35.046
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhq0f+001548
  16:01:19 STEP 1549 H=1549:00 +CPU= 19.906
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:41 STEP 1550 H=1550:00 +CPU= 21.175
  16:03:13 STEP 1551 H=1551:00 +CPU= 32.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:07:53 1519251 22257734 oifs_43r3_ps_0541_1987050100_123_956_12173185_0 1,728,000 12,459 0.0072
09 May 2023 16:07:53 1519251 22257734 oifs_43r3_ps_0541_1987050100_123_956_12173185_0 1,728,000 12,459 0.0072
09 May 2023 16:07:49 1519251 22257734 oifs_43r3_ps_0541_1987050100_123_956_12173185_0 864,000 6,675 0.0077


©2024 cpdn.org