<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
port/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:223303:1672954383:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:34:54 STEP 1392 H=1392:00 +CPU= 33.750
22:35:15 STEP 1393 H=1393:00 +CPU= 19.078
22:35:35 STEP 1394 H=1394:00 +CPU= 19.358
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:36:17 STEP 1395 H=1395:00 +CPU= 28.974
22:36:37 STEP 1396 H=1396:00 +CPU= 18.623
22:36:56 STEP 1397 H=1397:00 +CPU= 18.017
22:37:26 STEP 1398 H=1398:00 +CPU= 28.311
22:37:45 STEP 1399 H=1399:00 +CPU= 17.642
22:38:04 STEP 1400 H=1400:00 +CPU= 18.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:38:46 STEP 1401 H=1401:00 +CPU= 29.393
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:39 STEP 1402 H=1402:00 +CPU= 20.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:40:11 STEP 1403 H=1403:00 +CPU= 19.719
22:40:46 STEP 1404 H=1404:00 +CPU= 32.148
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001404
22:41:05 STEP 1405 H=1405:00 +CPU= 18.254
22:41:25 STEP 1406 H=1406:00 +CPU= 18.805
22:41:55 STEP 1407 H=1407:00 +CPU= 28.503
22:42:15 STEP 1408 H=1408:00 +CPU= 18.661
22:42:35 STEP 1409 H=1409:00 +CPU= 18.178
22:43:06 STEP 1410 H=1410:00 +CPU= 29.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:43:38 STEP 1411 H=1411:00 +CPU= 19.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:10 STEP 1412 H=1412:00 +CPU= 19.739
22:44:39 STEP 1413 H=1413:00 +CPU= 27.790
22:44:58 STEP 1414 H=1414:00 +CPU= 18.105
22:45:18 STEP 1415 H=1415:00 +CPU= 18.598
22:45:54 STEP 1416 H=1416:00 +CPU= 33.521
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_58.zip
Uploading the intermediate file: upload_file_58.zip
22:46:13 STEP 1417 H=1417:00 +CPU= 17.978
22:46:33 STEP 1418 H=1418:00 +CPU= 18.637
22:47:04 STEP 1419 H=1419:00 +CPU= 28.615
22:47:23 STEP 1420 H=1420:00 +CPU= 18.324
22:47:43 STEP 1421 H=1421:00 +CPU= 18.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:48:26 STEP 1422 H=1422:00 +CPU= 30.373
22:48:47 STEP 1423 H=1423:00 +CPU= 18.982
22:49:07 STEP 1424 H=1424:00 +CPU= 18.807
22:49:37 STEP 1425 H=1425:00 +CPU= 28.564
22:49:58 STEP 1426 H=1426:00 +CPU= 19.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:50:29 STEP 1427 H=1427:00 +CPU= 18.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:28 STEP 1428 H=1428:00 +CPU= 34.420
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001428
22:51:49 STEP 1429 H=1429:00 +CPU= 19.461
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:52:22 STEP 1430 H=1430:00 +CPU= 19.901
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:05 STEP 1431 H=1431:00 +CPU= 29.777
22:53:25 STEP 1432 H=1432:00 +CPU= 19.037
22:53:45 STEP 1433 H=1433:00 +CPU= 18.116
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:54:28 STEP 1434 H=1434:00 +CPU= 30.493
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:55:21 STEP 1435 H=1435:00 +CPU= 19.494
22:55:41 STEP 1436 H=1436:00 +CPU= 18.634
22:56:12 STEP 1437 H=1437:00 +CPU= 28.722
22:56:31 STEP 1438 H=1438:00 +CPU= 18.725
22:56:51 STEP 1439 H=1439:00 +CPU= 18.705
22:57:27 STEP 1440 H=1440:00 +CPU= 33.856
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_59.zip
Uploading the intermediate file: upload_file_59.zip
22:57:49 STEP 1441 H=1441:00 +CPU= 19.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:58:11 STEP 1442 H=1442:00 +CPU= 18.964
22:58:41 STEP 1443 H=1443:00 +CPU= 28.516
22:59:00 STEP 1444 H=1444:00 +CPU= 18.296
22:59:20 STEP 1445 H=1445:00 +CPU= 18.555
22:59:51 STEP 1446 H=1446:00 +CPU= 29.119
23:00:10 STEP 1447 H=1447:00 +CPU= 17.956
23:00:30 STEP 1448 H=1448:00 +CPU= 19.024
23:00:59 STEP 1449 H=1449:00 +CPU= 27.688
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:01:30 STEP 1450 H=1450:00 +CPU= 19.081
23:01:50 STEP 1451 H=1451:00 +CPU= 19.114
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:02:36 STEP 1452 H=1452:00 +CPU= 32.632
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001452
23:02:56 STEP 1453 H=1453:00 +CPU= 18.096
23:03:16 STEP 1454 H=1454:00 +CPU= 18.876
23:03:47 STEP 1455 H=1455:00 +CPU= 29.088
23:04:07 STEP 1456 H=1456:00 +CPU= 19.003
23:04:26 STEP 1457 H=1457:00 +CPU= 18.449
23:04:58 STEP 1458 H=1458:00 +CPU= 29.382
23:05:17 STEP 1459 H=1459:00 +CPU= 18.654
23:05:38 STEP 1460 H=1460:00 +CPU= 18.882
23:06:08 STEP 1461 H=1461:00 +CPU= 28.793
23:06:28 STEP 1462 H=1462:00 +CPU= 18.704
23:06:48 STEP 1463 H=1463:00 +CPU= 19.061
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:07:37 STEP 1464 H=1464:00 +CPU= 34.413
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_60.zip
Uploading the intermediate file: upload_file_60.zip
23:07:57 STEP 1465 H=1465:00 +CPU= 18.748
23:08:17 STEP 1466 H=1466:00 +CPU= 19.053
23:08:48 STEP 1467 H=1467:00 +CPU= 29.183
23:09:09 STEP 1468 H=1468:00 +CPU= 19.055
23:09:28 STEP 1469 H=1469:00 +CPU= 18.638
23:10:00 STEP 1470 H=1470:00 +CPU= 29.566
23:10:20 STEP 1471 H=1471:00 +CPU= 18.783
23:10:40 STEP 1472 H=1472:00 +CPU= 19.265
23:11:12 STEP 1473 H=1473:00 +CPU= 29.422
23:11:32 STEP 1474 H=1474:00 +CPU= 19.268
23:11:52 STEP 1475 H=1475:00 +CPU= 18.787
23:12:26 STEP 1476 H=1476:00 +CPU= 32.407
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001476
23:12:47 STEP 1477 H=1477:00 +CPU= 18.890
23:13:07 STEP 1478 H=1478:00 +CPU= 19.010
23:13:38 STEP 1479 H=1479:00 +CPU= 29.271
23:13:59 STEP 1480 H=1480:00 +CPU= 19.103
23:14:18 STEP 1481 H=1481:00 +CPU= 18.376
23:14:50 STEP 1482 H=1482:00 +CPU= 29.634
23:15:10 STEP 1483 H=1483:00 +CPU= 18.761
23:15:30 STEP 1484 H=1484:00 +CPU= 19.016
23:16:01 STEP 1485 H=1485:00 +CPU= 29.188
23:16:22 STEP 1486 H=1486:00 +CPU= 19.287
23:16:42 STEP 1487 H=1487:00 +CPU= 19.129
23:17:18 STEP 1488 H=1488:00 +CPU= 34.232
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_61.zip
Uploading the intermediate file: upload_file_61.zip
23:17:38 STEP 1489 H=1489:00 +CPU= 18.521
23:17:59 STEP 1490 H=1490:00 +CPU= 19.196
23:18:30 STEP 1491 H=1491:00 +CPU= 29.456
23:18:50 STEP 1492 H=1492:00 +CPU= 19.073
23:19:10 STEP 1493 H=1493:00 +CPU= 18.513
23:19:41 STEP 1494 H=1494:00 +CPU= 29.763
23:20:01 STEP 1495 H=1495:00 +CPU= 18.631
23:20:22 STEP 1496 H=1496:00 +CPU= 19.334
23:20:53 STEP 1497 H=1497:00 +CPU= 29.503
23:21:13 STEP 1498 H=1498:00 +CPU= 19.171
23:21:33 STEP 1499 H=1499:00 +CPU= 18.742
23:22:05 STEP 1500 H=1500:00 +CPU= 30.638
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001500
23:22:23 STEP 1501 H=1501:00 +CPU= 17.436
23:22:42 STEP 1502 H=1502:00 +CPU= 17.776
23:23:10 STEP 1503 H=1503:00 +CPU= 27.212
23:23:28 STEP 1504 H=1504:00 +CPU= 17.661
23:23:47 STEP 1505 H=1505:00 +CPU= 17.460
23:24:16 STEP 1506 H=1506:00 +CPU= 27.895
23:24:34 STEP 1507 H=1507:00 +CPU= 17.309
23:24:52 STEP 1508 H=1508:00 +CPU= 17.654
23:25:22 STEP 1509 H=1509:00 +CPU= 28.524
23:25:45 STEP 1510 H=1510:00 +CPU= 21.349
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:26:15 STEP 1511 H=1511:00 +CPU= 18.075
23:26:47 STEP 1512 H=1512:00 +CPU= 31.082
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_62.zip
Uploading the intermediate file: upload_file_62.zip
23:27:05 STEP 1513 H=1513:00 +CPU= 17.454
23:27:24 STEP 1514 H=1514:00 +CPU= 18.117
23:27:53 STEP 1515 H=1515:00 +CPU= 27.763
23:28:11 STEP 1516 H=1516:00 +CPU= 17.906
23:28:29 STEP 1517 H=1517:00 +CPU= 17.336
23:28:59 STEP 1518 H=1518:00 +CPU= 28.777
23:29:18 STEP 1519 H=1519:00 +CPU= 18.058
23:29:37 STEP 1520 H=1520:00 +CPU= 17.849
23:30:06 STEP 1521 H=1521:00 +CPU= 27.885
23:30:26 STEP 1522 H=1522:00 +CPU= 19.074
23:30:46 STEP 1523 H=1523:00 +CPU= 18.470
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:30 STEP 1524 H=1524:00 +CPU= 31.733
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001524
23:31:49 STEP 1525 H=1525:00 +CPU= 17.997
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:32:21 STEP 1526 H=1526:00 +CPU= 19.658
23:32:50 STEP 1527 H=1527:00 +CPU= 27.734
23:33:10 STEP 1528 H=1528:00 +CPU= 19.081
23:33:29 STEP 1529 H=1529:00 +CPU= 17.616
23:33:58 STEP 1530 H=1530:00 +CPU= 28.060
23:34:17 STEP 1531 H=1531:00 +CPU= 18.178
23:34:38 STEP 1532 H=1532:00 +CPU= 19.445
23:35:07 STEP 1533 H=1533:00 +CPU= 27.929
23:35:26 STEP 1534 H=1534:00 +CPU= 18.241
23:35:47 STEP 1535 H=1535:00 +CPU= 19.800
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:36:59 STEP 1536 H=1536:00 +CPU= 36.310
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_63.zip
Uploading the intermediate file: upload_file_63.zip
23:37:18 STEP 1537 H=1537:00 +CPU= 18.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:38:51 STEP 1538 H=1538:00 +CPU= 20.101
23:39:22 STEP 1539 H=1539:00 +CPU= 29.094
23:39:42 STEP 1540 H=1540:00 +CPU= 19.073
23:40:01 STEP 1541 H=1541:00 +CPU= 17.751
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:40:54 STEP 1542 H=1542:00 +CPU= 30.202
23:41:12 STEP 1543 H=1543:00 +CPU= 17.667
23:41:32 STEP 1544 H=1544:00 +CPU= 19.210
23:42:03 STEP 1545 H=1545:00 +CPU= 28.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:42:35 STEP 1546 H=1546:00 +CPU= 19.414
23:42:54 STEP 1547 H=1547:00 +CPU= 17.744
23:43:27 STEP 1548 H=1548:00 +CPU= 31.077
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001548
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:43:58 STEP 1549 H=1549:00 +CPU= 19.604
23:44:18 STEP 1550 H=1550:00 +CPU= 19.039
23:44:48 STEP 1551 H=1551:00 +CPU= 28.228
23:45:08 STEP 1552 H=1552:00 +CPU= 18.989
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:45:39 STEP 1553 H=1553:00 +CPU= 18.648
23:46:09 STEP 1554 H=1554:00 +CPU= 29.038
23:46:29 STEP 1555 H=1555:00 +CPU= 18.265
23:46:48 STEP 1556 H=1556:00 +CPU= 18.158
23:47:16 STEP 1557 H=1557:00 +CPU= 27.463
23:47:35 STEP 1558 H=1558:00 +CPU= 17.879
23:47:55 STEP 1559 H=1559:00 +CPU= 18.671
23:48:29 STEP 1560 H=1560:00 +CPU= 32.518
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_64.zip
Uploading the intermediate file: upload_file_64.zip
23:48:49 STEP 1561 H=1561:00 +CPU= 19.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:45 STEP 1562 H=1562:00 +CPU= 19.632
23:50:14 STEP 1563 H=1563:00 +CPU= 27.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:50:45 STEP 1564 H=1564:00 +CPU= 18.770
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.678] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001)
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235059:1672959059:4675.685] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235100:1672959060:4676.685] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:4908:4908] [20230105:235100:1672959060:4676.685] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DB4720 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001020C3C cloudsc_ 1028 cloudsc.F90
oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90
oifs_43r3_model.e 0000000000F10C1B callpar_ 1011 callpar.F90
oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2016050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0120
(argv4) batchid: 985
(argv5) wuid: 12201764
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7cabd2623b3f788ecf64d6fa854034ae to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0120_2016050100_123_985_12201764.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0120_2016050100_123_985_12201764.zip
ic_ancil_file: ic_ancil_12201764
ifsdata_file: ifsdata_12201764
climate_data_file: clim_data_12201764
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_39d3b031e696818e3625e2d8af949d9c to: /var/lib/boinc-client/slots/0/ic_ancil_12201764.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12201764.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_131e660ce6ff80ac745fc0dbe9804c1e to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201764.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201764.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201764.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201764.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12201764.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12201764.xml
-- Model is restarting --
Adjusting last_iter, 1565, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12201764.xml
last_cpu_time: 37308
upload_file_number: 65
last_iter: 1559
last_upload: 5616000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916
The child process has been launched with process id: 2123
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:084027:1672990827:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:41:50 STEP 1560 H=1560:00 +CPU= 33.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:43:13 STEP 1561 H=1561:00 +CPU= 18.836
08:43:31 STEP 1562 H=1562:00 +CPU= 17.537
08:43:59 STEP 1563 H=1563:00 +CPU= 27.122
08:44:17 STEP 1564 H=1564:00 +CPU= 17.774
08:44:36 STEP 1565 H=1565:00 +CPU= 17.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:46:00 STEP 1566 H=1566:00 +CPU= 30.729
08:46:18 STEP 1567 H=1567:00 +CPU= 17.352
08:46:37 STEP 1568 H=1568:00 +CPU= 18.015
08:47:08 STEP 1569 H=1569:00 +CPU= 28.960
08:47:27 STEP 1570 H=1570:00 +CPU= 17.882
08:47:46 STEP 1571 H=1571:00 +CPU= 18.234
08:48:19 STEP 1572 H=1572:00 +CPU= 31.032
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001572
08:48:38 STEP 1573 H=1573:00 +CPU= 17.682
08:48:57 STEP 1574 H=1574:00 +CPU= 18.260
08:49:27 STEP 1575 H=1575:00 +CPU= 28.174
08:49:47 STEP 1576 H=1576:00 +CPU= 19.031
08:50:06 STEP 1577 H=1577:00 +CPU= 17.817
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:51:23 STEP 1578 H=1578:00 +CPU= 31.861
08:51:40 STEP 1579 H=1579:00 +CPU= 16.935
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:52:14 STEP 1580 H=1580:00 +CPU= 21.384
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:53:08 STEP 1581 H=1581:00 +CPU= 29.646
08:53:26 STEP 1582 H=1582:00 +CPU= 17.951
08:53:44 STEP 1583 H=1583:00 +CPU= 17.293
08:54:21 STEP 1584 H=1584:00 +CPU= 34.906
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916_65.zip
Uploading the intermediate file: upload_file_65.zip
08:54:53 STEP 1585 H=1585:00 +CPU= 19.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
08:55:28 STEP 1586 H=1586:00 +CPU= 19.367
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS CLDPP T/S (1002)
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFOUTER
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFMAIN
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085549:1672991749:922.284] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFDIFH
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085550:1672991750:923.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Olymp:1:1:2123:2123] [20230106:085550:1672991750:923.284] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000FBE5B0 vdfdifh_ 294 vdfdifh.F90
oifs_43r3_model.e 0000000000FADFE1 vdfmain_ 1171 vdfmain.F90
oifs_43r3_model.e 0000000000F92699 vdfouter_ 739 vdfouter.F90
oifs_43r3_model.e 0000000000F88512 turbulence_layer_ 158 turbulence_layer.F90
oifs_43r3_model.e 0000000000F0B683 callpar_ 869 callpar.F90
oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2016050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0120
(argv4) batchid: 985
(argv5) wuid: 12201764
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201764
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7cabd2623b3f788ecf64d6fa854034ae to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0120_2016050100_123_985_12201764.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0120_2016050100_123_985_12201764.zip
ic_ancil_file: ic_ancil_12201764
ifsdata_file: ifsdata_12201764
climate_data_file: clim_data_12201764
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_39d3b031e696818e3625e2d8af949d9c to: /var/lib/boinc-client/slots/0/ic_ancil_12201764.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12201764.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_131e660ce6ff80ac745fc0dbe9804c1e to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201764.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201764.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201764.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201764.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12201764.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12201764.xml
-- Model is restarting --
Adjusting last_iter, 1587, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12201764.xml
last_cpu_time: 37929
upload_file_number: 66
last_iter: 1583
last_upload: 5702400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0120_2016050100_123_985_12201764_0_r1354201916
The child process has been launched with process id: 2331
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Olymp:1:1:2331:2331] [20230106:120532:1673003132:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:06:53 STEP 1584 H=1584:00 +CPU= 31.685
12:07:11 STEP 1585 H=1585:00 +CPU= 16.919
12:07:29 STEP 1586 H=1586:00 +CPU= 17.556
12:07:58 STEP 1587 H=1587:00 +CPU= 27.255
12:08:15 STEP 1588 H=1588:00 +CPU= 17.199
12:08:33 STEP 1589 H=1589:00 +CPU= 16.775
12:09:00 STEP 1590 H=1590:00 +CPU= 27.102
12:09:18 STEP 1591 H=1591:00 +CPU= 16.800
12:09:36 STEP 1592 H=1592:00 +CPU= 17.288
12:10:03 STEP 1593 H=1593:00 +CPU= 26.552
12:10:21 STEP 1594 H=1594:00 +CPU= 17.297
12:10:38 STEP 1595 H=1595:00 +CPU= 16.784
12:11:08 STEP 1596 H=1596:00 +CPU= 29.394
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001596
12:11:26 STEP 1597 H=1597:00 +CPU= 16.831
12:11:43 STEP 1598 H=1598:00 +CPU= 17.327
12:12:11 STEP 1599 H=1599:00 +CPU= 26.632
12:12:29 STEP 1600 H=1600:00 +CPU= 17.295
12:12:46 STEP 1601 H=1601:00 +CPU= 16.767
12:13:14 STEP 1602 H=1602:00 +CPU= 27.034
12:13:31 STEP 1603 H=1603:00 +CPU= 16.850
12:13:49 STEP 1604 H=1604:00 +CPU= 17.501
12:14:17 STEP 1605 H=1605:00 +CPU= 27.012
12:14:36 STEP 1606 H=1606:00 +CPU= 17.641
12:14:54 STEP 1607 H=1607:00 +CPU= 17.830
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
