climateprediction.net home page
Task 22262585

Task 22262585

Name oifs_43r3_ps_0113_1992050100_123_961_12177757_0
Workunit 12177757
Created 21 Dec 2022, 23:34:51 UTC
Sent 4 Jan 2023, 14:25:03 UTC
Report deadline 3 Feb 2023, 14:25:03 UTC
Received 11 Jan 2023, 15:32:31 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1535230
Run time 12 hours 33 min 43 sec
CPU time 12 hours 23 min 2 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 6.81 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.79 MB
Peak swap size 4,978.08 MB
Peak disk usage 4,769.97 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
n: 10627200
result_base_name: oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307
The child process has been launched with process id: 3718
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:125354:1673006034:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:54:46 STEP 2568 H=2568:00 +CPU= 22.692
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_106.zip
Uploading the intermediate file: upload_file_106.zip
  12:54:58 STEP 2569 H=2569:00 +CPU= 12.268
  12:55:11 STEP 2570 H=2570:00 +CPU= 12.228
  12:55:31 STEP 2571 H=2571:00 +CPU= 20.467
  12:55:45 STEP 2572 H=2572:00 +CPU= 14.007
  12:55:59 STEP 2573 H=2573:00 +CPU= 12.955
  12:56:19 STEP 2574 H=2574:00 +CPU= 20.109
  12:56:32 STEP 2575 H=2575:00 +CPU= 13.168
  12:56:47 STEP 2576 H=2576:00 +CPU= 13.993
  12:57:09 STEP 2577 H=2577:00 +CPU= 21.665
  12:57:21 STEP 2578 H=2578:00 +CPU= 12.590
  12:57:34 STEP 2579 H=2579:00 +CPU= 12.132
  12:57:55 STEP 2580 H=2580:00 +CPU= 20.809
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002580
  12:58:06 STEP 2581 H=2581:00 +CPU= 11.805
  12:58:20 STEP 2582 H=2582:00 +CPU= 13.012
  12:58:40 STEP 2583 H=2583:00 +CPU= 20.104
  12:58:54 STEP 2584 H=2584:00 +CPU= 13.607
  12:59:07 STEP 2585 H=2585:00 +CPU= 12.809
  12:59:28 STEP 2586 H=2586:00 +CPU= 20.686
  12:59:41 STEP 2587 H=2587:00 +CPU= 13.282
  12:59:54 STEP 2588 H=2588:00 +CPU= 12.177
  13:00:13 STEP 2589 H=2589:00 +CPU= 19.513
  13:00:28 STEP 2590 H=2590:00 +CPU= 14.496
  13:00:42 STEP 2591 H=2591:00 +CPU= 13.876
  13:01:08 STEP 2592 H=2592:00 +CPU= 25.584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_107.zip
Uploading the intermediate file: upload_file_107.zip
  13:01:22 STEP 2593 H=2593:00 +CPU= 13.523
  13:01:36 STEP 2594 H=2594:00 +CPU= 13.693
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:02:29 STEP 2595 H=2595:00 +CPU= 22.881
  13:02:45 STEP 2596 H=2596:00 +CPU= 15.899
  13:03:00 STEP 2597 H=2597:00 +CPU= 14.696
  13:03:23 STEP 2598 H=2598:00 +CPU= 22.394
  13:03:36 STEP 2599 H=2599:00 +CPU= 13.052
  13:03:50 STEP 2600 H=2600:00 +CPU= 13.992
  13:04:12 STEP 2601 H=2601:00 +CPU= 21.563
  13:04:27 STEP 2602 H=2602:00 +CPU= 14.805
  13:04:40 STEP 2603 H=2603:00 +CPU= 13.456
  13:05:04 STEP 2604 H=2604:00 +CPU= 23.017
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002604
  13:05:18 STEP 2605 H=2605:00 +CPU= 14.795
  13:05:32 STEP 2606 H=2606:00 +CPU= 13.569
  13:05:53 STEP 2607 H=2607:00 +CPU= 20.813
  13:06:07 STEP 2608 H=2608:00 +CPU= 14.149
  13:06:22 STEP 2609 H=2609:00 +CPU= 14.789
  13:06:44 STEP 2610 H=2610:00 +CPU= 21.811
  13:06:59 STEP 2611 H=2611:00 +CPU= 14.332
  13:07:14 STEP 2612 H=2612:00 +CPU= 15.501
  13:07:36 STEP 2613 H=2613:00 +CPU= 21.434
  13:07:50 STEP 2614 H=2614:00 +CPU= 13.881
  13:08:04 STEP 2615 H=2615:00 +CPU= 13.747
  13:08:30 STEP 2616 H=2616:00 +CPU= 25.617
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_108.zip
Uploading the intermediate file: upload_file_108.zip
  13:08:44 STEP 2617 H=2617:00 +CPU= 14.129
  13:08:59 STEP 2618 H=2618:00 +CPU= 14.858
  13:09:20 STEP 2619 H=2619:00 +CPU= 20.608
  13:09:34 STEP 2620 H=2620:00 +CPU= 13.533
  13:09:48 STEP 2621 H=2621:00 +CPU= 14.068
  13:10:10 STEP 2622 H=2622:00 +CPU= 22.091
  13:10:25 STEP 2623 H=2623:00 +CPU= 14.419
  13:10:39 STEP 2624 H=2624:00 +CPU= 14.015
  13:11:00 STEP 2625 H=2625:00 +CPU= 21.241
  13:11:15 STEP 2626 H=2626:00 +CPU= 14.525
  13:11:29 STEP 2627 H=2627:00 +CPU= 14.158
  13:11:53 STEP 2628 H=2628:00 +CPU= 24.012
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002628
  13:12:07 STEP 2629 H=2629:00 +CPU= 13.996
  13:12:23 STEP 2630 H=2630:00 +CPU= 15.503
  13:12:45 STEP 2631 H=2631:00 +CPU= 21.996
  13:13:00 STEP 2632 H=2632:00 +CPU= 14.244
  13:13:14 STEP 2633 H=2633:00 +CPU= 14.034
  13:13:35 STEP 2634 H=2634:00 +CPU= 21.425
  13:13:50 STEP 2635 H=2635:00 +CPU= 14.864
  13:14:05 STEP 2636 H=2636:00 +CPU= 14.193
  13:14:25 STEP 2637 H=2637:00 +CPU= 20.522
  13:14:39 STEP 2638 H=2638:00 +CPU= 13.733
  13:14:54 STEP 2639 H=2639:00 +CPU= 14.392
  13:15:19 STEP 2640 H=2640:00 +CPU= 24.982
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_109.zip
Uploading the intermediate file: upload_file_109.zip
  13:15:33 STEP 2641 H=2641:00 +CPU= 13.827
  13:15:48 STEP 2642 H=2642:00 +CPU= 14.773
  13:16:09 STEP 2643 H=2643:00 +CPU= 21.056
  13:16:23 STEP 2644 H=2644:00 +CPU= 13.957
  13:16:37 STEP 2645 H=2645:00 +CPU= 13.262
  13:16:58 STEP 2646 H=2646:00 +CPU= 21.555
  13:17:14 STEP 2647 H=2647:00 +CPU= 15.157
  13:17:29 STEP 2648 H=2648:00 +CPU= 14.707
  13:17:50 STEP 2649 H=2649:00 +CPU= 21.356
  13:18:06 STEP 2650 H=2650:00 +CPU= 15.581
  13:18:20 STEP 2651 H=2651:00 +CPU= 13.607
  13:18:43 STEP 2652 H=2652:00 +CPU= 22.845
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002652
  13:18:57 STEP 2653 H=2653:00 +CPU= 14.408
  13:19:12 STEP 2654 H=2654:00 +CPU= 14.785
  13:19:34 STEP 2655 H=2655:00 +CPU= 21.382
  13:19:48 STEP 2656 H=2656:00 +CPU= 14.580
  13:20:02 STEP 2657 H=2657:00 +CPU= 13.270
  13:20:23 STEP 2658 H=2658:00 +CPU= 21.397
  13:20:36 STEP 2659 H=2659:00 +CPU= 13.050
  13:20:51 STEP 2660 H=2660:00 +CPU= 14.603
  13:21:13 STEP 2661 H=2661:00 +CPU= 21.310
  13:21:29 STEP 2662 H=2662:00 +CPU= 16.011
  13:21:43 STEP 2663 H=2663:00 +CPU= 13.846
  13:22:09 STEP 2664 H=2664:00 +CPU= 25.846
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_110.zip
Uploading the intermediate file: upload_file_110.zip
  13:22:23 STEP 2665 H=2665:00 +CPU= 13.410
  13:22:38 STEP 2666 H=2666:00 +CPU= 14.803
  13:22:59 STEP 2667 H=2667:00 +CPU= 21.125
  13:23:13 STEP 2668 H=2668:00 +CPU= 13.735
  13:23:26 STEP 2669 H=2669:00 +CPU= 13.585
  13:23:49 STEP 2670 H=2670:00 +CPU= 22.241
  13:24:03 STEP 2671 H=2671:00 +CPU= 13.757
  13:24:17 STEP 2672 H=2672:00 +CPU= 14.226
  13:24:38 STEP 2673 H=2673:00 +CPU= 20.722
  13:24:51 STEP 2674 H=2674:00 +CPU= 13.202
  13:25:05 STEP 2675 H=2675:00 +CPU= 13.329
  13:25:29 STEP 2676 H=2676:00 +CPU= 23.586
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002676
  13:25:42 STEP 2677 H=2677:00 +CPU= 13.411
  13:25:56 STEP 2678 H=2678:00 +CPU= 13.704
  13:26:16 STEP 2679 H=2679:00 +CPU= 20.343
  13:26:31 STEP 2680 H=2680:00 +CPU= 14.394
  13:26:45 STEP 2681 H=2681:00 +CPU= 13.791
  13:27:07 STEP 2682 H=2682:00 +CPU= 22.084
  13:27:21 STEP 2683 H=2683:00 +CPU= 13.646
  13:27:36 STEP 2684 H=2684:00 +CPU= 14.738
  13:27:59 STEP 2685 H=2685:00 +CPU= 23.121
  13:28:13 STEP 2686 H=2686:00 +CPU= 14.157
  13:28:27 STEP 2687 H=2687:00 +CPU= 13.923
  13:28:54 STEP 2688 H=2688:00 +CPU= 26.354
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_111.zip
Uploading the intermediate file: upload_file_111.zip
  13:29:08 STEP 2689 H=2689:00 +CPU= 13.190
  13:29:22 STEP 2690 H=2690:00 +CPU= 13.984
  13:29:43 STEP 2691 H=2691:00 +CPU= 21.257
  13:29:57 STEP 2692 H=2692:00 +CPU= 13.611
  13:30:11 STEP 2693 H=2693:00 +CPU= 14.268
  13:30:33 STEP 2694 H=2694:00 +CPU= 21.846
  13:30:46 STEP 2695 H=2695:00 +CPU= 12.957
  13:31:00 STEP 2696 H=2696:00 +CPU= 13.323
  13:31:23 STEP 2697 H=2697:00 +CPU= 22.752
  13:31:37 STEP 2698 H=2698:00 +CPU= 13.983
  13:31:51 STEP 2699 H=2699:00 +CPU= 13.727
  13:32:14 STEP 2700 H=2700:00 +CPU= 23.080
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002700
  13:32:27 STEP 2701 H=2701:00 +CPU= 13.184
  13:32:41 STEP 2702 H=2702:00 +CPU= 13.474
  13:33:02 STEP 2703 H=2703:00 +CPU= 20.640
  13:33:16 STEP 2704 H=2704:00 +CPU= 13.972
  13:33:30 STEP 2705 H=2705:00 +CPU= 14.002
  13:33:54 STEP 2706 H=2706:00 +CPU= 23.528
  13:34:07 STEP 2707 H=2707:00 +CPU= 13.735
  13:34:22 STEP 2708 H=2708:00 +CPU= 14.417
  13:34:44 STEP 2709 H=2709:00 +CPU= 22.155
  13:34:59 STEP 2710 H=2710:00 +CPU= 14.464
  13:35:12 STEP 2711 H=2711:00 +CPU= 13.403
  13:35:38 STEP 2712 H=2712:00 +CPU= 24.754
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_112.zip
Uploading the intermediate file: upload_file_112.zip
  13:35:52 STEP 2713 H=2713:00 +CPU= 13.876
  13:36:06 STEP 2714 H=2714:00 +CPU= 14.641
  13:36:28 STEP 2715 H=2715:00 +CPU= 21.017
  13:36:41 STEP 2716 H=2716:00 +CPU= 13.481
  13:36:55 STEP 2717 H=2717:00 +CPU= 14.000
  13:37:17 STEP 2718 H=2718:00 +CPU= 21.497
  13:37:30 STEP 2719 H=2719:00 +CPU= 12.981
  13:37:44 STEP 2720 H=2720:00 +CPU= 14.052
  13:38:07 STEP 2721 H=2721:00 +CPU= 22.836
  13:38:22 STEP 2722 H=2722:00 +CPU= 14.767
  13:38:35 STEP 2723 H=2723:00 +CPU= 13.142
  13:39:00 STEP 2724 H=2724:00 +CPU= 24.014
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002724
  13:39:13 STEP 2725 H=2725:00 +CPU= 13.143
  13:39:27 STEP 2726 H=2726:00 +CPU= 14.071
  13:39:49 STEP 2727 H=2727:00 +CPU= 21.701
  13:40:03 STEP 2728 H=2728:00 +CPU= 14.048
  13:40:17 STEP 2729 H=2729:00 +CPU= 13.610
  13:40:38 STEP 2730 H=2730:00 +CPU= 21.397
  13:40:52 STEP 2731 H=2731:00 +CPU= 13.275
  13:41:06 STEP 2732 H=2732:00 +CPU= 14.284
  13:41:28 STEP 2733 H=2733:00 +CPU= 21.641
  13:41:41 STEP 2734 H=2734:00 +CPU= 13.234
  13:41:54 STEP 2735 H=2735:00 +CPU= 12.010
  13:42:16 STEP 2736 H=2736:00 +CPU= 22.044
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_113.zip
Uploading the intermediate file: upload_file_113.zip
  13:42:28 STEP 2737 H=2737:00 +CPU= 12.062
  13:42:41 STEP 2738 H=2738:00 +CPU= 12.485
  13:42:59 STEP 2739 H=2739:00 +CPU= 18.583
  13:43:12 STEP 2740 H=2740:00 +CPU= 12.092
  13:43:23 STEP 2741 H=2741:00 +CPU= 11.639
  13:43:42 STEP 2742 H=2742:00 +CPU= 19.017
  13:43:54 STEP 2743 H=2743:00 +CPU= 11.820
  13:44:07 STEP 2744 H=2744:00 +CPU= 12.306
  13:44:26 STEP 2745 H=2745:00 +CPU= 18.809
  13:44:39 STEP 2746 H=2746:00 +CPU= 13.054
  13:44:52 STEP 2747 H=2747:00 +CPU= 12.486
  13:45:12 STEP 2748 H=2748:00 +CPU= 20.732
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002748
  13:45:24 STEP 2749 H=2749:00 +CPU= 11.524
  13:45:36 STEP 2750 H=2750:00 +CPU= 11.873
  13:45:55 STEP 2751 H=2751:00 +CPU= 19.158
  13:46:08 STEP 2752 H=2752:00 +CPU= 12.972
  13:46:20 STEP 2753 H=2753:00 +CPU= 11.786
  13:46:40 STEP 2754 H=2754:00 +CPU= 19.265
  13:46:52 STEP 2755 H=2755:00 +CPU= 11.866
  13:47:05 STEP 2756 H=2756:00 +CPU= 13.021
  13:47:23 STEP 2757 H=2757:00 +CPU= 18.473
  13:47:35 STEP 2758 H=2758:00 +CPU= 11.896
  13:47:48 STEP 2759 H=2759:00 +CPU= 12.683
  13:48:13 STEP 2760 H=2760:00 +CPU= 24.219
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_114.zip
Uploading the intermediate file: upload_file_114.zip
  13:48:24 STEP 2761 H=2761:00 +CPU= 11.565
  13:48:36 STEP 2762 H=2762:00 +CPU= 11.853
  13:48:55 STEP 2763 H=2763:00 +CPU= 18.368
  13:49:07 STEP 2764 H=2764:00 +CPU= 11.999
  13:49:19 STEP 2765 H=2765:00 +CPU= 11.749
  13:49:39 STEP 2766 H=2766:00 +CPU= 20.307
  13:49:52 STEP 2767 H=2767:00 +CPU= 12.499
  13:50:05 STEP 2768 H=2768:00 +CPU= 12.953
  13:50:24 STEP 2769 H=2769:00 +CPU= 18.884
  13:50:36 STEP 2770 H=2770:00 +CPU= 12.161
  13:50:48 STEP 2771 H=2771:00 +CPU= 11.699
  13:51:08 STEP 2772 H=2772:00 +CPU= 20.467
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002772
  13:51:20 STEP 2773 H=2773:00 +CPU= 11.716
  13:51:32 STEP 2774 H=2774:00 +CPU= 11.907
  13:51:50 STEP 2775 H=2775:00 +CPU= 18.295
  13:52:02 STEP 2776 H=2776:00 +CPU= 11.901
  13:52:14 STEP 2777 H=2777:00 +CPU= 11.633
  13:52:33 STEP 2778 H=2778:00 +CPU= 18.945
  13:52:45 STEP 2779 H=2779:00 +CPU= 11.583
  13:52:57 STEP 2780 H=2780:00 +CPU= 11.907
  13:53:15 STEP 2781 H=2781:00 +CPU= 18.318
  13:53:27 STEP 2782 H=2782:00 +CPU= 11.888
  13:53:39 STEP 2783 H=2783:00 +CPU= 11.944
  13:54:01 STEP 2784 H=2784:00 +CPU= 21.964
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_115.zip
Uploading the intermediate file: upload_file_115.zip
  13:54:13 STEP 2785 H=2785:00 +CPU= 11.558
  13:54:25 STEP 2786 H=2786:00 +CPU= 11.916
Uploading trickle at timestep: 10022400
  13:54:43 STEP 2787 H=2787:00 +CPU= 18.344
  13:54:55 STEP 2788 H=2788:00 +CPU= 11.873
  13:55:07 STEP 2789 H=2789:00 +CPU= 11.593
  13:55:25 STEP 2790 H=2790:00 +CPU= 18.739
  13:55:37 STEP 2791 H=2791:00 +CPU= 11.581
  13:55:49 STEP 2792 H=2792:00 +CPU= 11.891
  13:56:07 STEP 2793 H=2793:00 +CPU= 18.349
  13:56:19 STEP 2794 H=2794:00 +CPU= 11.858
  13:56:31 STEP 2795 H=2795:00 +CPU= 11.552
  13:56:51 STEP 2796 H=2796:00 +CPU= 20.326
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002796
  13:57:03 STEP 2797 H=2797:00 +CPU= 11.668
  13:57:15 STEP 2798 H=2798:00 +CPU= 12.071
  13:57:34 STEP 2799 H=2799:00 +CPU= 18.987
  13:57:47 STEP 2800 H=2800:00 +CPU= 12.749
  13:58:00 STEP 2801 H=2801:00 +CPU= 12.741
  13:58:21 STEP 2802 H=2802:00 +CPU= 20.773
  13:58:32 STEP 2803 H=2803:00 +CPU= 11.535
  13:58:44 STEP 2804 H=2804:00 +CPU= 11.856
  13:59:03 STEP 2805 H=2805:00 +CPU= 18.275
  13:59:14 STEP 2806 H=2806:00 +CPU= 11.862
  13:59:27 STEP 2807 H=2807:00 +CPU= 12.328
  13:59:51 STEP 2808 H=2808:00 +CPU= 23.644
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_116.zip
Uploading the intermediate file: upload_file_116.zip
  14:00:03 STEP 2809 H=2809:00 +CPU= 11.592
  14:00:14 STEP 2810 H=2810:00 +CPU= 11.828
  14:00:33 STEP 2811 H=2811:00 +CPU= 18.263
  14:00:45 STEP 2812 H=2812:00 +CPU= 11.916
  14:00:56 STEP 2813 H=2813:00 +CPU= 11.576
  14:01:16 STEP 2814 H=2814:00 +CPU= 19.786
  14:01:29 STEP 2815 H=2815:00 +CPU= 12.412
  14:01:42 STEP 2816 H=2816:00 +CPU= 13.200
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS CLDPP T/S    (1002) 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CONVECTION_LAYER 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 CUCALLN 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                  CUMASTRN 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                   CUBASEN 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222657:1673040417:34383.534] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                    CUADJTQ 
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222658:1673040418:34384.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:aspire:1:1:3718:3718] [20230106:222658:1673040418:34384.534] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000FBC238  cuadjtq_                  162  cuadjtq.F90
oifs_43r3_model.e  00000000010E68BD  cubasen_                  455  cubasen.F90
oifs_43r3_model.e  00000000010D0551  cumastrn_                 386  cumastrn.F90
oifs_43r3_model.e  00000000010C7792  cucalln_                  352  cucalln.F90
oifs_43r3_model.e  00000000010C4C15  convection_layer_         121  convection_layer.F90
oifs_43r3_model.e  0000000000F11F5B  callpar_                 1064  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1992050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0113
(argv4) batchid: 961
(argv5) wuid: 12177757
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_66d98eb2aa41ca6daaa2c2e52c639943 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0113_1992050100_123_961_12177757.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0113_1992050100_123_961_12177757.zip
ic_ancil_file: ic_ancil_12177757
ifsdata_file: ifsdata_12177757
climate_data_file: clim_data_12177757
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a01003235473fca505e7f8a435b64ab8 to: /var/lib/boinc-client/slots/0/ic_ancil_12177757.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12177757.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3e068bd42a73301947061f0993f37ca6 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12177757.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12177757.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12177757.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12177757.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12177757.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12177757.xml
-- Model is restarting --
Adjusting last_iter, 2817, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12177757.xml
last_cpu_time: 42530
upload_file_number: 117
last_iter: 2807
last_upload: 10108800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307
The child process has been launched with process id: 15132
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:aspire:1:1:15132:15132] [20230110:113940:1673347180:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  11:40:05 STEP 2808 H=2808:00 +CPU= 20.675
  11:40:17 STEP 2809 H=2809:00 +CPU= 11.487
  11:40:28 STEP 2810 H=2810:00 +CPU= 11.757
  11:40:47 STEP 2811 H=2811:00 +CPU= 18.181
  11:40:58 STEP 2812 H=2812:00 +CPU= 11.748
  11:41:10 STEP 2813 H=2813:00 +CPU= 11.484
  11:41:28 STEP 2814 H=2814:00 +CPU= 18.489
  11:41:40 STEP 2815 H=2815:00 +CPU= 11.482
  11:41:52 STEP 2816 H=2816:00 +CPU= 11.769
  11:42:10 STEP 2817 H=2817:00 +CPU= 18.155
  11:42:22 STEP 2818 H=2818:00 +CPU= 11.761
  11:42:33 STEP 2819 H=2819:00 +CPU= 11.471
  11:42:53 STEP 2820 H=2820:00 +CPU= 20.053
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002820
  11:43:05 STEP 2821 H=2821:00 +CPU= 11.488
  11:43:17 STEP 2822 H=2822:00 +CPU= 11.818
  11:43:35 STEP 2823 H=2823:00 +CPU= 18.173
  11:43:47 STEP 2824 H=2824:00 +CPU= 11.799
  11:43:58 STEP 2825 H=2825:00 +CPU= 11.426
  11:44:17 STEP 2826 H=2826:00 +CPU= 18.567
  11:44:28 STEP 2827 H=2827:00 +CPU= 11.417
  11:44:40 STEP 2828 H=2828:00 +CPU= 11.823
  11:44:58 STEP 2829 H=2829:00 +CPU= 18.114
  11:45:10 STEP 2830 H=2830:00 +CPU= 11.815
  11:45:22 STEP 2831 H=2831:00 +CPU= 11.773
  11:45:44 STEP 2832 H=2832:00 +CPU= 21.705
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_117.zip
Uploading the intermediate file: upload_file_117.zip
  11:45:55 STEP 2833 H=2833:00 +CPU= 11.459
  11:46:07 STEP 2834 H=2834:00 +CPU= 11.795
  11:46:26 STEP 2835 H=2835:00 +CPU= 18.262
  11:46:37 STEP 2836 H=2836:00 +CPU= 11.827
  11:46:49 STEP 2837 H=2837:00 +CPU= 11.593
  11:47:08 STEP 2838 H=2838:00 +CPU= 18.542
  11:47:19 STEP 2839 H=2839:00 +CPU= 11.502
  11:47:31 STEP 2840 H=2840:00 +CPU= 11.767
  11:47:49 STEP 2841 H=2841:00 +CPU= 18.235
  11:48:01 STEP 2842 H=2842:00 +CPU= 11.787
  11:48:13 STEP 2843 H=2843:00 +CPU= 11.504
  11:48:33 STEP 2844 H=2844:00 +CPU= 20.075
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002844
  11:48:44 STEP 2845 H=2845:00 +CPU= 11.544
  11:48:56 STEP 2846 H=2846:00 +CPU= 11.808
  11:49:14 STEP 2847 H=2847:00 +CPU= 18.205
  11:49:26 STEP 2848 H=2848:00 +CPU= 11.776
  11:49:38 STEP 2849 H=2849:00 +CPU= 11.483
  11:49:56 STEP 2850 H=2850:00 +CPU= 18.570
  11:50:08 STEP 2851 H=2851:00 +CPU= 11.467
  11:50:20 STEP 2852 H=2852:00 +CPU= 11.830
  11:50:38 STEP 2853 H=2853:00 +CPU= 18.124
  11:50:50 STEP 2854 H=2854:00 +CPU= 11.828
  11:51:02 STEP 2855 H=2855:00 +CPU= 11.785
  11:51:24 STEP 2856 H=2856:00 +CPU= 21.793
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_118.zip
Uploading the intermediate file: upload_file_118.zip
  11:51:35 STEP 2857 H=2857:00 +CPU= 11.425
  11:51:47 STEP 2858 H=2858:00 +CPU= 11.864
  11:52:05 STEP 2859 H=2859:00 +CPU= 18.130
  11:52:17 STEP 2860 H=2860:00 +CPU= 11.841
  11:52:29 STEP 2861 H=2861:00 +CPU= 11.416
  11:52:47 STEP 2862 H=2862:00 +CPU= 18.611
  11:52:59 STEP 2863 H=2863:00 +CPU= 11.439
  11:53:11 STEP 2864 H=2864:00 +CPU= 11.805
  11:53:29 STEP 2865 H=2865:00 +CPU= 18.167
  11:53:41 STEP 2866 H=2866:00 +CPU= 11.828
  11:53:52 STEP 2867 H=2867:00 +CPU= 11.524
  11:54:12 STEP 2868 H=2868:00 +CPU= 20.070
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002868
  11:54:24 STEP 2869 H=2869:00 +CPU= 11.490
  11:54:36 STEP 2870 H=2870:00 +CPU= 11.764
  11:54:54 STEP 2871 H=2871:00 +CPU= 18.222
  11:55:06 STEP 2872 H=2872:00 +CPU= 11.773
  11:55:17 STEP 2873 H=2873:00 +CPU= 11.532
  11:55:36 STEP 2874 H=2874:00 +CPU= 18.487
  11:55:47 STEP 2875 H=2875:00 +CPU= 11.535
  11:55:59 STEP 2876 H=2876:00 +CPU= 11.775
  11:56:17 STEP 2877 H=2877:00 +CPU= 18.175
  11:56:29 STEP 2878 H=2878:00 +CPU= 11.761
  11:56:41 STEP 2879 H=2879:00 +CPU= 11.804
  11:57:03 STEP 2880 H=2880:00 +CPU= 21.848
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_119.zip
Uploading the intermediate file: upload_file_119.zip
  11:57:15 STEP 2881 H=2881:00 +CPU= 11.450
  11:57:27 STEP 2882 H=2882:00 +CPU= 11.810
  11:57:45 STEP 2883 H=2883:00 +CPU= 18.164
  11:57:57 STEP 2884 H=2884:00 +CPU= 11.869
  11:58:08 STEP 2885 H=2885:00 +CPU= 11.445
  11:58:27 STEP 2886 H=2886:00 +CPU= 18.572
  11:58:38 STEP 2887 H=2887:00 +CPU= 11.452
  11:58:50 STEP 2888 H=2888:00 +CPU= 11.891
  11:59:08 STEP 2889 H=2889:00 +CPU= 18.138
  11:59:20 STEP 2890 H=2890:00 +CPU= 11.876
  11:59:32 STEP 2891 H=2891:00 +CPU= 11.454
  11:59:52 STEP 2892 H=2892:00 +CPU= 20.157
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002892
  12:00:04 STEP 2893 H=2893:00 +CPU= 11.467
  12:00:15 STEP 2894 H=2894:00 +CPU= 11.842
  12:00:34 STEP 2895 H=2895:00 +CPU= 18.186
  12:00:46 STEP 2896 H=2896:00 +CPU= 11.807
  12:00:57 STEP 2897 H=2897:00 +CPU= 11.566
  12:01:16 STEP 2898 H=2898:00 +CPU= 18.553
  12:01:27 STEP 2899 H=2899:00 +CPU= 11.484
  12:01:39 STEP 2900 H=2900:00 +CPU= 11.750
  12:01:57 STEP 2901 H=2901:00 +CPU= 18.224
  12:02:09 STEP 2902 H=2902:00 +CPU= 11.760
  12:02:21 STEP 2903 H=2903:00 +CPU= 11.859
  12:02:43 STEP 2904 H=2904:00 +CPU= 21.737
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_120.zip
Uploading the intermediate file: upload_file_120.zip
  12:02:55 STEP 2905 H=2905:00 +CPU= 11.545
  12:03:07 STEP 2906 H=2906:00 +CPU= 11.762
  12:03:25 STEP 2907 H=2907:00 +CPU= 18.229
  12:03:37 STEP 2908 H=2908:00 +CPU= 11.774
  12:03:48 STEP 2909 H=2909:00 +CPU= 11.486
  12:04:07 STEP 2910 H=2910:00 +CPU= 18.581
  12:04:18 STEP 2911 H=2911:00 +CPU= 11.495
  12:04:30 STEP 2912 H=2912:00 +CPU= 11.857
  12:04:48 STEP 2913 H=2913:00 +CPU= 18.180
  12:05:00 STEP 2914 H=2914:00 +CPU= 11.879
  12:05:12 STEP 2915 H=2915:00 +CPU= 11.460
  12:05:32 STEP 2916 H=2916:00 +CPU= 20.151
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002916
  12:05:43 STEP 2917 H=2917:00 +CPU= 11.449
  12:05:55 STEP 2918 H=2918:00 +CPU= 11.888
  12:06:14 STEP 2919 H=2919:00 +CPU= 18.142
  12:06:25 STEP 2920 H=2920:00 +CPU= 11.861
  12:06:37 STEP 2921 H=2921:00 +CPU= 11.450
  12:06:56 STEP 2922 H=2922:00 +CPU= 18.560
  12:07:07 STEP 2923 H=2923:00 +CPU= 11.480
  12:07:19 STEP 2924 H=2924:00 +CPU= 11.805
  12:07:37 STEP 2925 H=2925:00 +CPU= 18.161
  12:07:49 STEP 2926 H=2926:00 +CPU= 11.807
  12:08:01 STEP 2927 H=2927:00 +CPU= 11.869
  12:08:23 STEP 2928 H=2928:00 +CPU= 21.785
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_121.zip
Uploading the intermediate file: upload_file_121.zip
  12:08:35 STEP 2929 H=2929:00 +CPU= 11.521
  12:08:46 STEP 2930 H=2930:00 +CPU= 11.765
  12:09:05 STEP 2931 H=2931:00 +CPU= 18.205
  12:09:16 STEP 2932 H=2932:00 +CPU= 11.767
  12:09:28 STEP 2933 H=2933:00 +CPU= 11.517
  12:09:47 STEP 2934 H=2934:00 +CPU= 18.522
  12:09:58 STEP 2935 H=2935:00 +CPU= 11.530
  12:10:10 STEP 2936 H=2936:00 +CPU= 11.795
  12:10:28 STEP 2937 H=2937:00 +CPU= 18.223
  12:10:40 STEP 2938 H=2938:00 +CPU= 11.789
  12:10:52 STEP 2939 H=2939:00 +CPU= 11.481
  12:11:12 STEP 2940 H=2940:00 +CPU= 20.129
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002940
  12:11:23 STEP 2941 H=2941:00 +CPU= 11.527
  12:11:35 STEP 2942 H=2942:00 +CPU= 11.843
  12:11:53 STEP 2943 H=2943:00 +CPU= 18.168
  12:12:05 STEP 2944 H=2944:00 +CPU= 11.836
  12:12:17 STEP 2945 H=2945:00 +CPU= 11.441
  12:12:35 STEP 2946 H=2946:00 +CPU= 18.576
  12:12:47 STEP 2947 H=2947:00 +CPU= 11.449
  12:12:59 STEP 2948 H=2948:00 +CPU= 11.853
  12:13:17 STEP 2949 H=2949:00 +CPU= 18.156
  12:13:29 STEP 2950 H=2950:00 +CPU= 11.831
  12:13:41 STEP 2951 H=2951:00 +CPU= 11.801
  12:14:02 STEP 2952 H=2952:00 +CPU= 21.081
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177757/ICMUAhq0f+002928
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0113_1992050100_123_961_12177757_0_r1372723307_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
12:17:25 (15127): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:14:37 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 10,623,600 44,582 0.0042
09 May 2023 16:10:26 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 10,022,400 42,071 0.0042
09 May 2023 16:09:04 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 9,072,000 37,871 0.0042
09 May 2023 16:09:00 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 8,208,000 34,415 0.0042
09 May 2023 16:08:57 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 7,344,000 30,939 0.0042
09 May 2023 16:08:52 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 6,480,000 27,329 0.0042
09 May 2023 16:08:48 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 5,616,000 23,530 0.0042
09 May 2023 16:08:44 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 4,752,000 19,374 0.0041
09 May 2023 16:08:41 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 3,456,000 14,031 0.0041
09 May 2023 16:08:02 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 2,592,000 10,577 0.0041
09 May 2023 16:08:00 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 1,728,000 7,063 0.0041
09 May 2023 16:07:56 1535230 22262585 oifs_43r3_ps_0113_1992050100_123_961_12177757_0 864,000 3,548 0.0041


©2024 cpdn.org