climateprediction.net home page
Task 22315146

Task 22315146

Name oifs_43r3_009z_2019110100_123_993_12213797_0
Workunit 12213797
Created 24 Feb 2023, 12:44:42 UTC
Sent 24 Feb 2023, 19:15:22 UTC
Report deadline 25 Apr 2023, 19:15:22 UTC
Received 27 Feb 2023, 18:18:42 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1519251
Run time 21 hours 41 min 39 sec
CPU time 21 hours 6 min 40 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 4.98 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,423.46 MB
Peak swap size 4,984.14 MB
Peak disk usage 1,540.54 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002760
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_114.zip
Uploading the intermediate file: upload_file_114.zip
  22:58:53 STEP 2762 H=2762:00 +CPU= 19.916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:01:16 STEP 2763 H=2763:00 +CPU= 31.901
  23:01:36 STEP 2764 H=2764:00 +CPU= 20.183
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:03:07 STEP 2765 H=2765:00 +CPU= 19.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:04:42 STEP 2766 H=2766:00 +CPU= 33.931
  23:05:02 STEP 2767 H=2767:00 +CPU= 20.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:06:55 STEP 2768 H=2768:00 +CPU= 20.775
  23:07:28 STEP 2769 H=2769:00 +CPU= 32.872
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:09:01 STEP 2770 H=2770:00 +CPU= 21.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:10:42 STEP 2771 H=2771:00 +CPU= 20.231
  23:11:16 STEP 2772 H=2772:00 +CPU= 33.932
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002772
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002772
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:12:58 STEP 2773 H=2773:00 +CPU= 20.006
  23:13:18 STEP 2774 H=2774:00 +CPU= 20.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:15:10 STEP 2775 H=2775:00 +CPU= 31.600
  23:15:31 STEP 2776 H=2776:00 +CPU= 20.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:17:11 STEP 2777 H=2777:00 +CPU= 19.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:18:46 STEP 2778 H=2778:00 +CPU= 32.720
  23:19:06 STEP 2779 H=2779:00 +CPU= 19.357
  23:19:26 STEP 2780 H=2780:00 +CPU= 20.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:21:19 STEP 2781 H=2781:00 +CPU= 31.823
  23:21:39 STEP 2782 H=2782:00 +CPU= 20.295
  23:21:59 STEP 2783 H=2783:00 +CPU= 20.146
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:24:58 STEP 2784 H=2784:00 +CPU= 37.735
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002784
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002784
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002784
  23:25:18 STEP 2785 H=2785:00 +CPU= 19.942
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002784
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_115.zip
Uploading the intermediate file: upload_file_115.zip
  23:27:10 STEP 2786 H=2786:00 +CPU= 20.858
  23:27:41 STEP 2787 H=2787:00 +CPU= 30.917
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:29:22 STEP 2788 H=2788:00 +CPU= 21.000
  23:29:43 STEP 2789 H=2789:00 +CPU= 21.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:31:58 STEP 2790 H=2790:00 +CPU= 33.520
  23:32:18 STEP 2791 H=2791:00 +CPU= 19.683
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:34:10 STEP 2792 H=2792:00 +CPU= 20.529
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:35:54 STEP 2793 H=2793:00 +CPU= 32.903
  23:36:14 STEP 2794 H=2794:00 +CPU= 20.853
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:38:17 STEP 2795 H=2795:00 +CPU= 19.932
  23:38:52 STEP 2796 H=2796:00 +CPU= 35.731
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002796
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002796
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002796
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:40:24 STEP 2797 H=2797:00 +CPU= 20.394
  23:40:45 STEP 2798 H=2798:00 +CPU= 21.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:42:38 STEP 2799 H=2799:00 +CPU= 32.381
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:44:09 STEP 2800 H=2800:00 +CPU= 21.037
  23:44:29 STEP 2801 H=2801:00 +CPU= 19.946
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:46:23 STEP 2802 H=2802:00 +CPU= 32.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:48:05 STEP 2803 H=2803:00 +CPU= 19.894
  23:48:26 STEP 2804 H=2804:00 +CPU= 21.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:50:28 STEP 2805 H=2805:00 +CPU= 30.900
  23:50:50 STEP 2806 H=2806:00 +CPU= 21.956
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:52:22 STEP 2807 H=2807:00 +CPU= 20.509
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 009z
(argv4) batchid: 993
(argv5) wuid: 12213797
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_88f624dba5097c4bd2242ad796053ca0 to: /var/lib/boinc-client/slots/1/oifs_43r3_009z_2019110100_123_993_12213797.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_009z_2019110100_123_993_12213797.zip
ic_ancil_file: ic_ancil_12213797
ifsdata_file: ifsdata_12213797
climate_data_file: clim_data_12213797
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4999ed04afa5ec71afd905d71fd51a2b to: /var/lib/boinc-client/slots/1/ic_ancil_12213797.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12213797.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_76c636b1d41b05b3d9572d082b176e38 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12213797.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12213797.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12213797.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12213797.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12213797.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12213797.xml
-- Model is restarting --
Adjusting last_iter, 2808, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12213797.xml
last_cpu_time: 72405
upload_file_number: 116
last_iter: 2809
last_upload: 10026000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948
The child process has been launched with process id: 1687752
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1687752:1687752] [20230227:235430:1677509670:0.001] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_116.zip
Uploading the intermediate file: upload_file_116.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:56:11 STEP 2808 H=2808:00 +CPU= 36.968
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002808
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002808
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002808
  23:56:31 STEP 2809 H=2809:00 +CPU= 19.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:58:03 STEP 2810 H=2810:00 +CPU= 20.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:59:55 STEP 2811 H=2811:00 +CPU= 31.778
  00:00:16 STEP 2812 H=2812:00 +CPU= 21.021
  00:00:36 STEP 2813 H=2813:00 +CPU= 19.651
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:02:41 STEP 2814 H=2814:00 +CPU= 32.947
  00:03:00 STEP 2815 H=2815:00 +CPU= 19.498
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           28  MAX ETADOT VERTICAL VEL.=   2.643061827177811E-007
  LON  =    70.0000000000000       degrees
  LAT  =    37.5698795904715       degrees
  00:03:20 STEP 2816 H=2816:00 +CPU= 20.123
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           29  MAX ETADOT VERTICAL VEL.=   3.223899681188556E-007
  LON  =    68.7500000000000       degrees
  LAT  =    37.5698795904715       degrees
  00:05:14 STEP 2817 H=2817:00 +CPU= 32.277
  00:05:34 STEP 2818 H=2818:00 +CPU= 20.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:07:06 STEP 2819 H=2819:00 +CPU= 20.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:08:52 STEP 2820 H=2820:00 +CPU= 35.423
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           32  MAX ETADOT VERTICAL VEL.=   5.452716201830634E-007
  LON  =    65.0000000000000       degrees
  LAT  =    38.6913661222016       degrees
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002820
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002820
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002820
  00:09:12 STEP 2821 H=2821:00 +CPU= 19.607
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            2  MAX ETADOT VERTICAL VEL.=   4.461242239458300E-007
  LON  =    62.5000000000000       degrees
  LAT  =    39.8128523737712       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:10:53 STEP 2822 H=2822:00 +CPU= 20.310
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            2  MAX ETADOT VERTICAL VEL.=   3.651869690502573E-007
  LON  =    62.5000000000000       degrees
  LAT  =    39.8128523737712       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:12:16 STEP 2823 H=2823:00 +CPU= 31.954
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            3  MAX ETADOT VERTICAL VEL.=   4.104153299475276E-007
  LON  =    61.2500000000000       degrees
  LAT  =    39.8128523737712       degrees
  00:12:37 STEP 2824 H=2824:00 +CPU= 20.962
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            4  MAX ETADOT VERTICAL VEL.=   4.644335022654241E-007
  LON  =    60.0000000000000       degrees
  LAT  =    40.9343383242788       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:14:18 STEP 2825 H=2825:00 +CPU= 19.750
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            5  MAX ETADOT VERTICAL VEL.=   5.573208660333174E-007
  LON  =    58.7500000000000       degrees
  LAT  =    40.9343383242788       degrees
  00:14:51 STEP 2826 H=2826:00 +CPU= 32.327
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           11  MAX ETADOT VERTICAL VEL.=   3.517882977855022E-007
  LON  =    57.6562500000000       degrees
  LAT  =    42.0558239509352       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:16:22 STEP 2827 H=2827:00 +CPU= 19.598
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           12  MAX ETADOT VERTICAL VEL.=   3.325395890411642E-007
  LON  =    56.2500000000000       degrees
  LAT  =    42.0558239509352       degrees
  00:16:45 STEP 2828 H=2828:00 +CPU= 22.121
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           12  MAX ETADOT VERTICAL VEL.=   2.335038087957125E-007
  LON  =    56.2500000000000       degrees
  LAT  =    43.1773092288349       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:18:38 STEP 2829 H=2829:00 +CPU= 31.788
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           13  MAX ETADOT VERTICAL VEL.=   3.054701868641101E-007
  LON  =    54.8437500000000       degrees
  LAT  =    43.1773092288349       degrees
  00:18:59 STEP 2830 H=2830:00 +CPU= 21.077
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           14  MAX ETADOT VERTICAL VEL.=   1.826055837006958E-007
  LON  =    53.4375000000000       degrees
  LAT  =    44.2987941306940       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:20:39 STEP 2831 H=2831:00 +CPU= 20.265
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           14  MAX ETADOT VERTICAL VEL.=   1.765027908296261E-007
  LON  =    53.4375000000000       degrees
  LAT  =    45.4202786265483       degrees
  00:21:16 STEP 2832 H=2832:00 +CPU= 37.163
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           15  MAX ETADOT VERTICAL VEL.=   1.618077021772547E-007
  LON  =    52.0312500000000       degrees
  LAT  =    45.4202786265483       degrees
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002832
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002832
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002832
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:22:57 STEP 2833 H=2833:00 +CPU= 20.032
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           17  MAX ETADOT VERTICAL VEL.=   3.640454171486692E-007
  LON  =    52.5000000000000       degrees
  LAT  =    46.5417626834057       degrees
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002832
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_117.zip
Uploading the intermediate file: upload_file_117.zip
  00:23:19 STEP 2834 H=2834:00 +CPU= 21.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:25:01 STEP 2835 H=2835:00 +CPU= 31.643
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            2  MAX ETADOT VERTICAL VEL.=   1.243197973563536E-007
  LON  =    51.0000000000000       degrees
  LAT  =    47.6632462648426       degrees
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:26:42 STEP 2836 H=2836:00 +CPU= 20.430
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           18  MAX ETADOT VERTICAL VEL.=   3.552674148860974E-007
  LON  =    51.0000000000000       degrees
  LAT  =    48.7847293305350       degrees
  00:27:03 STEP 2837 H=2837:00 +CPU= 20.770
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:28:46 STEP 2838 H=2838:00 +CPU= 31.976
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            4  MAX ETADOT VERTICAL VEL.=   1.618608562970057E-007
  LON  =    51.2000000000000       degrees
  LAT  =    49.9062118357110       degrees
  00:29:07 STEP 2839 H=2839:00 +CPU= 20.345
  00:29:26 STEP 2840 H=2840:00 +CPU= 19.801
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:31:21 STEP 2841 H=2841:00 +CPU= 32.520
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:33:11 STEP 2842 H=2842:00 +CPU= 19.954
  00:33:32 STEP 2843 H=2843:00 +CPU= 20.470
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:35:29 STEP 2844 H=2844:00 +CPU= 35.891
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002844
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002844
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002844
  00:35:49 STEP 2845 H=2845:00 +CPU= 19.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:37:31 STEP 2846 H=2846:00 +CPU= 20.952
  00:38:02 STEP 2847 H=2847:00 +CPU= 30.816
  00:38:22 STEP 2848 H=2848:00 +CPU= 19.982
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:40:03 STEP 2849 H=2849:00 +CPU= 20.385
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:41:37 STEP 2850 H=2850:00 +CPU= 32.390
  00:41:57 STEP 2851 H=2851:00 +CPU= 20.611
  00:42:18 STEP 2852 H=2852:00 +CPU= 20.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:44:02 STEP 2853 H=2853:00 +CPU= 32.371
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:45:23 STEP 2854 H=2854:00 +CPU= 20.808
  00:45:43 STEP 2855 H=2855:00 +CPU= 20.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:47:43 STEP 2856 H=2856:00 +CPU= 37.910
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002856
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002856
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:49:02 STEP 2857 H=2857:00 +CPU= 19.691
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002856
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_118.zip
Uploading the intermediate file: upload_file_118.zip
  00:49:23 STEP 2858 H=2858:00 +CPU= 20.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:51:07 STEP 2859 H=2859:00 +CPU= 31.856
  00:51:28 STEP 2860 H=2860:00 +CPU= 20.649
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:52:48 STEP 2861 H=2861:00 +CPU= 19.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:54:42 STEP 2862 H=2862:00 +CPU= 32.613
  00:55:03 STEP 2863 H=2863:00 +CPU= 21.822
  00:55:24 STEP 2864 H=2864:00 +CPU= 20.510
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:17 STEP 2865 H=2865:00 +CPU= 32.678
  00:57:37 STEP 2866 H=2866:00 +CPU= 20.544
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:10 STEP 2867 H=2867:00 +CPU= 20.799
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:00:56 STEP 2868 H=2868:00 +CPU= 35.264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002868
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002868
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002868
  01:01:16 STEP 2869 H=2869:00 +CPU= 19.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:08 STEP 2870 H=2870:00 +CPU= 20.715
  01:03:39 STEP 2871 H=2871:00 +CPU= 31.627
  01:04:00 STEP 2872 H=2872:00 +CPU= 20.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:31 STEP 2873 H=2873:00 +CPU= 20.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:05 STEP 2874 H=2874:00 +CPU= 32.133
  01:07:25 STEP 2875 H=2875:00 +CPU= 20.201
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:08:48 STEP 2876 H=2876:00 +CPU= 20.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:10:11 STEP 2877 H=2877:00 +CPU= 32.425
  01:10:32 STEP 2878 H=2878:00 +CPU= 20.456
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:12:23 STEP 2879 H=2879:00 +CPU= 20.382
  01:13:01 STEP 2880 H=2880:00 +CPU= 37.343
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002880
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002880
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002880
  01:13:20 STEP 2881 H=2881:00 +CPU= 19.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002880
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_119.zip
Uploading the intermediate file: upload_file_119.zip
  01:15:03 STEP 2882 H=2882:00 +CPU= 21.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:37 STEP 2883 H=2883:00 +CPU= 32.318
  01:16:57 STEP 2884 H=2884:00 +CPU= 20.109
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:18:37 STEP 2885 H=2885:00 +CPU= 19.928
  01:19:11 STEP 2886 H=2886:00 +CPU= 33.532
  01:19:31 STEP 2887 H=2887:00 +CPU= 19.511
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:21:24 STEP 2888 H=2888:00 +CPU= 20.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:23:17 STEP 2889 H=2889:00 +CPU= 32.036
  01:23:38 STEP 2890 H=2890:00 +CPU= 20.701
  01:23:57 STEP 2891 H=2891:00 +CPU= 19.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:03 STEP 2892 H=2892:00 +CPU= 35.479
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002892
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002892
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002892
  01:26:23 STEP 2893 H=2893:00 +CPU= 19.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:27:46 STEP 2894 H=2894:00 +CPU= 21.183
  01:28:20 STEP 2895 H=2895:00 +CPU= 33.381
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:01 STEP 2896 H=2896:00 +CPU= 20.311
  01:30:22 STEP 2897 H=2897:00 +CPU= 20.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:32:06 STEP 2898 H=2898:00 +CPU= 32.459
  01:32:26 STEP 2899 H=2899:00 +CPU= 20.057
  01:32:47 STEP 2900 H=2900:00 +CPU= 21.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:31 STEP 2901 H=2901:00 +CPU= 32.278
  01:34:51 STEP 2902 H=2902:00 +CPU= 20.243
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:36:23 STEP 2903 H=2903:00 +CPU= 21.249
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:23 STEP 2904 H=2904:00 +CPU= 37.826
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002904
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002904
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002904
  01:38:43 STEP 2905 H=2905:00 +CPU= 19.895
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002904
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_120.zip
Uploading the intermediate file: upload_file_120.zip
  01:39:02 STEP 2906 H=2906:00 +CPU= 18.703
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:24 STEP 2907 H=2907:00 +CPU= 29.512
  01:40:44 STEP 2908 H=2908:00 +CPU= 20.576
  01:41:04 STEP 2909 H=2909:00 +CPU= 19.821
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:59 STEP 2910 H=2910:00 +CPU= 33.361
  01:43:19 STEP 2911 H=2911:00 +CPU= 20.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:44:52 STEP 2912 H=2912:00 +CPU= 21.204
  01:45:24 STEP 2913 H=2913:00 +CPU= 31.330
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:46:54 STEP 2914 H=2914:00 +CPU= 20.349
  01:47:16 STEP 2915 H=2915:00 +CPU= 21.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:49:02 STEP 2916 H=2916:00 +CPU= 33.947
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002916
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002916
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002916
  01:49:22 STEP 2917 H=2917:00 +CPU= 20.048
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:50:54 STEP 2918 H=2918:00 +CPU= 21.228
  01:51:27 STEP 2919 H=2919:00 +CPU= 32.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:53:09 STEP 2920 H=2920:00 +CPU= 20.761
  01:53:28 STEP 2921 H=2921:00 +CPU= 19.721
  01:54:01 STEP 2922 H=2922:00 +CPU= 32.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:55:31 STEP 2923 H=2923:00 +CPU= 20.455
  01:55:52 STEP 2924 H=2924:00 +CPU= 21.030
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:58:29 STEP 2925 H=2925:00 +CPU= 33.401
  01:58:50 STEP 2926 H=2926:00 +CPU= 21.197
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:00:33 STEP 2927 H=2927:00 +CPU= 21.226
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 009z
(argv4) batchid: 993
(argv5) wuid: 12213797
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_88f624dba5097c4bd2242ad796053ca0 to: /var/lib/boinc-client/slots/1/oifs_43r3_009z_2019110100_123_993_12213797.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_009z_2019110100_123_993_12213797.zip
ic_ancil_file: ic_ancil_12213797
ifsdata_file: ifsdata_12213797
climate_data_file: clim_data_12213797
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4999ed04afa5ec71afd905d71fd51a2b to: /var/lib/boinc-client/slots/1/ic_ancil_12213797.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12213797.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_76c636b1d41b05b3d9572d082b176e38 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12213797.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12213797.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12213797.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12213797.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12213797.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12213797.xml
-- Model is restarting --
Adjusting last_iter, 2928, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12213797.xml
last_cpu_time: 75375
upload_file_number: 121
last_iter: 2929
last_upload: 10458000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948
The child process has been launched with process id: 1706517
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:p500:1:1:1706517:1706517] [20230228:020233:1677517353:0.001] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_121.zip
Uploading the intermediate file: upload_file_121.zip
  02:03:12 STEP 2928 H=2928:00 +CPU= 36.552
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002928
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002928
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002928
  02:03:32 STEP 2929 H=2929:00 +CPU= 19.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:05:14 STEP 2930 H=2930:00 +CPU= 20.803
  02:05:45 STEP 2931 H=2931:00 +CPU= 31.608
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:07:27 STEP 2932 H=2932:00 +CPU= 20.071
  02:07:47 STEP 2933 H=2933:00 +CPU= 19.922
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:29 STEP 2934 H=2934:00 +CPU= 31.763
  02:09:50 STEP 2935 H=2935:00 +CPU= 20.574
  02:10:11 STEP 2936 H=2936:00 +CPU= 20.884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:12:04 STEP 2937 H=2937:00 +CPU= 31.897
  02:12:24 STEP 2938 H=2938:00 +CPU= 20.211
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:13:46 STEP 2939 H=2939:00 +CPU= 20.305
  02:14:21 STEP 2940 H=2940:00 +CPU= 34.852
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002940
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002940
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:16:02 STEP 2941 H=2941:00 +CPU= 19.641
  02:16:24 STEP 2942 H=2942:00 +CPU= 21.381
  02:16:55 STEP 2943 H=2943:00 +CPU= 31.483
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:18:36 STEP 2944 H=2944:00 +CPU= 20.909
  02:18:56 STEP 2945 H=2945:00 +CPU= 19.518
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:20:29 STEP 2946 H=2946:00 +CPU= 32.476
  02:20:48 STEP 2947 H=2947:00 +CPU= 19.727
  02:21:09 STEP 2948 H=2948:00 +CPU= 20.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:23:02 STEP 2949 H=2949:00 +CPU= 31.864
  02:23:22 STEP 2950 H=2950:00 +CPU= 20.358
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:24:54 STEP 2951 H=2951:00 +CPU= 20.481
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:26:30 STEP 2952 H=2952:00 +CPU= 35.654
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 4.865283731608E-01 1.865286143845E-01 6.362598939121E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.260318722649E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.48989191816039E-02 -1.72994898449055E+00  1.89761310057317E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.19925486346338E-02 -1.00000000000000E+00  1.00000000000000E+00
  MAXGPFV : MAX. VALUE =    1.18049843669510     
  MAXGPFV : MAX. VALUE =   0.734386800951908     
 NSTEP =  2952 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002952 MODE=a
 IO-STREAM CLOSED - ICMGGhnw4+002952
 MPI-TASK:   1 -    349260106 BYTES IN      8 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0083( 3.6%)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302231500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302233000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302234500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200303000000  !!!!!!!!!!!! 
 02:26:30 STEP 2952 H=2952:00 +CPU= 35.654
 END CNT3
 NSTEP =  2952 CNT0   000000000  31.406  31.406  92.560   0.339               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           35/   804992476 WRITTEN          373/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     12    0       35       786125      373 ************     2.5     1.1     1.4     1.6     2.3
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 1437404.8 1437404.8    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 1434770.9 1434770.9     99.82      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   27    4950.5    4950.5      9.30      0.00
   9 SPCM     - SPECTRAL COMP.                      24     340.6     340.6      0.57      0.00
  10 SCAN2M - PHYSICS                               25   16351.7   16351.7     28.44      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  2    1098.4    1098.4      0.15      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  2      14.9      14.9      0.00      0.00
  13 SCAN2M - RADIATION CALC.                        9   27348.8   27348.8     17.12      0.00
  17 GRIDFPOS IN CNT4                                2      38.4      38.4      0.01      0.00
  27 RESTART FILES - WRITING                         1    2196.7    2196.7      0.15      0.00
  28 RESTART FILES - READING                         1    2146.3    2146.3      0.15      0.00
  29 SU4FPOS IN CNT4                                 2       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                 2    1929.5    1929.5      0.27      0.00
  37 CPGLAG   - SL COMPUTATIONS                     25    9637.0    9637.0     16.76      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL            25   14135.4   14135.4     24.58      0.00
  39 SU0YOMB                                         1     343.4     343.4      0.02      0.00
  51 SCAN2M   - SL COMM. PART 1                     25      24.9      24.9      0.04      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               24     180.0     180.0      0.30      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               24      33.2      33.2      0.06      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  2       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM       88     137.4     137.4      0.84      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM        50     125.0     125.0      0.43      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM         50      54.6      54.6      0.19      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM        88      50.3      50.3      0.31      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2      16.7      16.7      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION             88      19.4      19.4      0.12      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION             50      16.4      16.4      0.06      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION             88      21.5      21.5      0.13      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION             50      11.7      11.7      0.04      0.00
 400 GSTATS                                       6030       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                  5706       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     1437.4 CPU TIME     625.1 VECTOR TIME      625.1


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     7406.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhnw4+002952
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhnw4+002952
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhnw4+002952
Adding to the zip: /var/lib/boinc-client/slots/1/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/1/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMUAhnw4+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMSHhnw4+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12213797/ICMGGhnw4+002544
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_009z_2019110100_123_993_12213797_0_r1517877948_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
02:30:03 (1706491): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:54:51 1519251 22315146 oifs_43r3_009z_2019110100_123_993_12213797_0 10,623,600 76,000 0.0072


©2024 climateprediction.net