climateprediction.net home page
Task 22310650

Task 22310650

Name oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0
Workunit 12210435
Created 13 Feb 2023, 18:43:39 UTC
Sent 14 Feb 2023, 9:54:13 UTC
Report deadline 15 Apr 2023, 9:54:13 UTC
Received 24 Feb 2023, 10:16:05 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536616
Run time 10 hours 31 min 55 sec
CPU time 10 hours 23 min 48 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 4.72 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,076.06 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,363.17 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
lots/6/oifs_43r3_bl_a2tc_2016092300_15_991_12210435.zip
ic_ancil_file: ic_ancil_12210435
ifsdata_file: ifsdata_12210435
climate_data_file: clim_data_12210435
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client50002/slots/6/ic_ancil_12210435.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client50002/slots/6/ic_ancil_12210435.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0c302ec3e62d48f5329b1eaee83b0d3a to: /var/lib/boinc-client50002/slots/6/ifsdata/ifsdata_12210435.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client50002/slots/6/ifsdata/ifsdata_12210435.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client50002/slots/6/159l_2/clim_data_12210435.zip
Unzipping the climate data zip file: /var/lib/boinc-client50002/slots/6/159l_2/clim_data_12210435.zip
Checking for progress XML file: /var/lib/boinc-client50002/slots/6/progress_file_12210435.xml
Opened progress file ok : /var/lib/boinc-client50002/slots/6/progress_file_12210435.xml
-- Model is restarting --
Adjusting last_iter, 960, to previous model restart step.
Creating progress file: /var/lib/boinc-client50002/slots/6/progress_file_12210435.xml
last_cpu_time: 24325
upload_file_number: 9
last_iter: 961
last_upload: 777600
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930
The child process has been launched with process id: 1530691
Executing the command: /var/lib/boinc-client50002/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530691:1530691] [20230224:053833:1677217113:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930_9.zip
Uploading the intermediate file: upload_file_9.zip
  05:39:02 STEP  960 H= 240:00 +CPU= 23.904
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+000960
  05:39:27 STEP  961 H= 240:15 +CPU= 24.205
  05:39:50 STEP  962 H= 240:30 +CPU= 23.686
  05:40:15 STEP  963 H= 240:45 +CPU= 24.388
  05:40:40 STEP  964 H= 241:00 +CPU= 24.880
  05:41:03 STEP  965 H= 241:15 +CPU= 23.407
  05:41:28 STEP  966 H= 241:30 +CPU= 24.197
  05:41:51 STEP  967 H= 241:45 +CPU= 23.434
  05:42:15 STEP  968 H= 242:00 +CPU= 24.124
  05:42:39 STEP  969 H= 242:15 +CPU= 23.826
  05:43:04 STEP  970 H= 242:30 +CPU= 24.905
  05:43:28 STEP  971 H= 242:45 +CPU= 23.526
  05:43:59 STEP  972 H= 243:00 +CPU= 31.347
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+000972
  05:44:23 STEP  973 H= 243:15 +CPU= 23.754
  05:44:47 STEP  974 H= 243:30 +CPU= 23.398
  05:45:10 STEP  975 H= 243:45 +CPU= 23.810
  05:45:35 STEP  976 H= 244:00 +CPU= 24.308
  05:45:59 STEP  977 H= 244:15 +CPU= 24.079
  05:46:23 STEP  978 H= 244:30 +CPU= 23.748
  05:46:47 STEP  979 H= 244:45 +CPU= 23.736
  05:47:11 STEP  980 H= 245:00 +CPU= 23.996
  05:47:35 STEP  981 H= 245:15 +CPU= 23.667
  05:47:57 STEP  982 H= 245:30 +CPU= 22.708
  05:48:21 STEP  983 H= 245:45 +CPU= 23.743
  05:48:52 STEP  984 H= 246:00 +CPU= 30.784
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+000984
  05:49:16 STEP  985 H= 246:15 +CPU= 23.928
  05:49:40 STEP  986 H= 246:30 +CPU= 23.369
  05:50:04 STEP  987 H= 246:45 +CPU= 24.700
  05:50:29 STEP  988 H= 247:00 +CPU= 24.927
  05:50:52 STEP  989 H= 247:15 +CPU= 23.034
  05:51:16 STEP  990 H= 247:30 +CPU= 23.024
  05:51:40 STEP  991 H= 247:45 +CPU= 24.491
  05:52:05 STEP  992 H= 248:00 +CPU= 24.850
  05:52:28 STEP  993 H= 248:15 +CPU= 23.029
  05:52:52 STEP  994 H= 248:30 +CPU= 23.869
  05:53:16 STEP  995 H= 248:45 +CPU= 23.479
  05:53:47 STEP  996 H= 249:00 +CPU= 30.674
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+000996
  05:54:11 STEP  997 H= 249:15 +CPU= 24.599
  05:54:36 STEP  998 H= 249:30 +CPU= 24.419
  05:55:00 STEP  999 H= 249:45 +CPU= 24.158
  05:55:25 STEP 1000 H= 250:00 +CPU= 24.889
  05:55:50 STEP 1001 H= 250:15 +CPU= 24.413
  05:56:13 STEP 1002 H= 250:30 +CPU= 23.407
  05:56:37 STEP 1003 H= 250:45 +CPU= 24.362
  05:57:01 STEP 1004 H= 251:00 +CPU= 23.676
  05:57:24 STEP 1005 H= 251:15 +CPU= 23.171
  05:57:48 STEP 1006 H= 251:30 +CPU= 23.702
  05:58:13 STEP 1007 H= 251:45 +CPU= 24.213
  05:58:47 STEP 1008 H= 252:00 +CPU= 34.476
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001008
  05:59:12 STEP 1009 H= 252:15 +CPU= 24.430
  05:59:36 STEP 1010 H= 252:30 +CPU= 24.436
  06:00:01 STEP 1011 H= 252:45 +CPU= 24.543
  06:00:25 STEP 1012 H= 253:00 +CPU= 23.962
  06:00:49 STEP 1013 H= 253:15 +CPU= 24.332
  06:01:14 STEP 1014 H= 253:30 +CPU= 24.335
  06:01:37 STEP 1015 H= 253:45 +CPU= 23.319
  06:02:02 STEP 1016 H= 254:00 +CPU= 24.600
  06:02:27 STEP 1017 H= 254:15 +CPU= 24.555
  06:02:50 STEP 1018 H= 254:30 +CPU= 23.302
  06:03:14 STEP 1019 H= 254:45 +CPU= 23.841
  06:03:45 STEP 1020 H= 255:00 +CPU= 30.711
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001020
  06:04:09 STEP 1021 H= 255:15 +CPU= 24.135
  06:04:33 STEP 1022 H= 255:30 +CPU= 23.879
  06:04:57 STEP 1023 H= 255:45 +CPU= 24.251
  06:05:22 STEP 1024 H= 256:00 +CPU= 24.279
  06:05:46 STEP 1025 H= 256:15 +CPU= 24.547
  06:06:10 STEP 1026 H= 256:30 +CPU= 23.548
  06:06:33 STEP 1027 H= 256:45 +CPU= 23.216
  06:06:58 STEP 1028 H= 257:00 +CPU= 25.151
  06:07:23 STEP 1029 H= 257:15 +CPU= 24.335
  06:07:47 STEP 1030 H= 257:30 +CPU= 23.564
  06:08:10 STEP 1031 H= 257:45 +CPU= 23.851
  06:08:41 STEP 1032 H= 258:00 +CPU= 30.839
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001032
  06:09:05 STEP 1033 H= 258:15 +CPU= 23.177
  06:09:28 STEP 1034 H= 258:30 +CPU= 23.638
  06:09:52 STEP 1035 H= 258:45 +CPU= 23.671
  06:10:16 STEP 1036 H= 259:00 +CPU= 23.645
  06:10:40 STEP 1037 H= 259:15 +CPU= 23.779
  06:11:04 STEP 1038 H= 259:30 +CPU= 24.197
  06:11:28 STEP 1039 H= 259:45 +CPU= 23.425
  06:11:52 STEP 1040 H= 260:00 +CPU= 24.658
  06:12:16 STEP 1041 H= 260:15 +CPU= 23.730
  06:12:40 STEP 1042 H= 260:30 +CPU= 23.791
  06:13:04 STEP 1043 H= 260:45 +CPU= 23.513
  06:13:35 STEP 1044 H= 261:00 +CPU= 30.902
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001044
  06:13:58 STEP 1045 H= 261:15 +CPU= 23.254
  06:14:22 STEP 1046 H= 261:30 +CPU= 24.012
  06:14:47 STEP 1047 H= 261:45 +CPU= 24.704
  06:15:12 STEP 1048 H= 262:00 +CPU= 24.824
  06:15:36 STEP 1049 H= 262:15 +CPU= 23.992
  06:16:00 STEP 1050 H= 262:30 +CPU= 23.535
  06:16:23 STEP 1051 H= 262:45 +CPU= 23.755
  06:16:48 STEP 1052 H= 263:00 +CPU= 24.215
  06:17:13 STEP 1053 H= 263:15 +CPU= 24.931
  06:17:36 STEP 1054 H= 263:30 +CPU= 23.378
  06:18:01 STEP 1055 H= 263:45 +CPU= 24.427
  06:18:35 STEP 1056 H= 264:00 +CPU= 34.531
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001056
  06:19:00 STEP 1057 H= 264:15 +CPU= 24.267
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001056
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930_10.zip
Uploading the intermediate file: upload_file_10.zip
  06:19:25 STEP 1058 H= 264:30 +CPU= 25.227
  06:19:49 STEP 1059 H= 264:45 +CPU= 24.222
  06:20:15 STEP 1060 H= 265:00 +CPU= 25.154
  06:20:39 STEP 1061 H= 265:15 +CPU= 24.460
  06:21:04 STEP 1062 H= 265:30 +CPU= 24.693
  06:21:29 STEP 1063 H= 265:45 +CPU= 24.827
  06:21:54 STEP 1064 H= 266:00 +CPU= 25.027
  06:22:19 STEP 1065 H= 266:15 +CPU= 24.333
  06:22:43 STEP 1066 H= 266:30 +CPU= 24.082
  06:23:06 STEP 1067 H= 266:45 +CPU= 23.264
  06:23:37 STEP 1068 H= 267:00 +CPU= 30.830
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001068
  06:24:01 STEP 1069 H= 267:15 +CPU= 24.247
  06:24:25 STEP 1070 H= 267:30 +CPU= 23.987
  06:24:49 STEP 1071 H= 267:45 +CPU= 23.174
  06:25:14 STEP 1072 H= 268:00 +CPU= 24.985
  06:25:37 STEP 1073 H= 268:15 +CPU= 23.557
  06:26:01 STEP 1074 H= 268:30 +CPU= 23.616
  06:26:24 STEP 1075 H= 268:45 +CPU= 23.113
  06:26:49 STEP 1076 H= 269:00 +CPU= 24.817
  06:27:14 STEP 1077 H= 269:15 +CPU= 24.295
  06:27:38 STEP 1078 H= 269:30 +CPU= 23.983
  06:28:01 STEP 1079 H= 269:45 +CPU= 23.723
  06:28:32 STEP 1080 H= 270:00 +CPU= 30.244
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001080
  06:28:55 STEP 1081 H= 270:15 +CPU= 23.498
  06:29:19 STEP 1082 H= 270:30 +CPU= 23.646
  06:29:43 STEP 1083 H= 270:45 +CPU= 23.776
  06:30:07 STEP 1084 H= 271:00 +CPU= 23.982
  06:30:30 STEP 1085 H= 271:15 +CPU= 22.932
  06:30:53 STEP 1086 H= 271:30 +CPU= 22.906
  06:31:18 STEP 1087 H= 271:45 +CPU= 24.504
  06:31:42 STEP 1088 H= 272:00 +CPU= 23.928
  06:32:06 STEP 1089 H= 272:15 +CPU= 23.885
  06:32:29 STEP 1090 H= 272:30 +CPU= 23.364
  06:32:53 STEP 1091 H= 272:45 +CPU= 23.788
  06:33:24 STEP 1092 H= 273:00 +CPU= 30.606
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001092
  06:33:47 STEP 1093 H= 273:15 +CPU= 23.482
  06:34:11 STEP 1094 H= 273:30 +CPU= 23.564
  06:34:35 STEP 1095 H= 273:45 +CPU= 23.805
  06:34:59 STEP 1096 H= 274:00 +CPU= 24.347
  06:35:23 STEP 1097 H= 274:15 +CPU= 23.287
  06:35:46 STEP 1098 H= 274:30 +CPU= 23.411
  06:36:10 STEP 1099 H= 274:45 +CPU= 24.090
  06:36:34 STEP 1100 H= 275:00 +CPU= 23.960
  06:36:58 STEP 1101 H= 275:15 +CPU= 23.996
  06:37:25 STEP 1102 H= 275:30 +CPU= 25.635
  06:37:54 STEP 1103 H= 275:45 +CPU= 28.363
  06:38:37 STEP 1104 H= 276:00 +CPU= 42.242
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001104
  06:39:06 STEP 1105 H= 276:15 +CPU= 28.250
  06:39:34 STEP 1106 H= 276:30 +CPU= 27.314
  06:40:03 STEP 1107 H= 276:45 +CPU= 28.143
  06:40:34 STEP 1108 H= 277:00 +CPU= 29.782
  06:41:04 STEP 1109 H= 277:15 +CPU= 29.607
  06:41:33 STEP 1110 H= 277:30 +CPU= 28.941
  06:42:01 STEP 1111 H= 277:45 +CPU= 27.677
  06:42:30 STEP 1112 H= 278:00 +CPU= 28.603
  06:42:58 STEP 1113 H= 278:15 +CPU= 27.816
  06:43:26 STEP 1114 H= 278:30 +CPU= 27.086
  06:43:54 STEP 1115 H= 278:45 +CPU= 27.721
  06:44:31 STEP 1116 H= 279:00 +CPU= 35.939
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001116
  06:44:58 STEP 1117 H= 279:15 +CPU= 26.295
  06:45:26 STEP 1118 H= 279:30 +CPU= 27.755
  06:45:54 STEP 1119 H= 279:45 +CPU= 27.085
  06:46:23 STEP 1120 H= 280:00 +CPU= 28.766
  06:46:52 STEP 1121 H= 280:15 +CPU= 28.312
  06:47:21 STEP 1122 H= 280:30 +CPU= 28.522
  06:47:48 STEP 1123 H= 280:45 +CPU= 27.427
  06:48:17 STEP 1124 H= 281:00 +CPU= 28.501
  06:48:46 STEP 1125 H= 281:15 +CPU= 28.060
  06:49:14 STEP 1126 H= 281:30 +CPU= 27.874
  06:49:42 STEP 1127 H= 281:45 +CPU= 27.733
  06:50:19 STEP 1128 H= 282:00 +CPU= 36.055
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001128
  06:50:47 STEP 1129 H= 282:15 +CPU= 27.706
  06:51:15 STEP 1130 H= 282:30 +CPU= 27.173
  06:51:43 STEP 1131 H= 282:45 +CPU= 27.727
  06:52:13 STEP 1132 H= 283:00 +CPU= 29.157
  06:52:40 STEP 1133 H= 283:15 +CPU= 26.986
  06:53:08 STEP 1134 H= 283:30 +CPU= 27.625
  06:53:37 STEP 1135 H= 283:45 +CPU= 28.355
  06:54:06 STEP 1136 H= 284:00 +CPU= 29.144
  06:54:35 STEP 1137 H= 284:15 +CPU= 28.265
  06:55:04 STEP 1138 H= 284:30 +CPU= 28.059
  06:55:32 STEP 1139 H= 284:45 +CPU= 27.648
  06:56:08 STEP 1140 H= 285:00 +CPU= 35.108
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001140
  06:56:36 STEP 1141 H= 285:15 +CPU= 27.390
  06:57:04 STEP 1142 H= 285:30 +CPU= 28.036
  06:57:32 STEP 1143 H= 285:45 +CPU= 27.822
  06:58:03 STEP 1144 H= 286:00 +CPU= 29.838
  06:58:31 STEP 1145 H= 286:15 +CPU= 28.530
  06:59:02 STEP 1146 H= 286:30 +CPU= 29.852
  06:59:29 STEP 1147 H= 286:45 +CPU= 26.862
  06:59:58 STEP 1148 H= 287:00 +CPU= 28.361
  07:00:26 STEP 1149 H= 287:15 +CPU= 27.749
  07:00:55 STEP 1150 H= 287:30 +CPU= 28.183
  07:01:23 STEP 1151 H= 287:45 +CPU= 27.710
  07:02:05 STEP 1152 H= 288:00 +CPU= 40.125
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001152
  07:02:32 STEP 1153 H= 288:15 +CPU= 27.244
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001152
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930_11.zip
Uploading the intermediate file: upload_file_11.zip
  07:03:02 STEP 1154 H= 288:30 +CPU= 29.427
  07:03:31 STEP 1155 H= 288:45 +CPU= 27.735
  07:04:01 STEP 1156 H= 289:00 +CPU= 30.106
  07:04:30 STEP 1157 H= 289:15 +CPU= 28.330
  07:04:58 STEP 1158 H= 289:30 +CPU= 27.688
  07:05:29 STEP 1159 H= 289:45 +CPU= 30.089
  07:05:58 STEP 1160 H= 290:00 +CPU= 27.654
  07:06:25 STEP 1161 H= 290:15 +CPU= 27.403
  07:06:54 STEP 1162 H= 290:30 +CPU= 28.251
  07:07:23 STEP 1163 H= 290:45 +CPU= 28.008
  07:08:00 STEP 1164 H= 291:00 +CPU= 36.873
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001164
  07:08:29 STEP 1165 H= 291:15 +CPU= 28.662
  07:08:57 STEP 1166 H= 291:30 +CPU= 27.566
  07:09:26 STEP 1167 H= 291:45 +CPU= 28.157
  07:09:56 STEP 1168 H= 292:00 +CPU= 29.320
  07:10:26 STEP 1169 H= 292:15 +CPU= 30.010
  07:10:54 STEP 1170 H= 292:30 +CPU= 27.754
  07:11:24 STEP 1171 H= 292:45 +CPU= 29.167
  07:11:53 STEP 1172 H= 293:00 +CPU= 28.691
  07:12:21 STEP 1173 H= 293:15 +CPU= 27.152
  07:12:50 STEP 1174 H= 293:30 +CPU= 28.786
  07:13:18 STEP 1175 H= 293:45 +CPU= 27.547
  07:13:55 STEP 1176 H= 294:00 +CPU= 36.015
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001176
  07:14:24 STEP 1177 H= 294:15 +CPU= 28.903
  07:14:53 STEP 1178 H= 294:30 +CPU= 28.913
  07:15:20 STEP 1179 H= 294:45 +CPU= 26.731
  07:15:50 STEP 1180 H= 295:00 +CPU= 28.916
  07:16:19 STEP 1181 H= 295:15 +CPU= 28.380
  07:16:47 STEP 1182 H= 295:30 +CPU= 27.665
  07:17:15 STEP 1183 H= 295:45 +CPU= 27.367
  07:17:44 STEP 1184 H= 296:00 +CPU= 28.649
  07:18:12 STEP 1185 H= 296:15 +CPU= 27.313
  07:18:41 STEP 1186 H= 296:30 +CPU= 28.553
  07:19:09 STEP 1187 H= 296:45 +CPU= 28.331
  07:19:46 STEP 1188 H= 297:00 +CPU= 35.654
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001188
  07:20:14 STEP 1189 H= 297:15 +CPU= 27.548
  07:20:42 STEP 1190 H= 297:30 +CPU= 27.546
  07:21:10 STEP 1191 H= 297:45 +CPU= 27.793
  07:21:41 STEP 1192 H= 298:00 +CPU= 29.935
  07:22:11 STEP 1193 H= 298:15 +CPU= 30.285
  07:22:40 STEP 1194 H= 298:30 +CPU= 27.845
  07:23:08 STEP 1195 H= 298:45 +CPU= 27.867
  07:23:36 STEP 1196 H= 299:00 +CPU= 27.554
  07:24:04 STEP 1197 H= 299:15 +CPU= 27.462
  07:24:32 STEP 1198 H= 299:30 +CPU= 27.331
  07:25:00 STEP 1199 H= 299:45 +CPU= 27.476
  07:25:44 STEP 1200 H= 300:00 +CPU= 42.463
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001200
  07:26:13 STEP 1201 H= 300:15 +CPU= 28.323
  07:26:42 STEP 1202 H= 300:30 +CPU= 28.169
  07:27:11 STEP 1203 H= 300:45 +CPU= 29.361
  07:27:41 STEP 1204 H= 301:00 +CPU= 29.583
  07:28:11 STEP 1205 H= 301:15 +CPU= 28.724
  07:28:41 STEP 1206 H= 301:30 +CPU= 29.021
  07:29:09 STEP 1207 H= 301:45 +CPU= 27.857
  07:29:39 STEP 1208 H= 302:00 +CPU= 29.450
  07:30:08 STEP 1209 H= 302:15 +CPU= 28.099
  07:30:36 STEP 1210 H= 302:30 +CPU= 27.900
  07:31:05 STEP 1211 H= 302:45 +CPU= 28.092
  07:31:40 STEP 1212 H= 303:00 +CPU= 34.791
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001212
  07:32:08 STEP 1213 H= 303:15 +CPU= 27.470
  07:32:36 STEP 1214 H= 303:30 +CPU= 27.242
  07:33:04 STEP 1215 H= 303:45 +CPU= 27.230
  07:33:33 STEP 1216 H= 304:00 +CPU= 28.761
  07:34:02 STEP 1217 H= 304:15 +CPU= 28.325
  07:34:32 STEP 1218 H= 304:30 +CPU= 29.579
  07:34:59 STEP 1219 H= 304:45 +CPU= 26.873
  07:35:28 STEP 1220 H= 305:00 +CPU= 28.642
  07:35:57 STEP 1221 H= 305:15 +CPU= 28.866
  07:36:27 STEP 1222 H= 305:30 +CPU= 28.878
  07:36:55 STEP 1223 H= 305:45 +CPU= 27.993
  07:37:31 STEP 1224 H= 306:00 +CPU= 34.926
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001224
  07:38:00 STEP 1225 H= 306:15 +CPU= 28.933
  07:38:29 STEP 1226 H= 306:30 +CPU= 28.208
  07:38:58 STEP 1227 H= 306:45 +CPU= 28.876
  07:39:27 STEP 1228 H= 307:00 +CPU= 28.432
  07:39:57 STEP 1229 H= 307:15 +CPU= 30.247
  07:40:27 STEP 1230 H= 307:30 +CPU= 29.096
  07:40:55 STEP 1231 H= 307:45 +CPU= 28.024
  07:41:27 STEP 1232 H= 308:00 +CPU= 31.171
  07:41:56 STEP 1233 H= 308:15 +CPU= 28.447
  07:42:25 STEP 1234 H= 308:30 +CPU= 28.686
  07:42:52 STEP 1235 H= 308:45 +CPU= 26.855
  07:43:30 STEP 1236 H= 309:00 +CPU= 36.809
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001236
  07:43:58 STEP 1237 H= 309:15 +CPU= 27.740
  07:44:28 STEP 1238 H= 309:30 +CPU= 29.221
  07:44:57 STEP 1239 H= 309:45 +CPU= 28.228
  07:45:27 STEP 1240 H= 310:00 +CPU= 29.127
  07:45:57 STEP 1241 H= 310:15 +CPU= 30.028
  07:46:26 STEP 1242 H= 310:30 +CPU= 28.504
  07:46:55 STEP 1243 H= 310:45 +CPU= 28.207
  07:47:23 STEP 1244 H= 311:00 +CPU= 28.313
  07:47:52 STEP 1245 H= 311:15 +CPU= 27.454
  07:48:21 STEP 1246 H= 311:30 +CPU= 28.179
  07:48:50 STEP 1247 H= 311:45 +CPU= 29.474
  07:49:32 STEP 1248 H= 312:00 +CPU= 40.346
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001248
  07:50:01 STEP 1249 H= 312:15 +CPU= 29.103
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001248
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930_12.zip
Uploading the intermediate file: upload_file_12.zip
  07:50:30 STEP 1250 H= 312:30 +CPU= 27.983
  07:50:58 STEP 1251 H= 312:45 +CPU= 27.849
  07:51:28 STEP 1252 H= 313:00 +CPU= 29.415
  07:51:56 STEP 1253 H= 313:15 +CPU= 27.533
  07:52:24 STEP 1254 H= 313:30 +CPU= 27.588
  07:52:54 STEP 1255 H= 313:45 +CPU= 30.000
  07:53:24 STEP 1256 H= 314:00 +CPU= 29.224
  07:53:54 STEP 1257 H= 314:15 +CPU= 29.242
  07:54:21 STEP 1258 H= 314:30 +CPU= 27.050
  07:54:49 STEP 1259 H= 314:45 +CPU= 27.206
  07:55:25 STEP 1260 H= 315:00 +CPU= 35.386
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001260
  07:55:54 STEP 1261 H= 315:15 +CPU= 28.792
  07:56:22 STEP 1262 H= 315:30 +CPU= 27.947
  07:56:52 STEP 1263 H= 315:45 +CPU= 29.143
  07:57:20 STEP 1264 H= 316:00 +CPU= 27.582
  07:57:48 STEP 1265 H= 316:15 +CPU= 28.328
  07:58:17 STEP 1266 H= 316:30 +CPU= 28.058
  07:58:46 STEP 1267 H= 316:45 +CPU= 28.288
  07:59:16 STEP 1268 H= 317:00 +CPU= 29.663
  07:59:44 STEP 1269 H= 317:15 +CPU= 27.888
  08:00:11 STEP 1270 H= 317:30 +CPU= 27.187
  08:00:40 STEP 1271 H= 317:45 +CPU= 28.527
  08:01:17 STEP 1272 H= 318:00 +CPU= 36.139
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001272
  08:01:46 STEP 1273 H= 318:15 +CPU= 28.217
  08:02:14 STEP 1274 H= 318:30 +CPU= 27.725
  08:02:44 STEP 1275 H= 318:45 +CPU= 29.102
  08:03:14 STEP 1276 H= 319:00 +CPU= 29.828
  08:03:42 STEP 1277 H= 319:15 +CPU= 27.723
  08:04:11 STEP 1278 H= 319:30 +CPU= 28.136
  08:04:39 STEP 1279 H= 319:45 +CPU= 27.534
  08:05:08 STEP 1280 H= 320:00 +CPU= 28.669
  08:05:38 STEP 1281 H= 320:15 +CPU= 29.271
  08:06:05 STEP 1282 H= 320:30 +CPU= 26.579
  08:06:33 STEP 1283 H= 320:45 +CPU= 27.837
  08:07:08 STEP 1284 H= 321:00 +CPU= 34.311
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001284
  08:07:38 STEP 1285 H= 321:15 +CPU= 29.129
  08:08:07 STEP 1286 H= 321:30 +CPU= 29.151
  08:08:35 STEP 1287 H= 321:45 +CPU= 27.793
  08:09:05 STEP 1288 H= 322:00 +CPU= 29.729
  08:09:34 STEP 1289 H= 322:15 +CPU= 28.814
  08:10:03 STEP 1290 H= 322:30 +CPU= 28.492
  08:10:32 STEP 1291 H= 322:45 +CPU= 28.354
  08:11:02 STEP 1292 H= 323:00 +CPU= 29.346
  08:11:29 STEP 1293 H= 323:15 +CPU= 26.914
  08:11:58 STEP 1294 H= 323:30 +CPU= 27.835
  08:12:26 STEP 1295 H= 323:45 +CPU= 27.956
  08:13:08 STEP 1296 H= 324:00 +CPU= 41.084
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001296
  08:13:36 STEP 1297 H= 324:15 +CPU= 27.032
  08:14:04 STEP 1298 H= 324:30 +CPU= 27.533
  08:14:33 STEP 1299 H= 324:45 +CPU= 28.329
  08:15:02 STEP 1300 H= 325:00 +CPU= 28.262
  08:15:30 STEP 1301 H= 325:15 +CPU= 26.839
  08:15:57 STEP 1302 H= 325:30 +CPU= 26.985
  08:16:26 STEP 1303 H= 325:45 +CPU= 27.889
  08:16:54 STEP 1304 H= 326:00 +CPU= 27.716
  08:17:21 STEP 1305 H= 326:15 +CPU= 26.811
  08:17:49 STEP 1306 H= 326:30 +CPU= 27.393
  08:18:17 STEP 1307 H= 326:45 +CPU= 27.330
  08:18:53 STEP 1308 H= 327:00 +CPU= 34.859
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001308
  08:19:21 STEP 1309 H= 327:15 +CPU= 28.320
  08:19:49 STEP 1310 H= 327:30 +CPU= 27.211
  08:20:17 STEP 1311 H= 327:45 +CPU= 27.812
  08:20:46 STEP 1312 H= 328:00 +CPU= 28.753
  08:21:15 STEP 1313 H= 328:15 +CPU= 28.320
  08:21:44 STEP 1314 H= 328:30 +CPU= 28.128
  08:22:11 STEP 1315 H= 328:45 +CPU= 26.703
  08:22:40 STEP 1316 H= 329:00 +CPU= 28.597
  08:23:06 STEP 1317 H= 329:15 +CPU= 25.659
  08:23:33 STEP 1318 H= 329:30 +CPU= 27.065
  08:24:02 STEP 1319 H= 329:45 +CPU= 28.503
  08:24:38 STEP 1320 H= 330:00 +CPU= 35.498
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001320
  08:25:06 STEP 1321 H= 330:15 +CPU= 26.898
  08:25:33 STEP 1322 H= 330:30 +CPU= 27.131
  08:26:01 STEP 1323 H= 330:45 +CPU= 27.233
  08:26:29 STEP 1324 H= 331:00 +CPU= 28.163
  08:26:57 STEP 1325 H= 331:15 +CPU= 27.527
  08:27:25 STEP 1326 H= 331:30 +CPU= 27.353
  08:27:53 STEP 1327 H= 331:45 +CPU= 27.591
  08:28:21 STEP 1328 H= 332:00 +CPU= 27.702
  08:28:49 STEP 1329 H= 332:15 +CPU= 26.771
  08:29:17 STEP 1330 H= 332:30 +CPU= 27.929
  08:29:45 STEP 1331 H= 332:45 +CPU= 28.148
  08:30:20 STEP 1332 H= 333:00 +CPU= 33.714
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001332
  08:30:48 STEP 1333 H= 333:15 +CPU= 28.216
  08:31:16 STEP 1334 H= 333:30 +CPU= 26.976
  08:31:44 STEP 1335 H= 333:45 +CPU= 27.662
  08:32:12 STEP 1336 H= 334:00 +CPU= 28.428
  08:32:40 STEP 1337 H= 334:15 +CPU= 27.678
  08:33:08 STEP 1338 H= 334:30 +CPU= 27.109
  08:33:36 STEP 1339 H= 334:45 +CPU= 27.687
  08:34:04 STEP 1340 H= 335:00 +CPU= 27.673
  08:34:32 STEP 1341 H= 335:15 +CPU= 27.156
  08:34:59 STEP 1342 H= 335:30 +CPU= 27.186
  08:35:28 STEP 1343 H= 335:45 +CPU= 27.897
  08:36:09 STEP 1344 H= 336:00 +CPU= 40.168
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001344
  08:36:37 STEP 1345 H= 336:15 +CPU= 28.328
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001344
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930_13.zip
Uploading the intermediate file: upload_file_13.zip
  08:37:05 STEP 1346 H= 336:30 +CPU= 27.340
  08:37:33 STEP 1347 H= 336:45 +CPU= 27.193
  08:38:01 STEP 1348 H= 337:00 +CPU= 27.897
  08:38:30 STEP 1349 H= 337:15 +CPU= 28.630
  08:38:59 STEP 1350 H= 337:30 +CPU= 28.370
  08:39:28 STEP 1351 H= 337:45 +CPU= 28.215
  08:39:57 STEP 1352 H= 338:00 +CPU= 29.065
  08:40:27 STEP 1353 H= 338:15 +CPU= 29.489
  08:40:55 STEP 1354 H= 338:30 +CPU= 27.521
  08:41:22 STEP 1355 H= 338:45 +CPU= 27.259
  08:41:58 STEP 1356 H= 339:00 +CPU= 35.672
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001356
  08:42:25 STEP 1357 H= 339:15 +CPU= 26.463
  08:42:53 STEP 1358 H= 339:30 +CPU= 27.256
  08:43:20 STEP 1359 H= 339:45 +CPU= 27.128
  08:43:49 STEP 1360 H= 340:00 +CPU= 28.052
  08:44:16 STEP 1361 H= 340:15 +CPU= 26.953
  08:44:43 STEP 1362 H= 340:30 +CPU= 27.206
  08:45:10 STEP 1363 H= 340:45 +CPU= 26.236
  08:45:38 STEP 1364 H= 341:00 +CPU= 27.652
  08:46:05 STEP 1365 H= 341:15 +CPU= 26.439
  08:46:32 STEP 1366 H= 341:30 +CPU= 26.182
  08:46:58 STEP 1367 H= 341:45 +CPU= 25.402
  08:47:32 STEP 1368 H= 342:00 +CPU= 33.501
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001368
  08:47:58 STEP 1369 H= 342:15 +CPU= 26.450
  08:48:25 STEP 1370 H= 342:30 +CPU= 26.692
  08:48:52 STEP 1371 H= 342:45 +CPU= 26.085
  08:49:19 STEP 1372 H= 343:00 +CPU= 26.985
  08:49:46 STEP 1373 H= 343:15 +CPU= 26.907
  08:50:14 STEP 1374 H= 343:30 +CPU= 26.813
  08:50:40 STEP 1375 H= 343:45 +CPU= 26.252
  08:51:08 STEP 1376 H= 344:00 +CPU= 27.626
  08:51:35 STEP 1377 H= 344:15 +CPU= 26.769
  08:52:02 STEP 1378 H= 344:30 +CPU= 27.047
  08:52:30 STEP 1379 H= 344:45 +CPU= 27.372
  08:53:04 STEP 1380 H= 345:00 +CPU= 34.082
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001380
  08:53:31 STEP 1381 H= 345:15 +CPU= 26.484
  08:53:58 STEP 1382 H= 345:30 +CPU= 26.690
  08:54:25 STEP 1383 H= 345:45 +CPU= 26.817
  08:54:53 STEP 1384 H= 346:00 +CPU= 27.171
  08:55:19 STEP 1385 H= 346:15 +CPU= 26.166
  08:55:46 STEP 1386 H= 346:30 +CPU= 26.197
  08:56:12 STEP 1387 H= 346:45 +CPU= 25.892
  08:56:38 STEP 1388 H= 347:00 +CPU= 25.820
  08:57:06 STEP 1389 H= 347:15 +CPU= 27.158
  08:57:32 STEP 1390 H= 347:30 +CPU= 25.801
  08:57:58 STEP 1391 H= 347:45 +CPU= 26.131
  08:58:37 STEP 1392 H= 348:00 +CPU= 37.960
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001392
  08:59:03 STEP 1393 H= 348:15 +CPU= 25.822
  08:59:29 STEP 1394 H= 348:30 +CPU= 25.420
  08:59:55 STEP 1395 H= 348:45 +CPU= 26.305
  09:00:21 STEP 1396 H= 349:00 +CPU= 25.714
  09:00:48 STEP 1397 H= 349:15 +CPU= 26.241
  09:01:14 STEP 1398 H= 349:30 +CPU= 25.494
  09:01:39 STEP 1399 H= 349:45 +CPU= 24.972
  09:02:06 STEP 1400 H= 350:00 +CPU= 26.717
  09:02:31 STEP 1401 H= 350:15 +CPU= 24.865
  09:02:57 STEP 1402 H= 350:30 +CPU= 25.016
  09:03:22 STEP 1403 H= 350:45 +CPU= 24.861
  09:03:55 STEP 1404 H= 351:00 +CPU= 32.642
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001404
  09:04:20 STEP 1405 H= 351:15 +CPU= 25.424
  09:04:46 STEP 1406 H= 351:30 +CPU= 25.753
  09:05:12 STEP 1407 H= 351:45 +CPU= 25.220
  09:05:38 STEP 1408 H= 352:00 +CPU= 26.393
  09:06:04 STEP 1409 H= 352:15 +CPU= 25.461
  09:06:29 STEP 1410 H= 352:30 +CPU= 25.080
  09:06:54 STEP 1411 H= 352:45 +CPU= 25.061
  09:07:21 STEP 1412 H= 353:00 +CPU= 26.415
  09:07:47 STEP 1413 H= 353:15 +CPU= 25.467
  09:08:13 STEP 1414 H= 353:30 +CPU= 25.831
  09:08:38 STEP 1415 H= 353:45 +CPU= 25.292
  09:09:12 STEP 1416 H= 354:00 +CPU= 33.158
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001416
  09:09:37 STEP 1417 H= 354:15 +CPU= 25.291
  09:10:02 STEP 1418 H= 354:30 +CPU= 24.952
  09:10:28 STEP 1419 H= 354:45 +CPU= 25.372
  09:10:54 STEP 1420 H= 355:00 +CPU= 25.902
  09:11:20 STEP 1421 H= 355:15 +CPU= 25.829
  09:11:45 STEP 1422 H= 355:30 +CPU= 24.933
  09:12:11 STEP 1423 H= 355:45 +CPU= 25.488
  09:12:37 STEP 1424 H= 356:00 +CPU= 25.901
  09:13:02 STEP 1425 H= 356:15 +CPU= 25.303
  09:13:28 STEP 1426 H= 356:30 +CPU= 25.381
  09:13:54 STEP 1427 H= 356:45 +CPU= 25.494
  09:14:27 STEP 1428 H= 357:00 +CPU= 32.668
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001428
  09:14:52 STEP 1429 H= 357:15 +CPU= 25.271
  09:15:18 STEP 1430 H= 357:30 +CPU= 25.363
  09:15:43 STEP 1431 H= 357:45 +CPU= 25.411
  09:16:09 STEP 1432 H= 358:00 +CPU= 25.069
  09:16:33 STEP 1433 H= 358:15 +CPU= 24.655
  09:16:59 STEP 1434 H= 358:30 +CPU= 25.438
  09:17:24 STEP 1435 H= 358:45 +CPU= 24.613
  09:17:50 STEP 1436 H= 359:00 +CPU= 25.632
  09:18:15 STEP 1437 H= 359:15 +CPU= 25.143
  09:18:40 STEP 1438 H= 359:30 +CPU= 24.816
  09:19:05 STEP 1439 H= 359:45 +CPU= 24.529
  09:19:40 STEP 1440 H= 360:00 +CPU= 34.953
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.7306205250E-21      I       0.5296910759E-06      I
ICloud fractionI      0.0000000000E+00      I       0.4211188682E-02      I
I  Relat. Hum. I      0.3394116316E+00      I       0.3559841269E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6695437782E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    294164833 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS588329.7 Mbytes/s, TOTAL TIME=    0.0109( 4.6%)
 09:19:40 STEP 1440 H= 360:00 +CPU= 34.953
 END CNT3
 NSTEP =  1440 CNT0   000000000  24.379  24.379  24.568   0.992               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        14180/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    253    0       34      1044068    14180 ************    34.3     1.9    32.4     3.1    59.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.96      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  521    5061.7    5061.7     19.88      0.00
   9 SPCM     - SPECTRAL COMP.                     480     835.3     835.3      3.02      0.00
  10 SCAN2M - PHYSICS                              481   12376.5   12376.5     44.87      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 40    1117.4    1117.4      0.34      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 21      71.2      71.2      0.01      0.00
  17 GRIDFPOS IN CNT4                               40      31.0      31.0      0.01      0.00
  27 RESTART FILES - WRITING                        10    4469.5    4469.5      0.34      0.00
  28 RESTART FILES - READING                         1    4341.5    4341.5      0.03      0.00
  29 SU4FPOS IN CNT4                                40       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                40    6584.6    6584.6      1.99      0.00
  31 POSDDH IN STEPO                                 3       1.1       1.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    481    5481.0    5481.0     19.87      0.00
  39 SU0YOMB                                         1     429.7     429.7      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    481      26.9      26.9      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              480     248.2     248.2      0.90      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              480      61.5      61.5      0.22      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 21       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1044     445.0     445.0      3.50      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1000     255.8     255.8      1.93      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1000      93.2      93.2      0.70      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1044     138.0     138.0      1.09      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      44.4      44.4      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1044      60.3      60.3      0.47      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1000      31.8      31.8      0.24      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1044      90.5      90.5      0.71      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1000      42.1      42.1      0.32      0.00
 400 GSTATS                                      56453       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 52473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    13267.0 CPU TIME   13102.8 VECTOR TIME    13102.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       97.7
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client50002/slots/6/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/slots/6/NODE.001_01
Adding to the zip: /var/lib/boinc-client50002/slots/6/ifs.stat
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12210435/ICMGGh7zg+000960
Zipping up the final file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0_r450729930_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
09:23:13 (1530643): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:52:37 1536616 22310650 oifs_43r3_bl_a2tc_2016092300_15_991_12210435_0 1,295,100 37,428 0.0289


©2024 cpdn.org