climateprediction.net home page
Task 22313500

Task 22313500

Name oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0
Workunit 12213215
Created 13 Feb 2023, 23:50:09 UTC
Sent 14 Feb 2023, 8:50:21 UTC
Report deadline 15 Apr 2023, 8:50:21 UTC
Received 24 Feb 2023, 10:16:05 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536616
Run time 10 hours 33 min 25 sec
CPU time 10 hours 33 min 25 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 4.72 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,075.80 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,368.55 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
g+000948
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645_9.zip
Uploading the intermediate file: upload_file_9.zip
  07:18:19 STEP  961 H= 240:15 +CPU= 24.404
Uploading trickle at timestep: 864000
  07:18:44 STEP  962 H= 240:30 +CPU= 24.789
  07:19:08 STEP  963 H= 240:45 +CPU= 24.407
  07:19:33 STEP  964 H= 241:00 +CPU= 24.522
  07:19:57 STEP  965 H= 241:15 +CPU= 23.990
  07:20:21 STEP  966 H= 241:30 +CPU= 23.874
  07:20:46 STEP  967 H= 241:45 +CPU= 24.810
  07:21:11 STEP  968 H= 242:00 +CPU= 25.275
  07:21:35 STEP  969 H= 242:15 +CPU= 24.030
  07:22:00 STEP  970 H= 242:30 +CPU= 24.629
  07:22:25 STEP  971 H= 242:45 +CPU= 24.998
  07:22:57 STEP  972 H= 243:00 +CPU= 31.919
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+000972
  07:23:22 STEP  973 H= 243:15 +CPU= 24.987
  07:23:47 STEP  974 H= 243:30 +CPU= 24.753
  07:24:11 STEP  975 H= 243:45 +CPU= 24.314
  07:24:36 STEP  976 H= 244:00 +CPU= 25.019
  07:25:01 STEP  977 H= 244:15 +CPU= 24.587
  07:25:26 STEP  978 H= 244:30 +CPU= 24.812
  07:25:51 STEP  979 H= 244:45 +CPU= 25.083
  07:26:17 STEP  980 H= 245:00 +CPU= 25.448
  07:26:42 STEP  981 H= 245:15 +CPU= 25.183
  07:27:07 STEP  982 H= 245:30 +CPU= 24.329
  07:27:32 STEP  983 H= 245:45 +CPU= 25.506
  07:28:04 STEP  984 H= 246:00 +CPU= 31.789
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+000984
  07:28:29 STEP  985 H= 246:15 +CPU= 24.607
  07:28:54 STEP  986 H= 246:30 +CPU= 25.070
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4yk
(argv4) batchid: 991
(argv5) wuid: 12213215
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client50002/slots/3
Project directory is: /var/lib/boinc-client50002/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client50002/slots/3/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client50002/slots/3/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_958d6ef948ed02a00fa45b6a46cd79ff to: /var/lib/boinc-client50002/slots/3/oifs_43r3_bl_a4yk_2016092300_15_991_12213215.zip
Unzipping the namelist zip file: /var/lib/boinc-client50002/slots/3/oifs_43r3_bl_a4yk_2016092300_15_991_12213215.zip
ic_ancil_file: ic_ancil_12213215
ifsdata_file: ifsdata_12213215
climate_data_file: clim_data_12213215
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client50002/slots/3/ic_ancil_12213215.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client50002/slots/3/ic_ancil_12213215.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_d2786c04e5693a94cc9784ca3fc471d1 to: /var/lib/boinc-client50002/slots/3/ifsdata/ifsdata_12213215.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client50002/slots/3/ifsdata/ifsdata_12213215.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client50002/slots/3/159l_2/clim_data_12213215.zip
Unzipping the climate data zip file: /var/lib/boinc-client50002/slots/3/159l_2/clim_data_12213215.zip
Checking for progress XML file: /var/lib/boinc-client50002/slots/3/progress_file_12213215.xml
Opened progress file ok : /var/lib/boinc-client50002/slots/3/progress_file_12213215.xml
-- Model is restarting --
Adjusting last_iter, 987, to previous model restart step.
Creating progress file: /var/lib/boinc-client50002/slots/3/progress_file_12213215.xml
last_cpu_time: 24883
upload_file_number: 10
last_iter: 961
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645
The child process has been launched with process id: 1530682
Executing the command: /var/lib/boinc-client50002/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530682:1530682] [20230224:053833:1677217113:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:39:03 STEP  960 H= 240:00 +CPU= 24.654
  05:39:27 STEP  961 H= 240:15 +CPU= 23.580
  05:39:51 STEP  962 H= 240:30 +CPU= 24.528
  05:40:16 STEP  963 H= 240:45 +CPU= 24.316
  05:40:40 STEP  964 H= 241:00 +CPU= 24.749
  05:41:04 STEP  965 H= 241:15 +CPU= 23.568
  05:41:28 STEP  966 H= 241:30 +CPU= 23.671
  05:41:52 STEP  967 H= 241:45 +CPU= 23.548
  05:42:17 STEP  968 H= 242:00 +CPU= 25.188
  05:42:41 STEP  969 H= 242:15 +CPU= 24.159
  05:43:05 STEP  970 H= 242:30 +CPU= 23.348
  05:43:27 STEP  971 H= 242:45 +CPU= 22.448
  05:43:57 STEP  972 H= 243:00 +CPU= 30.117
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+000972
  05:44:21 STEP  973 H= 243:15 +CPU= 23.545
  05:44:44 STEP  974 H= 243:30 +CPU= 23.203
  05:45:08 STEP  975 H= 243:45 +CPU= 23.324
  05:45:32 STEP  976 H= 244:00 +CPU= 24.373
  05:45:56 STEP  977 H= 244:15 +CPU= 23.890
  05:46:19 STEP  978 H= 244:30 +CPU= 23.146
  05:46:43 STEP  979 H= 244:45 +CPU= 23.612
  05:47:07 STEP  980 H= 245:00 +CPU= 23.875
  05:47:30 STEP  981 H= 245:15 +CPU= 23.382
  05:47:54 STEP  982 H= 245:30 +CPU= 23.812
  05:48:19 STEP  983 H= 245:45 +CPU= 24.858
  05:48:51 STEP  984 H= 246:00 +CPU= 31.211
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+000984
  05:49:14 STEP  985 H= 246:15 +CPU= 23.768
  05:49:39 STEP  986 H= 246:30 +CPU= 24.057
  05:50:03 STEP  987 H= 246:45 +CPU= 24.312
  05:50:27 STEP  988 H= 247:00 +CPU= 24.153
  05:50:51 STEP  989 H= 247:15 +CPU= 23.959
  05:51:15 STEP  990 H= 247:30 +CPU= 23.239
  05:51:39 STEP  991 H= 247:45 +CPU= 24.543
  05:52:04 STEP  992 H= 248:00 +CPU= 24.761
  05:52:28 STEP  993 H= 248:15 +CPU= 23.986
  05:52:51 STEP  994 H= 248:30 +CPU= 23.068
  05:53:15 STEP  995 H= 248:45 +CPU= 23.854
  05:53:47 STEP  996 H= 249:00 +CPU= 31.157
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+000996
  05:54:10 STEP  997 H= 249:15 +CPU= 23.537
  05:54:33 STEP  998 H= 249:30 +CPU= 23.038
  05:54:57 STEP  999 H= 249:45 +CPU= 23.471
  05:55:21 STEP 1000 H= 250:00 +CPU= 24.010
  05:55:44 STEP 1001 H= 250:15 +CPU= 23.291
  05:56:08 STEP 1002 H= 250:30 +CPU= 23.944
  05:56:32 STEP 1003 H= 250:45 +CPU= 23.586
  05:56:57 STEP 1004 H= 251:00 +CPU= 24.978
  05:57:21 STEP 1005 H= 251:15 +CPU= 23.448
  05:57:45 STEP 1006 H= 251:30 +CPU= 24.217
  05:58:09 STEP 1007 H= 251:45 +CPU= 24.122
  05:58:45 STEP 1008 H= 252:00 +CPU= 35.452
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001008
  05:59:09 STEP 1009 H= 252:15 +CPU= 24.103
  05:59:34 STEP 1010 H= 252:30 +CPU= 24.482
  05:59:58 STEP 1011 H= 252:45 +CPU= 24.314
  06:00:23 STEP 1012 H= 253:00 +CPU= 24.680
  06:00:46 STEP 1013 H= 253:15 +CPU= 23.515
  06:01:09 STEP 1014 H= 253:30 +CPU= 22.972
  06:01:34 STEP 1015 H= 253:45 +CPU= 24.042
  06:01:58 STEP 1016 H= 254:00 +CPU= 24.240
  06:02:22 STEP 1017 H= 254:15 +CPU= 24.494
  06:02:46 STEP 1018 H= 254:30 +CPU= 23.613
  06:03:10 STEP 1019 H= 254:45 +CPU= 24.080
  06:03:42 STEP 1020 H= 255:00 +CPU= 31.277
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001020
  06:04:06 STEP 1021 H= 255:15 +CPU= 23.900
  06:04:30 STEP 1022 H= 255:30 +CPU= 23.865
  06:04:54 STEP 1023 H= 255:45 +CPU= 24.076
  06:05:19 STEP 1024 H= 256:00 +CPU= 25.013
  06:05:43 STEP 1025 H= 256:15 +CPU= 23.575
  06:06:06 STEP 1026 H= 256:30 +CPU= 23.422
  06:06:30 STEP 1027 H= 256:45 +CPU= 24.069
  06:06:55 STEP 1028 H= 257:00 +CPU= 24.970
  06:07:20 STEP 1029 H= 257:15 +CPU= 24.235
  06:07:43 STEP 1030 H= 257:30 +CPU= 23.541
  06:08:07 STEP 1031 H= 257:45 +CPU= 23.905
  06:08:38 STEP 1032 H= 258:00 +CPU= 30.672
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001032
  06:09:02 STEP 1033 H= 258:15 +CPU= 23.800
  06:09:26 STEP 1034 H= 258:30 +CPU= 23.586
  06:09:49 STEP 1035 H= 258:45 +CPU= 23.651
  06:10:15 STEP 1036 H= 259:00 +CPU= 25.181
  06:10:38 STEP 1037 H= 259:15 +CPU= 23.605
  06:11:02 STEP 1038 H= 259:30 +CPU= 23.339
  06:11:25 STEP 1039 H= 259:45 +CPU= 22.723
  06:11:49 STEP 1040 H= 260:00 +CPU= 24.456
  06:12:14 STEP 1041 H= 260:15 +CPU= 24.617
  06:12:38 STEP 1042 H= 260:30 +CPU= 23.874
  06:13:02 STEP 1043 H= 260:45 +CPU= 24.559
  06:13:33 STEP 1044 H= 261:00 +CPU= 30.259
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001044
  06:13:56 STEP 1045 H= 261:15 +CPU= 23.352
  06:14:20 STEP 1046 H= 261:30 +CPU= 23.979
  06:14:44 STEP 1047 H= 261:45 +CPU= 23.964
  06:15:09 STEP 1048 H= 262:00 +CPU= 24.428
  06:15:33 STEP 1049 H= 262:15 +CPU= 23.883
  06:15:57 STEP 1050 H= 262:30 +CPU= 23.764
  06:16:22 STEP 1051 H= 262:45 +CPU= 24.815
  06:16:46 STEP 1052 H= 263:00 +CPU= 24.564
  06:17:10 STEP 1053 H= 263:15 +CPU= 23.614
  06:17:34 STEP 1054 H= 263:30 +CPU= 23.418
  06:17:57 STEP 1055 H= 263:45 +CPU= 22.913
  06:18:32 STEP 1056 H= 264:00 +CPU= 34.939
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645_10.zip
Uploading the intermediate file: upload_file_10.zip
  06:18:56 STEP 1057 H= 264:15 +CPU= 23.935
  06:19:20 STEP 1058 H= 264:30 +CPU= 24.536
  06:19:45 STEP 1059 H= 264:45 +CPU= 24.573
  06:20:10 STEP 1060 H= 265:00 +CPU= 25.158
  06:20:34 STEP 1061 H= 265:15 +CPU= 23.938
  06:20:58 STEP 1062 H= 265:30 +CPU= 23.779
  06:21:23 STEP 1063 H= 265:45 +CPU= 24.375
  06:21:47 STEP 1064 H= 266:00 +CPU= 23.945
  06:22:10 STEP 1065 H= 266:15 +CPU= 23.655
  06:22:35 STEP 1066 H= 266:30 +CPU= 24.725
  06:22:59 STEP 1067 H= 266:45 +CPU= 23.569
  06:23:29 STEP 1068 H= 267:00 +CPU= 30.277
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001068
  06:23:54 STEP 1069 H= 267:15 +CPU= 24.449
  06:24:18 STEP 1070 H= 267:30 +CPU= 23.664
  06:24:42 STEP 1071 H= 267:45 +CPU= 24.083
  06:25:07 STEP 1072 H= 268:00 +CPU= 25.275
  06:25:31 STEP 1073 H= 268:15 +CPU= 23.765
  06:25:55 STEP 1074 H= 268:30 +CPU= 23.518
  06:26:19 STEP 1075 H= 268:45 +CPU= 23.807
  06:26:43 STEP 1076 H= 269:00 +CPU= 24.064
  06:27:06 STEP 1077 H= 269:15 +CPU= 23.328
  06:27:30 STEP 1078 H= 269:30 +CPU= 23.806
  06:27:54 STEP 1079 H= 269:45 +CPU= 24.129
  06:28:26 STEP 1080 H= 270:00 +CPU= 31.716
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001080
  06:28:50 STEP 1081 H= 270:15 +CPU= 23.377
  06:29:14 STEP 1082 H= 270:30 +CPU= 24.245
  06:29:37 STEP 1083 H= 270:45 +CPU= 22.900
  06:30:01 STEP 1084 H= 271:00 +CPU= 24.454
  06:30:25 STEP 1085 H= 271:15 +CPU= 23.241
  06:30:49 STEP 1086 H= 271:30 +CPU= 24.671
  06:31:13 STEP 1087 H= 271:45 +CPU= 23.687
  06:31:38 STEP 1088 H= 272:00 +CPU= 24.888
  06:32:03 STEP 1089 H= 272:15 +CPU= 24.267
  06:32:27 STEP 1090 H= 272:30 +CPU= 24.401
  06:32:51 STEP 1091 H= 272:45 +CPU= 23.704
  06:33:22 STEP 1092 H= 273:00 +CPU= 31.050
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001092
  06:33:45 STEP 1093 H= 273:15 +CPU= 23.297
  06:34:09 STEP 1094 H= 273:30 +CPU= 23.374
  06:34:33 STEP 1095 H= 273:45 +CPU= 23.606
  06:34:57 STEP 1096 H= 274:00 +CPU= 24.388
  06:35:21 STEP 1097 H= 274:15 +CPU= 24.267
  06:35:45 STEP 1098 H= 274:30 +CPU= 23.818
  06:36:08 STEP 1099 H= 274:45 +CPU= 22.982
  06:36:33 STEP 1100 H= 275:00 +CPU= 24.533
  06:36:58 STEP 1101 H= 275:15 +CPU= 24.486
  06:37:24 STEP 1102 H= 275:30 +CPU= 25.894
  06:37:53 STEP 1103 H= 275:45 +CPU= 28.440
  06:38:35 STEP 1104 H= 276:00 +CPU= 40.640
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001104
  06:39:03 STEP 1105 H= 276:15 +CPU= 27.856
  06:39:31 STEP 1106 H= 276:30 +CPU= 27.488
  06:39:59 STEP 1107 H= 276:45 +CPU= 28.069
  06:40:28 STEP 1108 H= 277:00 +CPU= 27.811
  06:40:55 STEP 1109 H= 277:15 +CPU= 26.437
  06:41:24 STEP 1110 H= 277:30 +CPU= 28.400
  06:41:52 STEP 1111 H= 277:45 +CPU= 27.582
  06:42:21 STEP 1112 H= 278:00 +CPU= 28.896
  06:42:50 STEP 1113 H= 278:15 +CPU= 27.940
  06:43:18 STEP 1114 H= 278:30 +CPU= 27.562
  06:43:46 STEP 1115 H= 278:45 +CPU= 27.789
  06:44:23 STEP 1116 H= 279:00 +CPU= 36.420
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001116
  06:44:51 STEP 1117 H= 279:15 +CPU= 27.780
  06:45:19 STEP 1118 H= 279:30 +CPU= 27.788
  06:45:48 STEP 1119 H= 279:45 +CPU= 28.406
  06:46:17 STEP 1120 H= 280:00 +CPU= 28.645
  06:46:45 STEP 1121 H= 280:15 +CPU= 27.753
  06:47:14 STEP 1122 H= 280:30 +CPU= 28.145
  06:47:42 STEP 1123 H= 280:45 +CPU= 27.670
  06:48:11 STEP 1124 H= 281:00 +CPU= 28.588
  06:48:39 STEP 1125 H= 281:15 +CPU= 27.941
  06:49:08 STEP 1126 H= 281:30 +CPU= 28.628
  06:49:35 STEP 1127 H= 281:45 +CPU= 26.855
  06:50:11 STEP 1128 H= 282:00 +CPU= 35.486
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001128
  06:50:40 STEP 1129 H= 282:15 +CPU= 27.939
  06:51:09 STEP 1130 H= 282:30 +CPU= 28.361
  06:51:37 STEP 1131 H= 282:45 +CPU= 27.714
  06:52:05 STEP 1132 H= 283:00 +CPU= 28.321
  06:52:33 STEP 1133 H= 283:15 +CPU= 26.973
  06:53:00 STEP 1134 H= 283:30 +CPU= 27.158
  06:53:29 STEP 1135 H= 283:45 +CPU= 28.028
  06:53:58 STEP 1136 H= 284:00 +CPU= 29.039
  06:54:26 STEP 1137 H= 284:15 +CPU= 26.931
  06:54:54 STEP 1138 H= 284:30 +CPU= 27.958
  06:55:23 STEP 1139 H= 284:45 +CPU= 27.591
  06:55:58 STEP 1140 H= 285:00 +CPU= 35.142
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001140
  06:56:27 STEP 1141 H= 285:15 +CPU= 27.180
  06:56:55 STEP 1142 H= 285:30 +CPU= 27.438
  06:57:23 STEP 1143 H= 285:45 +CPU= 27.452
  06:57:53 STEP 1144 H= 286:00 +CPU= 29.357
  06:58:22 STEP 1145 H= 286:15 +CPU= 29.170
  06:58:50 STEP 1146 H= 286:30 +CPU= 27.607
  06:59:19 STEP 1147 H= 286:45 +CPU= 28.507
  06:59:48 STEP 1148 H= 287:00 +CPU= 27.901
  07:00:16 STEP 1149 H= 287:15 +CPU= 27.844
  07:00:43 STEP 1150 H= 287:30 +CPU= 27.189
  07:01:14 STEP 1151 H= 287:45 +CPU= 29.660
  07:01:56 STEP 1152 H= 288:00 +CPU= 41.572
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645_11.zip
Uploading the intermediate file: upload_file_11.zip
  07:02:24 STEP 1153 H= 288:15 +CPU= 27.579
  07:02:53 STEP 1154 H= 288:30 +CPU= 28.591
  07:03:23 STEP 1155 H= 288:45 +CPU= 28.874
  07:03:50 STEP 1156 H= 289:00 +CPU= 27.221
  07:04:18 STEP 1157 H= 289:15 +CPU= 27.387
  07:04:47 STEP 1158 H= 289:30 +CPU= 27.932
  07:05:16 STEP 1159 H= 289:45 +CPU= 29.138
  07:05:46 STEP 1160 H= 290:00 +CPU= 28.993
  07:06:14 STEP 1161 H= 290:15 +CPU= 28.540
  07:06:44 STEP 1162 H= 290:30 +CPU= 28.814
  07:07:11 STEP 1163 H= 290:45 +CPU= 26.952
  07:07:48 STEP 1164 H= 291:00 +CPU= 36.029
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001164
  07:08:16 STEP 1165 H= 291:15 +CPU= 28.016
  07:08:45 STEP 1166 H= 291:30 +CPU= 27.944
  07:09:14 STEP 1167 H= 291:45 +CPU= 29.368
  07:09:44 STEP 1168 H= 292:00 +CPU= 29.236
  07:10:13 STEP 1169 H= 292:15 +CPU= 28.674
  07:10:43 STEP 1170 H= 292:30 +CPU= 28.806
  07:11:11 STEP 1171 H= 292:45 +CPU= 28.039
  07:11:41 STEP 1172 H= 293:00 +CPU= 28.893
  07:12:09 STEP 1173 H= 293:15 +CPU= 27.392
  07:12:37 STEP 1174 H= 293:30 +CPU= 27.668
  07:13:04 STEP 1175 H= 293:45 +CPU= 26.753
  07:13:40 STEP 1176 H= 294:00 +CPU= 35.867
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001176
  07:14:10 STEP 1177 H= 294:15 +CPU= 28.800
  07:14:38 STEP 1178 H= 294:30 +CPU= 27.741
  07:15:08 STEP 1179 H= 294:45 +CPU= 29.454
  07:15:37 STEP 1180 H= 295:00 +CPU= 28.407
  07:16:04 STEP 1181 H= 295:15 +CPU= 27.474
  07:16:32 STEP 1182 H= 295:30 +CPU= 27.571
  07:17:01 STEP 1183 H= 295:45 +CPU= 27.726
  07:17:30 STEP 1184 H= 296:00 +CPU= 29.168
  07:17:59 STEP 1185 H= 296:15 +CPU= 27.873
  07:18:28 STEP 1186 H= 296:30 +CPU= 28.413
  07:18:56 STEP 1187 H= 296:45 +CPU= 27.885
  07:19:31 STEP 1188 H= 297:00 +CPU= 34.886
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001188
  07:20:01 STEP 1189 H= 297:15 +CPU= 29.124
  07:20:31 STEP 1190 H= 297:30 +CPU= 29.061
  07:20:59 STEP 1191 H= 297:45 +CPU= 28.105
  07:21:28 STEP 1192 H= 298:00 +CPU= 28.948
  07:21:56 STEP 1193 H= 298:15 +CPU= 27.371
  07:22:24 STEP 1194 H= 298:30 +CPU= 27.910
  07:22:54 STEP 1195 H= 298:45 +CPU= 29.115
  07:23:24 STEP 1196 H= 299:00 +CPU= 29.794
  07:23:52 STEP 1197 H= 299:15 +CPU= 28.076
  07:24:22 STEP 1198 H= 299:30 +CPU= 29.489
  07:24:52 STEP 1199 H= 299:45 +CPU= 29.361
  07:25:34 STEP 1200 H= 300:00 +CPU= 40.830
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001200
  07:26:02 STEP 1201 H= 300:15 +CPU= 27.656
  07:26:31 STEP 1202 H= 300:30 +CPU= 28.124
  07:27:01 STEP 1203 H= 300:45 +CPU= 28.645
  07:27:29 STEP 1204 H= 301:00 +CPU= 27.985
  07:27:57 STEP 1205 H= 301:15 +CPU= 27.364
  07:28:27 STEP 1206 H= 301:30 +CPU= 29.261
  07:28:57 STEP 1207 H= 301:45 +CPU= 29.096
  07:29:26 STEP 1208 H= 302:00 +CPU= 28.754
  07:29:55 STEP 1209 H= 302:15 +CPU= 27.997
  07:30:24 STEP 1210 H= 302:30 +CPU= 28.881
  07:30:55 STEP 1211 H= 302:45 +CPU= 30.259
  07:31:32 STEP 1212 H= 303:00 +CPU= 36.375
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001212
  07:32:00 STEP 1213 H= 303:15 +CPU= 27.484
  07:32:29 STEP 1214 H= 303:30 +CPU= 28.578
  07:32:57 STEP 1215 H= 303:45 +CPU= 28.029
  07:33:28 STEP 1216 H= 304:00 +CPU= 30.496
  07:33:56 STEP 1217 H= 304:15 +CPU= 28.219
  07:34:26 STEP 1218 H= 304:30 +CPU= 28.803
  07:34:54 STEP 1219 H= 304:45 +CPU= 28.392
  07:35:24 STEP 1220 H= 305:00 +CPU= 29.238
  07:35:53 STEP 1221 H= 305:15 +CPU= 28.659
  07:36:23 STEP 1222 H= 305:30 +CPU= 29.727
  07:36:51 STEP 1223 H= 305:45 +CPU= 27.680
  07:37:28 STEP 1224 H= 306:00 +CPU= 36.034
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001224
  07:37:58 STEP 1225 H= 306:15 +CPU= 29.394
  07:38:27 STEP 1226 H= 306:30 +CPU= 29.215
  07:38:55 STEP 1227 H= 306:45 +CPU= 27.055
  07:39:25 STEP 1228 H= 307:00 +CPU= 29.304
  07:39:53 STEP 1229 H= 307:15 +CPU= 27.839
  07:40:25 STEP 1230 H= 307:30 +CPU= 31.215
  07:40:53 STEP 1231 H= 307:45 +CPU= 28.056
  07:41:23 STEP 1232 H= 308:00 +CPU= 28.920
  07:41:52 STEP 1233 H= 308:15 +CPU= 29.108
  07:42:21 STEP 1234 H= 308:30 +CPU= 28.037
  07:42:49 STEP 1235 H= 308:45 +CPU= 27.612
  07:43:25 STEP 1236 H= 309:00 +CPU= 36.031
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001236
  07:43:54 STEP 1237 H= 309:15 +CPU= 27.978
  07:44:21 STEP 1238 H= 309:30 +CPU= 27.199
  07:44:50 STEP 1239 H= 309:45 +CPU= 28.194
  07:45:19 STEP 1240 H= 310:00 +CPU= 28.928
  07:45:48 STEP 1241 H= 310:15 +CPU= 28.042
  07:46:17 STEP 1242 H= 310:30 +CPU= 28.539
  07:46:46 STEP 1243 H= 310:45 +CPU= 29.093
  07:47:14 STEP 1244 H= 311:00 +CPU= 27.410
  07:47:42 STEP 1245 H= 311:15 +CPU= 27.370
  07:48:12 STEP 1246 H= 311:30 +CPU= 29.582
  07:48:41 STEP 1247 H= 311:45 +CPU= 27.912
  07:49:22 STEP 1248 H= 312:00 +CPU= 40.550
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645_12.zip
Uploading the intermediate file: upload_file_12.zip
  07:49:52 STEP 1249 H= 312:15 +CPU= 29.414
  07:50:21 STEP 1250 H= 312:30 +CPU= 28.610
  07:50:50 STEP 1251 H= 312:45 +CPU= 28.917
  07:51:21 STEP 1252 H= 313:00 +CPU= 29.637
  07:51:50 STEP 1253 H= 313:15 +CPU= 29.499
  07:52:18 STEP 1254 H= 313:30 +CPU= 27.033
  07:52:48 STEP 1255 H= 313:45 +CPU= 29.166
  07:53:17 STEP 1256 H= 314:00 +CPU= 28.541
  07:53:47 STEP 1257 H= 314:15 +CPU= 28.821
  07:54:16 STEP 1258 H= 314:30 +CPU= 28.758
  07:54:45 STEP 1259 H= 314:45 +CPU= 28.780
  07:55:21 STEP 1260 H= 315:00 +CPU= 35.510
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001260
  07:55:50 STEP 1261 H= 315:15 +CPU= 28.117
  07:56:18 STEP 1262 H= 315:30 +CPU= 27.874
  07:56:47 STEP 1263 H= 315:45 +CPU= 28.285
  07:57:16 STEP 1264 H= 316:00 +CPU= 28.270
  07:57:44 STEP 1265 H= 316:15 +CPU= 27.781
  07:58:12 STEP 1266 H= 316:30 +CPU= 27.929
  07:58:41 STEP 1267 H= 316:45 +CPU= 28.051
  07:59:10 STEP 1268 H= 317:00 +CPU= 28.636
  07:59:38 STEP 1269 H= 317:15 +CPU= 27.741
  08:00:08 STEP 1270 H= 317:30 +CPU= 29.365
  08:00:38 STEP 1271 H= 317:45 +CPU= 28.804
  08:01:13 STEP 1272 H= 318:00 +CPU= 35.157
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001272
  08:01:41 STEP 1273 H= 318:15 +CPU= 27.704
  08:02:09 STEP 1274 H= 318:30 +CPU= 27.630
  08:02:38 STEP 1275 H= 318:45 +CPU= 28.377
  08:03:07 STEP 1276 H= 319:00 +CPU= 28.523
  08:03:36 STEP 1277 H= 319:15 +CPU= 27.987
  08:04:03 STEP 1278 H= 319:30 +CPU= 27.228
  08:04:32 STEP 1279 H= 319:45 +CPU= 28.264
  08:05:00 STEP 1280 H= 320:00 +CPU= 27.549
  08:05:28 STEP 1281 H= 320:15 +CPU= 27.447
  08:05:58 STEP 1282 H= 320:30 +CPU= 29.165
  08:06:28 STEP 1283 H= 320:45 +CPU= 29.154
  08:07:03 STEP 1284 H= 321:00 +CPU= 35.483
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001284
  08:07:32 STEP 1285 H= 321:15 +CPU= 28.118
  08:08:01 STEP 1286 H= 321:30 +CPU= 29.003
  08:08:29 STEP 1287 H= 321:45 +CPU= 27.740
  08:08:59 STEP 1288 H= 322:00 +CPU= 29.815
  08:09:28 STEP 1289 H= 322:15 +CPU= 28.096
  08:09:58 STEP 1290 H= 322:30 +CPU= 29.137
  08:10:27 STEP 1291 H= 322:45 +CPU= 28.554
  08:10:56 STEP 1292 H= 323:00 +CPU= 29.225
  08:11:25 STEP 1293 H= 323:15 +CPU= 27.767
  08:11:53 STEP 1294 H= 323:30 +CPU= 27.675
  08:12:22 STEP 1295 H= 323:45 +CPU= 29.311
  08:13:04 STEP 1296 H= 324:00 +CPU= 40.558
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001296
  08:13:33 STEP 1297 H= 324:15 +CPU= 28.502
  08:14:02 STEP 1298 H= 324:30 +CPU= 28.370
  08:14:30 STEP 1299 H= 324:45 +CPU= 27.863
  08:15:00 STEP 1300 H= 325:00 +CPU= 29.210
  08:15:27 STEP 1301 H= 325:15 +CPU= 26.328
  08:15:55 STEP 1302 H= 325:30 +CPU= 27.546
  08:16:23 STEP 1303 H= 325:45 +CPU= 27.311
  08:16:52 STEP 1304 H= 326:00 +CPU= 28.822
  08:17:20 STEP 1305 H= 326:15 +CPU= 26.875
  08:17:47 STEP 1306 H= 326:30 +CPU= 26.949
  08:18:15 STEP 1307 H= 326:45 +CPU= 27.667
  08:18:51 STEP 1308 H= 327:00 +CPU= 35.174
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001308
  08:19:20 STEP 1309 H= 327:15 +CPU= 28.116
  08:19:47 STEP 1310 H= 327:30 +CPU= 26.741
  08:20:15 STEP 1311 H= 327:45 +CPU= 27.742
  08:20:43 STEP 1312 H= 328:00 +CPU= 27.832
  08:21:12 STEP 1313 H= 328:15 +CPU= 28.532
  08:21:39 STEP 1314 H= 328:30 +CPU= 26.381
  08:22:06 STEP 1315 H= 328:45 +CPU= 27.233
  08:22:35 STEP 1316 H= 329:00 +CPU= 28.015
  08:23:04 STEP 1317 H= 329:15 +CPU= 28.443
  08:23:32 STEP 1318 H= 329:30 +CPU= 27.738
  08:23:59 STEP 1319 H= 329:45 +CPU= 26.776
  08:24:34 STEP 1320 H= 330:00 +CPU= 34.552
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001320
  08:25:02 STEP 1321 H= 330:15 +CPU= 27.282
  08:25:30 STEP 1322 H= 330:30 +CPU= 27.828
  08:25:57 STEP 1323 H= 330:45 +CPU= 26.934
  08:26:27 STEP 1324 H= 331:00 +CPU= 28.862
  08:26:54 STEP 1325 H= 331:15 +CPU= 26.785
  08:27:21 STEP 1326 H= 331:30 +CPU= 26.920
  08:27:50 STEP 1327 H= 331:45 +CPU= 28.593
  08:28:20 STEP 1328 H= 332:00 +CPU= 29.039
  08:28:48 STEP 1329 H= 332:15 +CPU= 27.719
  08:29:16 STEP 1330 H= 332:30 +CPU= 27.769
  08:29:45 STEP 1331 H= 332:45 +CPU= 28.199
  08:30:21 STEP 1332 H= 333:00 +CPU= 35.663
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001332
  08:30:49 STEP 1333 H= 333:15 +CPU= 27.606
  08:31:17 STEP 1334 H= 333:30 +CPU= 27.191
  08:31:44 STEP 1335 H= 333:45 +CPU= 27.067
  08:32:12 STEP 1336 H= 334:00 +CPU= 27.893
  08:32:40 STEP 1337 H= 334:15 +CPU= 27.711
  08:33:08 STEP 1338 H= 334:30 +CPU= 27.638
  08:33:37 STEP 1339 H= 334:45 +CPU= 28.379
  08:34:05 STEP 1340 H= 335:00 +CPU= 27.551
  08:34:33 STEP 1341 H= 335:15 +CPU= 27.576
  08:35:02 STEP 1342 H= 335:30 +CPU= 27.902
  08:35:30 STEP 1343 H= 335:45 +CPU= 28.110
  08:36:11 STEP 1344 H= 336:00 +CPU= 39.807
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645_13.zip
Uploading the intermediate file: upload_file_13.zip
  08:36:38 STEP 1345 H= 336:15 +CPU= 26.948
  08:37:07 STEP 1346 H= 336:30 +CPU= 28.165
  08:37:35 STEP 1347 H= 336:45 +CPU= 27.268
  08:38:04 STEP 1348 H= 337:00 +CPU= 28.798
  08:38:31 STEP 1349 H= 337:15 +CPU= 26.255
  08:38:59 STEP 1350 H= 337:30 +CPU= 28.013
  08:39:27 STEP 1351 H= 337:45 +CPU= 27.513
  08:39:55 STEP 1352 H= 338:00 +CPU= 27.747
  08:40:24 STEP 1353 H= 338:15 +CPU= 28.422
  08:40:52 STEP 1354 H= 338:30 +CPU= 27.522
  08:41:20 STEP 1355 H= 338:45 +CPU= 27.324
  08:41:55 STEP 1356 H= 339:00 +CPU= 35.116
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001356
  08:42:23 STEP 1357 H= 339:15 +CPU= 27.341
  08:42:49 STEP 1358 H= 339:30 +CPU= 25.907
  08:43:16 STEP 1359 H= 339:45 +CPU= 25.975
  08:43:45 STEP 1360 H= 340:00 +CPU= 28.727
  08:44:12 STEP 1361 H= 340:15 +CPU= 27.346
  08:44:39 STEP 1362 H= 340:30 +CPU= 26.608
  08:45:06 STEP 1363 H= 340:45 +CPU= 26.688
  08:45:34 STEP 1364 H= 341:00 +CPU= 27.710
  08:46:02 STEP 1365 H= 341:15 +CPU= 27.056
  08:46:28 STEP 1366 H= 341:30 +CPU= 26.219
  08:46:56 STEP 1367 H= 341:45 +CPU= 27.806
  08:47:31 STEP 1368 H= 342:00 +CPU= 33.993
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001368
  08:47:56 STEP 1369 H= 342:15 +CPU= 25.157
  08:48:24 STEP 1370 H= 342:30 +CPU= 26.762
  08:48:51 STEP 1371 H= 342:45 +CPU= 27.287
  08:49:18 STEP 1372 H= 343:00 +CPU= 27.052
  08:49:46 STEP 1373 H= 343:15 +CPU= 27.281
  08:50:14 STEP 1374 H= 343:30 +CPU= 27.490
  08:50:40 STEP 1375 H= 343:45 +CPU= 25.798
  08:51:07 STEP 1376 H= 344:00 +CPU= 27.211
  08:51:35 STEP 1377 H= 344:15 +CPU= 26.908
  08:52:01 STEP 1378 H= 344:30 +CPU= 26.400
  08:52:29 STEP 1379 H= 344:45 +CPU= 27.120
  08:53:04 STEP 1380 H= 345:00 +CPU= 34.499
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001380
  08:53:30 STEP 1381 H= 345:15 +CPU= 26.483
  08:53:57 STEP 1382 H= 345:30 +CPU= 26.027
  08:54:24 STEP 1383 H= 345:45 +CPU= 26.475
  08:54:51 STEP 1384 H= 346:00 +CPU= 27.231
  08:55:18 STEP 1385 H= 346:15 +CPU= 26.099
  08:55:44 STEP 1386 H= 346:30 +CPU= 26.536
  08:56:11 STEP 1387 H= 346:45 +CPU= 26.131
  08:56:38 STEP 1388 H= 347:00 +CPU= 26.739
  08:57:03 STEP 1389 H= 347:15 +CPU= 25.197
  08:57:31 STEP 1390 H= 347:30 +CPU= 26.965
  08:57:57 STEP 1391 H= 347:45 +CPU= 26.364
  08:58:36 STEP 1392 H= 348:00 +CPU= 37.818
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001392
  08:59:02 STEP 1393 H= 348:15 +CPU= 25.814
  08:59:28 STEP 1394 H= 348:30 +CPU= 26.463
  08:59:55 STEP 1395 H= 348:45 +CPU= 26.438
  09:00:23 STEP 1396 H= 349:00 +CPU= 27.410
  09:00:50 STEP 1397 H= 349:15 +CPU= 26.434
  09:01:16 STEP 1398 H= 349:30 +CPU= 25.756
  09:01:43 STEP 1399 H= 349:45 +CPU= 26.541
  09:02:09 STEP 1400 H= 350:00 +CPU= 26.530
  09:02:35 STEP 1401 H= 350:15 +CPU= 25.894
  09:03:01 STEP 1402 H= 350:30 +CPU= 25.438
  09:03:27 STEP 1403 H= 350:45 +CPU= 25.424
  09:04:00 STEP 1404 H= 351:00 +CPU= 33.024
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001404
  09:04:26 STEP 1405 H= 351:15 +CPU= 26.209
  09:04:52 STEP 1406 H= 351:30 +CPU= 25.301
  09:05:18 STEP 1407 H= 351:45 +CPU= 25.515
  09:05:44 STEP 1408 H= 352:00 +CPU= 26.117
  09:06:09 STEP 1409 H= 352:15 +CPU= 25.094
  09:06:35 STEP 1410 H= 352:30 +CPU= 25.296
  09:06:59 STEP 1411 H= 352:45 +CPU= 24.664
  09:07:26 STEP 1412 H= 353:00 +CPU= 26.163
  09:07:51 STEP 1413 H= 353:15 +CPU= 25.460
  09:08:17 STEP 1414 H= 353:30 +CPU= 25.386
  09:08:42 STEP 1415 H= 353:45 +CPU= 25.388
  09:09:15 STEP 1416 H= 354:00 +CPU= 32.435
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001416
  09:09:41 STEP 1417 H= 354:15 +CPU= 25.201
  09:10:07 STEP 1418 H= 354:30 +CPU= 26.263
  09:10:33 STEP 1419 H= 354:45 +CPU= 25.879
  09:10:59 STEP 1420 H= 355:00 +CPU= 26.004
  09:11:26 STEP 1421 H= 355:15 +CPU= 26.326
  09:11:52 STEP 1422 H= 355:30 +CPU= 25.668
  09:12:18 STEP 1423 H= 355:45 +CPU= 25.715
  09:12:44 STEP 1424 H= 356:00 +CPU= 26.091
  09:13:10 STEP 1425 H= 356:15 +CPU= 25.218
  09:13:35 STEP 1426 H= 356:30 +CPU= 25.267
  09:14:01 STEP 1427 H= 356:45 +CPU= 25.343
  09:14:34 STEP 1428 H= 357:00 +CPU= 33.247
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001428
  09:15:00 STEP 1429 H= 357:15 +CPU= 25.258
  09:15:25 STEP 1430 H= 357:30 +CPU= 25.313
  09:15:51 STEP 1431 H= 357:45 +CPU= 25.570
  09:16:18 STEP 1432 H= 358:00 +CPU= 26.180
  09:16:43 STEP 1433 H= 358:15 +CPU= 25.411
  09:17:09 STEP 1434 H= 358:30 +CPU= 25.892
  09:17:35 STEP 1435 H= 358:45 +CPU= 25.825
  09:18:02 STEP 1436 H= 359:00 +CPU= 25.965
  09:18:27 STEP 1437 H= 359:15 +CPU= 25.648
  09:18:53 STEP 1438 H= 359:30 +CPU= 25.544
  09:19:20 STEP 1439 H= 359:45 +CPU= 26.118
  09:19:55 STEP 1440 H= 360:00 +CPU= 34.529
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.8837519187E-21      I       0.2802009817E-05      I
ICloud fractionI      0.0000000000E+00      I       0.1447834609E-01      I
I  Relat. Hum. I      0.5206238412E+00      I       0.5279515418E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2517545097E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    299399521 BYTES IN      7 RECORDS TRANSFERRED IN    0.0020 SECONDS149699.8 Mbytes/s, TOTAL TIME=    0.0130(15.4%)
 09:19:55 STEP 1440 H= 360:00 +CPU= 34.529
 END CNT3
 NSTEP =  1440 CNT0   000000000  23.843  23.843  24.260   0.983               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        14180/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    253    0       34      1044068    14180 ************    34.4     2.0    32.5     3.1    60.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.96      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  521    5068.4    5068.4     19.88      0.00
   9 SPCM     - SPECTRAL COMP.                     480     818.0     818.0      2.96      0.00
  10 SCAN2M - PHYSICS                              481   12373.0   12373.0     44.81      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 40    1104.5    1104.5      0.33      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 21      71.5      71.5      0.01      0.00
  17 GRIDFPOS IN CNT4                               40      30.5      30.5      0.01      0.00
  27 RESTART FILES - WRITING                        10    4418.1    4418.1      0.33      0.00
  28 RESTART FILES - READING                         1    4475.8    4475.8      0.03      0.00
  29 SU4FPOS IN CNT4                                40       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                40    6623.2    6623.2      1.99      0.00
  31 POSDDH IN STEPO                                 3       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    481    5552.2    5552.2     20.11      0.00
  39 SU0YOMB                                         1     399.2     399.2      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    481      26.6      26.6      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              480     241.8     241.8      0.87      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              480      59.9      59.9      0.22      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 21       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1044     441.7     441.7      3.47      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1000     254.8     254.8      1.92      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1000      92.6      92.6      0.70      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1044     137.0     137.0      1.08      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      44.3      44.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1044      59.7      59.7      0.47      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1000      31.1      31.1      0.23      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1044      89.7      89.7      0.71      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1000      41.1      41.1      0.31      0.00
 400 GSTATS                                      56453       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 52473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    13281.6 CPU TIME   13121.9 VECTOR TIME    13121.9


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       97.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client50002/slots/3/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/slots/3/NODE.001_01
Adding to the zip: /var/lib/boinc-client50002/slots/3/ifs.stat
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12213215/ICMSHh7zg+001368
Zipping up the final file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0_r178347645_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
09:23:27 (1530633): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:52:37 1536616 22313500 oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0 1,295,100 38,005 0.0293
09 May 2023 16:52:10 1536616 22313500 oifs_43r3_bl_a4yk_2016092300_15_991_12213215_0 864,000 24,214 0.0280


©2024 cpdn.org