climateprediction.net home page
Task 22309419

Task 22309419

Name oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1
Workunit 12207982
Created 13 Feb 2023, 16:29:54 UTC
Sent 14 Feb 2023, 8:50:21 UTC
Report deadline 15 Apr 2023, 8:50:21 UTC
Received 24 Feb 2023, 10:16:05 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536616
Run time 10 hours 38 min 46 sec
CPU time 10 hours 38 min 46 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 4.72 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,075.48 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,360.51 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
---------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 07:26:21 0AAA00AAA STEPO      977    26.265   26.265   26.362    414:14    415:53 0.75347464241459E-06       2GB       0MB
 07:26:46 0AAA00AAA STEPO      978    24.960   24.960   25.061    414:39    416:18 0.75347464241459E-06       2GB       0MB
 07:27:11 0AAA00AAA STEPO      979    24.081   24.081   24.178    415:03    416:43 0.75347464241459E-06       2GB       0MB
 07:27:35 0AAA00AAA STEPO      980    24.508   24.508   24.611    415:28    417:07 0.75347464241459E-06       2GB       0MB
 07:28:01 0AAA00AAA STEPO      981    25.584   25.584   25.698    415:53    417:33 0.75347464241459E-06       2GB       0MB
 07:28:27 0AAA00AAA STEPO      982    26.112   26.112   26.212    416:19    417:59 0.75347464241459E-06       2GB       0MB
 07:28:52 0AAA00AAA STEPO      983    25.241   25.241   25.333    416:45    418:24 0.75347464241459E-06       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     960",
 CTIME   = "00100000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         948, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client50002/slots/16/progress_file_12207982.xml

<running_values>
  <last_cpu_time>25019.550000</last_cpu_time>
  <upload_file_number>10</upload_file_number>
  <last_iter>983</last_iter>
  <last_upload>864000</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0x7
(argv4) batchid: 991
(argv5) wuid: 12207982
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client50002/slots/16
Project directory is: /var/lib/boinc-client50002/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client50002/slots/16/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client50002/slots/16/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_85dec83e504a39765f361c7661ac9460 to: /var/lib/boinc-client50002/slots/16/oifs_43r3_bl_a0x7_2016092300_15_991_12207982.zip
Unzipping the namelist zip file: /var/lib/boinc-client50002/slots/16/oifs_43r3_bl_a0x7_2016092300_15_991_12207982.zip
ic_ancil_file: ic_ancil_12207982
ifsdata_file: ifsdata_12207982
climate_data_file: clim_data_12207982
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client50002/slots/16/ic_ancil_12207982.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client50002/slots/16/ic_ancil_12207982.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e7150b81c767161eccd8b0d2b9354c25 to: /var/lib/boinc-client50002/slots/16/ifsdata/ifsdata_12207982.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client50002/slots/16/ifsdata/ifsdata_12207982.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client50002/slots/16/159l_2/clim_data_12207982.zip
Unzipping the climate data zip file: /var/lib/boinc-client50002/slots/16/159l_2/clim_data_12207982.zip
Checking for progress XML file: /var/lib/boinc-client50002/slots/16/progress_file_12207982.xml
Opened progress file ok : /var/lib/boinc-client50002/slots/16/progress_file_12207982.xml
-- Model is restarting --
Adjusting last_iter, 983, to previous model restart step.
Creating progress file: /var/lib/boinc-client50002/slots/16/progress_file_12207982.xml
last_cpu_time: 25019
upload_file_number: 10
last_iter: 961
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1_r1797776657
The child process has been launched with process id: 1530684
Executing the command: /var/lib/boinc-client50002/slots/16/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:epyc7v12:1:1:1530684:1530684] [20230224:053833:1677217113:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:39:04 STEP  960 H= 240:00 +CPU= 25.569
  05:39:29 STEP  961 H= 240:15 +CPU= 25.229
  05:39:55 STEP  962 H= 240:30 +CPU= 25.361
  05:40:20 STEP  963 H= 240:45 +CPU= 24.749
  05:40:45 STEP  964 H= 241:00 +CPU= 24.841
  05:41:08 STEP  965 H= 241:15 +CPU= 23.263
  05:41:32 STEP  966 H= 241:30 +CPU= 24.009
  05:41:56 STEP  967 H= 241:45 +CPU= 23.765
  05:42:22 STEP  968 H= 242:00 +CPU= 25.586
  05:42:46 STEP  969 H= 242:15 +CPU= 24.391
  05:43:10 STEP  970 H= 242:30 +CPU= 24.163
  05:43:34 STEP  971 H= 242:45 +CPU= 23.423
  05:44:05 STEP  972 H= 243:00 +CPU= 30.739
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+000972
  05:44:29 STEP  973 H= 243:15 +CPU= 24.672
  05:44:53 STEP  974 H= 243:30 +CPU= 23.249
  05:45:17 STEP  975 H= 243:45 +CPU= 24.498
  05:45:42 STEP  976 H= 244:00 +CPU= 24.688
  05:46:06 STEP  977 H= 244:15 +CPU= 24.130
  05:46:30 STEP  978 H= 244:30 +CPU= 23.203
  05:46:53 STEP  979 H= 244:45 +CPU= 23.830
  05:47:18 STEP  980 H= 245:00 +CPU= 24.190
  05:47:42 STEP  981 H= 245:15 +CPU= 24.609
  05:48:06 STEP  982 H= 245:30 +CPU= 23.918
  05:48:31 STEP  983 H= 245:45 +CPU= 24.273
  05:49:02 STEP  984 H= 246:00 +CPU= 30.884
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+000984
  05:49:27 STEP  985 H= 246:15 +CPU= 24.860
  05:49:50 STEP  986 H= 246:30 +CPU= 23.549
  05:50:14 STEP  987 H= 246:45 +CPU= 23.679
  05:50:39 STEP  988 H= 247:00 +CPU= 24.470
  05:51:03 STEP  989 H= 247:15 +CPU= 24.045
  05:51:27 STEP  990 H= 247:30 +CPU= 24.309
  05:51:52 STEP  991 H= 247:45 +CPU= 24.937
  05:52:17 STEP  992 H= 248:00 +CPU= 24.519
  05:52:41 STEP  993 H= 248:15 +CPU= 24.255
  05:53:05 STEP  994 H= 248:30 +CPU= 23.318
  05:53:30 STEP  995 H= 248:45 +CPU= 25.233
  05:54:02 STEP  996 H= 249:00 +CPU= 32.009
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+000996
  05:54:27 STEP  997 H= 249:15 +CPU= 24.754
  05:54:51 STEP  998 H= 249:30 +CPU= 24.435
  05:55:17 STEP  999 H= 249:45 +CPU= 25.135
  05:55:41 STEP 1000 H= 250:00 +CPU= 24.642
  05:56:05 STEP 1001 H= 250:15 +CPU= 23.862
  05:56:29 STEP 1002 H= 250:30 +CPU= 24.101
  05:56:54 STEP 1003 H= 250:45 +CPU= 23.997
  05:57:19 STEP 1004 H= 251:00 +CPU= 25.353
  05:57:44 STEP 1005 H= 251:15 +CPU= 24.736
  05:58:07 STEP 1006 H= 251:30 +CPU= 23.409
  05:58:32 STEP 1007 H= 251:45 +CPU= 24.497
  05:59:08 STEP 1008 H= 252:00 +CPU= 35.971
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001008
  05:59:33 STEP 1009 H= 252:15 +CPU= 24.777
  05:59:57 STEP 1010 H= 252:30 +CPU= 23.941
  06:00:21 STEP 1011 H= 252:45 +CPU= 23.746
  06:00:47 STEP 1012 H= 253:00 +CPU= 25.470
  06:01:11 STEP 1013 H= 253:15 +CPU= 24.146
  06:01:34 STEP 1014 H= 253:30 +CPU= 23.517
  06:01:58 STEP 1015 H= 253:45 +CPU= 23.965
  06:02:24 STEP 1016 H= 254:00 +CPU= 25.750
  06:02:49 STEP 1017 H= 254:15 +CPU= 24.312
  06:03:13 STEP 1018 H= 254:30 +CPU= 23.842
  06:03:37 STEP 1019 H= 254:45 +CPU= 23.983
  06:04:08 STEP 1020 H= 255:00 +CPU= 31.312
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001020
  06:04:32 STEP 1021 H= 255:15 +CPU= 23.938
  06:04:56 STEP 1022 H= 255:30 +CPU= 24.223
  06:05:20 STEP 1023 H= 255:45 +CPU= 23.597
  06:05:45 STEP 1024 H= 256:00 +CPU= 24.422
  06:06:10 STEP 1025 H= 256:15 +CPU= 25.362
  06:06:34 STEP 1026 H= 256:30 +CPU= 23.794
  06:06:58 STEP 1027 H= 256:45 +CPU= 24.165
  06:07:23 STEP 1028 H= 257:00 +CPU= 24.919
  06:07:47 STEP 1029 H= 257:15 +CPU= 23.489
  06:08:11 STEP 1030 H= 257:30 +CPU= 24.474
  06:08:35 STEP 1031 H= 257:45 +CPU= 24.029
  06:09:07 STEP 1032 H= 258:00 +CPU= 31.739
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001032
  06:09:31 STEP 1033 H= 258:15 +CPU= 23.746
  06:09:55 STEP 1034 H= 258:30 +CPU= 24.134
  06:10:19 STEP 1035 H= 258:45 +CPU= 23.765
  06:10:43 STEP 1036 H= 259:00 +CPU= 24.175
  06:11:08 STEP 1037 H= 259:15 +CPU= 24.698
  06:11:32 STEP 1038 H= 259:30 +CPU= 23.649
  06:11:56 STEP 1039 H= 259:45 +CPU= 24.145
  06:12:21 STEP 1040 H= 260:00 +CPU= 24.344
  06:12:45 STEP 1041 H= 260:15 +CPU= 24.145
  06:13:09 STEP 1042 H= 260:30 +CPU= 24.306
  06:13:33 STEP 1043 H= 260:45 +CPU= 23.366
  06:14:05 STEP 1044 H= 261:00 +CPU= 31.879
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001044
  06:14:30 STEP 1045 H= 261:15 +CPU= 25.423
  06:14:55 STEP 1046 H= 261:30 +CPU= 24.495
  06:15:20 STEP 1047 H= 261:45 +CPU= 24.736
  06:15:45 STEP 1048 H= 262:00 +CPU= 25.359
  06:16:10 STEP 1049 H= 262:15 +CPU= 24.301
  06:16:34 STEP 1050 H= 262:30 +CPU= 24.091
  06:16:58 STEP 1051 H= 262:45 +CPU= 24.436
  06:17:22 STEP 1052 H= 263:00 +CPU= 24.022
  06:17:46 STEP 1053 H= 263:15 +CPU= 23.973
  06:18:11 STEP 1054 H= 263:30 +CPU= 24.405
  06:18:35 STEP 1055 H= 263:45 +CPU= 24.129
  06:19:10 STEP 1056 H= 264:00 +CPU= 34.741
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1_r1797776657_10.zip
Uploading the intermediate file: upload_file_10.zip
  06:19:35 STEP 1057 H= 264:15 +CPU= 24.574
  06:20:00 STEP 1058 H= 264:30 +CPU= 24.920
  06:20:23 STEP 1059 H= 264:45 +CPU= 23.651
  06:20:48 STEP 1060 H= 265:00 +CPU= 24.489
  06:21:12 STEP 1061 H= 265:15 +CPU= 23.444
  06:21:36 STEP 1062 H= 265:30 +CPU= 24.201
  06:22:01 STEP 1063 H= 265:45 +CPU= 24.666
  06:22:25 STEP 1064 H= 266:00 +CPU= 24.078
  06:22:48 STEP 1065 H= 266:15 +CPU= 23.229
  06:23:11 STEP 1066 H= 266:30 +CPU= 23.290
  06:23:36 STEP 1067 H= 266:45 +CPU= 24.148
  06:24:07 STEP 1068 H= 267:00 +CPU= 31.191
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001068
  06:24:32 STEP 1069 H= 267:15 +CPU= 24.548
  06:24:56 STEP 1070 H= 267:30 +CPU= 24.196
  06:25:20 STEP 1071 H= 267:45 +CPU= 23.907
  06:25:45 STEP 1072 H= 268:00 +CPU= 24.743
  06:26:09 STEP 1073 H= 268:15 +CPU= 23.972
  06:26:33 STEP 1074 H= 268:30 +CPU= 24.103
  06:26:57 STEP 1075 H= 268:45 +CPU= 23.775
  06:27:21 STEP 1076 H= 269:00 +CPU= 24.077
  06:27:46 STEP 1077 H= 269:15 +CPU= 24.427
  06:28:10 STEP 1078 H= 269:30 +CPU= 24.410
  06:28:35 STEP 1079 H= 269:45 +CPU= 24.951
  06:29:07 STEP 1080 H= 270:00 +CPU= 31.755
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001080
  06:29:31 STEP 1081 H= 270:15 +CPU= 24.197
  06:29:55 STEP 1082 H= 270:30 +CPU= 24.106
  06:30:20 STEP 1083 H= 270:45 +CPU= 24.949
  06:30:46 STEP 1084 H= 271:00 +CPU= 25.497
  06:31:11 STEP 1085 H= 271:15 +CPU= 24.429
  06:31:35 STEP 1086 H= 271:30 +CPU= 24.052
  06:31:58 STEP 1087 H= 271:45 +CPU= 23.420
  06:32:24 STEP 1088 H= 272:00 +CPU= 25.458
  06:32:48 STEP 1089 H= 272:15 +CPU= 24.627
  06:33:12 STEP 1090 H= 272:30 +CPU= 23.504
  06:33:36 STEP 1091 H= 272:45 +CPU= 24.030
  06:34:08 STEP 1092 H= 273:00 +CPU= 31.886
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001092
  06:34:32 STEP 1093 H= 273:15 +CPU= 23.722
  06:34:56 STEP 1094 H= 273:30 +CPU= 24.005
  06:35:21 STEP 1095 H= 273:45 +CPU= 24.281
  06:35:46 STEP 1096 H= 274:00 +CPU= 25.503
  06:36:10 STEP 1097 H= 274:15 +CPU= 23.654
  06:36:34 STEP 1098 H= 274:30 +CPU= 24.209
  06:36:58 STEP 1099 H= 274:45 +CPU= 24.057
  06:37:26 STEP 1100 H= 275:00 +CPU= 27.181
  06:37:56 STEP 1101 H= 275:15 +CPU= 29.675
  06:38:24 STEP 1102 H= 275:30 +CPU= 27.404
  06:38:53 STEP 1103 H= 275:45 +CPU= 26.704
  06:39:34 STEP 1104 H= 276:00 +CPU= 40.666
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001104
  06:40:04 STEP 1105 H= 276:15 +CPU= 28.853
  06:40:33 STEP 1106 H= 276:30 +CPU= 28.674
  06:41:01 STEP 1107 H= 276:45 +CPU= 28.166
  06:41:32 STEP 1108 H= 277:00 +CPU= 30.550
  06:42:00 STEP 1109 H= 277:15 +CPU= 27.477
  06:42:29 STEP 1110 H= 277:30 +CPU= 28.565
  06:42:58 STEP 1111 H= 277:45 +CPU= 28.664
  06:43:29 STEP 1112 H= 278:00 +CPU= 29.811
  06:43:58 STEP 1113 H= 278:15 +CPU= 28.918
  06:44:26 STEP 1114 H= 278:30 +CPU= 26.977
  06:44:54 STEP 1115 H= 278:45 +CPU= 27.882
  06:45:30 STEP 1116 H= 279:00 +CPU= 34.977
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001116
  06:45:58 STEP 1117 H= 279:15 +CPU= 28.031
  06:46:28 STEP 1118 H= 279:30 +CPU= 29.444
  06:46:58 STEP 1119 H= 279:45 +CPU= 28.622
  06:47:26 STEP 1120 H= 280:00 +CPU= 27.786
  06:47:55 STEP 1121 H= 280:15 +CPU= 28.790
  06:48:23 STEP 1122 H= 280:30 +CPU= 27.691
  06:48:53 STEP 1123 H= 280:45 +CPU= 28.998
  06:49:22 STEP 1124 H= 281:00 +CPU= 28.672
  06:49:52 STEP 1125 H= 281:15 +CPU= 29.566
  06:50:21 STEP 1126 H= 281:30 +CPU= 27.828
  06:50:49 STEP 1127 H= 281:45 +CPU= 27.703
  06:51:25 STEP 1128 H= 282:00 +CPU= 36.167
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001128
  06:51:53 STEP 1129 H= 282:15 +CPU= 27.240
  06:52:22 STEP 1130 H= 282:30 +CPU= 28.538
  06:52:50 STEP 1131 H= 282:45 +CPU= 27.322
  06:53:21 STEP 1132 H= 283:00 +CPU= 29.964
  06:53:50 STEP 1133 H= 283:15 +CPU= 28.930
  06:54:19 STEP 1134 H= 283:30 +CPU= 28.554
  06:54:48 STEP 1135 H= 283:45 +CPU= 28.759
  06:55:18 STEP 1136 H= 284:00 +CPU= 29.710
  06:55:46 STEP 1137 H= 284:15 +CPU= 27.848
  06:56:15 STEP 1138 H= 284:30 +CPU= 28.626
  06:56:44 STEP 1139 H= 284:45 +CPU= 28.421
  06:57:22 STEP 1140 H= 285:00 +CPU= 37.003
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001140
  06:57:50 STEP 1141 H= 285:15 +CPU= 28.157
  06:58:18 STEP 1142 H= 285:30 +CPU= 27.602
  06:58:47 STEP 1143 H= 285:45 +CPU= 28.607
  06:59:18 STEP 1144 H= 286:00 +CPU= 29.865
  06:59:47 STEP 1145 H= 286:15 +CPU= 28.954
  07:00:16 STEP 1146 H= 286:30 +CPU= 28.014
  07:00:45 STEP 1147 H= 286:45 +CPU= 28.866
  07:01:16 STEP 1148 H= 287:00 +CPU= 30.398
  07:01:45 STEP 1149 H= 287:15 +CPU= 28.671
  07:02:14 STEP 1150 H= 287:30 +CPU= 28.367
  07:02:43 STEP 1151 H= 287:45 +CPU= 28.367
  07:03:27 STEP 1152 H= 288:00 +CPU= 43.034
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1_r1797776657_11.zip
Uploading the intermediate file: upload_file_11.zip
  07:03:57 STEP 1153 H= 288:15 +CPU= 29.261
  07:04:26 STEP 1154 H= 288:30 +CPU= 29.263
  07:04:55 STEP 1155 H= 288:45 +CPU= 28.564
  07:05:24 STEP 1156 H= 289:00 +CPU= 28.657
  07:05:53 STEP 1157 H= 289:15 +CPU= 28.237
  07:06:23 STEP 1158 H= 289:30 +CPU= 29.022
  07:06:51 STEP 1159 H= 289:45 +CPU= 27.496
  07:07:21 STEP 1160 H= 290:00 +CPU= 29.605
  07:07:49 STEP 1161 H= 290:15 +CPU= 28.359
  07:08:18 STEP 1162 H= 290:30 +CPU= 27.837
  07:08:47 STEP 1163 H= 290:45 +CPU= 29.375
  07:09:25 STEP 1164 H= 291:00 +CPU= 36.952
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001164
  07:09:54 STEP 1165 H= 291:15 +CPU= 28.788
  07:10:25 STEP 1166 H= 291:30 +CPU= 30.071
  07:10:55 STEP 1167 H= 291:45 +CPU= 29.285
  07:11:26 STEP 1168 H= 292:00 +CPU= 30.559
  07:11:55 STEP 1169 H= 292:15 +CPU= 29.018
  07:12:25 STEP 1170 H= 292:30 +CPU= 29.260
  07:12:55 STEP 1171 H= 292:45 +CPU= 29.158
  07:13:24 STEP 1172 H= 293:00 +CPU= 29.526
  07:13:53 STEP 1173 H= 293:15 +CPU= 27.632
  07:14:20 STEP 1174 H= 293:30 +CPU= 26.453
  07:14:48 STEP 1175 H= 293:45 +CPU= 28.272
  07:15:25 STEP 1176 H= 294:00 +CPU= 36.324
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001176
  07:15:54 STEP 1177 H= 294:15 +CPU= 28.632
  07:16:23 STEP 1178 H= 294:30 +CPU= 28.554
  07:16:52 STEP 1179 H= 294:45 +CPU= 28.718
  07:17:22 STEP 1180 H= 295:00 +CPU= 29.714
  07:17:51 STEP 1181 H= 295:15 +CPU= 27.925
  07:18:20 STEP 1182 H= 295:30 +CPU= 29.355
  07:18:51 STEP 1183 H= 295:45 +CPU= 29.706
  07:19:20 STEP 1184 H= 296:00 +CPU= 28.547
  07:19:49 STEP 1185 H= 296:15 +CPU= 29.131
  07:20:17 STEP 1186 H= 296:30 +CPU= 27.684
  07:20:47 STEP 1187 H= 296:45 +CPU= 29.022
  07:21:24 STEP 1188 H= 297:00 +CPU= 36.454
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001188
  07:21:54 STEP 1189 H= 297:15 +CPU= 29.932
  07:22:23 STEP 1190 H= 297:30 +CPU= 28.279
  07:22:52 STEP 1191 H= 297:45 +CPU= 28.808
  07:23:22 STEP 1192 H= 298:00 +CPU= 29.490
  07:23:53 STEP 1193 H= 298:15 +CPU= 31.035
  07:24:22 STEP 1194 H= 298:30 +CPU= 28.843
  07:24:50 STEP 1195 H= 298:45 +CPU= 27.003
  07:25:20 STEP 1196 H= 299:00 +CPU= 29.362
  07:25:49 STEP 1197 H= 299:15 +CPU= 27.935
  07:26:19 STEP 1198 H= 299:30 +CPU= 28.624
  07:26:48 STEP 1199 H= 299:45 +CPU= 28.280
  07:27:30 STEP 1200 H= 300:00 +CPU= 42.075
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001200
  07:28:00 STEP 1201 H= 300:15 +CPU= 28.596
  07:28:28 STEP 1202 H= 300:30 +CPU= 27.055
  07:28:58 STEP 1203 H= 300:45 +CPU= 28.688
  07:29:29 STEP 1204 H= 301:00 +CPU= 30.892
  07:29:58 STEP 1205 H= 301:15 +CPU= 29.013
  07:30:28 STEP 1206 H= 301:30 +CPU= 28.730
  07:30:56 STEP 1207 H= 301:45 +CPU= 27.889
  07:31:26 STEP 1208 H= 302:00 +CPU= 29.413
  07:31:55 STEP 1209 H= 302:15 +CPU= 28.491
  07:32:25 STEP 1210 H= 302:30 +CPU= 29.955
  07:32:55 STEP 1211 H= 302:45 +CPU= 28.890
  07:33:32 STEP 1212 H= 303:00 +CPU= 36.330
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001212
  07:34:02 STEP 1213 H= 303:15 +CPU= 29.614
  07:34:31 STEP 1214 H= 303:30 +CPU= 28.577
  07:35:01 STEP 1215 H= 303:45 +CPU= 29.611
  07:35:30 STEP 1216 H= 304:00 +CPU= 29.263
  07:35:58 STEP 1217 H= 304:15 +CPU= 27.539
  07:36:27 STEP 1218 H= 304:30 +CPU= 28.721
  07:36:57 STEP 1219 H= 304:45 +CPU= 29.065
  07:37:26 STEP 1220 H= 305:00 +CPU= 27.971
  07:37:55 STEP 1221 H= 305:15 +CPU= 29.067
  07:38:26 STEP 1222 H= 305:30 +CPU= 30.067
  07:38:54 STEP 1223 H= 305:45 +CPU= 27.114
  07:39:32 STEP 1224 H= 306:00 +CPU= 36.798
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001224
  07:40:00 STEP 1225 H= 306:15 +CPU= 28.396
  07:40:30 STEP 1226 H= 306:30 +CPU= 28.749
  07:40:59 STEP 1227 H= 306:45 +CPU= 28.454
  07:41:27 STEP 1228 H= 307:00 +CPU= 28.084
  07:41:56 STEP 1229 H= 307:15 +CPU= 28.280
  07:42:25 STEP 1230 H= 307:30 +CPU= 28.914
  07:42:56 STEP 1231 H= 307:45 +CPU= 30.462
  07:43:26 STEP 1232 H= 308:00 +CPU= 29.544
  07:43:57 STEP 1233 H= 308:15 +CPU= 29.761
  07:44:25 STEP 1234 H= 308:30 +CPU= 27.892
  07:44:54 STEP 1235 H= 308:45 +CPU= 28.599
  07:45:31 STEP 1236 H= 309:00 +CPU= 36.561
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001236
  07:45:59 STEP 1237 H= 309:15 +CPU= 27.169
  07:46:28 STEP 1238 H= 309:30 +CPU= 29.087
  07:46:58 STEP 1239 H= 309:45 +CPU= 28.739
  07:47:27 STEP 1240 H= 310:00 +CPU= 29.002
  07:47:56 STEP 1241 H= 310:15 +CPU= 28.743
  07:48:27 STEP 1242 H= 310:30 +CPU= 29.906
  07:48:57 STEP 1243 H= 310:45 +CPU= 29.604
  07:49:26 STEP 1244 H= 311:00 +CPU= 29.045
  07:49:57 STEP 1245 H= 311:15 +CPU= 30.510
  07:50:28 STEP 1246 H= 311:30 +CPU= 29.698
  07:50:58 STEP 1247 H= 311:45 +CPU= 29.695
  07:51:39 STEP 1248 H= 312:00 +CPU= 40.718
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1_r1797776657_12.zip
Uploading the intermediate file: upload_file_12.zip
  07:52:09 STEP 1249 H= 312:15 +CPU= 29.552
  07:52:39 STEP 1250 H= 312:30 +CPU= 29.821
  07:53:09 STEP 1251 H= 312:45 +CPU= 29.084
  07:53:38 STEP 1252 H= 313:00 +CPU= 29.253
  07:54:08 STEP 1253 H= 313:15 +CPU= 29.097
  07:54:36 STEP 1254 H= 313:30 +CPU= 27.468
  07:55:05 STEP 1255 H= 313:45 +CPU= 27.709
  07:55:35 STEP 1256 H= 314:00 +CPU= 29.775
  07:56:03 STEP 1257 H= 314:15 +CPU= 27.791
  07:56:32 STEP 1258 H= 314:30 +CPU= 28.020
  07:57:01 STEP 1259 H= 314:45 +CPU= 28.381
  07:57:38 STEP 1260 H= 315:00 +CPU= 36.617
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001260
  07:58:06 STEP 1261 H= 315:15 +CPU= 28.097
  07:58:36 STEP 1262 H= 315:30 +CPU= 28.606
  07:59:05 STEP 1263 H= 315:45 +CPU= 28.846
  07:59:35 STEP 1264 H= 316:00 +CPU= 29.393
  08:00:04 STEP 1265 H= 316:15 +CPU= 28.503
  08:00:35 STEP 1266 H= 316:30 +CPU= 30.629
  08:01:04 STEP 1267 H= 316:45 +CPU= 28.569
  08:01:33 STEP 1268 H= 317:00 +CPU= 29.243
  08:02:02 STEP 1269 H= 317:15 +CPU= 28.575
  08:02:31 STEP 1270 H= 317:30 +CPU= 27.996
  08:02:58 STEP 1271 H= 317:45 +CPU= 26.351
  08:03:33 STEP 1272 H= 318:00 +CPU= 34.783
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001272
  08:04:03 STEP 1273 H= 318:15 +CPU= 29.195
  08:04:31 STEP 1274 H= 318:30 +CPU= 27.912
  08:05:00 STEP 1275 H= 318:45 +CPU= 28.606
  08:05:31 STEP 1276 H= 319:00 +CPU= 30.128
  08:06:02 STEP 1277 H= 319:15 +CPU= 30.418
  08:06:29 STEP 1278 H= 319:30 +CPU= 27.307
  08:06:57 STEP 1279 H= 319:45 +CPU= 27.527
  08:07:27 STEP 1280 H= 320:00 +CPU= 29.123
  08:07:55 STEP 1281 H= 320:15 +CPU= 27.459
  08:08:25 STEP 1282 H= 320:30 +CPU= 29.064
  08:08:53 STEP 1283 H= 320:45 +CPU= 27.721
  08:09:28 STEP 1284 H= 321:00 +CPU= 34.856
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001284
  08:09:58 STEP 1285 H= 321:15 +CPU= 29.298
  08:10:28 STEP 1286 H= 321:30 +CPU= 29.387
  08:10:58 STEP 1287 H= 321:45 +CPU= 29.954
  08:11:27 STEP 1288 H= 322:00 +CPU= 28.726
  08:11:57 STEP 1289 H= 322:15 +CPU= 28.734
  08:12:27 STEP 1290 H= 322:30 +CPU= 29.748
  08:12:57 STEP 1291 H= 322:45 +CPU= 29.596
  08:13:28 STEP 1292 H= 323:00 +CPU= 30.233
  08:13:57 STEP 1293 H= 323:15 +CPU= 28.082
  08:14:24 STEP 1294 H= 323:30 +CPU= 27.156
  08:14:53 STEP 1295 H= 323:45 +CPU= 28.609
  08:15:35 STEP 1296 H= 324:00 +CPU= 40.581
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001296
  08:16:04 STEP 1297 H= 324:15 +CPU= 28.768
  08:16:33 STEP 1298 H= 324:30 +CPU= 28.250
  08:17:01 STEP 1299 H= 324:45 +CPU= 28.317
  08:17:32 STEP 1300 H= 325:00 +CPU= 29.961
  08:18:01 STEP 1301 H= 325:15 +CPU= 28.527
  08:18:30 STEP 1302 H= 325:30 +CPU= 28.693
  08:18:58 STEP 1303 H= 325:45 +CPU= 27.432
  08:19:28 STEP 1304 H= 326:00 +CPU= 29.394
  08:19:55 STEP 1305 H= 326:15 +CPU= 27.415
  08:20:23 STEP 1306 H= 326:30 +CPU= 27.616
  08:20:52 STEP 1307 H= 326:45 +CPU= 28.167
  08:21:28 STEP 1308 H= 327:00 +CPU= 35.989
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001308
  08:21:56 STEP 1309 H= 327:15 +CPU= 27.854
  08:22:26 STEP 1310 H= 327:30 +CPU= 28.948
  08:22:54 STEP 1311 H= 327:45 +CPU= 28.265
  08:23:24 STEP 1312 H= 328:00 +CPU= 29.525
  08:23:53 STEP 1313 H= 328:15 +CPU= 28.209
  08:24:21 STEP 1314 H= 328:30 +CPU= 27.230
  08:24:48 STEP 1315 H= 328:45 +CPU= 27.573
  08:25:17 STEP 1316 H= 329:00 +CPU= 28.235
  08:25:46 STEP 1317 H= 329:15 +CPU= 28.043
  08:26:15 STEP 1318 H= 329:30 +CPU= 28.388
  08:26:42 STEP 1319 H= 329:45 +CPU= 26.847
  08:27:18 STEP 1320 H= 330:00 +CPU= 36.040
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001320
  08:27:48 STEP 1321 H= 330:15 +CPU= 29.542
  08:28:16 STEP 1322 H= 330:30 +CPU= 27.466
  08:28:45 STEP 1323 H= 330:45 +CPU= 28.299
  08:29:14 STEP 1324 H= 331:00 +CPU= 28.554
  08:29:42 STEP 1325 H= 331:15 +CPU= 27.876
  08:30:10 STEP 1326 H= 331:30 +CPU= 27.101
  08:30:37 STEP 1327 H= 331:45 +CPU= 26.883
  08:31:07 STEP 1328 H= 332:00 +CPU= 29.555
  08:31:35 STEP 1329 H= 332:15 +CPU= 27.706
  08:32:03 STEP 1330 H= 332:30 +CPU= 27.794
  08:32:31 STEP 1331 H= 332:45 +CPU= 27.264
  08:33:06 STEP 1332 H= 333:00 +CPU= 34.469
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001332
  08:33:34 STEP 1333 H= 333:15 +CPU= 28.028
  08:34:02 STEP 1334 H= 333:30 +CPU= 27.410
  08:34:31 STEP 1335 H= 333:45 +CPU= 28.332
  08:35:02 STEP 1336 H= 334:00 +CPU= 30.278
  08:35:31 STEP 1337 H= 334:15 +CPU= 28.887
  08:36:00 STEP 1338 H= 334:30 +CPU= 28.282
  08:36:28 STEP 1339 H= 334:45 +CPU= 27.427
  08:36:57 STEP 1340 H= 335:00 +CPU= 28.799
  08:37:26 STEP 1341 H= 335:15 +CPU= 28.421
  08:37:55 STEP 1342 H= 335:30 +CPU= 28.132
  08:38:23 STEP 1343 H= 335:45 +CPU= 27.985
  08:39:05 STEP 1344 H= 336:00 +CPU= 40.551
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1_r1797776657_13.zip
Uploading the intermediate file: upload_file_13.zip
  08:39:33 STEP 1345 H= 336:15 +CPU= 27.489
  08:40:02 STEP 1346 H= 336:30 +CPU= 28.643
  08:40:31 STEP 1347 H= 336:45 +CPU= 28.584
  08:40:59 STEP 1348 H= 337:00 +CPU= 28.197
  08:41:27 STEP 1349 H= 337:15 +CPU= 27.378
  08:41:56 STEP 1350 H= 337:30 +CPU= 28.268
  08:42:23 STEP 1351 H= 337:45 +CPU= 26.561
  08:42:51 STEP 1352 H= 338:00 +CPU= 28.582
  08:43:19 STEP 1353 H= 338:15 +CPU= 27.459
  08:43:48 STEP 1354 H= 338:30 +CPU= 28.185
  08:44:16 STEP 1355 H= 338:45 +CPU= 28.040
  08:44:52 STEP 1356 H= 339:00 +CPU= 35.589
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001356
  08:45:18 STEP 1357 H= 339:15 +CPU= 25.926
  08:45:46 STEP 1358 H= 339:30 +CPU= 27.142
  08:46:13 STEP 1359 H= 339:45 +CPU= 26.319
  08:46:41 STEP 1360 H= 340:00 +CPU= 27.882
  08:47:08 STEP 1361 H= 340:15 +CPU= 26.973
  08:47:35 STEP 1362 H= 340:30 +CPU= 26.499
  08:48:03 STEP 1363 H= 340:45 +CPU= 27.710
  08:48:32 STEP 1364 H= 341:00 +CPU= 27.983
  08:49:00 STEP 1365 H= 341:15 +CPU= 28.144
  08:49:27 STEP 1366 H= 341:30 +CPU= 25.878
  08:49:54 STEP 1367 H= 341:45 +CPU= 27.256
  08:50:29 STEP 1368 H= 342:00 +CPU= 34.161
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001368
  08:50:56 STEP 1369 H= 342:15 +CPU= 27.282
  08:51:24 STEP 1370 H= 342:30 +CPU= 26.989
  08:51:51 STEP 1371 H= 342:45 +CPU= 27.088
  08:52:19 STEP 1372 H= 343:00 +CPU= 27.705
  08:52:46 STEP 1373 H= 343:15 +CPU= 26.763
  08:53:13 STEP 1374 H= 343:30 +CPU= 26.533
  08:53:41 STEP 1375 H= 343:45 +CPU= 27.618
  08:54:08 STEP 1376 H= 344:00 +CPU= 27.167
  08:54:35 STEP 1377 H= 344:15 +CPU= 26.652
  08:55:03 STEP 1378 H= 344:30 +CPU= 27.035
  08:55:29 STEP 1379 H= 344:45 +CPU= 25.690
  08:56:03 STEP 1380 H= 345:00 +CPU= 33.577
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001380
  08:56:30 STEP 1381 H= 345:15 +CPU= 26.559
  08:56:57 STEP 1382 H= 345:30 +CPU= 27.302
  08:57:24 STEP 1383 H= 345:45 +CPU= 26.798
  08:57:53 STEP 1384 H= 346:00 +CPU= 27.426
  08:58:20 STEP 1385 H= 346:15 +CPU= 26.704
  08:58:47 STEP 1386 H= 346:30 +CPU= 27.042
  08:59:14 STEP 1387 H= 346:45 +CPU= 26.810
  08:59:41 STEP 1388 H= 347:00 +CPU= 26.852
  09:00:08 STEP 1389 H= 347:15 +CPU= 26.546
  09:00:35 STEP 1390 H= 347:30 +CPU= 26.548
  09:01:03 STEP 1391 H= 347:45 +CPU= 27.426
  09:01:41 STEP 1392 H= 348:00 +CPU= 38.474
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001392
  09:02:08 STEP 1393 H= 348:15 +CPU= 26.375
  09:02:34 STEP 1394 H= 348:30 +CPU= 25.850
  09:03:01 STEP 1395 H= 348:45 +CPU= 26.653
  09:03:28 STEP 1396 H= 349:00 +CPU= 26.753
  09:03:54 STEP 1397 H= 349:15 +CPU= 25.504
  09:04:20 STEP 1398 H= 349:30 +CPU= 26.275
  09:04:46 STEP 1399 H= 349:45 +CPU= 25.373
  09:05:12 STEP 1400 H= 350:00 +CPU= 25.967
  09:05:38 STEP 1401 H= 350:15 +CPU= 25.804
  09:06:04 STEP 1402 H= 350:30 +CPU= 26.411
  09:06:30 STEP 1403 H= 350:45 +CPU= 25.105
  09:07:03 STEP 1404 H= 351:00 +CPU= 33.241
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001404
  09:07:29 STEP 1405 H= 351:15 +CPU= 25.887
  09:07:55 STEP 1406 H= 351:30 +CPU= 26.026
  09:08:21 STEP 1407 H= 351:45 +CPU= 25.486
  09:08:48 STEP 1408 H= 352:00 +CPU= 26.348
  09:09:13 STEP 1409 H= 352:15 +CPU= 24.841
  09:09:39 STEP 1410 H= 352:30 +CPU= 26.062
  09:10:05 STEP 1411 H= 352:45 +CPU= 25.723
  09:10:31 STEP 1412 H= 353:00 +CPU= 26.223
  09:10:58 STEP 1413 H= 353:15 +CPU= 26.441
  09:11:24 STEP 1414 H= 353:30 +CPU= 25.767
  09:11:50 STEP 1415 H= 353:45 +CPU= 26.289
  09:12:24 STEP 1416 H= 354:00 +CPU= 33.364
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001416
  09:12:50 STEP 1417 H= 354:15 +CPU= 25.627
  09:13:16 STEP 1418 H= 354:30 +CPU= 26.144
  09:13:42 STEP 1419 H= 354:45 +CPU= 26.032
  09:14:09 STEP 1420 H= 355:00 +CPU= 26.195
  09:14:34 STEP 1421 H= 355:15 +CPU= 25.276
  09:15:00 STEP 1422 H= 355:30 +CPU= 25.357
  09:15:26 STEP 1423 H= 355:45 +CPU= 25.750
  09:15:53 STEP 1424 H= 356:00 +CPU= 26.941
  09:16:19 STEP 1425 H= 356:15 +CPU= 25.806
  09:16:46 STEP 1426 H= 356:30 +CPU= 26.506
  09:17:12 STEP 1427 H= 356:45 +CPU= 26.346
  09:17:46 STEP 1428 H= 357:00 +CPU= 33.431
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001428
  09:18:11 STEP 1429 H= 357:15 +CPU= 25.055
  09:18:37 STEP 1430 H= 357:30 +CPU= 25.977
  09:19:03 STEP 1431 H= 357:45 +CPU= 25.479
  09:19:30 STEP 1432 H= 358:00 +CPU= 26.544
  09:19:56 STEP 1433 H= 358:15 +CPU= 26.079
  09:20:21 STEP 1434 H= 358:30 +CPU= 24.783
  09:20:46 STEP 1435 H= 358:45 +CPU= 25.082
  09:21:13 STEP 1436 H= 359:00 +CPU= 26.402
  09:21:38 STEP 1437 H= 359:15 +CPU= 25.162
  09:22:03 STEP 1438 H= 359:30 +CPU= 25.129
  09:22:29 STEP 1439 H= 359:45 +CPU= 25.443
  09:23:03 STEP 1440 H= 360:00 +CPU= 34.165
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.7706610858E-21      I       0.8945509114E-06      I
ICloud fractionI      0.0000000000E+00      I       0.1823229791E-01      I
I  Relat. Hum. I      0.2507468432E+00      I       0.2986951424E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2028555109E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    312330625 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS624661.2 Mbytes/s, TOTAL TIME=    0.0114( 4.4%)
 09:23:03 STEP 1440 H= 360:00 +CPU= 34.165
 END CNT3
 NSTEP =  1440 CNT0   000000000  23.931  23.931  24.021   0.996               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        14180/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    253    0       34      1044068    14180 ************    34.8     2.1    32.8     3.3    61.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.96      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  521    5065.2    5065.2     19.59      0.00
   9 SPCM     - SPECTRAL COMP.                     480     848.0     848.0      3.02      0.00
  10 SCAN2M - PHYSICS                              481   12634.7   12634.7     45.12      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 40    1129.3    1129.3      0.34      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 21      74.3      74.3      0.01      0.00
  17 GRIDFPOS IN CNT4                               40      31.0      31.0      0.01      0.00
  27 RESTART FILES - WRITING                        10    4517.3    4517.3      0.34      0.00
  28 RESTART FILES - READING                         1    4528.8    4528.8      0.03      0.00
  29 SU4FPOS IN CNT4                                40       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                40    6613.9    6613.9      1.96      0.00
  31 POSDDH IN STEPO                                 3       1.1       1.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    481    5605.1    5605.1     20.02      0.00
  39 SU0YOMB                                         1     428.5     428.5      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    481      26.9      26.9      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              480     251.6     251.6      0.90      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              480      63.0      63.0      0.22      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 21       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1044     446.2     446.2      3.46      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1000     256.2     256.2      1.90      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1000      93.7      93.7      0.70      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1044     138.0     138.0      1.07      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      46.4      46.4      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1044      61.1      61.1      0.47      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1000      32.5      32.5      0.24      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1044      93.1      93.1      0.72      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1000      43.1      43.1      0.32      0.00
 400 GSTATS                                      56453       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 52473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    13470.0 CPU TIME   13307.8 VECTOR TIME    13307.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       96.3
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client50002/slots/16/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/slots/16/NODE.001_01
Adding to the zip: /var/lib/boinc-client50002/slots/16/ifs.stat
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_12207982/ICMSHh7zg+001368
Zipping up the final file: /var/lib/boinc-client50002/projects/climateprediction.net/oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1_r1797776657_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
09:26:36 (1530630): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:52:37 1536616 22309419 oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1 1,295,100 38,326 0.0296
09 May 2023 16:52:10 1536616 22309419 oifs_43r3_bl_a0x7_2016092300_15_991_12207982_1 864,000 24,449 0.0283


©2024 cpdn.org