climateprediction.net home page
Task 22281418

Task 22281418

Name oifs_43r3_ps_0849_2009050100_123_978_12195493_0
Workunit 12195493
Created 23 Dec 2022, 22:07:29 UTC
Sent 1 Jan 2023, 12:26:19 UTC
Report deadline 31 Jan 2023, 12:26:19 UTC
Received 2 Jan 2023, 15:23:15 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1466689
Run time 21 hours 32 min 15 sec
CPU time 21 hours 15 min 52 sec
Validate state Invalid
Credit 11,733.12
Device peak FLOPS 4.64 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.59 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,301.29 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
diction.net/oifs_43r3_ps_12195493/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_81.zip
Uploading the intermediate file: upload_file_81.zip
  04:39:39 STEP 1969 H=1969:00 +CPU= 23.051
  04:40:03 STEP 1970 H=1970:00 +CPU= 23.869
  04:40:40 STEP 1971 H=1971:00 +CPU= 37.076
  04:41:04 STEP 1972 H=1972:00 +CPU= 23.814
  04:41:27 STEP 1973 H=1973:00 +CPU= 23.109
  04:42:05 STEP 1974 H=1974:00 +CPU= 37.979
  04:42:29 STEP 1975 H=1975:00 +CPU= 23.207
  04:42:53 STEP 1976 H=1976:00 +CPU= 23.976
  04:43:31 STEP 1977 H=1977:00 +CPU= 37.206
  04:43:55 STEP 1978 H=1978:00 +CPU= 23.958
  04:44:18 STEP 1979 H=1979:00 +CPU= 23.157
  04:44:59 STEP 1980 H=1980:00 +CPU= 41.095
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001980
  04:45:23 STEP 1981 H=1981:00 +CPU= 23.252
  04:45:47 STEP 1982 H=1982:00 +CPU= 24.154
  04:46:25 STEP 1983 H=1983:00 +CPU= 37.292
  04:46:49 STEP 1984 H=1984:00 +CPU= 23.915
  04:47:12 STEP 1985 H=1985:00 +CPU= 23.362
  04:47:50 STEP 1986 H=1986:00 +CPU= 38.090
  04:48:14 STEP 1987 H=1987:00 +CPU= 23.268
  04:48:38 STEP 1988 H=1988:00 +CPU= 24.058
  04:49:15 STEP 1989 H=1989:00 +CPU= 37.168
  04:49:39 STEP 1990 H=1990:00 +CPU= 23.852
  04:50:03 STEP 1991 H=1991:00 +CPU= 23.939
  04:50:47 STEP 1992 H=1992:00 +CPU= 43.291
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_82.zip
Uploading the intermediate file: upload_file_82.zip
  04:51:11 STEP 1993 H=1993:00 +CPU= 23.304
  04:51:35 STEP 1994 H=1994:00 +CPU= 23.926
  04:52:12 STEP 1995 H=1995:00 +CPU= 37.354
  04:52:36 STEP 1996 H=1996:00 +CPU= 24.085
  04:53:00 STEP 1997 H=1997:00 +CPU= 23.224
  04:53:38 STEP 1998 H=1998:00 +CPU= 38.192
  04:54:02 STEP 1999 H=1999:00 +CPU= 23.386
  04:54:26 STEP 2000 H=2000:00 +CPU= 23.969
  04:55:04 STEP 2001 H=2001:00 +CPU= 37.381
  04:55:28 STEP 2002 H=2002:00 +CPU= 24.059
  04:55:52 STEP 2003 H=2003:00 +CPU= 23.243
  04:56:33 STEP 2004 H=2004:00 +CPU= 41.115
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002004
  04:56:56 STEP 2005 H=2005:00 +CPU= 23.206
  04:57:21 STEP 2006 H=2006:00 +CPU= 24.122
  04:57:58 STEP 2007 H=2007:00 +CPU= 37.223
  04:58:22 STEP 2008 H=2008:00 +CPU= 23.881
  04:58:46 STEP 2009 H=2009:00 +CPU= 23.407
  04:59:24 STEP 2010 H=2010:00 +CPU= 38.028
  04:59:48 STEP 2011 H=2011:00 +CPU= 23.247
  05:00:12 STEP 2012 H=2012:00 +CPU= 24.113
  05:00:50 STEP 2013 H=2013:00 +CPU= 37.282
  05:01:14 STEP 2014 H=2014:00 +CPU= 23.909
  05:01:38 STEP 2015 H=2015:00 +CPU= 24.058
  05:02:22 STEP 2016 H=2016:00 +CPU= 43.346
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_83.zip
Uploading the intermediate file: upload_file_83.zip
  05:02:45 STEP 2017 H=2017:00 +CPU= 23.307
  05:03:10 STEP 2018 H=2018:00 +CPU= 23.925
  05:03:47 STEP 2019 H=2019:00 +CPU= 37.219
  05:04:11 STEP 2020 H=2020:00 +CPU= 23.971
  05:04:35 STEP 2021 H=2021:00 +CPU= 23.154
  05:05:13 STEP 2022 H=2022:00 +CPU= 38.182
  05:05:37 STEP 2023 H=2023:00 +CPU= 23.283
  05:06:01 STEP 2024 H=2024:00 +CPU= 23.986
  05:06:38 STEP 2025 H=2025:00 +CPU= 37.250
  05:07:02 STEP 2026 H=2026:00 +CPU= 24.052
  05:07:26 STEP 2027 H=2027:00 +CPU= 23.255
  05:08:07 STEP 2028 H=2028:00 +CPU= 41.222
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002028
  05:08:31 STEP 2029 H=2029:00 +CPU= 23.226
  05:08:55 STEP 2030 H=2030:00 +CPU= 24.183
  05:09:33 STEP 2031 H=2031:00 +CPU= 37.297
  05:09:57 STEP 2032 H=2032:00 +CPU= 23.911
  05:10:20 STEP 2033 H=2033:00 +CPU= 23.414
  05:10:59 STEP 2034 H=2034:00 +CPU= 38.109
  05:11:22 STEP 2035 H=2035:00 +CPU= 23.274
  05:11:47 STEP 2036 H=2036:00 +CPU= 24.094
  05:12:24 STEP 2037 H=2037:00 +CPU= 37.269
  05:12:48 STEP 2038 H=2038:00 +CPU= 23.885
  05:13:13 STEP 2039 H=2039:00 +CPU= 24.091
  05:13:56 STEP 2040 H=2040:00 +CPU= 43.370
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_84.zip
Uploading the intermediate file: upload_file_84.zip
  05:14:20 STEP 2041 H=2041:00 +CPU= 23.185
  05:14:44 STEP 2042 H=2042:00 +CPU= 23.910
  05:15:21 STEP 2043 H=2043:00 +CPU= 37.318
  05:15:45 STEP 2044 H=2044:00 +CPU= 24.052
  05:16:09 STEP 2045 H=2045:00 +CPU= 23.241
  05:16:47 STEP 2046 H=2046:00 +CPU= 38.214
  05:17:11 STEP 2047 H=2047:00 +CPU= 23.382
  05:17:35 STEP 2048 H=2048:00 +CPU= 24.004
  05:18:13 STEP 2049 H=2049:00 +CPU= 37.432
  05:18:37 STEP 2050 H=2050:00 +CPU= 24.077
  05:19:00 STEP 2051 H=2051:00 +CPU= 23.252
  05:19:42 STEP 2052 H=2052:00 +CPU= 41.187
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002052
  05:20:05 STEP 2053 H=2053:00 +CPU= 23.254
  05:20:30 STEP 2054 H=2054:00 +CPU= 24.185
  05:21:08 STEP 2055 H=2055:00 +CPU= 37.267
  05:21:32 STEP 2056 H=2056:00 +CPU= 23.963
  05:21:55 STEP 2057 H=2057:00 +CPU= 23.406
  05:22:34 STEP 2058 H=2058:00 +CPU= 37.997
  05:22:57 STEP 2059 H=2059:00 +CPU= 23.278
  05:23:21 STEP 2060 H=2060:00 +CPU= 24.088
  05:23:59 STEP 2061 H=2061:00 +CPU= 37.202
  05:24:23 STEP 2062 H=2062:00 +CPU= 23.894
  05:24:47 STEP 2063 H=2063:00 +CPU= 24.045
  05:25:31 STEP 2064 H=2064:00 +CPU= 43.360
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_85.zip
Uploading the intermediate file: upload_file_85.zip
  05:25:54 STEP 2065 H=2065:00 +CPU= 23.218
  05:26:18 STEP 2066 H=2066:00 +CPU= 23.908
  05:26:56 STEP 2067 H=2067:00 +CPU= 37.264
  05:27:20 STEP 2068 H=2068:00 +CPU= 23.985
  05:27:44 STEP 2069 H=2069:00 +CPU= 23.212
  05:28:22 STEP 2070 H=2070:00 +CPU= 38.135
  05:28:46 STEP 2071 H=2071:00 +CPU= 23.257
  05:29:10 STEP 2072 H=2072:00 +CPU= 23.956
  05:29:47 STEP 2073 H=2073:00 +CPU= 37.194
  05:30:11 STEP 2074 H=2074:00 +CPU= 24.009
  05:30:35 STEP 2075 H=2075:00 +CPU= 23.248
  05:31:16 STEP 2076 H=2076:00 +CPU= 41.097
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002076
  05:31:40 STEP 2077 H=2077:00 +CPU= 23.295
  05:32:04 STEP 2078 H=2078:00 +CPU= 24.164
  05:32:41 STEP 2079 H=2079:00 +CPU= 37.263
  05:33:05 STEP 2080 H=2080:00 +CPU= 23.904
  05:33:29 STEP 2081 H=2081:00 +CPU= 23.423
  05:34:07 STEP 2082 H=2082:00 +CPU= 38.007
  05:34:30 STEP 2083 H=2083:00 +CPU= 23.235
  05:34:55 STEP 2084 H=2084:00 +CPU= 24.129
  05:35:32 STEP 2085 H=2085:00 +CPU= 37.197
  05:35:56 STEP 2086 H=2086:00 +CPU= 23.821
  05:36:20 STEP 2087 H=2087:00 +CPU= 23.990
  05:37:04 STEP 2088 H=2088:00 +CPU= 43.359
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_86.zip
Uploading the intermediate file: upload_file_86.zip
  05:37:28 STEP 2089 H=2089:00 +CPU= 23.248
  05:37:52 STEP 2090 H=2090:00 +CPU= 23.977
  05:38:29 STEP 2091 H=2091:00 +CPU= 37.316
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2009050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0849
(argv4) batchid: 978
(argv5) wuid: 12195493
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_67c4ce0d7f3e99ceadcc2c3a0d71df5a to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0849_2009050100_123_978_12195493.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0849_2009050100_123_978_12195493.zip
ic_ancil_file: ic_ancil_12195493
ifsdata_file: ifsdata_12195493
climate_data_file: clim_data_12195493
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_ea4f4b772af1976552a3fc204b924041 to: /var/lib/boinc-client/slots/2/ic_ancil_12195493.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12195493.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_da44ffd655b18a686bfe2857ff419fac to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12195493.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12195493.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12195493.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12195493.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12195493.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12195493.xml
-- Model is restarting --
Adjusting last_iter, 2092, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12195493.xml
last_cpu_time: 63466
upload_file_number: 87
last_iter: 2087
last_upload: 7516800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847
The child process has been launched with process id: 10398
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:10398:10398] [20230102:054223:1672656143:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:43:08 STEP 2088 H=2088:00 +CPU= 41.717
  05:43:31 STEP 2089 H=2089:00 +CPU= 23.152
  05:43:55 STEP 2090 H=2090:00 +CPU= 23.927
  05:44:32 STEP 2091 H=2091:00 +CPU= 37.193
  05:44:56 STEP 2092 H=2092:00 +CPU= 23.890
  05:45:20 STEP 2093 H=2093:00 +CPU= 23.324
  05:45:58 STEP 2094 H=2094:00 +CPU= 38.059
  05:46:21 STEP 2095 H=2095:00 +CPU= 23.258
  05:46:46 STEP 2096 H=2096:00 +CPU= 24.119
  05:47:23 STEP 2097 H=2097:00 +CPU= 37.281
  05:47:47 STEP 2098 H=2098:00 +CPU= 24.040
  05:48:11 STEP 2099 H=2099:00 +CPU= 23.410
  05:48:52 STEP 2100 H=2100:00 +CPU= 41.072
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002100
  05:49:16 STEP 2101 H=2101:00 +CPU= 23.420
  05:49:40 STEP 2102 H=2102:00 +CPU= 23.944
  05:50:18 STEP 2103 H=2103:00 +CPU= 37.364
  05:50:42 STEP 2104 H=2104:00 +CPU= 24.155
  05:51:06 STEP 2105 H=2105:00 +CPU= 23.240
  05:51:44 STEP 2106 H=2106:00 +CPU= 38.175
  05:52:08 STEP 2107 H=2107:00 +CPU= 23.449
  05:52:32 STEP 2108 H=2108:00 +CPU= 23.933
  05:53:10 STEP 2109 H=2109:00 +CPU= 37.398
  05:53:34 STEP 2110 H=2110:00 +CPU= 24.127
  05:53:58 STEP 2111 H=2111:00 +CPU= 23.950
  05:54:42 STEP 2112 H=2112:00 +CPU= 43.236
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_87.zip
Uploading the intermediate file: upload_file_87.zip
  05:55:05 STEP 2113 H=2113:00 +CPU= 23.128
  05:55:30 STEP 2114 H=2114:00 +CPU= 24.021
  05:56:07 STEP 2115 H=2115:00 +CPU= 37.218
  05:56:31 STEP 2116 H=2116:00 +CPU= 23.922
  05:56:55 STEP 2117 H=2117:00 +CPU= 23.365
  05:57:33 STEP 2118 H=2118:00 +CPU= 38.042
  05:57:56 STEP 2119 H=2119:00 +CPU= 23.235
  05:58:20 STEP 2120 H=2120:00 +CPU= 24.092
  05:58:58 STEP 2121 H=2121:00 +CPU= 37.268
  05:59:22 STEP 2122 H=2122:00 +CPU= 24.022
  05:59:46 STEP 2123 H=2123:00 +CPU= 23.399
  06:00:27 STEP 2124 H=2124:00 +CPU= 41.146
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002124
  06:00:51 STEP 2125 H=2125:00 +CPU= 23.389
  06:01:15 STEP 2126 H=2126:00 +CPU= 23.913
  06:01:53 STEP 2127 H=2127:00 +CPU= 37.309
  06:02:17 STEP 2128 H=2128:00 +CPU= 24.076
  06:02:40 STEP 2129 H=2129:00 +CPU= 23.251
  06:03:19 STEP 2130 H=2130:00 +CPU= 38.097
  06:03:42 STEP 2131 H=2131:00 +CPU= 23.368
  06:04:06 STEP 2132 H=2132:00 +CPU= 23.922
  06:04:44 STEP 2133 H=2133:00 +CPU= 37.351
  06:05:08 STEP 2134 H=2134:00 +CPU= 24.108
  06:05:32 STEP 2135 H=2135:00 +CPU= 23.925
  06:06:16 STEP 2136 H=2136:00 +CPU= 43.340
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_88.zip
Uploading the intermediate file: upload_file_88.zip
  06:06:40 STEP 2137 H=2137:00 +CPU= 23.198
  06:07:04 STEP 2138 H=2138:00 +CPU= 24.014
  06:07:41 STEP 2139 H=2139:00 +CPU= 37.176
  06:08:05 STEP 2140 H=2140:00 +CPU= 23.921
  06:08:29 STEP 2141 H=2141:00 +CPU= 23.362
  06:09:07 STEP 2142 H=2142:00 +CPU= 38.017
  06:09:34 STEP 2143 H=2143:00 +CPU= 23.306
  06:09:58 STEP 2144 H=2144:00 +CPU= 23.967
  06:10:35 STEP 2145 H=2145:00 +CPU= 37.229
  06:11:00 STEP 2146 H=2146:00 +CPU= 23.993
  06:11:23 STEP 2147 H=2147:00 +CPU= 23.268
  06:12:05 STEP 2148 H=2148:00 +CPU= 41.205
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002148
  06:12:29 STEP 2149 H=2149:00 +CPU= 23.426
  06:12:53 STEP 2150 H=2150:00 +CPU= 24.018
  06:13:30 STEP 2151 H=2151:00 +CPU= 37.287
  06:13:54 STEP 2152 H=2152:00 +CPU= 24.162
  06:14:18 STEP 2153 H=2153:00 +CPU= 23.309
  06:14:56 STEP 2154 H=2154:00 +CPU= 38.191
  06:15:20 STEP 2155 H=2155:00 +CPU= 23.395
  06:15:44 STEP 2156 H=2156:00 +CPU= 23.979
  06:16:22 STEP 2157 H=2157:00 +CPU= 37.307
  06:16:46 STEP 2158 H=2158:00 +CPU= 24.119
  06:17:10 STEP 2159 H=2159:00 +CPU= 23.960
  06:17:54 STEP 2160 H=2160:00 +CPU= 43.332
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_89.zip
Uploading the intermediate file: upload_file_89.zip
  06:18:18 STEP 2161 H=2161:00 +CPU= 23.191
  06:18:42 STEP 2162 H=2162:00 +CPU= 23.931
  06:19:20 STEP 2163 H=2163:00 +CPU= 37.266
  06:19:44 STEP 2164 H=2164:00 +CPU= 24.000
  06:20:07 STEP 2165 H=2165:00 +CPU= 23.191
  06:20:45 STEP 2166 H=2166:00 +CPU= 38.013
  06:21:09 STEP 2167 H=2167:00 +CPU= 23.326
  06:21:33 STEP 2168 H=2168:00 +CPU= 23.919
  06:22:11 STEP 2169 H=2169:00 +CPU= 37.277
  06:22:35 STEP 2170 H=2170:00 +CPU= 24.051
  06:22:58 STEP 2171 H=2171:00 +CPU= 23.252
  06:23:40 STEP 2172 H=2172:00 +CPU= 41.172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002172
  06:24:03 STEP 2173 H=2173:00 +CPU= 23.336
  06:24:28 STEP 2174 H=2174:00 +CPU= 24.049
  06:25:05 STEP 2175 H=2175:00 +CPU= 37.150
  06:25:30 STEP 2176 H=2176:00 +CPU= 23.953
  06:25:53 STEP 2177 H=2177:00 +CPU= 23.268
  06:26:31 STEP 2178 H=2178:00 +CPU= 37.949
  06:26:55 STEP 2179 H=2179:00 +CPU= 23.205
  06:27:19 STEP 2180 H=2180:00 +CPU= 23.912
  06:27:58 STEP 2181 H=2181:00 +CPU= 38.957
  06:28:24 STEP 2182 H=2182:00 +CPU= 25.517
  06:28:50 STEP 2183 H=2183:00 +CPU= 25.906
  06:29:37 STEP 2184 H=2184:00 +CPU= 45.938
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_90.zip
Uploading the intermediate file: upload_file_90.zip
  06:30:02 STEP 2185 H=2185:00 +CPU= 24.999
  06:30:28 STEP 2186 H=2186:00 +CPU= 25.909
  06:31:08 STEP 2187 H=2187:00 +CPU= 39.193
  06:31:34 STEP 2188 H=2188:00 +CPU= 26.002
  06:32:00 STEP 2189 H=2189:00 +CPU= 25.057
  06:32:40 STEP 2190 H=2190:00 +CPU= 40.192
  06:33:05 STEP 2191 H=2191:00 +CPU= 24.994
  06:33:32 STEP 2192 H=2192:00 +CPU= 26.133
  06:34:11 STEP 2193 H=2193:00 +CPU= 39.237
  06:34:37 STEP 2194 H=2194:00 +CPU= 25.732
  06:35:03 STEP 2195 H=2195:00 +CPU= 25.433
  06:35:47 STEP 2196 H=2196:00 +CPU= 43.407
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002196
  06:36:12 STEP 2197 H=2197:00 +CPU= 25.288
  06:36:38 STEP 2198 H=2198:00 +CPU= 25.874
  06:37:18 STEP 2199 H=2199:00 +CPU= 39.292
  06:37:44 STEP 2200 H=2200:00 +CPU= 25.861
  06:38:09 STEP 2201 H=2201:00 +CPU= 25.186
  06:38:50 STEP 2202 H=2202:00 +CPU= 40.059
  06:39:15 STEP 2203 H=2203:00 +CPU= 25.199
  06:39:41 STEP 2204 H=2204:00 +CPU= 25.852
  06:40:21 STEP 2205 H=2205:00 +CPU= 39.396
  06:40:47 STEP 2206 H=2206:00 +CPU= 25.521
  06:41:13 STEP 2207 H=2207:00 +CPU= 25.906
  06:41:59 STEP 2208 H=2208:00 +CPU= 46.010
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_91.zip
Uploading the intermediate file: upload_file_91.zip
  06:42:25 STEP 2209 H=2209:00 +CPU= 24.841
  06:42:51 STEP 2210 H=2210:00 +CPU= 26.057
  06:43:30 STEP 2211 H=2211:00 +CPU= 39.309
  06:43:56 STEP 2212 H=2212:00 +CPU= 25.883
  06:44:22 STEP 2213 H=2213:00 +CPU= 24.986
  06:45:02 STEP 2214 H=2214:00 +CPU= 40.362
  06:45:28 STEP 2215 H=2215:00 +CPU= 25.244
  06:45:54 STEP 2216 H=2216:00 +CPU= 25.946
  06:46:34 STEP 2217 H=2217:00 +CPU= 39.437
  06:47:00 STEP 2218 H=2218:00 +CPU= 25.842
  06:47:25 STEP 2219 H=2219:00 +CPU= 25.011
  06:48:09 STEP 2220 H=2220:00 +CPU= 43.644
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002220
  06:48:34 STEP 2221 H=2221:00 +CPU= 25.117
  06:49:01 STEP 2222 H=2222:00 +CPU= 26.129
  06:49:40 STEP 2223 H=2223:00 +CPU= 39.328
  06:50:06 STEP 2224 H=2224:00 +CPU= 25.890
  06:50:32 STEP 2225 H=2225:00 +CPU= 25.145
  06:51:13 STEP 2226 H=2226:00 +CPU= 40.560
  06:51:38 STEP 2227 H=2227:00 +CPU= 25.036
  06:52:04 STEP 2228 H=2228:00 +CPU= 26.002
  06:52:44 STEP 2229 H=2229:00 +CPU= 39.510
  06:53:10 STEP 2230 H=2230:00 +CPU= 25.768
  06:53:36 STEP 2231 H=2231:00 +CPU= 25.679
  06:54:22 STEP 2232 H=2232:00 +CPU= 45.889
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_92.zip
Uploading the intermediate file: upload_file_92.zip
  06:54:48 STEP 2233 H=2233:00 +CPU= 25.098
  06:55:14 STEP 2234 H=2234:00 +CPU= 25.726
  06:55:54 STEP 2235 H=2235:00 +CPU= 39.576
  06:56:20 STEP 2236 H=2236:00 +CPU= 25.916
  06:56:45 STEP 2237 H=2237:00 +CPU= 25.118
  06:57:25 STEP 2238 H=2238:00 +CPU= 40.271
  06:57:51 STEP 2239 H=2239:00 +CPU= 25.127
  06:58:17 STEP 2240 H=2240:00 +CPU= 25.889
  06:58:57 STEP 2241 H=2241:00 +CPU= 39.490
  06:59:23 STEP 2242 H=2242:00 +CPU= 26.055
  06:59:48 STEP 2243 H=2243:00 +CPU= 25.063
  07:00:32 STEP 2244 H=2244:00 +CPU= 43.480
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002244
  07:00:58 STEP 2245 H=2245:00 +CPU= 25.214
  07:01:24 STEP 2246 H=2246:00 +CPU= 25.773
  07:02:04 STEP 2247 H=2247:00 +CPU= 39.662
  07:02:30 STEP 2248 H=2248:00 +CPU= 25.937
  07:02:55 STEP 2249 H=2249:00 +CPU= 25.184
  07:03:36 STEP 2250 H=2250:00 +CPU= 40.325
  07:04:01 STEP 2251 H=2251:00 +CPU= 25.122
  07:04:27 STEP 2252 H=2252:00 +CPU= 25.884
  07:05:07 STEP 2253 H=2253:00 +CPU= 39.581
  07:05:33 STEP 2254 H=2254:00 +CPU= 25.894
  07:06:00 STEP 2255 H=2255:00 +CPU= 25.809
  07:06:46 STEP 2256 H=2256:00 +CPU= 45.955
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_93.zip
Uploading the intermediate file: upload_file_93.zip
  07:07:12 STEP 2257 H=2257:00 +CPU= 25.333
  07:07:38 STEP 2258 H=2258:00 +CPU= 25.885
  07:08:18 STEP 2259 H=2259:00 +CPU= 39.350
  07:08:44 STEP 2260 H=2260:00 +CPU= 25.751
  07:09:09 STEP 2261 H=2261:00 +CPU= 25.361
  07:09:50 STEP 2262 H=2262:00 +CPU= 40.186
  07:10:15 STEP 2263 H=2263:00 +CPU= 25.232
  07:10:42 STEP 2264 H=2264:00 +CPU= 25.856
  07:11:22 STEP 2265 H=2265:00 +CPU= 39.648
  07:11:48 STEP 2266 H=2266:00 +CPU= 25.805
  07:12:13 STEP 2267 H=2267:00 +CPU= 25.200
  07:12:57 STEP 2268 H=2268:00 +CPU= 43.523
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002268
  07:13:23 STEP 2269 H=2269:00 +CPU= 25.416
  07:13:49 STEP 2270 H=2270:00 +CPU= 26.006
  07:14:28 STEP 2271 H=2271:00 +CPU= 39.424
  07:14:55 STEP 2272 H=2272:00 +CPU= 25.835
  07:15:20 STEP 2273 H=2273:00 +CPU= 25.324
  07:16:01 STEP 2274 H=2274:00 +CPU= 40.186
  07:16:26 STEP 2275 H=2275:00 +CPU= 25.245
  07:16:53 STEP 2276 H=2276:00 +CPU= 25.930
  07:17:32 STEP 2277 H=2277:00 +CPU= 39.518
  07:17:58 STEP 2278 H=2278:00 +CPU= 25.666
  07:18:25 STEP 2279 H=2279:00 +CPU= 25.953
  07:19:11 STEP 2280 H=2280:00 +CPU= 45.988
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_94.zip
Uploading the intermediate file: upload_file_94.zip
  07:19:37 STEP 2281 H=2281:00 +CPU= 24.886
  07:20:03 STEP 2282 H=2282:00 +CPU= 26.062
  07:20:43 STEP 2283 H=2283:00 +CPU= 39.298
  07:21:09 STEP 2284 H=2284:00 +CPU= 25.934
  07:21:34 STEP 2285 H=2285:00 +CPU= 24.991
  07:22:15 STEP 2286 H=2286:00 +CPU= 40.482
  07:22:40 STEP 2287 H=2287:00 +CPU= 25.064
  07:23:06 STEP 2288 H=2288:00 +CPU= 25.881
  07:23:46 STEP 2289 H=2289:00 +CPU= 39.357
  07:24:12 STEP 2290 H=2290:00 +CPU= 25.820
  07:24:37 STEP 2291 H=2291:00 +CPU= 25.073
  07:25:21 STEP 2292 H=2292:00 +CPU= 43.734
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002292
  07:25:46 STEP 2293 H=2293:00 +CPU= 24.953
  07:26:12 STEP 2294 H=2294:00 +CPU= 26.110
  07:26:52 STEP 2295 H=2295:00 +CPU= 39.326
  07:27:18 STEP 2296 H=2296:00 +CPU= 25.913
  07:27:43 STEP 2297 H=2297:00 +CPU= 24.895
  07:28:24 STEP 2298 H=2298:00 +CPU= 40.470
  07:28:49 STEP 2299 H=2299:00 +CPU= 25.065
  07:29:16 STEP 2300 H=2300:00 +CPU= 25.906
  07:29:55 STEP 2301 H=2301:00 +CPU= 39.267
  07:30:21 STEP 2302 H=2302:00 +CPU= 25.769
  07:30:47 STEP 2303 H=2303:00 +CPU= 25.800
  07:31:34 STEP 2304 H=2304:00 +CPU= 46.102
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_95.zip
Uploading the intermediate file: upload_file_95.zip
  07:31:59 STEP 2305 H=2305:00 +CPU= 24.886
  07:32:25 STEP 2306 H=2306:00 +CPU= 25.782
  07:33:05 STEP 2307 H=2307:00 +CPU= 39.338
  07:33:31 STEP 2308 H=2308:00 +CPU= 26.076
  07:33:56 STEP 2309 H=2309:00 +CPU= 24.977
  07:34:37 STEP 2310 H=2310:00 +CPU= 40.257
  07:35:02 STEP 2311 H=2311:00 +CPU= 24.838
  07:35:28 STEP 2312 H=2312:00 +CPU= 25.997
  07:36:08 STEP 2313 H=2313:00 +CPU= 39.287
  07:36:34 STEP 2314 H=2314:00 +CPU= 25.931
  07:36:59 STEP 2315 H=2315:00 +CPU= 25.157
  07:37:46 STEP 2316 H=2316:00 +CPU= 43.721
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002316
  07:38:13 STEP 2317 H=2317:00 +CPU= 25.066
  07:38:40 STEP 2318 H=2318:00 +CPU= 25.936
  07:39:27 STEP 2319 H=2319:00 +CPU= 39.533
  07:39:54 STEP 2320 H=2320:00 +CPU= 26.069
  07:40:20 STEP 2321 H=2321:00 +CPU= 24.942
  07:41:01 STEP 2322 H=2322:00 +CPU= 40.464
  07:41:29 STEP 2323 H=2323:00 +CPU= 24.910
  07:41:59 STEP 2324 H=2324:00 +CPU= 26.080
  07:42:42 STEP 2325 H=2325:00 +CPU= 39.514
  07:43:09 STEP 2326 H=2326:00 +CPU= 25.874
  07:43:38 STEP 2327 H=2327:00 +CPU= 25.856
  07:44:27 STEP 2328 H=2328:00 +CPU= 46.118
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_96.zip
Uploading the intermediate file: upload_file_96.zip
Uploading trickle at timestep: 8380800
  07:44:53 STEP 2329 H=2329:00 +CPU= 24.999
  07:45:19 STEP 2330 H=2330:00 +CPU= 25.687
  07:46:00 STEP 2331 H=2331:00 +CPU= 39.747
  07:46:26 STEP 2332 H=2332:00 +CPU= 25.754
  07:46:52 STEP 2333 H=2333:00 +CPU= 25.174
  07:47:32 STEP 2334 H=2334:00 +CPU= 40.289
  07:47:58 STEP 2335 H=2335:00 +CPU= 25.083
  07:48:24 STEP 2336 H=2336:00 +CPU= 25.698
  07:49:03 STEP 2337 H=2337:00 +CPU= 39.444
  07:49:30 STEP 2338 H=2338:00 +CPU= 26.066
  07:49:55 STEP 2339 H=2339:00 +CPU= 25.214
  07:50:39 STEP 2340 H=2340:00 +CPU= 43.568
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002340
  07:51:04 STEP 2341 H=2341:00 +CPU= 25.003
  07:51:30 STEP 2342 H=2342:00 +CPU= 25.718
  07:52:10 STEP 2343 H=2343:00 +CPU= 39.524
  07:52:36 STEP 2344 H=2344:00 +CPU= 25.913
  07:53:01 STEP 2345 H=2345:00 +CPU= 25.014
  07:53:42 STEP 2346 H=2346:00 +CPU= 40.284
  07:54:07 STEP 2347 H=2347:00 +CPU= 24.900
  07:54:33 STEP 2348 H=2348:00 +CPU= 25.763
  07:55:13 STEP 2349 H=2349:00 +CPU= 39.297
  07:55:39 STEP 2350 H=2350:00 +CPU= 26.033
  07:56:05 STEP 2351 H=2351:00 +CPU= 25.959
  07:56:52 STEP 2352 H=2352:00 +CPU= 45.883
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_97.zip
Uploading the intermediate file: upload_file_97.zip
  07:57:17 STEP 2353 H=2353:00 +CPU= 25.113
  07:57:43 STEP 2354 H=2354:00 +CPU= 25.966
  07:58:23 STEP 2355 H=2355:00 +CPU= 39.310
  07:58:49 STEP 2356 H=2356:00 +CPU= 25.718
  07:59:14 STEP 2357 H=2357:00 +CPU= 25.201
  07:59:55 STEP 2358 H=2358:00 +CPU= 40.283
  08:00:21 STEP 2359 H=2359:00 +CPU= 25.054
  08:00:47 STEP 2360 H=2360:00 +CPU= 25.773
  08:01:26 STEP 2361 H=2361:00 +CPU= 39.409
  08:01:53 STEP 2362 H=2362:00 +CPU= 25.807
  08:02:18 STEP 2363 H=2363:00 +CPU= 25.357
  08:03:02 STEP 2364 H=2364:00 +CPU= 43.620
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002364
  08:03:28 STEP 2365 H=2365:00 +CPU= 25.240
  08:03:54 STEP 2366 H=2366:00 +CPU= 25.891
  08:04:34 STEP 2367 H=2367:00 +CPU= 39.391
  08:05:00 STEP 2368 H=2368:00 +CPU= 25.708
  08:05:25 STEP 2369 H=2369:00 +CPU= 25.158
  08:06:06 STEP 2370 H=2370:00 +CPU= 40.260
  08:06:31 STEP 2371 H=2371:00 +CPU= 25.170
  08:06:57 STEP 2372 H=2372:00 +CPU= 25.662
  08:07:37 STEP 2373 H=2373:00 +CPU= 39.442
  08:08:03 STEP 2374 H=2374:00 +CPU= 25.662
  08:08:29 STEP 2375 H=2375:00 +CPU= 25.834
  08:09:16 STEP 2376 H=2376:00 +CPU= 46.083
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_98.zip
Uploading the intermediate file: upload_file_98.zip
  08:09:41 STEP 2377 H=2377:00 +CPU= 24.874
  08:10:07 STEP 2378 H=2378:00 +CPU= 25.891
  08:10:47 STEP 2379 H=2379:00 +CPU= 39.382
  08:11:13 STEP 2380 H=2380:00 +CPU= 25.843
  08:11:38 STEP 2381 H=2381:00 +CPU= 24.939
  08:12:18 STEP 2382 H=2382:00 +CPU= 40.299
  08:12:44 STEP 2383 H=2383:00 +CPU= 25.159
  08:13:10 STEP 2384 H=2384:00 +CPU= 25.872
  08:13:50 STEP 2385 H=2385:00 +CPU= 39.339
  08:14:16 STEP 2386 H=2386:00 +CPU= 25.883
  08:14:41 STEP 2387 H=2387:00 +CPU= 25.300
  08:15:25 STEP 2388 H=2388:00 +CPU= 43.834
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002388
  08:15:50 STEP 2389 H=2389:00 +CPU= 24.921
  08:16:16 STEP 2390 H=2390:00 +CPU= 25.901
  08:16:56 STEP 2391 H=2391:00 +CPU= 39.481
  08:17:22 STEP 2392 H=2392:00 +CPU= 25.834
  08:17:48 STEP 2393 H=2393:00 +CPU= 24.937
  08:18:28 STEP 2394 H=2394:00 +CPU= 40.339
  08:18:54 STEP 2395 H=2395:00 +CPU= 25.237
  08:19:20 STEP 2396 H=2396:00 +CPU= 25.822
  08:19:59 STEP 2397 H=2397:00 +CPU= 39.367
  08:20:25 STEP 2398 H=2398:00 +CPU= 25.676
  08:20:52 STEP 2399 H=2399:00 +CPU= 25.982
  08:21:38 STEP 2400 H=2400:00 +CPU= 46.217
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_99.zip
Uploading the intermediate file: upload_file_99.zip
  08:22:04 STEP 2401 H=2401:00 +CPU= 24.907
  08:22:30 STEP 2402 H=2402:00 +CPU= 25.808
  08:23:10 STEP 2403 H=2403:00 +CPU= 39.506
  08:23:36 STEP 2404 H=2404:00 +CPU= 25.844
  08:24:01 STEP 2405 H=2405:00 +CPU= 25.014
  08:24:42 STEP 2406 H=2406:00 +CPU= 40.478
  08:25:07 STEP 2407 H=2407:00 +CPU= 24.944
  08:25:34 STEP 2408 H=2408:00 +CPU= 25.969
  08:26:13 STEP 2409 H=2409:00 +CPU= 39.404
  08:26:40 STEP 2410 H=2410:00 +CPU= 25.941
  08:27:05 STEP 2411 H=2411:00 +CPU= 25.094
  08:27:49 STEP 2412 H=2412:00 +CPU= 43.907
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002412
  08:28:15 STEP 2413 H=2413:00 +CPU= 25.053
  08:28:41 STEP 2414 H=2414:00 +CPU= 25.861
  08:29:21 STEP 2415 H=2415:00 +CPU= 39.507
  08:29:47 STEP 2416 H=2416:00 +CPU= 25.933
  08:30:12 STEP 2417 H=2417:00 +CPU= 25.062
  08:30:53 STEP 2418 H=2418:00 +CPU= 40.502
  08:31:18 STEP 2419 H=2419:00 +CPU= 25.005
  08:31:45 STEP 2420 H=2420:00 +CPU= 26.091
  08:32:24 STEP 2421 H=2421:00 +CPU= 39.342
  08:32:51 STEP 2422 H=2422:00 +CPU= 25.806
  08:33:17 STEP 2423 H=2423:00 +CPU= 25.736
  08:34:04 STEP 2424 H=2424:00 +CPU= 46.902
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_100.zip
Uploading the intermediate file: upload_file_100.zip
  08:34:29 STEP 2425 H=2425:00 +CPU= 25.111
  08:34:56 STEP 2426 H=2426:00 +CPU= 25.971
  08:35:36 STEP 2427 H=2427:00 +CPU= 39.600
  08:36:02 STEP 2428 H=2428:00 +CPU= 26.030
  08:36:27 STEP 2429 H=2429:00 +CPU= 25.139
  08:37:08 STEP 2430 H=2430:00 +CPU= 40.623
  08:37:34 STEP 2431 H=2431:00 +CPU= 25.091
  08:38:00 STEP 2432 H=2432:00 +CPU= 26.099
  08:38:40 STEP 2433 H=2433:00 +CPU= 39.413
  08:39:06 STEP 2434 H=2434:00 +CPU= 26.012
  08:39:31 STEP 2435 H=2435:00 +CPU= 25.022
  08:40:15 STEP 2436 H=2436:00 +CPU= 43.916
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002436
  08:40:41 STEP 2437 H=2437:00 +CPU= 25.054
  08:41:07 STEP 2438 H=2438:00 +CPU= 25.918
  08:41:47 STEP 2439 H=2439:00 +CPU= 39.579
  08:42:13 STEP 2440 H=2440:00 +CPU= 26.065
  08:42:39 STEP 2441 H=2441:00 +CPU= 25.252
  08:43:20 STEP 2442 H=2442:00 +CPU= 40.504
  08:43:45 STEP 2443 H=2443:00 +CPU= 25.206
  08:44:12 STEP 2444 H=2444:00 +CPU= 26.069
  08:44:51 STEP 2445 H=2445:00 +CPU= 39.389
  08:45:18 STEP 2446 H=2446:00 +CPU= 25.952
  08:45:44 STEP 2447 H=2447:00 +CPU= 26.004
  08:46:31 STEP 2448 H=2448:00 +CPU= 46.378
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_101.zip
Uploading the intermediate file: upload_file_101.zip
  08:46:57 STEP 2449 H=2449:00 +CPU= 25.464
  08:47:23 STEP 2450 H=2450:00 +CPU= 26.012
  08:48:03 STEP 2451 H=2451:00 +CPU= 39.419
  08:48:29 STEP 2452 H=2452:00 +CPU= 25.791
  08:48:55 STEP 2453 H=2453:00 +CPU= 25.404
  08:49:36 STEP 2454 H=2454:00 +CPU= 40.500
  08:50:01 STEP 2455 H=2455:00 +CPU= 25.387
  08:50:28 STEP 2456 H=2456:00 +CPU= 26.053
  08:51:08 STEP 2457 H=2457:00 +CPU= 39.563
  08:51:34 STEP 2458 H=2458:00 +CPU= 25.652
  08:51:59 STEP 2459 H=2459:00 +CPU= 25.243
  08:52:43 STEP 2460 H=2460:00 +CPU= 43.797
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002460
  08:53:09 STEP 2461 H=2461:00 +CPU= 25.114
  08:53:35 STEP 2462 H=2462:00 +CPU= 25.936
  08:54:15 STEP 2463 H=2463:00 +CPU= 39.600
  08:54:41 STEP 2464 H=2464:00 +CPU= 25.926
  08:55:07 STEP 2465 H=2465:00 +CPU= 25.162
  08:55:48 STEP 2466 H=2466:00 +CPU= 40.578
  08:56:13 STEP 2467 H=2467:00 +CPU= 25.321
  08:56:39 STEP 2468 H=2468:00 +CPU= 25.866
  08:57:19 STEP 2469 H=2469:00 +CPU= 39.448
  08:57:45 STEP 2470 H=2470:00 +CPU= 25.779
  08:58:11 STEP 2471 H=2471:00 +CPU= 25.899
  08:58:58 STEP 2472 H=2472:00 +CPU= 46.473
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_102.zip
Uploading the intermediate file: upload_file_102.zip
  08:59:24 STEP 2473 H=2473:00 +CPU= 25.108
  08:59:50 STEP 2474 H=2474:00 +CPU= 25.796
  09:00:30 STEP 2475 H=2475:00 +CPU= 39.851
  09:00:57 STEP 2476 H=2476:00 +CPU= 26.083
  09:01:22 STEP 2477 H=2477:00 +CPU= 25.305
  09:02:03 STEP 2478 H=2478:00 +CPU= 40.466
  09:02:29 STEP 2479 H=2479:00 +CPU= 25.150
  09:02:55 STEP 2480 H=2480:00 +CPU= 25.941
  09:03:35 STEP 2481 H=2481:00 +CPU= 39.516
  09:04:01 STEP 2482 H=2482:00 +CPU= 25.984
  09:04:26 STEP 2483 H=2483:00 +CPU= 25.071
  09:05:10 STEP 2484 H=2484:00 +CPU= 43.658
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002484
  09:05:36 STEP 2485 H=2485:00 +CPU= 25.251
  09:06:02 STEP 2486 H=2486:00 +CPU= 25.930
  09:06:42 STEP 2487 H=2487:00 +CPU= 39.817
  09:07:08 STEP 2488 H=2488:00 +CPU= 25.871
  09:07:34 STEP 2489 H=2489:00 +CPU= 25.418
  09:08:15 STEP 2490 H=2490:00 +CPU= 40.380
  09:08:40 STEP 2491 H=2491:00 +CPU= 25.250
  09:09:06 STEP 2492 H=2492:00 +CPU= 25.766
  09:09:46 STEP 2493 H=2493:00 +CPU= 39.525
  09:10:12 STEP 2494 H=2494:00 +CPU= 25.830
  09:10:38 STEP 2495 H=2495:00 +CPU= 25.850
  09:11:25 STEP 2496 H=2496:00 +CPU= 46.366
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195493/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0849_2009050100_123_978_12195493_0_r516290847_103.zip
Uploading the intermediate file: upload_file_103.zip
  09:11:50 STEP 2497 H=2497:00 +CPU= 25.103
  09:12:17 STEP 2498 H=2498:00 +CPU= 26.010
  09:12:57 STEP 2499 H=2499:00 +CPU= 39.587
  09:13:23 STEP 2500 H=2500:00 +CPU= 25.922
  09:13:48 STEP 2501 H=2501:00 +CPU= 25.094
  09:14:29 STEP 2502 H=2502:00 +CPU= 40.531
  09:14:55 STEP 2503 H=2503:00 +CPU= 25.376
  09:15:21 STEP 2504 H=2504:00 +CPU= 26.007
  09:16:01 STEP 2505 H=2505:00 +CPU= 39.517
  09:16:27 STEP 2506 H=2506:00 +CPU= 25.866
  09:16:53 STEP 2507 H=2507:00 +CPU= 24.943
  09:17:37 STEP 2508 H=2508:00 +CPU= 43.788
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002508
  09:18:02 STEP 2509 H=2509:00 +CPU= 25.100
  09:18:28 STEP 2510 H=2510:00 +CPU= 26.140
  09:19:08 STEP 2511 H=2511:00 +CPU= 39.621
  09:19:35 STEP 2512 H=2512:00 +CPU= 25.989
  09:20:00 STEP 2513 H=2513:00 +CPU= 25.101
  09:20:41 STEP 2514 H=2514:00 +CPU= 40.615
  09:21:06 STEP 2515 H=2515:00 +CPU= 25.135
  09:21:32 STEP 2516 H=2516:00 +CPU= 26.123
  09:22:12 STEP 2517 H=2517:00 +CPU= 39.405
  09:22:38 STEP 2518 H=2518:00 +CPU= 25.866
  09:23:04 STEP 2519 H=2519:00 +CPU= 25.784
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:06:00 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 8,380,800 70,694 0.0084
09 May 2023 16:05:42 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 6,048,000 51,325 0.0085
09 May 2023 16:05:37 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 5,184,000 44,869 0.0087
09 May 2023 16:05:37 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 5,184,000 44,869 0.0087
09 May 2023 16:05:27 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 3,715,200 32,487 0.0087
09 May 2023 16:05:27 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 3,715,200 32,487 0.0087
09 May 2023 16:05:22 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 2,851,200 25,136 0.0088
09 May 2023 16:05:17 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 1,987,200 17,500 0.0088
09 May 2023 16:05:08 1466689 22281418 oifs_43r3_ps_0849_2009050100_123_978_12195493_0 864,000 7,247 0.0084


©2024 climateprediction.net