climateprediction.net home page
Task 22306480

Task 22306480

Name oifs_43r3_ps_0469_2015050100_123_984_12201113_1
Workunit 12201113
Created 7 Feb 2023, 12:52:10 UTC
Sent 7 Feb 2023, 12:52:53 UTC
Report deadline 8 Apr 2023, 12:52:53 UTC
Received 8 Feb 2023, 9:53:46 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1524359
Run time 19 hours 24 min 45 sec
CPU time 18 hours 22 min 29 sec
Validate state Invalid
Credit 7,015.68
Device peak FLOPS 3.53 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,399.16 MB
Peak swap size 4,978.07 MB
Peak disk usage 2,019.18 MB
Stderr
<core_client_version>7.9.3</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
/var/lib/boinc-client/slots/4/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001020
  17:12:42 STEP 1021 H=1021:00 +CPU= 34.661
  17:13:18 STEP 1022 H=1022:00 +CPU= 35.801
  17:14:14 STEP 1023 H=1023:00 +CPU= 53.985
  17:14:50 STEP 1024 H=1024:00 +CPU= 35.697
  17:15:25 STEP 1025 H=1025:00 +CPU= 34.664
  17:16:21 STEP 1026 H=1026:00 +CPU= 55.246
  17:16:56 STEP 1027 H=1027:00 +CPU= 34.450
  17:17:32 STEP 1028 H=1028:00 +CPU= 35.408
  17:18:28 STEP 1029 H=1029:00 +CPU= 53.781
  17:19:05 STEP 1030 H=1030:00 +CPU= 35.908
  17:19:43 STEP 1031 H=1031:00 +CPU= 36.123
  17:20:49 STEP 1032 H=1032:00 +CPU= 62.929
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_42.zip
Uploading the intermediate file: upload_file_42.zip
  17:21:24 STEP 1033 H=1033:00 +CPU= 34.499
  17:22:00 STEP 1034 H=1034:00 +CPU= 35.618
  17:22:56 STEP 1035 H=1035:00 +CPU= 53.881
  17:23:32 STEP 1036 H=1036:00 +CPU= 35.538
  17:24:07 STEP 1037 H=1037:00 +CPU= 34.401
  17:25:06 STEP 1038 H=1038:00 +CPU= 55.541
  17:25:40 STEP 1039 H=1039:00 +CPU= 34.210
  17:26:17 STEP 1040 H=1040:00 +CPU= 35.645
  17:27:12 STEP 1041 H=1041:00 +CPU= 53.673
  17:27:49 STEP 1042 H=1042:00 +CPU= 35.856
  17:28:25 STEP 1043 H=1043:00 +CPU= 34.615
  17:29:26 STEP 1044 H=1044:00 +CPU= 59.719
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001044
  17:30:02 STEP 1045 H=1045:00 +CPU= 34.676
  17:30:38 STEP 1046 H=1046:00 +CPU= 35.511
  17:31:32 STEP 1047 H=1047:00 +CPU= 53.559
  17:32:08 STEP 1048 H=1048:00 +CPU= 35.592
  17:32:43 STEP 1049 H=1049:00 +CPU= 34.467
  17:33:40 STEP 1050 H=1050:00 +CPU= 55.329
  17:34:15 STEP 1051 H=1051:00 +CPU= 34.482
  17:34:51 STEP 1052 H=1052:00 +CPU= 35.562
  17:35:45 STEP 1053 H=1053:00 +CPU= 53.549
  17:36:22 STEP 1054 H=1054:00 +CPU= 35.625
  17:36:58 STEP 1055 H=1055:00 +CPU= 35.510
  17:38:04 STEP 1056 H=1056:00 +CPU= 62.745
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_43.zip
Uploading the intermediate file: upload_file_43.zip
  17:38:39 STEP 1057 H=1057:00 +CPU= 34.504
  17:39:15 STEP 1058 H=1058:00 +CPU= 35.619
  17:40:10 STEP 1059 H=1059:00 +CPU= 53.669
  17:40:46 STEP 1060 H=1060:00 +CPU= 35.643
  17:41:21 STEP 1061 H=1061:00 +CPU= 34.638
  17:42:17 STEP 1062 H=1062:00 +CPU= 55.447
  17:42:53 STEP 1063 H=1063:00 +CPU= 34.613
  17:43:28 STEP 1064 H=1064:00 +CPU= 35.550
  17:44:22 STEP 1065 H=1065:00 +CPU= 53.889
  17:44:58 STEP 1066 H=1066:00 +CPU= 35.486
  17:45:35 STEP 1067 H=1067:00 +CPU= 34.771
  17:46:37 STEP 1068 H=1068:00 +CPU= 60.417
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001068
  17:47:12 STEP 1069 H=1069:00 +CPU= 34.575
  17:47:48 STEP 1070 H=1070:00 +CPU= 35.861
  17:48:44 STEP 1071 H=1071:00 +CPU= 53.955
  17:49:19 STEP 1072 H=1072:00 +CPU= 35.473
  17:49:54 STEP 1073 H=1073:00 +CPU= 34.244
  17:50:49 STEP 1074 H=1074:00 +CPU= 54.889
  17:51:24 STEP 1075 H=1075:00 +CPU= 34.304
  17:52:00 STEP 1076 H=1076:00 +CPU= 35.472
  17:52:55 STEP 1077 H=1077:00 +CPU= 53.928
  17:53:31 STEP 1078 H=1078:00 +CPU= 35.442
  17:54:07 STEP 1079 H=1079:00 +CPU= 35.766
  17:55:12 STEP 1080 H=1080:00 +CPU= 62.563
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001068
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_44.zip
Uploading the intermediate file: upload_file_44.zip
  17:55:48 STEP 1081 H=1081:00 +CPU= 34.389
  17:56:24 STEP 1082 H=1082:00 +CPU= 35.648
  17:57:20 STEP 1083 H=1083:00 +CPU= 54.125
  17:57:57 STEP 1084 H=1084:00 +CPU= 35.750
  17:58:32 STEP 1085 H=1085:00 +CPU= 34.272
  17:59:28 STEP 1086 H=1086:00 +CPU= 55.182
  18:00:03 STEP 1087 H=1087:00 +CPU= 34.489
  18:00:39 STEP 1088 H=1088:00 +CPU= 35.791
  18:01:34 STEP 1089 H=1089:00 +CPU= 53.900
  18:02:10 STEP 1090 H=1090:00 +CPU= 35.492
  18:02:46 STEP 1091 H=1091:00 +CPU= 34.793
  18:03:48 STEP 1092 H=1092:00 +CPU= 60.012
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001092
  18:04:23 STEP 1093 H=1093:00 +CPU= 34.662
  18:04:59 STEP 1094 H=1094:00 +CPU= 35.724
  18:05:54 STEP 1095 H=1095:00 +CPU= 53.924
  18:06:31 STEP 1096 H=1096:00 +CPU= 35.806
  18:07:06 STEP 1097 H=1097:00 +CPU= 34.421
  18:08:03 STEP 1098 H=1098:00 +CPU= 55.447
  18:08:38 STEP 1099 H=1099:00 +CPU= 34.372
  18:09:15 STEP 1100 H=1100:00 +CPU= 35.563
  18:10:10 STEP 1101 H=1101:00 +CPU= 53.608
  18:10:45 STEP 1102 H=1102:00 +CPU= 35.553
  18:11:21 STEP 1103 H=1103:00 +CPU= 35.725
  18:12:26 STEP 1104 H=1104:00 +CPU= 62.720
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001092
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_45.zip
Uploading the intermediate file: upload_file_45.zip
  18:13:01 STEP 1105 H=1105:00 +CPU= 34.048
  18:13:36 STEP 1106 H=1106:00 +CPU= 35.213
  18:14:31 STEP 1107 H=1107:00 +CPU= 53.579
  18:15:08 STEP 1108 H=1108:00 +CPU= 35.706
  18:15:43 STEP 1109 H=1109:00 +CPU= 34.274
  18:16:38 STEP 1110 H=1110:00 +CPU= 55.050
  18:17:14 STEP 1111 H=1111:00 +CPU= 34.565
  18:17:50 STEP 1112 H=1112:00 +CPU= 35.653
  18:18:45 STEP 1113 H=1113:00 +CPU= 53.989
  18:19:21 STEP 1114 H=1114:00 +CPU= 35.695
  18:19:56 STEP 1115 H=1115:00 +CPU= 33.697
  18:20:57 STEP 1116 H=1116:00 +CPU= 59.685
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001116
  18:21:35 STEP 1117 H=1117:00 +CPU= 34.851
  18:22:12 STEP 1118 H=1118:00 +CPU= 35.667
  18:23:11 STEP 1119 H=1119:00 +CPU= 54.713
  18:23:49 STEP 1120 H=1120:00 +CPU= 35.991
  18:24:25 STEP 1121 H=1121:00 +CPU= 34.815
  18:25:23 STEP 1122 H=1122:00 +CPU= 55.308
  18:25:57 STEP 1123 H=1123:00 +CPU= 34.073
  18:26:33 STEP 1124 H=1124:00 +CPU= 35.621
  18:27:28 STEP 1125 H=1125:00 +CPU= 53.883
  18:28:04 STEP 1126 H=1126:00 +CPU= 35.560
  18:28:41 STEP 1127 H=1127:00 +CPU= 35.764
  18:29:48 STEP 1128 H=1128:00 +CPU= 63.185
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_46.zip
Uploading the intermediate file: upload_file_46.zip
  18:30:24 STEP 1129 H=1129:00 +CPU= 34.556
  18:31:00 STEP 1130 H=1130:00 +CPU= 35.745
  18:31:54 STEP 1131 H=1131:00 +CPU= 53.715
  18:32:30 STEP 1132 H=1132:00 +CPU= 35.654
  18:33:05 STEP 1133 H=1133:00 +CPU= 34.624
  18:34:01 STEP 1134 H=1134:00 +CPU= 55.262
  18:34:36 STEP 1135 H=1135:00 +CPU= 34.687
  18:35:13 STEP 1136 H=1136:00 +CPU= 35.855
  18:36:09 STEP 1137 H=1137:00 +CPU= 53.979
  18:36:45 STEP 1138 H=1138:00 +CPU= 35.311
  18:37:21 STEP 1139 H=1139:00 +CPU= 34.612
  18:38:22 STEP 1140 H=1140:00 +CPU= 59.906
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001140
  18:38:57 STEP 1141 H=1141:00 +CPU= 34.537
  18:39:33 STEP 1142 H=1142:00 +CPU= 35.638
  18:40:28 STEP 1143 H=1143:00 +CPU= 53.987
  18:41:04 STEP 1144 H=1144:00 +CPU= 35.930
  18:41:40 STEP 1145 H=1145:00 +CPU= 34.759
  18:42:36 STEP 1146 H=1146:00 +CPU= 55.428
  18:43:11 STEP 1147 H=1147:00 +CPU= 34.284
  18:43:48 STEP 1148 H=1148:00 +CPU= 35.619
  18:44:42 STEP 1149 H=1149:00 +CPU= 53.523
  18:45:19 STEP 1150 H=1150:00 +CPU= 35.729
  18:45:55 STEP 1151 H=1151:00 +CPU= 35.843
  18:47:01 STEP 1152 H=1152:00 +CPU= 63.169
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_47.zip
Uploading the intermediate file: upload_file_47.zip
  18:47:37 STEP 1153 H=1153:00 +CPU= 34.439
  18:48:13 STEP 1154 H=1154:00 +CPU= 35.712
  18:49:07 STEP 1155 H=1155:00 +CPU= 54.030
  18:49:43 STEP 1156 H=1156:00 +CPU= 35.494
  18:50:19 STEP 1157 H=1157:00 +CPU= 34.367
  18:51:15 STEP 1158 H=1158:00 +CPU= 54.822
  18:51:51 STEP 1159 H=1159:00 +CPU= 34.805
  18:52:28 STEP 1160 H=1160:00 +CPU= 35.605
  18:53:25 STEP 1161 H=1161:00 +CPU= 54.330
  18:54:02 STEP 1162 H=1162:00 +CPU= 35.694
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0469
(argv4) batchid: 984
(argv5) wuid: 12201113
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_d95a7af6e70c409e2631e82e552ea4f3 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0469_2015050100_123_984_12201113.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0469_2015050100_123_984_12201113.zip
ic_ancil_file: ic_ancil_12201113
ifsdata_file: ifsdata_12201113
climate_data_file: clim_data_12201113
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_22315c3977e7f92d0893fc294a21dfe8 to: /var/lib/boinc-client/slots/4/ic_ancil_12201113.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12201113.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_42b51d404c711653a6bf4a10f30db3e2 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12201113.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12201113.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12201113.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12201113.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12201113.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12201113.xml
-- Model is restarting --
Adjusting last_iter, 1162, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12201113.xml
last_cpu_time: 49136
upload_file_number: 48
last_iter: 1151
last_upload: 4147200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328
The child process has been launched with process id: 6268
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:torrents:1:1:6268:6268] [20230207:190011:1675825211:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:04:28 STEP 1152 H=1152:00 +CPU= 55.620
  19:06:02 STEP 1153 H=1153:00 +CPU= 33.668
  19:06:33 STEP 1154 H=1154:00 +CPU= 29.418
  19:07:27 STEP 1155 H=1155:00 +CPU= 53.840
  19:07:51 STEP 1156 H=1156:00 +CPU= 23.791
  19:08:16 STEP 1157 H=1157:00 +CPU= 24.661
  19:10:11 STEP 1158 H=1158:00 +CPU= 52.683
  19:11:35 STEP 1159 H=1159:00 +CPU= 35.798
  19:14:05 STEP 1160 H=1160:00 +CPU= 36.868
  19:15:06 STEP 1161 H=1161:00 +CPU= 56.056
  19:15:44 STEP 1162 H=1162:00 +CPU= 37.246
  19:16:21 STEP 1163 H=1163:00 +CPU= 36.214
  19:18:08 STEP 1164 H=1164:00 +CPU= 61.473
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001164
  19:18:46 STEP 1165 H=1165:00 +CPU= 35.956
  19:19:45 STEP 1166 H=1166:00 +CPU= 37.173
  19:20:40 STEP 1167 H=1167:00 +CPU= 50.885
  19:21:15 STEP 1168 H=1168:00 +CPU= 33.992
  19:21:51 STEP 1169 H=1169:00 +CPU= 35.226
  19:22:43 STEP 1170 H=1170:00 +CPU= 50.994
  19:23:19 STEP 1171 H=1171:00 +CPU= 34.959
  19:23:56 STEP 1172 H=1172:00 +CPU= 36.677
  19:25:22 STEP 1173 H=1173:00 +CPU= 54.561
  19:25:51 STEP 1174 H=1174:00 +CPU= 28.417
  19:26:29 STEP 1175 H=1175:00 +CPU= 36.323
  19:28:42 STEP 1176 H=1176:00 +CPU= 65.737
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_48.zip
Uploading the intermediate file: upload_file_48.zip
  19:29:18 STEP 1177 H=1177:00 +CPU= 34.396
  19:29:56 STEP 1178 H=1178:00 +CPU= 36.629
  19:30:51 STEP 1179 H=1179:00 +CPU= 53.217
  19:31:29 STEP 1180 H=1180:00 +CPU= 37.201
  19:32:07 STEP 1181 H=1181:00 +CPU= 36.096
  19:33:45 STEP 1182 H=1182:00 +CPU= 56.152
  19:34:17 STEP 1183 H=1183:00 +CPU= 32.264
  19:34:51 STEP 1184 H=1184:00 +CPU= 33.048
  19:35:45 STEP 1185 H=1185:00 +CPU= 52.757
  19:36:52 STEP 1186 H=1186:00 +CPU= 33.902
  19:37:26 STEP 1187 H=1187:00 +CPU= 33.238
  19:38:29 STEP 1188 H=1188:00 +CPU= 58.819
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001188
  19:39:03 STEP 1189 H=1189:00 +CPU= 33.423
  19:39:42 STEP 1190 H=1190:00 +CPU= 37.427
  19:40:39 STEP 1191 H=1191:00 +CPU= 55.547
  19:41:17 STEP 1192 H=1192:00 +CPU= 37.086
  19:41:54 STEP 1193 H=1193:00 +CPU= 36.247
  19:42:52 STEP 1194 H=1194:00 +CPU= 57.746
  19:43:26 STEP 1195 H=1195:00 +CPU= 32.494
  19:44:01 STEP 1196 H=1196:00 +CPU= 33.172
  19:44:53 STEP 1197 H=1197:00 +CPU= 51.646
  19:45:29 STEP 1198 H=1198:00 +CPU= 34.806
  19:46:07 STEP 1199 H=1199:00 +CPU= 37.237
  19:47:18 STEP 1200 H=1200:00 +CPU= 64.560
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_49.zip
Uploading the intermediate file: upload_file_49.zip
  19:47:53 STEP 1201 H=1201:00 +CPU= 34.160
  19:49:56 STEP 1202 H=1202:00 +CPU= 37.010
  19:50:52 STEP 1203 H=1203:00 +CPU= 52.527
  19:51:26 STEP 1204 H=1204:00 +CPU= 32.425
  19:52:00 STEP 1205 H=1205:00 +CPU= 33.528
  19:53:00 STEP 1206 H=1206:00 +CPU= 56.876
  19:53:34 STEP 1207 H=1207:00 +CPU= 32.816
  19:54:09 STEP 1208 H=1208:00 +CPU= 34.721
  19:55:07 STEP 1209 H=1209:00 +CPU= 55.521
  19:55:42 STEP 1210 H=1210:00 +CPU= 34.227
  19:56:18 STEP 1211 H=1211:00 +CPU= 35.681
  19:57:26 STEP 1212 H=1212:00 +CPU= 63.360
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001212
  19:58:02 STEP 1213 H=1213:00 +CPU= 36.165
  19:58:36 STEP 1214 H=1214:00 +CPU= 33.333
  19:59:32 STEP 1215 H=1215:00 +CPU= 54.631
  20:00:11 STEP 1216 H=1216:00 +CPU= 37.385
  20:00:49 STEP 1217 H=1217:00 +CPU= 36.105
  20:01:49 STEP 1218 H=1218:00 +CPU= 58.186
  20:02:24 STEP 1219 H=1219:00 +CPU= 35.159
  20:03:02 STEP 1220 H=1220:00 +CPU= 37.100
  20:03:59 STEP 1221 H=1221:00 +CPU= 53.738
  20:04:37 STEP 1222 H=1222:00 +CPU= 37.225
  20:05:14 STEP 1223 H=1223:00 +CPU= 36.113
  20:07:24 STEP 1224 H=1224:00 +CPU= 63.509
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_50.zip
Uploading the intermediate file: upload_file_50.zip
  20:08:02 STEP 1225 H=1225:00 +CPU= 33.361
  20:08:40 STEP 1226 H=1226:00 +CPU= 36.036
  20:09:33 STEP 1227 H=1227:00 +CPU= 51.934
  20:10:07 STEP 1228 H=1228:00 +CPU= 33.868
  20:10:46 STEP 1229 H=1229:00 +CPU= 36.292
  20:11:44 STEP 1230 H=1230:00 +CPU= 55.946
  20:12:22 STEP 1231 H=1231:00 +CPU= 35.745
  20:13:25 STEP 1232 H=1232:00 +CPU= 34.378
  20:14:51 STEP 1233 H=1233:00 +CPU= 51.886
  20:15:26 STEP 1234 H=1234:00 +CPU= 33.492
  20:15:59 STEP 1235 H=1235:00 +CPU= 32.172
  20:17:10 STEP 1236 H=1236:00 +CPU= 61.117
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001236
  20:17:47 STEP 1237 H=1237:00 +CPU= 36.060
  20:18:22 STEP 1238 H=1238:00 +CPU= 33.443
  20:19:19 STEP 1239 H=1239:00 +CPU= 55.801
  20:19:56 STEP 1240 H=1240:00 +CPU= 37.391
  20:20:32 STEP 1241 H=1241:00 +CPU= 35.766
  20:21:25 STEP 1242 H=1242:00 +CPU= 51.364
  20:21:59 STEP 1243 H=1243:00 +CPU= 33.365
  20:22:37 STEP 1244 H=1244:00 +CPU= 36.411
  20:23:31 STEP 1245 H=1245:00 +CPU= 53.077
  20:24:04 STEP 1246 H=1246:00 +CPU= 32.876
  20:24:41 STEP 1247 H=1247:00 +CPU= 34.809
  20:25:51 STEP 1248 H=1248:00 +CPU= 61.929
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_51.zip
Uploading the intermediate file: upload_file_51.zip
  20:26:26 STEP 1249 H=1249:00 +CPU= 32.917
  20:27:03 STEP 1250 H=1250:00 +CPU= 36.995
  20:28:01 STEP 1251 H=1251:00 +CPU= 56.265
  20:28:39 STEP 1252 H=1252:00 +CPU= 36.983
  20:29:15 STEP 1253 H=1253:00 +CPU= 35.408
  20:30:13 STEP 1254 H=1254:00 +CPU= 57.148
  20:30:45 STEP 1255 H=1255:00 +CPU= 31.902
  20:31:20 STEP 1256 H=1256:00 +CPU= 33.793
  20:32:17 STEP 1257 H=1257:00 +CPU= 55.427
  20:32:54 STEP 1258 H=1258:00 +CPU= 35.768
  20:33:30 STEP 1259 H=1259:00 +CPU= 35.302
  20:34:29 STEP 1260 H=1260:00 +CPU= 57.084
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001260
  20:35:03 STEP 1261 H=1261:00 +CPU= 33.104
  20:35:39 STEP 1262 H=1262:00 +CPU= 34.105
  20:36:32 STEP 1263 H=1263:00 +CPU= 51.315
  20:37:09 STEP 1264 H=1264:00 +CPU= 35.975
  20:37:42 STEP 1265 H=1265:00 +CPU= 32.125
  20:38:40 STEP 1266 H=1266:00 +CPU= 56.659
  20:39:16 STEP 1267 H=1267:00 +CPU= 35.824
  20:39:54 STEP 1268 H=1268:00 +CPU= 37.106
  20:40:50 STEP 1269 H=1269:00 +CPU= 56.305
  20:41:32 STEP 1270 H=1270:00 +CPU= 35.049
  20:42:07 STEP 1271 H=1271:00 +CPU= 34.767
  20:45:43 STEP 1272 H=1272:00 +CPU= 65.436
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_52.zip
Uploading the intermediate file: upload_file_52.zip
  20:46:26 STEP 1273 H=1273:00 +CPU= 33.491
  20:47:53 STEP 1274 H=1274:00 +CPU= 31.218
  20:48:53 STEP 1275 H=1275:00 +CPU= 53.907
  20:49:31 STEP 1276 H=1276:00 +CPU= 37.197
  20:50:04 STEP 1277 H=1277:00 +CPU= 32.216
  20:50:57 STEP 1278 H=1278:00 +CPU= 51.677
  20:51:30 STEP 1279 H=1279:00 +CPU= 32.537
  20:52:04 STEP 1280 H=1280:00 +CPU= 33.094
  20:52:59 STEP 1281 H=1281:00 +CPU= 54.346
  20:53:37 STEP 1282 H=1282:00 +CPU= 36.594
  20:54:15 STEP 1283 H=1283:00 +CPU= 36.205
  20:55:20 STEP 1284 H=1284:00 +CPU= 62.487
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001284
  20:55:57 STEP 1285 H=1285:00 +CPU= 36.200
  20:57:52 STEP 1286 H=1286:00 +CPU= 37.277
  20:58:54 STEP 1287 H=1287:00 +CPU= 54.907
  20:59:27 STEP 1288 H=1288:00 +CPU= 33.357
  20:59:57 STEP 1289 H=1289:00 +CPU= 29.153
  21:00:54 STEP 1290 H=1290:00 +CPU= 56.006
  21:01:17 STEP 1291 H=1291:00 +CPU= 22.689
  21:01:54 STEP 1292 H=1292:00 +CPU= 36.797
  21:02:57 STEP 1293 H=1293:00 +CPU= 55.791
  21:03:36 STEP 1294 H=1294:00 +CPU= 37.227
  21:04:12 STEP 1295 H=1295:00 +CPU= 33.801
  21:05:28 STEP 1296 H=1296:00 +CPU= 63.563
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_53.zip
Uploading the intermediate file: upload_file_53.zip
  21:06:04 STEP 1297 H=1297:00 +CPU= 34.577
  21:06:42 STEP 1298 H=1298:00 +CPU= 35.590
  21:07:38 STEP 1299 H=1299:00 +CPU= 54.307
  21:08:14 STEP 1300 H=1300:00 +CPU= 33.590
  21:08:50 STEP 1301 H=1301:00 +CPU= 34.198
  21:09:48 STEP 1302 H=1302:00 +CPU= 57.384
  21:10:21 STEP 1303 H=1303:00 +CPU= 33.211
  21:10:55 STEP 1304 H=1304:00 +CPU= 32.883
  21:11:45 STEP 1305 H=1305:00 +CPU= 49.981
  21:12:22 STEP 1306 H=1306:00 +CPU= 36.423
  21:12:55 STEP 1307 H=1307:00 +CPU= 32.093
  21:13:57 STEP 1308 H=1308:00 +CPU= 59.882
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001308
  21:14:33 STEP 1309 H=1309:00 +CPU= 35.009
  21:15:10 STEP 1310 H=1310:00 +CPU= 37.040
  21:16:05 STEP 1311 H=1311:00 +CPU= 53.458
  21:16:41 STEP 1312 H=1312:00 +CPU= 35.743
  21:17:15 STEP 1313 H=1313:00 +CPU= 32.869
  21:18:10 STEP 1314 H=1314:00 +CPU= 53.559
  21:18:46 STEP 1315 H=1315:00 +CPU= 35.277
  21:19:24 STEP 1316 H=1316:00 +CPU= 36.874
  21:20:21 STEP 1317 H=1317:00 +CPU= 55.709
  21:20:57 STEP 1318 H=1318:00 +CPU= 35.049
  21:21:31 STEP 1319 H=1319:00 +CPU= 32.900
  21:23:10 STEP 1320 H=1320:00 +CPU= 60.890
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_54.zip
Uploading the intermediate file: upload_file_54.zip
  21:23:46 STEP 1321 H=1321:00 +CPU= 34.777
  21:24:24 STEP 1322 H=1322:00 +CPU= 37.104
  21:26:42 STEP 1323 H=1323:00 +CPU= 56.015
  21:27:20 STEP 1324 H=1324:00 +CPU= 35.665
  21:27:55 STEP 1325 H=1325:00 +CPU= 34.082
  21:28:51 STEP 1326 H=1326:00 +CPU= 54.015
  21:29:29 STEP 1327 H=1327:00 +CPU= 35.218
  21:30:05 STEP 1328 H=1328:00 +CPU= 33.642
  21:31:01 STEP 1329 H=1329:00 +CPU= 55.014
  21:31:38 STEP 1330 H=1330:00 +CPU= 36.815
  21:32:15 STEP 1331 H=1331:00 +CPU= 35.667
  21:33:17 STEP 1332 H=1332:00 +CPU= 60.697
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001332
  21:35:00 STEP 1333 H=1333:00 +CPU= 34.298
  21:35:37 STEP 1334 H=1334:00 +CPU= 35.777
  21:36:23 STEP 1335 H=1335:00 +CPU= 44.884
  21:36:57 STEP 1336 H=1336:00 +CPU= 32.304
  21:37:32 STEP 1337 H=1337:00 +CPU= 33.630
  21:38:31 STEP 1338 H=1338:00 +CPU= 56.947
  21:39:06 STEP 1339 H=1339:00 +CPU= 34.857
  21:39:43 STEP 1340 H=1340:00 +CPU= 36.668
  21:40:38 STEP 1341 H=1341:00 +CPU= 54.356
  21:41:13 STEP 1342 H=1342:00 +CPU= 34.773
  21:41:51 STEP 1343 H=1343:00 +CPU= 37.115
  21:43:40 STEP 1344 H=1344:00 +CPU= 64.529
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_55.zip
Uploading the intermediate file: upload_file_55.zip
  21:44:18 STEP 1345 H=1345:00 +CPU= 35.301
  21:44:55 STEP 1346 H=1346:00 +CPU= 34.916
  21:45:52 STEP 1347 H=1347:00 +CPU= 54.780
  21:46:29 STEP 1348 H=1348:00 +CPU= 34.450
  21:48:01 STEP 1349 H=1349:00 +CPU= 35.756
  21:49:16 STEP 1350 H=1350:00 +CPU= 54.625
  21:49:53 STEP 1351 H=1351:00 +CPU= 34.584
  21:50:31 STEP 1352 H=1352:00 +CPU= 36.876
  21:51:29 STEP 1353 H=1353:00 +CPU= 55.714
  21:52:09 STEP 1354 H=1354:00 +CPU= 37.213
  21:52:45 STEP 1355 H=1355:00 +CPU= 35.234
  21:53:51 STEP 1356 H=1356:00 +CPU= 61.922
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001356
  21:54:29 STEP 1357 H=1357:00 +CPU= 36.138
  21:55:07 STEP 1358 H=1358:00 +CPU= 37.099
  21:56:04 STEP 1359 H=1359:00 +CPU= 55.997
  21:56:39 STEP 1360 H=1360:00 +CPU= 34.846
  21:57:13 STEP 1361 H=1361:00 +CPU= 31.714
  21:58:05 STEP 1362 H=1362:00 +CPU= 50.523
  21:58:37 STEP 1363 H=1363:00 +CPU= 31.334
  21:59:11 STEP 1364 H=1364:00 +CPU= 32.391
  22:01:34 STEP 1365 H=1365:00 +CPU= 54.992
  22:02:15 STEP 1366 H=1366:00 +CPU= 34.338
  22:02:50 STEP 1367 H=1367:00 +CPU= 33.477
  22:04:09 STEP 1368 H=1368:00 +CPU= 58.266
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_56.zip
  22:05:20 STEP 1369 H=1369:00 +CPU= 30.576
Uploading the intermediate file: upload_file_56.zip
  22:06:03 STEP 1370 H=1370:00 +CPU= 33.037
  22:06:39 STEP 1371 H=1371:00 +CPU= 35.753
  22:07:02 STEP 1372 H=1372:00 +CPU= 23.028
  22:07:38 STEP 1373 H=1373:00 +CPU= 34.418
  22:08:28 STEP 1374 H=1374:00 +CPU= 50.582
  22:09:11 STEP 1375 H=1375:00 +CPU= 32.961
  22:11:56 STEP 1376 H=1376:00 +CPU= 37.391
  22:12:58 STEP 1377 H=1377:00 +CPU= 53.355
  22:13:31 STEP 1378 H=1378:00 +CPU= 33.477
  22:14:08 STEP 1379 H=1379:00 +CPU= 35.287
  22:15:13 STEP 1380 H=1380:00 +CPU= 62.500
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001380
  22:15:50 STEP 1381 H=1381:00 +CPU= 35.920
  22:16:28 STEP 1382 H=1382:00 +CPU= 36.703
  22:17:25 STEP 1383 H=1383:00 +CPU= 56.104
  22:18:02 STEP 1384 H=1384:00 +CPU= 36.026
  22:20:01 STEP 1385 H=1385:00 +CPU= 35.080
  22:21:26 STEP 1386 H=1386:00 +CPU= 53.606
  22:22:05 STEP 1387 H=1387:00 +CPU= 31.590
  22:22:28 STEP 1388 H=1388:00 +CPU= 22.347
  22:23:14 STEP 1389 H=1389:00 +CPU= 45.204
  22:23:55 STEP 1390 H=1390:00 +CPU= 36.685
  22:24:30 STEP 1391 H=1391:00 +CPU= 32.907
  22:26:27 STEP 1392 H=1392:00 +CPU= 64.072
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_57.zip
Uploading the intermediate file: upload_file_57.zip
Uploading trickle at timestep: 5011200
  22:27:04 STEP 1393 H=1393:00 +CPU= 34.955
  22:27:37 STEP 1394 H=1394:00 +CPU= 32.356
  22:28:34 STEP 1395 H=1395:00 +CPU= 54.926
  22:29:10 STEP 1396 H=1396:00 +CPU= 34.457
  22:29:44 STEP 1397 H=1397:00 +CPU= 31.793
  22:30:37 STEP 1398 H=1398:00 +CPU= 51.633
  22:31:12 STEP 1399 H=1399:00 +CPU= 35.476
  22:32:13 STEP 1400 H=1400:00 +CPU= 37.057
  22:33:07 STEP 1401 H=1401:00 +CPU= 52.806
  22:33:40 STEP 1402 H=1402:00 +CPU= 32.465
  22:34:17 STEP 1403 H=1403:00 +CPU= 33.974
  22:37:03 STEP 1404 H=1404:00 +CPU= 58.346
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001404
  22:37:40 STEP 1405 H=1405:00 +CPU= 35.279
  22:38:14 STEP 1406 H=1406:00 +CPU= 32.714
  22:39:08 STEP 1407 H=1407:00 +CPU= 51.141
  22:39:44 STEP 1408 H=1408:00 +CPU= 34.806
  22:40:20 STEP 1409 H=1409:00 +CPU= 35.248
  22:41:18 STEP 1410 H=1410:00 +CPU= 56.841
  22:42:12 STEP 1411 H=1411:00 +CPU= 33.344
  22:42:58 STEP 1412 H=1412:00 +CPU= 35.380
  22:43:55 STEP 1413 H=1413:00 +CPU= 54.117
  22:44:30 STEP 1414 H=1414:00 +CPU= 32.985
  22:45:08 STEP 1415 H=1415:00 +CPU= 36.789
  22:46:26 STEP 1416 H=1416:00 +CPU= 61.625
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_58.zip
Uploading the intermediate file: upload_file_58.zip
  22:47:02 STEP 1417 H=1417:00 +CPU= 34.245
  22:47:40 STEP 1418 H=1418:00 +CPU= 36.734
  22:48:57 STEP 1419 H=1419:00 +CPU= 52.724
  22:49:43 STEP 1420 H=1420:00 +CPU= 36.078
  22:50:20 STEP 1421 H=1421:00 +CPU= 35.717
  22:51:12 STEP 1422 H=1422:00 +CPU= 50.854
  22:51:46 STEP 1423 H=1423:00 +CPU= 33.965
  22:57:17 STEP 1424 H=1424:00 +CPU= 38.647
  22:58:13 STEP 1425 H=1425:00 +CPU= 53.792
  22:59:25 STEP 1426 H=1426:00 +CPU= 35.633
  23:00:02 STEP 1427 H=1427:00 +CPU= 34.885
  23:01:06 STEP 1428 H=1428:00 +CPU= 59.932
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001428
  23:01:44 STEP 1429 H=1429:00 +CPU= 35.885
  23:02:20 STEP 1430 H=1430:00 +CPU= 34.496
  23:04:24 STEP 1431 H=1431:00 +CPU= 54.513
  23:05:00 STEP 1432 H=1432:00 +CPU= 34.418
  23:05:25 STEP 1433 H=1433:00 +CPU= 24.707
  23:06:23 STEP 1434 H=1434:00 +CPU= 56.923
  23:07:01 STEP 1435 H=1435:00 +CPU= 35.754
  23:07:36 STEP 1436 H=1436:00 +CPU= 34.451
  23:08:29 STEP 1437 H=1437:00 +CPU= 51.363
  23:09:04 STEP 1438 H=1438:00 +CPU= 32.776
  23:09:39 STEP 1439 H=1439:00 +CPU= 32.842
  23:10:58 STEP 1440 H=1440:00 +CPU= 65.123
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_59.zip
Uploading the intermediate file: upload_file_59.zip
  23:11:35 STEP 1441 H=1441:00 +CPU= 35.319
  23:12:12 STEP 1442 H=1442:00 +CPU= 35.484
  23:13:06 STEP 1443 H=1443:00 +CPU= 51.431
  23:13:43 STEP 1444 H=1444:00 +CPU= 36.270
  23:14:18 STEP 1445 H=1445:00 +CPU= 33.037
  23:15:10 STEP 1446 H=1446:00 +CPU= 50.231
  23:15:45 STEP 1447 H=1447:00 +CPU= 34.218
  23:16:23 STEP 1448 H=1448:00 +CPU= 36.169
  23:17:19 STEP 1449 H=1449:00 +CPU= 53.921
  23:17:54 STEP 1450 H=1450:00 +CPU= 33.412
  23:18:31 STEP 1451 H=1451:00 +CPU= 35.381
  23:19:36 STEP 1452 H=1452:00 +CPU= 62.000
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001452
  23:20:11 STEP 1453 H=1453:00 +CPU= 35.514
  23:20:49 STEP 1454 H=1454:00 +CPU= 35.819
  23:21:42 STEP 1455 H=1455:00 +CPU= 51.258
  23:22:16 STEP 1456 H=1456:00 +CPU= 33.046
  23:22:50 STEP 1457 H=1457:00 +CPU= 33.183
  23:23:49 STEP 1458 H=1458:00 +CPU= 56.557
  23:24:26 STEP 1459 H=1459:00 +CPU= 35.096
  23:24:59 STEP 1460 H=1460:00 +CPU= 32.909
  23:25:50 STEP 1461 H=1461:00 +CPU= 49.884
  23:26:26 STEP 1462 H=1462:00 +CPU= 34.330
  23:27:04 STEP 1463 H=1463:00 +CPU= 35.529
  23:28:33 STEP 1464 H=1464:00 +CPU= 56.007
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_60.zip
Uploading the intermediate file: upload_file_60.zip
  23:29:17 STEP 1465 H=1465:00 +CPU= 33.578
  23:29:46 STEP 1466 H=1466:00 +CPU= 28.721
  23:30:39 STEP 1467 H=1467:00 +CPU= 53.136
  23:31:16 STEP 1468 H=1468:00 +CPU= 36.010
  23:31:51 STEP 1469 H=1469:00 +CPU= 35.274
  23:32:48 STEP 1470 H=1470:00 +CPU= 55.454
  23:33:26 STEP 1471 H=1471:00 +CPU= 35.420
  23:34:02 STEP 1472 H=1472:00 +CPU= 34.869
  23:35:00 STEP 1473 H=1473:00 +CPU= 55.433
  23:35:37 STEP 1474 H=1474:00 +CPU= 35.820
  23:36:12 STEP 1475 H=1475:00 +CPU= 32.584
  23:39:16 STEP 1476 H=1476:00 +CPU= 60.271
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001476
  23:39:52 STEP 1477 H=1477:00 +CPU= 33.651
  23:40:26 STEP 1478 H=1478:00 +CPU= 32.453
  23:41:20 STEP 1479 H=1479:00 +CPU= 53.062
  23:41:57 STEP 1480 H=1480:00 +CPU= 36.327
  23:42:33 STEP 1481 H=1481:00 +CPU= 34.856
  23:43:30 STEP 1482 H=1482:00 +CPU= 54.698
  23:44:02 STEP 1483 H=1483:00 +CPU= 31.750
  23:44:40 STEP 1484 H=1484:00 +CPU= 36.698
  23:45:31 STEP 1485 H=1485:00 +CPU= 50.340
  23:46:08 STEP 1486 H=1486:00 +CPU= 36.209
  23:46:45 STEP 1487 H=1487:00 +CPU= 36.900
  23:47:55 STEP 1488 H=1488:00 +CPU= 59.571
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_61.zip
Uploading the intermediate file: upload_file_61.zip
  23:48:32 STEP 1489 H=1489:00 +CPU= 34.948
  23:50:04 STEP 1490 H=1490:00 +CPU= 35.569
  23:51:02 STEP 1491 H=1491:00 +CPU= 54.409
  23:51:40 STEP 1492 H=1492:00 +CPU= 36.932
  23:52:17 STEP 1493 H=1493:00 +CPU= 35.236
  23:53:16 STEP 1494 H=1494:00 +CPU= 57.003
  23:53:52 STEP 1495 H=1495:00 +CPU= 34.618
  23:54:28 STEP 1496 H=1496:00 +CPU= 35.110
  23:55:24 STEP 1497 H=1497:00 +CPU= 54.950
  23:56:02 STEP 1498 H=1498:00 +CPU= 36.743
  23:56:36 STEP 1499 H=1499:00 +CPU= 32.712
  23:57:36 STEP 1500 H=1500:00 +CPU= 58.375
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001500
  23:58:10 STEP 1501 H=1501:00 +CPU= 33.481
  23:58:47 STEP 1502 H=1502:00 +CPU= 36.508
  23:59:43 STEP 1503 H=1503:00 +CPU= 55.336
  00:00:21 STEP 1504 H=1504:00 +CPU= 33.691
  00:03:44 STEP 1505 H=1505:00 +CPU= 36.945
  00:05:01 STEP 1506 H=1506:00 +CPU= 55.393
  00:05:34 STEP 1507 H=1507:00 +CPU= 33.108
  00:06:09 STEP 1508 H=1508:00 +CPU= 34.552
  00:06:56 STEP 1509 H=1509:00 +CPU= 46.511
  00:11:32 STEP 1510 H=1510:00 +CPU= 36.158
  00:12:21 STEP 1511 H=1511:00 +CPU= 36.556
  00:13:45 STEP 1512 H=1512:00 +CPU= 64.676
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_62.zip
Uploading the intermediate file: upload_file_62.zip
  00:15:05 STEP 1513 H=1513:00 +CPU= 35.139
  00:15:47 STEP 1514 H=1514:00 +CPU= 35.732
  00:16:41 STEP 1515 H=1515:00 +CPU= 51.150
  00:17:14 STEP 1516 H=1516:00 +CPU= 32.479
  00:17:49 STEP 1517 H=1517:00 +CPU= 33.841
  00:18:44 STEP 1518 H=1518:00 +CPU= 55.033
  00:19:20 STEP 1519 H=1519:00 +CPU= 35.616
  00:19:57 STEP 1520 H=1520:00 +CPU= 36.400
  00:20:54 STEP 1521 H=1521:00 +CPU= 55.550
  00:21:28 STEP 1522 H=1522:00 +CPU= 33.846
  00:22:02 STEP 1523 H=1523:00 +CPU= 33.277
  00:23:06 STEP 1524 H=1524:00 +CPU= 61.930
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001524
  00:23:42 STEP 1525 H=1525:00 +CPU= 35.698
  00:24:17 STEP 1526 H=1526:00 +CPU= 34.317
  00:25:13 STEP 1527 H=1527:00 +CPU= 54.424
  00:25:51 STEP 1528 H=1528:00 +CPU= 36.745
  00:26:27 STEP 1529 H=1529:00 +CPU= 35.543
  00:27:23 STEP 1530 H=1530:00 +CPU= 54.760
  00:27:59 STEP 1531 H=1531:00 +CPU= 35.151
  00:28:36 STEP 1532 H=1532:00 +CPU= 36.246
  00:29:31 STEP 1533 H=1533:00 +CPU= 53.649
  00:30:07 STEP 1534 H=1534:00 +CPU= 35.096
  00:30:45 STEP 1535 H=1535:00 +CPU= 37.178
  00:31:52 STEP 1536 H=1536:00 +CPU= 59.232
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_63.zip
Uploading the intermediate file: upload_file_63.zip
  00:32:26 STEP 1537 H=1537:00 +CPU= 32.770
  00:32:59 STEP 1538 H=1538:00 +CPU= 32.530
  00:33:49 STEP 1539 H=1539:00 +CPU= 49.259
  00:34:22 STEP 1540 H=1540:00 +CPU= 32.563
  00:34:56 STEP 1541 H=1541:00 +CPU= 34.442
  00:36:49 STEP 1542 H=1542:00 +CPU= 56.841
  00:38:06 STEP 1543 H=1543:00 +CPU= 34.407
  00:38:34 STEP 1544 H=1544:00 +CPU= 27.738
  00:39:27 STEP 1545 H=1545:00 +CPU= 50.816
  00:39:54 STEP 1546 H=1546:00 +CPU= 27.083
  00:40:27 STEP 1547 H=1547:00 +CPU= 31.648
  00:41:28 STEP 1548 H=1548:00 +CPU= 56.057
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001548
  00:43:56 STEP 1549 H=1549:00 +CPU= 35.272
  00:44:30 STEP 1550 H=1550:00 +CPU= 32.591
  00:45:30 STEP 1551 H=1551:00 +CPU= 55.410
  00:46:08 STEP 1552 H=1552:00 +CPU= 36.904
  00:46:46 STEP 1553 H=1553:00 +CPU= 35.975
  00:47:45 STEP 1554 H=1554:00 +CPU= 57.255
  00:48:18 STEP 1555 H=1555:00 +CPU= 32.502
  00:48:51 STEP 1556 H=1556:00 +CPU= 32.595
  00:49:43 STEP 1557 H=1557:00 +CPU= 50.176
  00:50:20 STEP 1558 H=1558:00 +CPU= 37.007
  00:50:54 STEP 1559 H=1559:00 +CPU= 30.134
  00:52:16 STEP 1560 H=1560:00 +CPU= 64.493
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201113/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2015050100_123_984_12201113_1_r1285354328_64.zip
Uploading the intermediate file: upload_file_64.zip
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:49:40 1524359 22306480 oifs_43r3_ps_0469_2015050100_123_984_12201113_1 5,011,200 59,176 0.0118
09 May 2023 16:49:31 1524359 22306480 oifs_43r3_ps_0469_2015050100_123_984_12201113_1 3,456,000 40,639 0.0118
09 May 2023 16:49:27 1524359 22306480 oifs_43r3_ps_0469_2015050100_123_984_12201113_1 2,592,000 30,455 0.0117
09 May 2023 16:49:24 1524359 22306480 oifs_43r3_ps_0469_2015050100_123_984_12201113_1 1,728,000 20,312 0.0118
09 May 2023 16:49:19 1524359 22306480 oifs_43r3_ps_0469_2015050100_123_984_12201113_1 864,000 10,188 0.0118


©2024 climateprediction.net