climateprediction.net home page
Task 22279310

Task 22279310

Name oifs_43r3_ps_0873_2007050100_123_976_12193517_0
Workunit 12193517
Created 23 Dec 2022, 16:56:32 UTC
Sent 1 Jan 2023, 16:59:11 UTC
Report deadline 31 Jan 2023, 16:59:11 UTC
Received 2 Jan 2023, 14:23:33 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1514495
Run time 10 hours 50 min 41 sec
CPU time 10 hours 40 min 21 sec
Validate state Invalid
Credit 6,048.00
Device peak FLOPS 5.68 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.24 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,713.09 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
20.934
  11:56:47 STEP 1013 H=1013:00 +CPU= 20.095
  11:57:20 STEP 1014 H=1014:00 +CPU= 32.546
  11:57:41 STEP 1015 H=1015:00 +CPU= 20.768
  11:58:04 STEP 1016 H=1016:00 +CPU= 21.541
  11:58:36 STEP 1017 H=1017:00 +CPU= 32.259
  11:58:58 STEP 1018 H=1018:00 +CPU= 20.970
  11:59:18 STEP 1019 H=1019:00 +CPU= 20.385
  11:59:54 STEP 1020 H=1020:00 +CPU= 35.857
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001020
  12:00:14 STEP 1021 H=1021:00 +CPU= 20.177
  12:00:35 STEP 1022 H=1022:00 +CPU= 20.999
  12:01:08 STEP 1023 H=1023:00 +CPU= 32.156
  12:01:29 STEP 1024 H=1024:00 +CPU= 21.270
  12:01:50 STEP 1025 H=1025:00 +CPU= 20.422
  12:02:22 STEP 1026 H=1026:00 +CPU= 32.729
  12:02:43 STEP 1027 H=1027:00 +CPU= 20.545
  12:03:04 STEP 1028 H=1028:00 +CPU= 21.059
  12:03:36 STEP 1029 H=1029:00 +CPU= 31.929
  12:03:57 STEP 1030 H=1030:00 +CPU= 20.847
  12:04:18 STEP 1031 H=1031:00 +CPU= 20.815
  12:04:56 STEP 1032 H=1032:00 +CPU= 37.928
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_42.zip
Uploading the intermediate file: upload_file_42.zip
  12:05:16 STEP 1033 H=1033:00 +CPU= 20.323
  12:05:37 STEP 1034 H=1034:00 +CPU= 20.811
  12:06:09 STEP 1035 H=1035:00 +CPU= 31.744
  12:06:30 STEP 1036 H=1036:00 +CPU= 20.887
  12:06:50 STEP 1037 H=1037:00 +CPU= 20.316
  12:07:23 STEP 1038 H=1038:00 +CPU= 32.608
  12:07:43 STEP 1039 H=1039:00 +CPU= 20.244
  12:08:04 STEP 1040 H=1040:00 +CPU= 20.968
  12:08:36 STEP 1041 H=1041:00 +CPU= 31.826
  12:08:57 STEP 1042 H=1042:00 +CPU= 21.147
  12:09:18 STEP 1043 H=1043:00 +CPU= 20.196
  12:09:54 STEP 1044 H=1044:00 +CPU= 36.053
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001044
  12:10:15 STEP 1045 H=1045:00 +CPU= 20.522
  12:10:36 STEP 1046 H=1046:00 +CPU= 20.768
  12:11:07 STEP 1047 H=1047:00 +CPU= 31.646
  12:11:28 STEP 1048 H=1048:00 +CPU= 21.093
  12:11:49 STEP 1049 H=1049:00 +CPU= 20.263
  12:12:22 STEP 1050 H=1050:00 +CPU= 32.521
  12:12:43 STEP 1051 H=1051:00 +CPU= 20.255
  12:13:04 STEP 1052 H=1052:00 +CPU= 21.005
  12:13:36 STEP 1053 H=1053:00 +CPU= 31.970
  12:13:57 STEP 1054 H=1054:00 +CPU= 21.078
  12:14:18 STEP 1055 H=1055:00 +CPU= 20.970
  12:14:56 STEP 1056 H=1056:00 +CPU= 38.169
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_43.zip
Uploading the intermediate file: upload_file_43.zip
  12:15:17 STEP 1057 H=1057:00 +CPU= 20.137
  12:15:38 STEP 1058 H=1058:00 +CPU= 21.006
  12:16:10 STEP 1059 H=1059:00 +CPU= 31.943
  12:16:31 STEP 1060 H=1060:00 +CPU= 20.956
  12:16:51 STEP 1061 H=1061:00 +CPU= 20.344
  12:17:24 STEP 1062 H=1062:00 +CPU= 32.821
  12:17:45 STEP 1063 H=1063:00 +CPU= 20.399
  12:18:06 STEP 1064 H=1064:00 +CPU= 21.044
  12:18:38 STEP 1065 H=1065:00 +CPU= 31.800
  12:18:59 STEP 1066 H=1066:00 +CPU= 20.995
  12:19:19 STEP 1067 H=1067:00 +CPU= 20.333
  12:19:55 STEP 1068 H=1068:00 +CPU= 35.747
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001068
  12:20:15 STEP 1069 H=1069:00 +CPU= 20.347
  12:20:36 STEP 1070 H=1070:00 +CPU= 20.857
  12:21:08 STEP 1071 H=1071:00 +CPU= 31.706
  12:21:29 STEP 1072 H=1072:00 +CPU= 20.890
  12:21:49 STEP 1073 H=1073:00 +CPU= 20.337
  12:22:22 STEP 1074 H=1074:00 +CPU= 32.792
  12:22:42 STEP 1075 H=1075:00 +CPU= 20.221
  12:23:03 STEP 1076 H=1076:00 +CPU= 20.943
  12:23:35 STEP 1077 H=1077:00 +CPU= 31.867
  12:23:56 STEP 1078 H=1078:00 +CPU= 20.964
  12:24:17 STEP 1079 H=1079:00 +CPU= 20.985
  12:24:56 STEP 1080 H=1080:00 +CPU= 38.328
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001068
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_44.zip
Uploading the intermediate file: upload_file_44.zip
  12:25:16 STEP 1081 H=1081:00 +CPU= 20.337
  12:25:38 STEP 1082 H=1082:00 +CPU= 21.090
  12:26:10 STEP 1083 H=1083:00 +CPU= 32.063
  12:26:31 STEP 1084 H=1084:00 +CPU= 21.026
  12:26:51 STEP 1085 H=1085:00 +CPU= 20.302
  12:27:25 STEP 1086 H=1086:00 +CPU= 33.256
  12:27:44 STEP 1087 H=1087:00 +CPU= 19.250
  12:28:04 STEP 1088 H=1088:00 +CPU= 20.302
  12:28:35 STEP 1089 H=1089:00 +CPU= 30.822
  12:28:55 STEP 1090 H=1090:00 +CPU= 19.712
  12:29:15 STEP 1091 H=1091:00 +CPU= 19.479
  12:29:48 STEP 1092 H=1092:00 +CPU= 33.557
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001092
  12:30:08 STEP 1093 H=1093:00 +CPU= 19.553
  12:30:30 STEP 1094 H=1094:00 +CPU= 21.637
  12:31:02 STEP 1095 H=1095:00 +CPU= 32.161
  12:31:23 STEP 1096 H=1096:00 +CPU= 21.125
  12:31:44 STEP 1097 H=1097:00 +CPU= 20.487
  12:32:18 STEP 1098 H=1098:00 +CPU= 33.704
  12:32:39 STEP 1099 H=1099:00 +CPU= 21.298
  12:33:00 STEP 1100 H=1100:00 +CPU= 21.202
  12:33:34 STEP 1101 H=1101:00 +CPU= 33.601
  12:33:56 STEP 1102 H=1102:00 +CPU= 22.016
  12:34:18 STEP 1103 H=1103:00 +CPU= 21.479
  12:34:59 STEP 1104 H=1104:00 +CPU= 40.769
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001092
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_45.zip
Uploading the intermediate file: upload_file_45.zip
  12:35:21 STEP 1105 H=1105:00 +CPU= 21.587
  12:35:42 STEP 1106 H=1106:00 +CPU= 21.435
  12:36:15 STEP 1107 H=1107:00 +CPU= 32.423
  12:36:36 STEP 1108 H=1108:00 +CPU= 21.232
  12:36:57 STEP 1109 H=1109:00 +CPU= 20.810
  12:37:30 STEP 1110 H=1110:00 +CPU= 32.862
  12:37:51 STEP 1111 H=1111:00 +CPU= 21.196
  12:38:13 STEP 1112 H=1112:00 +CPU= 21.145
  12:38:45 STEP 1113 H=1113:00 +CPU= 32.342
  12:39:07 STEP 1114 H=1114:00 +CPU= 22.377
  12:39:29 STEP 1115 H=1115:00 +CPU= 21.443
  12:40:07 STEP 1116 H=1116:00 +CPU= 37.629
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001116
  12:40:28 STEP 1117 H=1117:00 +CPU= 21.116
  12:40:49 STEP 1118 H=1118:00 +CPU= 21.309
  12:41:22 STEP 1119 H=1119:00 +CPU= 32.804
  12:41:44 STEP 1120 H=1120:00 +CPU= 22.080
  12:42:05 STEP 1121 H=1121:00 +CPU= 21.050
  12:42:40 STEP 1122 H=1122:00 +CPU= 33.989
  12:43:02 STEP 1123 H=1123:00 +CPU= 22.246
  12:43:24 STEP 1124 H=1124:00 +CPU= 22.366
  12:43:58 STEP 1125 H=1125:00 +CPU= 33.739
  12:44:20 STEP 1126 H=1126:00 +CPU= 21.868
  12:44:42 STEP 1127 H=1127:00 +CPU= 21.381
  12:45:22 STEP 1128 H=1128:00 +CPU= 40.108
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_46.zip
Uploading the intermediate file: upload_file_46.zip
  12:45:44 STEP 1129 H=1129:00 +CPU= 21.690
  12:46:06 STEP 1130 H=1130:00 +CPU= 22.109
  12:46:39 STEP 1131 H=1131:00 +CPU= 32.990
  12:47:01 STEP 1132 H=1132:00 +CPU= 22.084
  12:47:23 STEP 1133 H=1133:00 +CPU= 21.352
  12:47:57 STEP 1134 H=1134:00 +CPU= 33.447
  12:48:17 STEP 1135 H=1135:00 +CPU= 20.653
  12:48:40 STEP 1136 H=1136:00 +CPU= 22.428
  12:49:13 STEP 1137 H=1137:00 +CPU= 32.777
  12:49:36 STEP 1138 H=1138:00 +CPU= 22.623
  12:49:57 STEP 1139 H=1139:00 +CPU= 21.716
  12:50:36 STEP 1140 H=1140:00 +CPU= 38.665
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001140
  12:50:58 STEP 1141 H=1141:00 +CPU= 21.433
  12:51:21 STEP 1142 H=1142:00 +CPU= 21.707
  12:51:55 STEP 1143 H=1143:00 +CPU= 33.295
  12:52:17 STEP 1144 H=1144:00 +CPU= 22.244
  12:52:39 STEP 1145 H=1145:00 +CPU= 21.673
  12:53:13 STEP 1146 H=1146:00 +CPU= 34.236
  12:53:34 STEP 1147 H=1147:00 +CPU= 20.708
  12:53:55 STEP 1148 H=1148:00 +CPU= 21.190
  12:54:29 STEP 1149 H=1149:00 +CPU= 33.180
  12:54:51 STEP 1150 H=1150:00 +CPU= 22.115
  12:55:12 STEP 1151 H=1151:00 +CPU= 21.378
  12:55:52 STEP 1152 H=1152:00 +CPU= 39.250
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_47.zip
Uploading the intermediate file: upload_file_47.zip
  12:56:14 STEP 1153 H=1153:00 +CPU= 21.830
  12:56:36 STEP 1154 H=1154:00 +CPU= 22.418
  12:57:10 STEP 1155 H=1155:00 +CPU= 33.726
  12:57:32 STEP 1156 H=1156:00 +CPU= 22.043
  12:57:54 STEP 1157 H=1157:00 +CPU= 21.514
  12:58:28 STEP 1158 H=1158:00 +CPU= 34.011
  12:58:49 STEP 1159 H=1159:00 +CPU= 20.611
  12:59:10 STEP 1160 H=1160:00 +CPU= 21.344
  12:59:43 STEP 1161 H=1161:00 +CPU= 32.573
  13:00:05 STEP 1162 H=1162:00 +CPU= 21.428
  13:00:26 STEP 1163 H=1163:00 +CPU= 20.829
  13:01:03 STEP 1164 H=1164:00 +CPU= 37.110
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001164
  13:01:25 STEP 1165 H=1165:00 +CPU= 21.630
  13:01:46 STEP 1166 H=1166:00 +CPU= 21.387
  13:02:18 STEP 1167 H=1167:00 +CPU= 32.330
  13:02:40 STEP 1168 H=1168:00 +CPU= 21.491
  13:03:01 STEP 1169 H=1169:00 +CPU= 21.504
  13:03:36 STEP 1170 H=1170:00 +CPU= 33.901
  13:03:58 STEP 1171 H=1171:00 +CPU= 22.000
  13:04:20 STEP 1172 H=1172:00 +CPU= 21.704
  13:04:53 STEP 1173 H=1173:00 +CPU= 33.196
  13:05:15 STEP 1174 H=1174:00 +CPU= 21.836
  13:05:37 STEP 1175 H=1175:00 +CPU= 22.043
  13:06:16 STEP 1176 H=1176:00 +CPU= 39.246
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_48.zip
Uploading the intermediate file: upload_file_48.zip
  13:06:38 STEP 1177 H=1177:00 +CPU= 21.457
  13:07:00 STEP 1178 H=1178:00 +CPU= 21.446
  13:07:32 STEP 1179 H=1179:00 +CPU= 32.346
  13:07:54 STEP 1180 H=1180:00 +CPU= 21.353
  13:08:14 STEP 1181 H=1181:00 +CPU= 20.599
  13:08:48 STEP 1182 H=1182:00 +CPU= 33.106
  13:09:09 STEP 1183 H=1183:00 +CPU= 20.888
  13:09:31 STEP 1184 H=1184:00 +CPU= 21.781
  13:10:05 STEP 1185 H=1185:00 +CPU= 34.320
  13:10:28 STEP 1186 H=1186:00 +CPU= 22.821
  13:10:48 STEP 1187 H=1187:00 +CPU= 20.542
  13:11:25 STEP 1188 H=1188:00 +CPU= 36.564
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001188
  13:11:46 STEP 1189 H=1189:00 +CPU= 21.124
  13:12:09 STEP 1190 H=1190:00 +CPU= 22.418
  13:12:43 STEP 1191 H=1191:00 +CPU= 33.536
  13:13:05 STEP 1192 H=1192:00 +CPU= 22.138
  13:13:27 STEP 1193 H=1193:00 +CPU= 21.950
  13:14:01 STEP 1194 H=1194:00 +CPU= 33.814
  13:14:22 STEP 1195 H=1195:00 +CPU= 21.161
  13:14:45 STEP 1196 H=1196:00 +CPU= 22.358
  13:15:20 STEP 1197 H=1197:00 +CPU= 34.927
  13:15:41 STEP 1198 H=1198:00 +CPU= 21.307
  13:16:02 STEP 1199 H=1199:00 +CPU= 21.307
  13:16:43 STEP 1200 H=1200:00 +CPU= 40.331
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_49.zip
Uploading the intermediate file: upload_file_49.zip
  13:17:04 STEP 1201 H=1201:00 +CPU= 21.097
Uploading trickle at timestep: 4320000
  13:17:26 STEP 1202 H=1202:00 +CPU= 21.758
  13:17:58 STEP 1203 H=1203:00 +CPU= 32.412
  13:18:20 STEP 1204 H=1204:00 +CPU= 21.328
  13:18:41 STEP 1205 H=1205:00 +CPU= 20.977
  13:19:15 STEP 1206 H=1206:00 +CPU= 33.521
  13:19:36 STEP 1207 H=1207:00 +CPU= 21.577
  13:19:59 STEP 1208 H=1208:00 +CPU= 21.957
  13:20:32 STEP 1209 H=1209:00 +CPU= 33.262
  13:20:55 STEP 1210 H=1210:00 +CPU= 22.261
  13:21:16 STEP 1211 H=1211:00 +CPU= 20.992
  13:21:52 STEP 1212 H=1212:00 +CPU= 36.501
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001212
  13:22:14 STEP 1213 H=1213:00 +CPU= 21.244
  13:22:36 STEP 1214 H=1214:00 +CPU= 21.942
  13:23:08 STEP 1215 H=1215:00 +CPU= 32.479
  13:23:30 STEP 1216 H=1216:00 +CPU= 21.344
  13:23:51 STEP 1217 H=1217:00 +CPU= 20.855
  13:24:25 STEP 1218 H=1218:00 +CPU= 33.689
  13:24:46 STEP 1219 H=1219:00 +CPU= 21.219
  13:25:07 STEP 1220 H=1220:00 +CPU= 21.429
  13:25:41 STEP 1221 H=1221:00 +CPU= 33.067
  13:26:02 STEP 1222 H=1222:00 +CPU= 21.454
  13:26:24 STEP 1223 H=1223:00 +CPU= 21.432
  13:27:04 STEP 1224 H=1224:00 +CPU= 39.289
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_50.zip
Uploading the intermediate file: upload_file_50.zip
  13:27:24 STEP 1225 H=1225:00 +CPU= 20.555
  13:27:46 STEP 1226 H=1226:00 +CPU= 21.437
  13:28:19 STEP 1227 H=1227:00 +CPU= 33.660
  13:28:42 STEP 1228 H=1228:00 +CPU= 22.621
  13:29:03 STEP 1229 H=1229:00 +CPU= 20.814
  13:29:37 STEP 1230 H=1230:00 +CPU= 33.551
  13:29:59 STEP 1231 H=1231:00 +CPU= 21.443
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0873
(argv4) batchid: 976
(argv5) wuid: 12193517
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4cf6d0f3ab01d40e322824124c764a8c to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0873_2007050100_123_976_12193517.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0873_2007050100_123_976_12193517.zip
ic_ancil_file: ic_ancil_12193517
ifsdata_file: ifsdata_12193517
climate_data_file: clim_data_12193517
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a1e543e33697a13d49025a6fd773823c to: /var/lib/boinc-client/slots/3/ic_ancil_12193517.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12193517.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_68acaf3c4d954cba1fa2a44994b08c01 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12193517.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12193517.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12193517.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12193517.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12193517.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12193517.xml
-- Model is restarting --
Adjusting last_iter, 1231, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12193517.xml
last_cpu_time: 32042
upload_file_number: 51
last_iter: 1223
last_upload: 4406400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320
The child process has been launched with process id: 31333
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:31333:31333] [20230102:133106:1672662666:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  13:31:46 STEP 1224 H=1224:00 +CPU= 36.586
  13:32:06 STEP 1225 H=1225:00 +CPU= 20.236
  13:32:27 STEP 1226 H=1226:00 +CPU= 21.134
  13:33:00 STEP 1227 H=1227:00 +CPU= 32.018
  13:33:21 STEP 1228 H=1228:00 +CPU= 21.595
  13:33:42 STEP 1229 H=1229:00 +CPU= 21.036
  13:34:16 STEP 1230 H=1230:00 +CPU= 33.515
  13:34:37 STEP 1231 H=1231:00 +CPU= 20.745
  13:34:58 STEP 1232 H=1232:00 +CPU= 21.253
  13:35:31 STEP 1233 H=1233:00 +CPU= 32.147
  13:35:53 STEP 1234 H=1234:00 +CPU= 21.846
  13:36:13 STEP 1235 H=1235:00 +CPU= 20.651
  13:36:50 STEP 1236 H=1236:00 +CPU= 36.071
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001236
  13:37:10 STEP 1237 H=1237:00 +CPU= 20.410
  13:37:32 STEP 1238 H=1238:00 +CPU= 21.326
  13:38:05 STEP 1239 H=1239:00 +CPU= 32.796
  13:38:27 STEP 1240 H=1240:00 +CPU= 22.058
  13:38:49 STEP 1241 H=1241:00 +CPU= 21.148
  13:39:22 STEP 1242 H=1242:00 +CPU= 32.904
  13:39:43 STEP 1243 H=1243:00 +CPU= 20.972
  13:40:04 STEP 1244 H=1244:00 +CPU= 21.585
  13:40:37 STEP 1245 H=1245:00 +CPU= 32.051
  13:40:58 STEP 1246 H=1246:00 +CPU= 21.106
  13:41:19 STEP 1247 H=1247:00 +CPU= 21.318
  13:41:58 STEP 1248 H=1248:00 +CPU= 38.596
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_51.zip
Uploading the intermediate file: upload_file_51.zip
  13:42:19 STEP 1249 H=1249:00 +CPU= 20.920
  13:42:40 STEP 1250 H=1250:00 +CPU= 21.067
  13:43:14 STEP 1251 H=1251:00 +CPU= 33.476
  13:43:36 STEP 1252 H=1252:00 +CPU= 21.621
  13:43:57 STEP 1253 H=1253:00 +CPU= 20.999
  13:44:30 STEP 1254 H=1254:00 +CPU= 33.336
  13:44:52 STEP 1255 H=1255:00 +CPU= 21.099
  13:45:13 STEP 1256 H=1256:00 +CPU= 21.606
  13:45:46 STEP 1257 H=1257:00 +CPU= 32.622
  13:46:07 STEP 1258 H=1258:00 +CPU= 21.524
  13:46:28 STEP 1259 H=1259:00 +CPU= 20.878
  13:47:05 STEP 1260 H=1260:00 +CPU= 36.594
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001260
  13:47:26 STEP 1261 H=1261:00 +CPU= 20.428
  13:47:47 STEP 1262 H=1262:00 +CPU= 21.340
  13:48:22 STEP 1263 H=1263:00 +CPU= 33.153
  13:48:44 STEP 1264 H=1264:00 +CPU= 21.794
  13:49:06 STEP 1265 H=1265:00 +CPU= 21.485
  13:49:41 STEP 1266 H=1266:00 +CPU= 33.963
  13:50:01 STEP 1267 H=1267:00 +CPU= 20.629
  13:50:23 STEP 1268 H=1268:00 +CPU= 21.313
  13:50:59 STEP 1269 H=1269:00 +CPU= 34.291
  13:51:22 STEP 1270 H=1270:00 +CPU= 22.083
  13:51:45 STEP 1271 H=1271:00 +CPU= 22.364
  13:52:24 STEP 1272 H=1272:00 +CPU= 39.280
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_52.zip
Uploading the intermediate file: upload_file_52.zip
  13:52:45 STEP 1273 H=1273:00 +CPU= 20.700
  13:53:07 STEP 1274 H=1274:00 +CPU= 21.381
  13:53:41 STEP 1275 H=1275:00 +CPU= 33.309
  13:54:03 STEP 1276 H=1276:00 +CPU= 22.488
  13:54:26 STEP 1277 H=1277:00 +CPU= 21.220
  13:55:02 STEP 1278 H=1278:00 +CPU= 34.265
  13:55:25 STEP 1279 H=1279:00 +CPU= 21.342
  13:55:49 STEP 1280 H=1280:00 +CPU= 22.586
  13:56:25 STEP 1281 H=1281:00 +CPU= 34.171
  13:56:49 STEP 1282 H=1282:00 +CPU= 22.266
  13:57:11 STEP 1283 H=1283:00 +CPU= 21.161
  13:57:48 STEP 1284 H=1284:00 +CPU= 36.942
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001284
  13:58:09 STEP 1285 H=1285:00 +CPU= 20.917
  13:58:32 STEP 1286 H=1286:00 +CPU= 22.514
  13:59:05 STEP 1287 H=1287:00 +CPU= 32.894
  13:59:27 STEP 1288 H=1288:00 +CPU= 22.416
  13:59:49 STEP 1289 H=1289:00 +CPU= 21.820
  14:00:24 STEP 1290 H=1290:00 +CPU= 34.360
  14:00:45 STEP 1291 H=1291:00 +CPU= 21.805
  14:01:08 STEP 1292 H=1292:00 +CPU= 22.266
  14:01:41 STEP 1293 H=1293:00 +CPU= 33.437
  14:02:04 STEP 1294 H=1294:00 +CPU= 22.364
  14:02:27 STEP 1295 H=1295:00 +CPU= 22.476
  14:03:09 STEP 1296 H=1296:00 +CPU= 40.035
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_53.zip
Uploading the intermediate file: upload_file_53.zip
  14:03:31 STEP 1297 H=1297:00 +CPU= 21.732
  14:03:55 STEP 1298 H=1298:00 +CPU= 22.463
  14:04:32 STEP 1299 H=1299:00 +CPU= 33.556
  14:04:54 STEP 1300 H=1300:00 +CPU= 20.299
  14:05:10 STEP 1301 H=1301:00 +CPU= 16.002
  14:05:40 STEP 1302 H=1302:00 +CPU= 29.457
  14:06:02 STEP 1303 H=1303:00 +CPU= 20.746
  14:06:24 STEP 1304 H=1304:00 +CPU= 21.521
  14:06:58 STEP 1305 H=1305:00 +CPU= 32.409
  14:07:21 STEP 1306 H=1306:00 +CPU= 21.543
  14:07:44 STEP 1307 H=1307:00 +CPU= 21.474
  14:08:23 STEP 1308 H=1308:00 +CPU= 36.838
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001308
  14:08:44 STEP 1309 H=1309:00 +CPU= 20.910
  14:09:07 STEP 1310 H=1310:00 +CPU= 21.715
  14:09:42 STEP 1311 H=1311:00 +CPU= 33.017
  14:10:05 STEP 1312 H=1312:00 +CPU= 21.508
  14:10:26 STEP 1313 H=1313:00 +CPU= 20.861
  14:11:01 STEP 1314 H=1314:00 +CPU= 33.396
  14:11:22 STEP 1315 H=1315:00 +CPU= 20.823
  14:11:44 STEP 1316 H=1316:00 +CPU= 21.571
  14:12:18 STEP 1317 H=1317:00 +CPU= 32.611
  14:12:40 STEP 1318 H=1318:00 +CPU= 21.425
  14:13:02 STEP 1319 H=1319:00 +CPU= 21.465
  14:13:43 STEP 1320 H=1320:00 +CPU= 39.084
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_54.zip
Uploading the intermediate file: upload_file_54.zip
  14:14:04 STEP 1321 H=1321:00 +CPU= 20.754
  14:14:26 STEP 1322 H=1322:00 +CPU= 21.495
  14:15:00 STEP 1323 H=1323:00 +CPU= 32.468
  14:15:22 STEP 1324 H=1324:00 +CPU= 21.490
  14:15:44 STEP 1325 H=1325:00 +CPU= 20.884
  14:16:19 STEP 1326 H=1326:00 +CPU= 33.794
  14:16:42 STEP 1327 H=1327:00 +CPU= 21.255
  14:17:03 STEP 1328 H=1328:00 +CPU= 21.473
  14:17:38 STEP 1329 H=1329:00 +CPU= 34.383
  14:18:00 STEP 1330 H=1330:00 +CPU= 22.116
  14:18:22 STEP 1331 H=1331:00 +CPU= 21.324
  14:18:58 STEP 1332 H=1332:00 +CPU= 36.595
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001332
  14:19:19 STEP 1333 H=1333:00 +CPU= 20.510
  14:19:40 STEP 1334 H=1334:00 +CPU= 21.226
  14:20:13 STEP 1335 H=1335:00 +CPU= 32.743
  14:20:34 STEP 1336 H=1336:00 +CPU= 21.312
  14:20:55 STEP 1337 H=1337:00 +CPU= 20.744
  14:21:29 STEP 1338 H=1338:00 +CPU= 34.076
  14:21:51 STEP 1339 H=1339:00 +CPU= 21.712
  14:22:13 STEP 1340 H=1340:00 +CPU= 21.570
  14:22:46 STEP 1341 H=1341:00 +CPU= 32.811
  14:23:08 STEP 1342 H=1342:00 +CPU= 22.204
  14:23:30 STEP 1343 H=1343:00 +CPU= 21.450
  14:24:10 STEP 1344 H=1344:00 +CPU= 39.618
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_55.zip
Uploading the intermediate file: upload_file_55.zip
  14:24:31 STEP 1345 H=1345:00 +CPU= 21.688
  14:24:56 STEP 1346 H=1346:00 +CPU= 22.598
  14:25:31 STEP 1347 H=1347:00 +CPU= 33.959
  14:25:56 STEP 1348 H=1348:00 +CPU= 22.264
  14:26:19 STEP 1349 H=1349:00 +CPU= 22.184
  14:26:56 STEP 1350 H=1350:00 +CPU= 34.910
  14:27:18 STEP 1351 H=1351:00 +CPU= 21.460
  14:27:42 STEP 1352 H=1352:00 +CPU= 22.323
  14:28:17 STEP 1353 H=1353:00 +CPU= 33.547
  14:28:40 STEP 1354 H=1354:00 +CPU= 22.195
  14:29:02 STEP 1355 H=1355:00 +CPU= 21.607
  14:29:42 STEP 1356 H=1356:00 +CPU= 37.751
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001356
  14:30:04 STEP 1357 H=1357:00 +CPU= 21.741
  14:30:28 STEP 1358 H=1358:00 +CPU= 22.394
  14:31:03 STEP 1359 H=1359:00 +CPU= 33.794
  14:31:26 STEP 1360 H=1360:00 +CPU= 22.497
  14:31:48 STEP 1361 H=1361:00 +CPU= 21.369
  14:32:21 STEP 1362 H=1362:00 +CPU= 33.090
  14:32:41 STEP 1363 H=1363:00 +CPU= 20.492
  14:33:03 STEP 1364 H=1364:00 +CPU= 21.141
  14:33:35 STEP 1365 H=1365:00 +CPU= 32.228
  14:33:56 STEP 1366 H=1366:00 +CPU= 21.030
  14:34:17 STEP 1367 H=1367:00 +CPU= 21.124
  14:34:56 STEP 1368 H=1368:00 +CPU= 38.653
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_56.zip
Uploading the intermediate file: upload_file_56.zip
  14:35:18 STEP 1369 H=1369:00 +CPU= 21.755
  14:35:42 STEP 1370 H=1370:00 +CPU= 22.728
  14:36:18 STEP 1371 H=1371:00 +CPU= 33.950
  14:36:40 STEP 1372 H=1372:00 +CPU= 21.655
  14:37:01 STEP 1373 H=1373:00 +CPU= 21.307
  14:37:35 STEP 1374 H=1374:00 +CPU= 34.301
  14:37:57 STEP 1375 H=1375:00 +CPU= 21.625
  14:38:19 STEP 1376 H=1376:00 +CPU= 21.793
  14:38:53 STEP 1377 H=1377:00 +CPU= 33.618
  14:39:17 STEP 1378 H=1378:00 +CPU= 22.031
  14:39:42 STEP 1379 H=1379:00 +CPU= 22.274
  14:40:28 STEP 1380 H=1380:00 +CPU= 38.595
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001380
  14:40:51 STEP 1381 H=1381:00 +CPU= 22.005
  14:41:15 STEP 1382 H=1382:00 +CPU= 22.632
  14:41:51 STEP 1383 H=1383:00 +CPU= 33.823
  14:42:14 STEP 1384 H=1384:00 +CPU= 22.524
  14:42:37 STEP 1385 H=1385:00 +CPU= 21.491
  14:43:13 STEP 1386 H=1386:00 +CPU= 34.616
  14:43:35 STEP 1387 H=1387:00 +CPU= 21.230
  14:43:56 STEP 1388 H=1388:00 +CPU= 21.117
  14:44:28 STEP 1389 H=1389:00 +CPU= 32.250
  14:44:50 STEP 1390 H=1390:00 +CPU= 21.481
  14:45:12 STEP 1391 H=1391:00 +CPU= 21.579
  14:45:52 STEP 1392 H=1392:00 +CPU= 39.057
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_57.zip
Uploading the intermediate file: upload_file_57.zip
  14:46:12 STEP 1393 H=1393:00 +CPU= 20.597
  14:46:34 STEP 1394 H=1394:00 +CPU= 21.228
  14:47:07 STEP 1395 H=1395:00 +CPU= 33.410
  14:47:30 STEP 1396 H=1396:00 +CPU= 22.489
  14:47:52 STEP 1397 H=1397:00 +CPU= 21.851
  14:48:26 STEP 1398 H=1398:00 +CPU= 34.038
  14:48:47 STEP 1399 H=1399:00 +CPU= 21.245
  14:49:09 STEP 1400 H=1400:00 +CPU= 22.098
  14:49:43 STEP 1401 H=1401:00 +CPU= 33.062
  14:50:04 STEP 1402 H=1402:00 +CPU= 21.351
  14:50:26 STEP 1403 H=1403:00 +CPU= 21.346
  14:51:02 STEP 1404 H=1404:00 +CPU= 36.140
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001404
  14:51:24 STEP 1405 H=1405:00 +CPU= 21.649
  14:51:46 STEP 1406 H=1406:00 +CPU= 22.065
  14:52:21 STEP 1407 H=1407:00 +CPU= 34.593
  14:52:43 STEP 1408 H=1408:00 +CPU= 22.107
  14:53:04 STEP 1409 H=1409:00 +CPU= 20.935
  14:53:38 STEP 1410 H=1410:00 +CPU= 33.278
  14:53:59 STEP 1411 H=1411:00 +CPU= 20.815
  14:54:21 STEP 1412 H=1412:00 +CPU= 21.319
  14:54:54 STEP 1413 H=1413:00 +CPU= 32.787
  14:55:15 STEP 1414 H=1414:00 +CPU= 21.142
  14:55:37 STEP 1415 H=1415:00 +CPU= 21.933
  14:56:17 STEP 1416 H=1416:00 +CPU= 39.771
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_58.zip
Uploading the intermediate file: upload_file_58.zip
  14:56:38 STEP 1417 H=1417:00 +CPU= 20.708
  14:57:00 STEP 1418 H=1418:00 +CPU= 22.023
  14:57:33 STEP 1419 H=1419:00 +CPU= 32.996
  14:57:55 STEP 1420 H=1420:00 +CPU= 21.514
  14:58:16 STEP 1421 H=1421:00 +CPU= 20.536
  14:58:50 STEP 1422 H=1422:00 +CPU= 33.774
  14:59:11 STEP 1423 H=1423:00 +CPU= 20.452
  14:59:32 STEP 1424 H=1424:00 +CPU= 21.116
  15:00:06 STEP 1425 H=1425:00 +CPU= 33.101
  15:00:28 STEP 1426 H=1426:00 +CPU= 22.143
  15:00:49 STEP 1427 H=1427:00 +CPU= 20.769
  15:01:26 STEP 1428 H=1428:00 +CPU= 36.791
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001428
  15:01:47 STEP 1429 H=1429:00 +CPU= 21.479
  15:02:09 STEP 1430 H=1430:00 +CPU= 21.792
  15:02:42 STEP 1431 H=1431:00 +CPU= 32.253
  15:03:04 STEP 1432 H=1432:00 +CPU= 22.127
  15:03:24 STEP 1433 H=1433:00 +CPU= 20.269
  15:03:58 STEP 1434 H=1434:00 +CPU= 33.289
  15:04:18 STEP 1435 H=1435:00 +CPU= 20.321
  15:04:41 STEP 1436 H=1436:00 +CPU= 22.696
  15:05:16 STEP 1437 H=1437:00 +CPU= 34.679
  15:05:39 STEP 1438 H=1438:00 +CPU= 21.571
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0873
(argv4) batchid: 976
(argv5) wuid: 12193517
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4cf6d0f3ab01d40e322824124c764a8c to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0873_2007050100_123_976_12193517.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0873_2007050100_123_976_12193517.zip
ic_ancil_file: ic_ancil_12193517
ifsdata_file: ifsdata_12193517
climate_data_file: clim_data_12193517
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a1e543e33697a13d49025a6fd773823c to: /var/lib/boinc-client/slots/3/ic_ancil_12193517.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12193517.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_68acaf3c4d954cba1fa2a44994b08c01 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12193517.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12193517.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12193517.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12193517.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12193517.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12193517.xml
-- Model is restarting --
Adjusting last_iter, 1438, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12193517.xml
last_cpu_time: 37587
upload_file_number: 59
last_iter: 1415
last_upload: 5097600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320
The child process has been launched with process id: 39772
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-MS-7B86:1:1:39772:39772] [20230102:150655:1672668415:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:07:39 STEP 1416 H=1416:00 +CPU= 37.705
  15:08:01 STEP 1417 H=1417:00 +CPU= 21.134
  15:08:24 STEP 1418 H=1418:00 +CPU= 21.869
  15:09:00 STEP 1419 H=1419:00 +CPU= 33.586
  15:09:23 STEP 1420 H=1420:00 +CPU= 21.825
  15:09:43 STEP 1421 H=1421:00 +CPU= 20.526
  15:10:17 STEP 1422 H=1422:00 +CPU= 33.584
  15:10:39 STEP 1423 H=1423:00 +CPU= 21.354
  15:11:00 STEP 1424 H=1424:00 +CPU= 21.471
  15:11:33 STEP 1425 H=1425:00 +CPU= 32.848
  15:11:55 STEP 1426 H=1426:00 +CPU= 21.648
  15:12:15 STEP 1427 H=1427:00 +CPU= 20.153
  15:12:51 STEP 1428 H=1428:00 +CPU= 35.846
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001428
  15:13:11 STEP 1429 H=1429:00 +CPU= 20.426
  15:13:34 STEP 1430 H=1430:00 +CPU= 22.236
  15:14:06 STEP 1431 H=1431:00 +CPU= 32.475
  15:14:28 STEP 1432 H=1432:00 +CPU= 21.592
  15:14:49 STEP 1433 H=1433:00 +CPU= 20.802
  15:15:23 STEP 1434 H=1434:00 +CPU= 34.079
  15:15:45 STEP 1435 H=1435:00 +CPU= 21.785
  15:16:07 STEP 1436 H=1436:00 +CPU= 22.084
  15:16:40 STEP 1437 H=1437:00 +CPU= 32.691
  15:17:02 STEP 1438 H=1438:00 +CPU= 22.079
  15:17:25 STEP 1439 H=1439:00 +CPU= 22.366
  15:18:06 STEP 1440 H=1440:00 +CPU= 38.982
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193517/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0873_2007050100_123_976_12193517_0_r1406673320_59.zip
Uploading the intermediate file: upload_file_59.zip
  15:18:27 STEP 1441 H=1441:00 +CPU= 21.456
  15:18:49 STEP 1442 H=1442:00 +CPU= 21.900
  15:19:23 STEP 1443 H=1443:00 +CPU= 33.337
  15:19:44 STEP 1444 H=1444:00 +CPU= 21.342
  15:20:06 STEP 1445 H=1445:00 +CPU= 21.131
  15:20:40 STEP 1446 H=1446:00 +CPU= 33.425
  15:21:02 STEP 1447 H=1447:00 +CPU= 21.616
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:05:59 1514495 22279310 oifs_43r3_ps_0873_2007050100_123_976_12193517_0 4,320,000 31,259 0.0072
09 May 2023 16:05:54 1514495 22279310 oifs_43r3_ps_0873_2007050100_123_976_12193517_0 3,110,400 22,646 0.0073
09 May 2023 16:05:24 1514495 22279310 oifs_43r3_ps_0873_2007050100_123_976_12193517_0 2,073,600 15,025 0.0072
09 May 2023 16:05:21 1514495 22279310 oifs_43r3_ps_0873_2007050100_123_976_12193517_0 1,209,600 8,775 0.0073


©2024 climateprediction.net