climateprediction.net home page
Task 22440048

Task 22440048

Name oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1
Workunit 12287419
Created 7 Jun 2024, 21:24:54 UTC
Sent 7 Jun 2024, 21:32:31 UTC
Report deadline 6 Aug 2024, 21:32:31 UTC
Received 10 Jun 2024, 19:07:20 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1532385
Run time 11 hours 55 min 52 sec
CPU time 11 hours 55 min 52 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 5.61 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,564.23 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,382.80 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<stderr_txt>
boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_9.zip
Uploading the intermediate file: upload_file_9.zip
  10:45:52 STEP  961 H= 240:15 +CPU= 30.595
Uploading trickle at timestep: 864000
  10:46:21 STEP  962 H= 240:30 +CPU= 28.453
  10:46:50 STEP  963 H= 240:45 +CPU= 28.764
  10:47:19 STEP  964 H= 241:00 +CPU= 29.157
  10:47:47 STEP  965 H= 241:15 +CPU= 28.645
  10:48:19 STEP  966 H= 241:30 +CPU= 31.505
  10:48:55 STEP  967 H= 241:45 +CPU= 36.445
  10:49:28 STEP  968 H= 242:00 +CPU= 31.926
  10:49:57 STEP  969 H= 242:15 +CPU= 29.007
  10:50:25 STEP  970 H= 242:30 +CPU= 28.224
  10:50:54 STEP  971 H= 242:45 +CPU= 29.241
  10:51:28 STEP  972 H= 243:00 +CPU= 33.865
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+000972
  10:51:58 STEP  973 H= 243:15 +CPU= 30.113
  10:52:28 STEP  974 H= 243:30 +CPU= 30.140
  10:52:56 STEP  975 H= 243:45 +CPU= 27.326
  10:53:24 STEP  976 H= 244:00 +CPU= 27.954
  10:53:54 STEP  977 H= 244:15 +CPU= 30.364
  10:54:23 STEP  978 H= 244:30 +CPU= 28.502
  10:54:54 STEP  979 H= 244:45 +CPU= 30.844
  10:55:23 STEP  980 H= 245:00 +CPU= 28.837
  10:55:52 STEP  981 H= 245:15 +CPU= 29.198
  10:56:21 STEP  982 H= 245:30 +CPU= 29.137
  10:56:50 STEP  983 H= 245:45 +CPU= 29.296
  10:57:28 STEP  984 H= 246:00 +CPU= 37.303
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+000984
  10:57:57 STEP  985 H= 246:15 +CPU= 29.108
  10:58:26 STEP  986 H= 246:30 +CPU= 29.047
  10:58:57 STEP  987 H= 246:45 +CPU= 30.685
  10:59:27 STEP  988 H= 247:00 +CPU= 30.040
  10:59:58 STEP  989 H= 247:15 +CPU= 31.296
  11:00:26 STEP  990 H= 247:30 +CPU= 27.972
  11:00:56 STEP  991 H= 247:45 +CPU= 29.926
  11:01:24 STEP  992 H= 248:00 +CPU= 28.044
  11:01:54 STEP  993 H= 248:15 +CPU= 29.273
  11:02:22 STEP  994 H= 248:30 +CPU= 28.515
  11:02:53 STEP  995 H= 248:45 +CPU= 30.215
  11:03:28 STEP  996 H= 249:00 +CPU= 35.119
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+000996
  11:03:57 STEP  997 H= 249:15 +CPU= 29.423
  11:04:28 STEP  998 H= 249:30 +CPU= 30.833
  11:04:59 STEP  999 H= 249:45 +CPU= 30.978
  11:05:33 STEP 1000 H= 250:00 +CPU= 33.613
  11:06:07 STEP 1001 H= 250:15 +CPU= 34.278
  11:06:41 STEP 1002 H= 250:30 +CPU= 33.938
  11:07:13 STEP 1003 H= 250:45 +CPU= 32.319
  11:07:40 STEP 1004 H= 251:00 +CPU= 26.859
  11:08:11 STEP 1005 H= 251:15 +CPU= 30.869
  11:08:38 STEP 1006 H= 251:30 +CPU= 26.476
  11:09:11 STEP 1007 H= 251:45 +CPU= 32.710
  11:09:52 STEP 1008 H= 252:00 +CPU= 40.950
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001008
  11:10:23 STEP 1009 H= 252:15 +CPU= 30.951
  11:10:53 STEP 1010 H= 252:30 +CPU= 30.631
  11:11:21 STEP 1011 H= 252:45 +CPU= 27.609
  11:11:52 STEP 1012 H= 253:00 +CPU= 30.771
  11:12:22 STEP 1013 H= 253:15 +CPU= 30.036
  11:12:52 STEP 1014 H= 253:30 +CPU= 30.246
  11:13:21 STEP 1015 H= 253:45 +CPU= 28.318
  11:13:54 STEP 1016 H= 254:00 +CPU= 32.936
  11:14:24 STEP 1017 H= 254:15 +CPU= 30.192
  11:14:51 STEP 1018 H= 254:30 +CPU= 27.030
  11:15:23 STEP 1019 H= 254:45 +CPU= 31.977
  11:16:01 STEP 1020 H= 255:00 +CPU= 37.208
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001020
  11:16:34 STEP 1021 H= 255:15 +CPU= 32.615
  11:17:04 STEP 1022 H= 255:30 +CPU= 30.504
  11:17:36 STEP 1023 H= 255:45 +CPU= 31.460
  11:18:08 STEP 1024 H= 256:00 +CPU= 31.679
  11:18:36 STEP 1025 H= 256:15 +CPU= 28.202
  11:19:05 STEP 1026 H= 256:30 +CPU= 29.269
  11:19:33 STEP 1027 H= 256:45 +CPU= 26.993
  11:20:03 STEP 1028 H= 257:00 +CPU= 30.792
  11:20:35 STEP 1029 H= 257:15 +CPU= 31.587
  11:21:06 STEP 1030 H= 257:30 +CPU= 31.412
  11:21:37 STEP 1031 H= 257:45 +CPU= 30.441
  11:22:16 STEP 1032 H= 258:00 +CPU= 38.483
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001032
  11:22:47 STEP 1033 H= 258:15 +CPU= 31.564
  11:23:18 STEP 1034 H= 258:30 +CPU= 30.058
  11:23:51 STEP 1035 H= 258:45 +CPU= 33.443
  11:24:25 STEP 1036 H= 259:00 +CPU= 33.370
  11:24:53 STEP 1037 H= 259:15 +CPU= 28.304
  11:25:25 STEP 1038 H= 259:30 +CPU= 31.275
  11:25:53 STEP 1039 H= 259:45 +CPU= 28.767
  11:26:25 STEP 1040 H= 260:00 +CPU= 31.398
  11:26:54 STEP 1041 H= 260:15 +CPU= 29.387
  11:27:26 STEP 1042 H= 260:30 +CPU= 32.145
  11:27:59 STEP 1043 H= 260:45 +CPU= 33.117
  11:28:36 STEP 1044 H= 261:00 +CPU= 36.386
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001044
  11:29:06 STEP 1045 H= 261:15 +CPU= 30.157
  11:29:36 STEP 1046 H= 261:30 +CPU= 29.222
  11:30:07 STEP 1047 H= 261:45 +CPU= 30.966
  11:30:36 STEP 1048 H= 262:00 +CPU= 29.189
  11:31:07 STEP 1049 H= 262:15 +CPU= 31.133
  11:31:40 STEP 1050 H= 262:30 +CPU= 32.059
  11:32:12 STEP 1051 H= 262:45 +CPU= 32.255
  11:32:46 STEP 1052 H= 263:00 +CPU= 33.395
  11:33:19 STEP 1053 H= 263:15 +CPU= 32.620
  11:33:52 STEP 1054 H= 263:30 +CPU= 32.585
  11:34:26 STEP 1055 H= 263:45 +CPU= 34.812
  11:35:08 STEP 1056 H= 264:00 +CPU= 41.433
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_10.zip
Uploading the intermediate file: upload_file_10.zip
  11:35:42 STEP 1057 H= 264:15 +CPU= 31.663
  11:36:11 STEP 1058 H= 264:30 +CPU= 29.197
  11:36:39 STEP 1059 H= 264:45 +CPU= 28.330
  11:37:06 STEP 1060 H= 265:00 +CPU= 26.838
  11:37:32 STEP 1061 H= 265:15 +CPU= 25.354
  11:37:59 STEP 1062 H= 265:30 +CPU= 27.220
  11:38:29 STEP 1063 H= 265:45 +CPU= 29.442
  11:39:03 STEP 1064 H= 266:00 +CPU= 34.361
  11:39:33 STEP 1065 H= 266:15 +CPU= 29.793
  11:40:02 STEP 1066 H= 266:30 +CPU= 28.919
  11:40:32 STEP 1067 H= 266:45 +CPU= 29.529
  11:41:08 STEP 1068 H= 267:00 +CPU= 35.698
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001068
  11:41:39 STEP 1069 H= 267:15 +CPU= 31.177
  11:42:08 STEP 1070 H= 267:30 +CPU= 28.817
  11:42:38 STEP 1071 H= 267:45 +CPU= 30.160
  11:43:11 STEP 1072 H= 268:00 +CPU= 32.590
  11:43:42 STEP 1073 H= 268:15 +CPU= 30.248
  11:44:11 STEP 1074 H= 268:30 +CPU= 29.658
  11:44:42 STEP 1075 H= 268:45 +CPU= 30.095
  11:45:14 STEP 1076 H= 269:00 +CPU= 31.880
  11:45:41 STEP 1077 H= 269:15 +CPU= 26.984
  11:46:09 STEP 1078 H= 269:30 +CPU= 28.454
  11:46:37 STEP 1079 H= 269:45 +CPU= 28.295
  11:47:18 STEP 1080 H= 270:00 +CPU= 40.011
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001080
  11:47:49 STEP 1081 H= 270:15 +CPU= 30.977
  11:48:20 STEP 1082 H= 270:30 +CPU= 31.510
  11:48:53 STEP 1083 H= 270:45 +CPU= 33.183
  11:49:27 STEP 1084 H= 271:00 +CPU= 33.207
  11:50:00 STEP 1085 H= 271:15 +CPU= 33.723
  11:50:32 STEP 1086 H= 271:30 +CPU= 31.260
  11:51:02 STEP 1087 H= 271:45 +CPU= 29.970
  11:51:36 STEP 1088 H= 272:00 +CPU= 34.141
  11:52:05 STEP 1089 H= 272:15 +CPU= 28.619
  11:52:37 STEP 1090 H= 272:30 +CPU= 31.611
  11:53:04 STEP 1091 H= 272:45 +CPU= 27.025
  11:53:41 STEP 1092 H= 273:00 +CPU= 36.530
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001092
  11:54:11 STEP 1093 H= 273:15 +CPU= 30.209
  11:54:40 STEP 1094 H= 273:30 +CPU= 28.680
  11:55:08 STEP 1095 H= 273:45 +CPU= 27.872
  11:55:39 STEP 1096 H= 274:00 +CPU= 31.305
  11:56:09 STEP 1097 H= 274:15 +CPU= 29.299
  11:56:37 STEP 1098 H= 274:30 +CPU= 28.575
  11:57:07 STEP 1099 H= 274:45 +CPU= 29.659
  11:57:39 STEP 1100 H= 275:00 +CPU= 31.555
  11:58:10 STEP 1101 H= 275:15 +CPU= 31.651
  11:58:40 STEP 1102 H= 275:30 +CPU= 29.897
  11:59:09 STEP 1103 H= 275:45 +CPU= 28.522
  11:59:49 STEP 1104 H= 276:00 +CPU= 39.490
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001104
  12:00:14 STEP 1105 H= 276:15 +CPU= 25.633
  12:00:45 STEP 1106 H= 276:30 +CPU= 30.613
  12:01:16 STEP 1107 H= 276:45 +CPU= 31.209
  12:01:48 STEP 1108 H= 277:00 +CPU= 31.463
  12:02:17 STEP 1109 H= 277:15 +CPU= 29.245
  12:02:47 STEP 1110 H= 277:30 +CPU= 29.811
  12:03:19 STEP 1111 H= 277:45 +CPU= 32.481
  12:03:50 STEP 1112 H= 278:00 +CPU= 30.177
  12:04:20 STEP 1113 H= 278:15 +CPU= 30.182
  12:04:50 STEP 1114 H= 278:30 +CPU= 29.611
  12:05:22 STEP 1115 H= 278:45 +CPU= 31.712
  12:06:00 STEP 1116 H= 279:00 +CPU= 38.257
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001116
  12:06:31 STEP 1117 H= 279:15 +CPU= 31.206
  12:07:06 STEP 1118 H= 279:30 +CPU= 34.375
  12:07:35 STEP 1119 H= 279:45 +CPU= 29.249
  12:08:05 STEP 1120 H= 280:00 +CPU= 30.142
  12:08:36 STEP 1121 H= 280:15 +CPU= 30.197
  12:09:06 STEP 1122 H= 280:30 +CPU= 30.022
  12:09:39 STEP 1123 H= 280:45 +CPU= 32.884
  12:10:08 STEP 1124 H= 281:00 +CPU= 28.993
  12:10:37 STEP 1125 H= 281:15 +CPU= 29.574
  12:11:05 STEP 1126 H= 281:30 +CPU= 27.665
  12:11:35 STEP 1127 H= 281:45 +CPU= 30.209
  12:12:17 STEP 1128 H= 282:00 +CPU= 41.897
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001128
  12:12:47 STEP 1129 H= 282:15 +CPU= 29.587
  12:13:19 STEP 1130 H= 282:30 +CPU= 31.951
  12:13:47 STEP 1131 H= 282:45 +CPU= 27.926
  12:14:18 STEP 1132 H= 283:00 +CPU= 30.271
  12:14:45 STEP 1133 H= 283:15 +CPU= 27.461
  12:15:14 STEP 1134 H= 283:30 +CPU= 29.080
  12:15:45 STEP 1135 H= 283:45 +CPU= 31.321
  12:16:18 STEP 1136 H= 284:00 +CPU= 32.054
  12:16:47 STEP 1137 H= 284:15 +CPU= 29.457
  12:17:19 STEP 1138 H= 284:30 +CPU= 31.256
  12:17:50 STEP 1139 H= 284:45 +CPU= 31.531
  12:18:24 STEP 1140 H= 285:00 +CPU= 34.253
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001140
  12:18:53 STEP 1141 H= 285:15 +CPU= 28.716
  12:19:21 STEP 1142 H= 285:30 +CPU= 27.587
  12:19:50 STEP 1143 H= 285:45 +CPU= 29.812
  12:20:20 STEP 1144 H= 286:00 +CPU= 29.229
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4bc
(argv4) batchid: 1017
(argv5) wuid: 12287419
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/55
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/55/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/55/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4c150e1d1a3519c6a4a7430f892f3149 to: /var/lib/boinc-client/slots/55/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/55/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419.zip
ic_ancil_file: ic_ancil_12287419
ifsdata_file: ifsdata_12287419
climate_data_file: clim_data_12287419
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/55/ic_ancil_12287419.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/55/ic_ancil_12287419.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_04d72b1863dc88c3b4cf365b1063e4ed to: /var/lib/boinc-client/slots/55/ifsdata/ifsdata_12287419.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/55/ifsdata/ifsdata_12287419.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/55/159l_2/clim_data_12287419.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/55/159l_2/clim_data_12287419.zip
Checking for progress XML file: /var/lib/boinc-client/slots/55/progress_file_12287419.xml
Opened progress file ok : /var/lib/boinc-client/slots/55/progress_file_12287419.xml
-- Model is restarting --
Adjusting last_iter, 1144, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/55/progress_file_12287419.xml
last_cpu_time: 35509
upload_file_number: 11
last_iter: 1105
last_upload: 950400
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179
The child process has been launched with process id: 2238411
Executing the command: /var/lib/boinc-client/slots/55/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:PREMIUM:1:1:2238411:2238411] [20240610:122130:1718022090:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  12:21:59 STEP 1104 H= 276:00 +CPU= 26.623
  12:22:25 STEP 1105 H= 276:15 +CPU= 26.442
  12:22:53 STEP 1106 H= 276:30 +CPU= 27.305
  12:23:18 STEP 1107 H= 276:45 +CPU= 25.786
  12:23:46 STEP 1108 H= 277:00 +CPU= 27.472
  12:24:12 STEP 1109 H= 277:15 +CPU= 26.511
  12:24:40 STEP 1110 H= 277:30 +CPU= 27.449
  12:25:06 STEP 1111 H= 277:45 +CPU= 26.417
  12:25:33 STEP 1112 H= 278:00 +CPU= 27.150
  12:26:00 STEP 1113 H= 278:15 +CPU= 26.048
  12:26:25 STEP 1114 H= 278:30 +CPU= 25.599
  12:26:52 STEP 1115 H= 278:45 +CPU= 26.531
  12:27:25 STEP 1116 H= 279:00 +CPU= 32.982
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001116
  12:27:52 STEP 1117 H= 279:15 +CPU= 27.024
  12:28:20 STEP 1118 H= 279:30 +CPU= 27.730
  12:28:48 STEP 1119 H= 279:45 +CPU= 28.298
  12:29:16 STEP 1120 H= 280:00 +CPU= 28.237
  12:29:44 STEP 1121 H= 280:15 +CPU= 27.863
  12:30:11 STEP 1122 H= 280:30 +CPU= 26.512
  12:30:36 STEP 1123 H= 280:45 +CPU= 25.250
  12:31:02 STEP 1124 H= 281:00 +CPU= 26.651
  12:31:28 STEP 1125 H= 281:15 +CPU= 25.795
  12:31:54 STEP 1126 H= 281:30 +CPU= 26.044
  12:32:21 STEP 1127 H= 281:45 +CPU= 26.550
  12:32:54 STEP 1128 H= 282:00 +CPU= 33.029
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001128
  12:33:20 STEP 1129 H= 282:15 +CPU= 25.796
  12:33:46 STEP 1130 H= 282:30 +CPU= 25.944
  12:34:13 STEP 1131 H= 282:45 +CPU= 26.871
  12:34:41 STEP 1132 H= 283:00 +CPU= 28.324
  12:35:10 STEP 1133 H= 283:15 +CPU= 28.899
  12:35:37 STEP 1134 H= 283:30 +CPU= 26.796
  12:36:05 STEP 1135 H= 283:45 +CPU= 27.806
  12:36:34 STEP 1136 H= 284:00 +CPU= 29.030
  12:37:02 STEP 1137 H= 284:15 +CPU= 28.101
  12:37:32 STEP 1138 H= 284:30 +CPU= 29.467
  12:38:01 STEP 1139 H= 284:45 +CPU= 29.604
  12:38:38 STEP 1140 H= 285:00 +CPU= 36.534
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001140
  12:39:06 STEP 1141 H= 285:15 +CPU= 27.947
  12:39:32 STEP 1142 H= 285:30 +CPU= 26.645
  12:40:01 STEP 1143 H= 285:45 +CPU= 28.012
  12:40:31 STEP 1144 H= 286:00 +CPU= 30.189
  12:41:00 STEP 1145 H= 286:15 +CPU= 29.631
  12:41:28 STEP 1146 H= 286:30 +CPU= 27.628
  12:41:59 STEP 1147 H= 286:45 +CPU= 30.393
  12:42:30 STEP 1148 H= 287:00 +CPU= 31.488
  12:42:59 STEP 1149 H= 287:15 +CPU= 28.429
  12:43:27 STEP 1150 H= 287:30 +CPU= 28.830
  12:43:55 STEP 1151 H= 287:45 +CPU= 27.084
  12:44:37 STEP 1152 H= 288:00 +CPU= 42.492
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_11.zip
Uploading the intermediate file: upload_file_11.zip
  12:45:11 STEP 1153 H= 288:15 +CPU= 33.444
  12:45:39 STEP 1154 H= 288:30 +CPU= 28.769
  12:46:08 STEP 1155 H= 288:45 +CPU= 28.618
  12:46:40 STEP 1156 H= 289:00 +CPU= 31.424
  12:47:13 STEP 1157 H= 289:15 +CPU= 33.030
  12:47:42 STEP 1158 H= 289:30 +CPU= 29.278
  12:48:12 STEP 1159 H= 289:45 +CPU= 29.583
  12:48:42 STEP 1160 H= 290:00 +CPU= 30.487
  12:49:15 STEP 1161 H= 290:15 +CPU= 32.901
  12:49:44 STEP 1162 H= 290:30 +CPU= 28.301
  12:50:14 STEP 1163 H= 290:45 +CPU= 29.717
  12:50:50 STEP 1164 H= 291:00 +CPU= 36.594
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001164
  12:51:23 STEP 1165 H= 291:15 +CPU= 32.536
  12:51:54 STEP 1166 H= 291:30 +CPU= 31.013
  12:52:24 STEP 1167 H= 291:45 +CPU= 30.153
  12:52:55 STEP 1168 H= 292:00 +CPU= 30.722
  12:53:23 STEP 1169 H= 292:15 +CPU= 28.224
  12:53:56 STEP 1170 H= 292:30 +CPU= 32.285
  12:54:27 STEP 1171 H= 292:45 +CPU= 31.591
  12:54:56 STEP 1172 H= 293:00 +CPU= 28.940
  12:55:26 STEP 1173 H= 293:15 +CPU= 30.002
  12:55:59 STEP 1174 H= 293:30 +CPU= 32.517
  12:56:33 STEP 1175 H= 293:45 +CPU= 33.901
  12:57:16 STEP 1176 H= 294:00 +CPU= 43.608
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001176
  12:57:46 STEP 1177 H= 294:15 +CPU= 29.696
  12:58:15 STEP 1178 H= 294:30 +CPU= 28.518
  12:58:45 STEP 1179 H= 294:45 +CPU= 30.048
  12:59:16 STEP 1180 H= 295:00 +CPU= 31.452
  12:59:48 STEP 1181 H= 295:15 +CPU= 31.338
  13:00:16 STEP 1182 H= 295:30 +CPU= 27.711
  13:00:47 STEP 1183 H= 295:45 +CPU= 31.522
  13:01:17 STEP 1184 H= 296:00 +CPU= 29.833
  13:01:46 STEP 1185 H= 296:15 +CPU= 29.195
  13:02:14 STEP 1186 H= 296:30 +CPU= 27.578
  13:02:41 STEP 1187 H= 296:45 +CPU= 27.696
  13:03:17 STEP 1188 H= 297:00 +CPU= 35.485
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001188
  13:03:47 STEP 1189 H= 297:15 +CPU= 30.098
  13:04:18 STEP 1190 H= 297:30 +CPU= 30.777
  13:04:48 STEP 1191 H= 297:45 +CPU= 29.532
  13:05:19 STEP 1192 H= 298:00 +CPU= 31.050
  13:05:49 STEP 1193 H= 298:15 +CPU= 29.686
  13:06:17 STEP 1194 H= 298:30 +CPU= 28.132
  13:06:45 STEP 1195 H= 298:45 +CPU= 27.666
  13:07:11 STEP 1196 H= 299:00 +CPU= 25.881
  13:07:37 STEP 1197 H= 299:15 +CPU= 26.099
  13:08:01 STEP 1198 H= 299:30 +CPU= 24.241
  13:08:29 STEP 1199 H= 299:45 +CPU= 28.158
  13:09:06 STEP 1200 H= 300:00 +CPU= 36.519
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001200
  13:09:31 STEP 1201 H= 300:15 +CPU= 25.658
  13:09:58 STEP 1202 H= 300:30 +CPU= 26.399
  13:10:23 STEP 1203 H= 300:45 +CPU= 24.893
  13:10:51 STEP 1204 H= 301:00 +CPU= 28.306
  13:11:20 STEP 1205 H= 301:15 +CPU= 29.006
  13:11:45 STEP 1206 H= 301:30 +CPU= 25.254
  13:12:11 STEP 1207 H= 301:45 +CPU= 25.810
  13:12:38 STEP 1208 H= 302:00 +CPU= 26.515
  13:13:06 STEP 1209 H= 302:15 +CPU= 28.018
  13:13:33 STEP 1210 H= 302:30 +CPU= 27.111
  13:14:03 STEP 1211 H= 302:45 +CPU= 29.038
  13:14:37 STEP 1212 H= 303:00 +CPU= 33.957
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001212
  13:15:07 STEP 1213 H= 303:15 +CPU= 29.808
  13:15:36 STEP 1214 H= 303:30 +CPU= 29.544
  13:16:05 STEP 1215 H= 303:45 +CPU= 29.344
  13:16:40 STEP 1216 H= 304:00 +CPU= 33.917
  13:17:11 STEP 1217 H= 304:15 +CPU= 30.930
  13:17:40 STEP 1218 H= 304:30 +CPU= 28.999
  13:18:10 STEP 1219 H= 304:45 +CPU= 29.658
  13:18:40 STEP 1220 H= 305:00 +CPU= 30.602
  13:19:10 STEP 1221 H= 305:15 +CPU= 30.186
  13:19:41 STEP 1222 H= 305:30 +CPU= 30.803
  13:20:11 STEP 1223 H= 305:45 +CPU= 30.150
  13:20:50 STEP 1224 H= 306:00 +CPU= 38.343
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001224
  13:21:21 STEP 1225 H= 306:15 +CPU= 31.227
  13:21:51 STEP 1226 H= 306:30 +CPU= 29.518
  13:22:19 STEP 1227 H= 306:45 +CPU= 28.151
  13:22:47 STEP 1228 H= 307:00 +CPU= 28.523
  13:23:15 STEP 1229 H= 307:15 +CPU= 28.132
  13:23:42 STEP 1230 H= 307:30 +CPU= 26.867
  13:24:13 STEP 1231 H= 307:45 +CPU= 30.618
  13:24:38 STEP 1232 H= 308:00 +CPU= 25.275
  13:25:04 STEP 1233 H= 308:15 +CPU= 26.322
  13:25:29 STEP 1234 H= 308:30 +CPU= 24.157
  13:25:57 STEP 1235 H= 308:45 +CPU= 28.404
  13:26:29 STEP 1236 H= 309:00 +CPU= 32.071
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001236
  13:26:59 STEP 1237 H= 309:15 +CPU= 29.555
  13:27:28 STEP 1238 H= 309:30 +CPU= 29.083
  13:27:56 STEP 1239 H= 309:45 +CPU= 28.250
  13:28:22 STEP 1240 H= 310:00 +CPU= 25.489
  13:28:47 STEP 1241 H= 310:15 +CPU= 25.638
  13:29:12 STEP 1242 H= 310:30 +CPU= 25.019
  13:29:36 STEP 1243 H= 310:45 +CPU= 23.581
  13:30:01 STEP 1244 H= 311:00 +CPU= 25.343
  13:30:28 STEP 1245 H= 311:15 +CPU= 26.400
  13:30:49 STEP 1246 H= 311:30 +CPU= 20.981
  13:31:11 STEP 1247 H= 311:45 +CPU= 22.301
  13:31:44 STEP 1248 H= 312:00 +CPU= 33.415
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_12.zip
Uploading the intermediate file: upload_file_12.zip
  13:32:12 STEP 1249 H= 312:15 +CPU= 27.428
  13:32:41 STEP 1250 H= 312:30 +CPU= 28.810
  13:33:06 STEP 1251 H= 312:45 +CPU= 25.344
  13:33:29 STEP 1252 H= 313:00 +CPU= 22.564
  13:33:53 STEP 1253 H= 313:15 +CPU= 24.476
  13:34:13 STEP 1254 H= 313:30 +CPU= 20.044
  13:34:38 STEP 1255 H= 313:45 +CPU= 24.810
  13:35:01 STEP 1256 H= 314:00 +CPU= 22.841
  13:35:25 STEP 1257 H= 314:15 +CPU= 24.514
  13:35:46 STEP 1258 H= 314:30 +CPU= 20.542
  13:36:04 STEP 1259 H= 314:45 +CPU= 18.600
  13:36:32 STEP 1260 H= 315:00 +CPU= 27.770
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001260
  13:36:51 STEP 1261 H= 315:15 +CPU= 18.361
  13:37:08 STEP 1262 H= 315:30 +CPU= 17.343
  13:37:24 STEP 1263 H= 315:45 +CPU= 16.469
  13:37:45 STEP 1264 H= 316:00 +CPU= 20.189
  13:38:03 STEP 1265 H= 316:15 +CPU= 18.316
  13:38:26 STEP 1266 H= 316:30 +CPU= 22.924
  13:38:45 STEP 1267 H= 316:45 +CPU= 19.182
  13:39:04 STEP 1268 H= 317:00 +CPU= 18.643
  13:39:23 STEP 1269 H= 317:15 +CPU= 18.919
  13:39:40 STEP 1270 H= 317:30 +CPU= 16.945
  13:39:59 STEP 1271 H= 317:45 +CPU= 19.240
  13:40:24 STEP 1272 H= 318:00 +CPU= 25.685
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001272
  13:40:44 STEP 1273 H= 318:15 +CPU= 19.761
  13:41:02 STEP 1274 H= 318:30 +CPU= 18.010
  13:41:19 STEP 1275 H= 318:45 +CPU= 16.746
  13:41:36 STEP 1276 H= 319:00 +CPU= 17.111
  13:41:53 STEP 1277 H= 319:15 +CPU= 16.510
  13:42:10 STEP 1278 H= 319:30 +CPU= 17.794
  13:42:32 STEP 1279 H= 319:45 +CPU= 21.585
  13:42:49 STEP 1280 H= 320:00 +CPU= 17.483
  13:43:07 STEP 1281 H= 320:15 +CPU= 17.533
  13:43:25 STEP 1282 H= 320:30 +CPU= 17.718
  13:43:42 STEP 1283 H= 320:45 +CPU= 17.295
  13:44:04 STEP 1284 H= 321:00 +CPU= 22.145
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001284
  13:44:22 STEP 1285 H= 321:15 +CPU= 17.353
  13:44:38 STEP 1286 H= 321:30 +CPU= 16.932
  13:44:55 STEP 1287 H= 321:45 +CPU= 16.580
  13:45:12 STEP 1288 H= 322:00 +CPU= 16.758
  13:45:29 STEP 1289 H= 322:15 +CPU= 17.204
  13:45:45 STEP 1290 H= 322:30 +CPU= 16.265
  13:46:02 STEP 1291 H= 322:45 +CPU= 16.488
  13:46:19 STEP 1292 H= 323:00 +CPU= 17.165
  13:46:36 STEP 1293 H= 323:15 +CPU= 17.272
  13:46:55 STEP 1294 H= 323:30 +CPU= 18.665
  13:47:14 STEP 1295 H= 323:45 +CPU= 18.744
  13:47:38 STEP 1296 H= 324:00 +CPU= 24.416
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001296
  13:47:55 STEP 1297 H= 324:15 +CPU= 17.097
  13:48:13 STEP 1298 H= 324:30 +CPU= 18.217
  13:48:32 STEP 1299 H= 324:45 +CPU= 18.128
  13:48:51 STEP 1300 H= 325:00 +CPU= 19.491
  13:49:10 STEP 1301 H= 325:15 +CPU= 18.729
  13:49:28 STEP 1302 H= 325:30 +CPU= 18.570
  13:49:46 STEP 1303 H= 325:45 +CPU= 17.971
  13:50:05 STEP 1304 H= 326:00 +CPU= 18.653
  13:50:23 STEP 1305 H= 326:15 +CPU= 17.576
  13:50:39 STEP 1306 H= 326:30 +CPU= 16.529
  13:50:56 STEP 1307 H= 326:45 +CPU= 17.162
  13:51:18 STEP 1308 H= 327:00 +CPU= 21.943
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001308
  13:51:37 STEP 1309 H= 327:15 +CPU= 18.498
  13:51:55 STEP 1310 H= 327:30 +CPU= 18.489
  13:52:12 STEP 1311 H= 327:45 +CPU= 17.074
  13:52:29 STEP 1312 H= 328:00 +CPU= 16.597
  13:52:46 STEP 1313 H= 328:15 +CPU= 16.618
  13:53:02 STEP 1314 H= 328:30 +CPU= 16.451
  13:53:18 STEP 1315 H= 328:45 +CPU= 16.530
  13:53:35 STEP 1316 H= 329:00 +CPU= 16.730
  13:53:51 STEP 1317 H= 329:15 +CPU= 15.348
  13:54:06 STEP 1318 H= 329:30 +CPU= 15.402
  13:54:21 STEP 1319 H= 329:45 +CPU= 15.447
  13:54:42 STEP 1320 H= 330:00 +CPU= 20.072
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001320
  13:54:57 STEP 1321 H= 330:15 +CPU= 15.682
  13:55:13 STEP 1322 H= 330:30 +CPU= 15.652
  13:55:29 STEP 1323 H= 330:45 +CPU= 15.709
  13:55:45 STEP 1324 H= 331:00 +CPU= 16.101
  13:56:00 STEP 1325 H= 331:15 +CPU= 15.534
  13:56:16 STEP 1326 H= 331:30 +CPU= 15.348
  13:56:31 STEP 1327 H= 331:45 +CPU= 15.665
  13:56:47 STEP 1328 H= 332:00 +CPU= 15.366
  13:57:03 STEP 1329 H= 332:15 +CPU= 16.125
  13:57:19 STEP 1330 H= 332:30 +CPU= 16.126
  13:57:35 STEP 1331 H= 332:45 +CPU= 16.381
  13:57:56 STEP 1332 H= 333:00 +CPU= 20.875
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001332
  13:58:12 STEP 1333 H= 333:15 +CPU= 15.973
  13:58:28 STEP 1334 H= 333:30 +CPU= 15.815
  13:58:44 STEP 1335 H= 333:45 +CPU= 16.093
  13:59:00 STEP 1336 H= 334:00 +CPU= 15.755
  13:59:16 STEP 1337 H= 334:15 +CPU= 15.909
  13:59:31 STEP 1338 H= 334:30 +CPU= 15.431
  13:59:46 STEP 1339 H= 334:45 +CPU= 15.179
  14:00:02 STEP 1340 H= 335:00 +CPU= 15.613
  14:00:17 STEP 1341 H= 335:15 +CPU= 15.103
  14:00:33 STEP 1342 H= 335:30 +CPU= 15.707
  14:00:49 STEP 1343 H= 335:45 +CPU= 16.333
  14:01:12 STEP 1344 H= 336:00 +CPU= 22.518
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_13.zip
Uploading the intermediate file: upload_file_13.zip
  14:01:27 STEP 1345 H= 336:15 +CPU= 15.494
  14:01:42 STEP 1346 H= 336:30 +CPU= 15.073
  14:01:57 STEP 1347 H= 336:45 +CPU= 14.366
  14:02:12 STEP 1348 H= 337:00 +CPU= 15.611
  14:02:26 STEP 1349 H= 337:15 +CPU= 14.364
  14:02:42 STEP 1350 H= 337:30 +CPU= 15.193
  14:02:56 STEP 1351 H= 337:45 +CPU= 14.394
  14:03:12 STEP 1352 H= 338:00 +CPU= 15.631
  14:03:26 STEP 1353 H= 338:15 +CPU= 14.183
  14:03:41 STEP 1354 H= 338:30 +CPU= 15.130
  14:03:56 STEP 1355 H= 338:45 +CPU= 15.084
  14:04:16 STEP 1356 H= 339:00 +CPU= 19.650
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001356
  14:04:31 STEP 1357 H= 339:15 +CPU= 15.523
  14:04:47 STEP 1358 H= 339:30 +CPU= 15.549
  14:05:02 STEP 1359 H= 339:45 +CPU= 15.459
  14:05:18 STEP 1360 H= 340:00 +CPU= 15.967
  14:05:34 STEP 1361 H= 340:15 +CPU= 15.528
  14:05:49 STEP 1362 H= 340:30 +CPU= 14.935
  14:06:03 STEP 1363 H= 340:45 +CPU= 14.581
  14:06:19 STEP 1364 H= 341:00 +CPU= 15.528
  14:06:34 STEP 1365 H= 341:15 +CPU= 15.271
  14:06:50 STEP 1366 H= 341:30 +CPU= 15.398
  14:07:05 STEP 1367 H= 341:45 +CPU= 15.727
  14:07:25 STEP 1368 H= 342:00 +CPU= 19.753
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001368
  14:07:41 STEP 1369 H= 342:15 +CPU= 15.615
  14:07:56 STEP 1370 H= 342:30 +CPU= 15.609
  14:08:12 STEP 1371 H= 342:45 +CPU= 15.560
  14:08:28 STEP 1372 H= 343:00 +CPU= 15.762
  14:08:43 STEP 1373 H= 343:15 +CPU= 15.261
  14:08:58 STEP 1374 H= 343:30 +CPU= 15.132
  14:09:12 STEP 1375 H= 343:45 +CPU= 14.240
  14:09:27 STEP 1376 H= 344:00 +CPU= 14.348
  14:09:42 STEP 1377 H= 344:15 +CPU= 15.503
  14:09:57 STEP 1378 H= 344:30 +CPU= 15.138
  14:10:13 STEP 1379 H= 344:45 +CPU= 15.535
  14:10:32 STEP 1380 H= 345:00 +CPU= 19.404
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001380
  14:10:48 STEP 1381 H= 345:15 +CPU= 15.763
  14:11:04 STEP 1382 H= 345:30 +CPU= 15.952
  14:11:20 STEP 1383 H= 345:45 +CPU= 15.876
  14:11:37 STEP 1384 H= 346:00 +CPU= 16.769
  14:11:52 STEP 1385 H= 346:15 +CPU= 15.643
  14:12:07 STEP 1386 H= 346:30 +CPU= 14.859
  14:12:22 STEP 1387 H= 346:45 +CPU= 14.586
  14:12:37 STEP 1388 H= 347:00 +CPU= 15.273
  14:12:52 STEP 1389 H= 347:15 +CPU= 14.622
  14:13:06 STEP 1390 H= 347:30 +CPU= 14.790
  14:13:21 STEP 1391 H= 347:45 +CPU= 14.667
  14:13:43 STEP 1392 H= 348:00 +CPU= 21.914
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001392
  14:13:59 STEP 1393 H= 348:15 +CPU= 15.888
  14:14:14 STEP 1394 H= 348:30 +CPU= 15.198
  14:14:29 STEP 1395 H= 348:45 +CPU= 15.387
  14:14:45 STEP 1396 H= 349:00 +CPU= 16.054
  14:15:02 STEP 1397 H= 349:15 +CPU= 16.102
  14:15:17 STEP 1398 H= 349:30 +CPU= 15.284
  14:15:31 STEP 1399 H= 349:45 +CPU= 14.279
  14:15:46 STEP 1400 H= 350:00 +CPU= 14.944
  14:16:00 STEP 1401 H= 350:15 +CPU= 14.330
  14:16:15 STEP 1402 H= 350:30 +CPU= 14.534
  14:16:30 STEP 1403 H= 350:45 +CPU= 14.812
  14:16:49 STEP 1404 H= 351:00 +CPU= 19.130
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001404
  14:17:03 STEP 1405 H= 351:15 +CPU= 14.620
  14:17:18 STEP 1406 H= 351:30 +CPU= 14.928
  14:17:34 STEP 1407 H= 351:45 +CPU= 15.167
  14:17:49 STEP 1408 H= 352:00 +CPU= 15.075
  14:18:03 STEP 1409 H= 352:15 +CPU= 14.421
  14:18:18 STEP 1410 H= 352:30 +CPU= 14.982
  14:18:33 STEP 1411 H= 352:45 +CPU= 14.793
  14:18:48 STEP 1412 H= 353:00 +CPU= 15.146
  14:19:02 STEP 1413 H= 353:15 +CPU= 14.114
  14:19:16 STEP 1414 H= 353:30 +CPU= 13.629
  14:19:30 STEP 1415 H= 353:45 +CPU= 13.770
  14:19:48 STEP 1416 H= 354:00 +CPU= 18.301
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001416
  14:20:02 STEP 1417 H= 354:15 +CPU= 14.308
  14:20:16 STEP 1418 H= 354:30 +CPU= 14.156
  14:20:31 STEP 1419 H= 354:45 +CPU= 14.404
  14:20:46 STEP 1420 H= 355:00 +CPU= 14.961
  14:21:00 STEP 1421 H= 355:15 +CPU= 14.183
  14:21:14 STEP 1422 H= 355:30 +CPU= 14.318
  14:21:28 STEP 1423 H= 355:45 +CPU= 14.238
  14:21:43 STEP 1424 H= 356:00 +CPU= 14.793
  14:21:57 STEP 1425 H= 356:15 +CPU= 14.180
  14:22:12 STEP 1426 H= 356:30 +CPU= 14.144
  14:22:26 STEP 1427 H= 356:45 +CPU= 14.118
  14:22:44 STEP 1428 H= 357:00 +CPU= 18.264
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001428
  14:22:58 STEP 1429 H= 357:15 +CPU= 14.127
  14:23:12 STEP 1430 H= 357:30 +CPU= 14.261
  14:23:27 STEP 1431 H= 357:45 +CPU= 14.198
  14:23:41 STEP 1432 H= 358:00 +CPU= 14.843
  14:23:56 STEP 1433 H= 358:15 +CPU= 14.314
  14:24:10 STEP 1434 H= 358:30 +CPU= 14.322
  14:24:24 STEP 1435 H= 358:45 +CPU= 14.034
  14:24:39 STEP 1436 H= 359:00 +CPU= 14.767
  14:24:53 STEP 1437 H= 359:15 +CPU= 14.235
  14:25:07 STEP 1438 H= 359:30 +CPU= 14.229
  14:25:22 STEP 1439 H= 359:45 +CPU= 14.219
  14:25:40 STEP 1440 H= 360:00 +CPU= 18.861
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.3126498536E-21      I       0.1513301717E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4330632700E-01      I
I  Relat. Hum. I      0.4065088723E-01      I       0.3109248557E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.5853781219E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    324724868 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS811812.2 Mbytes/s, TOTAL TIME=    0.0062( 6.5%)
 14:25:40 STEP 1440 H= 360:00 +CPU= 18.861
 END CNT3
 NSTEP =  1440 CNT0   000000000  13.552  13.552  13.554   1.000               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        10010/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    178    0       46      1044068    10010 ************    13.8     1.1    12.6     1.5    26.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 7450768.6 7450768.6    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 7448237.5 7448237.5     99.97      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  365    2980.8    2980.8     14.60      0.00
   9 SPCM     - SPECTRAL COMP.                     336     734.1     734.1      3.31      0.00
  10 SCAN2M - PHYSICS                              337   11253.3   11253.3     50.90      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 28     620.0     620.0      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 15      43.8      43.8      0.01      0.00
  17 GRIDFPOS IN CNT4                               28      25.7      25.7      0.01      0.00
  27 RESTART FILES - WRITING                         7    2479.9    2479.9      0.23      0.00
  28 RESTART FILES - READING                         1    2148.7    2148.7      0.03      0.00
  29 SU4FPOS IN CNT4                                28       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                28    4761.3    4761.3      1.79      0.00
  31 POSDDH IN STEPO                                 2       0.8       0.8      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    337    4432.0    4432.0     20.05      0.00
  39 SU0YOMB                                         1     260.6     260.6      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    337      17.7      17.7      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              336     285.6     285.6      1.29      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              336      43.9      43.9      0.20      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 15       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      732     283.7     283.7      2.79      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       700     178.0     178.0      1.67      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        700      57.0      57.0      0.54      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       732      97.3      97.3      0.96      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      39.9      39.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            732      29.5      29.5      0.29      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            700      18.6      18.6      0.18      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            732      92.2      92.2      0.91      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            700      42.2      42.2      0.40      0.00
 400 GSTATS                                      39960       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 37126       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     7450.8 CPU TIME    7443.3 VECTOR TIME     7443.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      174.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/55/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/55/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/55/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287419/ICMUAh7zg+001344
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
14:29:12 (2238402): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1_r267924179_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
10 Jun 2024 19:08:44 1532385 22440048 oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1 1,295,100 42,952 0.0332
10 Jun 2024 11:12:34 1532385 22440048 oifs_43r3_bl_a4bc_2016092300_20_1017_12287419_1 864,000 29,837 0.0345


©2024 climateprediction.net