climateprediction.net home page
Task 22308474

Task 22308474

Name oifs_43r3_bl_a15w_2016092300_15_991_12208295_0
Workunit 12208295
Created 13 Feb 2023, 14:47:25 UTC
Sent 14 Feb 2023, 6:59:00 UTC
Report deadline 15 Apr 2023, 6:59:00 UTC
Received 14 Feb 2023, 21:53:57 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1531398
Run time 13 hours 23 min 56 sec
CPU time 13 hours 23 min 21 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 7.93 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,604.00 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,627.00 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
7:06 STEP  952 H= 238:00 +CPU= 36.070
  16:37:42 STEP  953 H= 238:15 +CPU= 36.037
  16:38:18 STEP  954 H= 238:30 +CPU= 35.346
  16:38:53 STEP  955 H= 238:45 +CPU= 35.417
  16:39:32 STEP  956 H= 239:00 +CPU= 38.111
  16:40:09 STEP  957 H= 239:15 +CPU= 36.631
  16:40:44 STEP  958 H= 239:30 +CPU= 35.169
  16:41:21 STEP  959 H= 239:45 +CPU= 36.455
  16:42:10 STEP  960 H= 240:00 +CPU= 48.813
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538_9.zip
Uploading the intermediate file: upload_file_9.zip
Uploading trickle at timestep: 864000
  16:42:47 STEP  961 H= 240:15 +CPU= 35.795
  16:43:23 STEP  962 H= 240:30 +CPU= 36.136
  16:43:59 STEP  963 H= 240:45 +CPU= 34.675
  16:44:37 STEP  964 H= 241:00 +CPU= 37.711
  16:45:12 STEP  965 H= 241:15 +CPU= 35.569
  16:45:48 STEP  966 H= 241:30 +CPU= 35.980
  16:46:24 STEP  967 H= 241:45 +CPU= 34.896
  16:47:01 STEP  968 H= 242:00 +CPU= 36.826
  16:47:38 STEP  969 H= 242:15 +CPU= 37.015
  16:48:14 STEP  970 H= 242:30 +CPU= 35.637
  16:48:51 STEP  971 H= 242:45 +CPU= 35.099
  16:49:35 STEP  972 H= 243:00 +CPU= 44.166
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000972
  16:50:11 STEP  973 H= 243:15 +CPU= 35.550
  16:50:47 STEP  974 H= 243:30 +CPU= 36.251
  16:51:24 STEP  975 H= 243:45 +CPU= 36.017
  16:52:01 STEP  976 H= 244:00 +CPU= 36.607
  16:52:36 STEP  977 H= 244:15 +CPU= 35.647
  16:53:13 STEP  978 H= 244:30 +CPU= 36.181
  16:53:49 STEP  979 H= 244:45 +CPU= 35.848
  16:54:26 STEP  980 H= 245:00 +CPU= 36.928
  16:55:03 STEP  981 H= 245:15 +CPU= 36.339
  16:55:39 STEP  982 H= 245:30 +CPU= 35.884
  16:56:15 STEP  983 H= 245:45 +CPU= 35.879
  16:57:03 STEP  984 H= 246:00 +CPU= 47.644
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000984
  16:57:39 STEP  985 H= 246:15 +CPU= 36.462
  16:58:15 STEP  986 H= 246:30 +CPU= 35.986
  16:58:51 STEP  987 H= 246:45 +CPU= 35.200
  16:59:30 STEP  988 H= 247:00 +CPU= 38.510
  17:00:06 STEP  989 H= 247:15 +CPU= 35.838
  17:00:42 STEP  990 H= 247:30 +CPU= 35.933
  17:01:19 STEP  991 H= 247:45 +CPU= 37.013
  17:01:57 STEP  992 H= 248:00 +CPU= 37.495
  17:02:32 STEP  993 H= 248:15 +CPU= 35.092
  17:03:08 STEP  994 H= 248:30 +CPU= 35.255
  17:03:44 STEP  995 H= 248:45 +CPU= 36.680
  17:04:32 STEP  996 H= 249:00 +CPU= 46.216
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000996
  17:05:08 STEP  997 H= 249:15 +CPU= 35.940
  17:05:44 STEP  998 H= 249:30 +CPU= 36.163
  17:06:20 STEP  999 H= 249:45 +CPU= 35.438
  17:06:57 STEP 1000 H= 250:00 +CPU= 36.445
  17:07:33 STEP 1001 H= 250:15 +CPU= 36.316
  17:08:10 STEP 1002 H= 250:30 +CPU= 36.847
  17:08:47 STEP 1003 H= 250:45 +CPU= 35.992
  17:09:23 STEP 1004 H= 251:00 +CPU= 36.302
  17:09:59 STEP 1005 H= 251:15 +CPU= 35.413
  17:10:35 STEP 1006 H= 251:30 +CPU= 35.954
  17:11:11 STEP 1007 H= 251:45 +CPU= 35.868
  17:12:00 STEP 1008 H= 252:00 +CPU= 48.622
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001008
  17:12:37 STEP 1009 H= 252:15 +CPU= 36.207
  17:13:14 STEP 1010 H= 252:30 +CPU= 37.183
  17:13:50 STEP 1011 H= 252:45 +CPU= 35.920
  17:14:27 STEP 1012 H= 253:00 +CPU= 36.931
  17:15:04 STEP 1013 H= 253:15 +CPU= 36.482
  17:15:40 STEP 1014 H= 253:30 +CPU= 35.908
  17:16:17 STEP 1015 H= 253:45 +CPU= 36.735
  17:16:54 STEP 1016 H= 254:00 +CPU= 36.850
  17:17:32 STEP 1017 H= 254:15 +CPU= 38.367
  17:18:08 STEP 1018 H= 254:30 +CPU= 35.636
  17:18:44 STEP 1019 H= 254:45 +CPU= 35.499
  17:19:29 STEP 1020 H= 255:00 +CPU= 44.467
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001020
  17:20:05 STEP 1021 H= 255:15 +CPU= 36.203
  17:20:41 STEP 1022 H= 255:30 +CPU= 35.684
  17:21:18 STEP 1023 H= 255:45 +CPU= 36.889
  17:21:55 STEP 1024 H= 256:00 +CPU= 36.449
  17:22:32 STEP 1025 H= 256:15 +CPU= 36.860
  17:23:08 STEP 1026 H= 256:30 +CPU= 35.997
  17:23:44 STEP 1027 H= 256:45 +CPU= 35.370
  17:24:20 STEP 1028 H= 257:00 +CPU= 36.624
  17:24:57 STEP 1029 H= 257:15 +CPU= 35.879
  17:25:34 STEP 1030 H= 257:30 +CPU= 36.474
  17:26:10 STEP 1031 H= 257:45 +CPU= 35.909
  17:26:56 STEP 1032 H= 258:00 +CPU= 44.993
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001032
  17:27:32 STEP 1033 H= 258:15 +CPU= 36.695
  17:28:11 STEP 1034 H= 258:30 +CPU= 38.577
  17:28:48 STEP 1035 H= 258:45 +CPU= 36.321
  17:29:25 STEP 1036 H= 259:00 +CPU= 37.451
  17:30:01 STEP 1037 H= 259:15 +CPU= 35.706
  17:30:37 STEP 1038 H= 259:30 +CPU= 35.791
  17:31:14 STEP 1039 H= 259:45 +CPU= 35.990
  17:31:50 STEP 1040 H= 260:00 +CPU= 35.219
  17:32:27 STEP 1041 H= 260:15 +CPU= 35.933
  17:33:04 STEP 1042 H= 260:30 +CPU= 35.884
  17:33:40 STEP 1043 H= 260:45 +CPU= 36.365
  17:34:24 STEP 1044 H= 261:00 +CPU= 43.903
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001044
  17:35:01 STEP 1045 H= 261:15 +CPU= 36.258
  17:35:38 STEP 1046 H= 261:30 +CPU= 36.644
  17:36:14 STEP 1047 H= 261:45 +CPU= 36.197
  17:36:50 STEP 1048 H= 262:00 +CPU= 36.032
  17:37:27 STEP 1049 H= 262:15 +CPU= 35.790
  17:38:02 STEP 1050 H= 262:30 +CPU= 35.574
  17:38:38 STEP 1051 H= 262:45 +CPU= 35.006
  17:39:15 STEP 1052 H= 263:00 +CPU= 36.573
  17:39:51 STEP 1053 H= 263:15 +CPU= 35.822
  17:40:27 STEP 1054 H= 263:30 +CPU= 36.406
  17:41:05 STEP 1055 H= 263:45 +CPU= 36.742
  17:41:54 STEP 1056 H= 264:00 +CPU= 48.717
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538_10.zip
Uploading the intermediate file: upload_file_10.zip
  17:42:30 STEP 1057 H= 264:15 +CPU= 35.218
  17:43:06 STEP 1058 H= 264:30 +CPU= 35.353
  17:43:42 STEP 1059 H= 264:45 +CPU= 36.330
  17:44:18 STEP 1060 H= 265:00 +CPU= 35.869
  17:44:54 STEP 1061 H= 265:15 +CPU= 35.559
  17:45:30 STEP 1062 H= 265:30 +CPU= 35.831
  17:46:07 STEP 1063 H= 265:45 +CPU= 36.088
  17:46:44 STEP 1064 H= 266:00 +CPU= 36.413
  17:47:21 STEP 1065 H= 266:15 +CPU= 36.374
  17:47:58 STEP 1066 H= 266:30 +CPU= 36.034
  17:48:34 STEP 1067 H= 266:45 +CPU= 36.155
  17:49:20 STEP 1068 H= 267:00 +CPU= 45.544
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001068
  17:49:56 STEP 1069 H= 267:15 +CPU= 36.007
  17:50:32 STEP 1070 H= 267:30 +CPU= 35.363
  17:51:08 STEP 1071 H= 267:45 +CPU= 36.059
  17:51:45 STEP 1072 H= 268:00 +CPU= 36.231
  17:52:21 STEP 1073 H= 268:15 +CPU= 36.402
  17:52:57 STEP 1074 H= 268:30 +CPU= 36.078
  17:53:34 STEP 1075 H= 268:45 +CPU= 36.564
  17:54:12 STEP 1076 H= 269:00 +CPU= 37.388
  17:54:48 STEP 1077 H= 269:15 +CPU= 36.405
  17:55:25 STEP 1078 H= 269:30 +CPU= 35.924
  17:56:02 STEP 1079 H= 269:45 +CPU= 37.177
  17:56:48 STEP 1080 H= 270:00 +CPU= 45.400
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001080
  17:57:24 STEP 1081 H= 270:15 +CPU= 35.795
  17:58:00 STEP 1082 H= 270:30 +CPU= 36.347
  17:58:38 STEP 1083 H= 270:45 +CPU= 37.391
  17:59:15 STEP 1084 H= 271:00 +CPU= 36.693
  17:59:52 STEP 1085 H= 271:15 +CPU= 37.045
  18:00:28 STEP 1086 H= 271:30 +CPU= 35.478
  18:01:05 STEP 1087 H= 271:45 +CPU= 37.136
  18:01:44 STEP 1088 H= 272:00 +CPU= 38.327
  18:02:21 STEP 1089 H= 272:15 +CPU= 37.153
  18:02:59 STEP 1090 H= 272:30 +CPU= 37.405
  18:03:35 STEP 1091 H= 272:45 +CPU= 35.955
  18:04:20 STEP 1092 H= 273:00 +CPU= 45.141
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001092
  18:04:56 STEP 1093 H= 273:15 +CPU= 35.536
  18:05:33 STEP 1094 H= 273:30 +CPU= 36.855
  18:06:11 STEP 1095 H= 273:45 +CPU= 37.490
  18:06:48 STEP 1096 H= 274:00 +CPU= 36.932
  18:07:24 STEP 1097 H= 274:15 +CPU= 35.619
  18:08:03 STEP 1098 H= 274:30 +CPU= 37.434
  18:08:40 STEP 1099 H= 274:45 +CPU= 36.980
  18:09:16 STEP 1100 H= 275:00 +CPU= 36.514
  18:09:53 STEP 1101 H= 275:15 +CPU= 36.618
  18:10:30 STEP 1102 H= 275:30 +CPU= 36.439
  18:11:06 STEP 1103 H= 275:45 +CPU= 36.464
  18:11:55 STEP 1104 H= 276:00 +CPU= 48.030
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001104
  18:12:31 STEP 1105 H= 276:15 +CPU= 36.552
  18:13:08 STEP 1106 H= 276:30 +CPU= 36.361
  18:13:44 STEP 1107 H= 276:45 +CPU= 35.811
  18:14:21 STEP 1108 H= 277:00 +CPU= 37.276
  18:14:57 STEP 1109 H= 277:15 +CPU= 34.739
  18:15:34 STEP 1110 H= 277:30 +CPU= 36.401
  18:16:11 STEP 1111 H= 277:45 +CPU= 36.461
  18:16:47 STEP 1112 H= 278:00 +CPU= 36.371
  18:17:23 STEP 1113 H= 278:15 +CPU= 35.049
  18:17:58 STEP 1114 H= 278:30 +CPU= 35.344
  18:18:35 STEP 1115 H= 278:45 +CPU= 36.145
  18:19:21 STEP 1116 H= 279:00 +CPU= 45.792
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001116
  18:19:58 STEP 1117 H= 279:15 +CPU= 37.258
  18:20:35 STEP 1118 H= 279:30 +CPU= 36.662
  18:21:11 STEP 1119 H= 279:45 +CPU= 35.277
  18:21:49 STEP 1120 H= 280:00 +CPU= 37.876
  18:22:24 STEP 1121 H= 280:15 +CPU= 35.079
  18:22:57 STEP 1122 H= 280:30 +CPU= 32.397
  18:23:29 STEP 1123 H= 280:45 +CPU= 32.627
  18:24:02 STEP 1124 H= 281:00 +CPU= 32.745
  18:24:35 STEP 1125 H= 281:15 +CPU= 32.556
  18:25:07 STEP 1126 H= 281:30 +CPU= 31.969
  18:25:39 STEP 1127 H= 281:45 +CPU= 32.213
  18:26:19 STEP 1128 H= 282:00 +CPU= 39.425
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001128
  18:26:51 STEP 1129 H= 282:15 +CPU= 32.515
  18:27:24 STEP 1130 H= 282:30 +CPU= 32.310
  18:27:57 STEP 1131 H= 282:45 +CPU= 32.767
  18:28:30 STEP 1132 H= 283:00 +CPU= 32.901
  18:29:02 STEP 1133 H= 283:15 +CPU= 32.013
  18:29:31 STEP 1134 H= 283:30 +CPU= 29.070
  18:29:58 STEP 1135 H= 283:45 +CPU= 26.707
  18:30:28 STEP 1136 H= 284:00 +CPU= 30.351
  18:30:56 STEP 1137 H= 284:15 +CPU= 28.236
  18:31:24 STEP 1138 H= 284:30 +CPU= 27.810
  18:31:55 STEP 1139 H= 284:45 +CPU= 30.268
  18:32:32 STEP 1140 H= 285:00 +CPU= 37.434
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001140
  18:33:01 STEP 1141 H= 285:15 +CPU= 28.981
  18:33:31 STEP 1142 H= 285:30 +CPU= 29.381
  18:34:00 STEP 1143 H= 285:45 +CPU= 28.795
  18:34:30 STEP 1144 H= 286:00 +CPU= 30.468
  18:35:00 STEP 1145 H= 286:15 +CPU= 29.614
  18:35:28 STEP 1146 H= 286:30 +CPU= 28.466
  18:35:57 STEP 1147 H= 286:45 +CPU= 28.778
  18:36:27 STEP 1148 H= 287:00 +CPU= 29.415
  18:36:56 STEP 1149 H= 287:15 +CPU= 29.120
  18:37:25 STEP 1150 H= 287:30 +CPU= 29.001
  18:37:55 STEP 1151 H= 287:45 +CPU= 29.505
  18:38:35 STEP 1152 H= 288:00 +CPU= 40.353
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538_11.zip
Uploading the intermediate file: upload_file_11.zip
  18:39:01 STEP 1153 H= 288:15 +CPU= 25.670
  18:39:27 STEP 1154 H= 288:30 +CPU= 26.042
  18:39:53 STEP 1155 H= 288:45 +CPU= 26.182
  18:40:19 STEP 1156 H= 289:00 +CPU= 25.434
  18:40:45 STEP 1157 H= 289:15 +CPU= 26.016
  18:41:11 STEP 1158 H= 289:30 +CPU= 25.890
  18:41:36 STEP 1159 H= 289:45 +CPU= 25.408
  18:42:02 STEP 1160 H= 290:00 +CPU= 25.699
  18:42:28 STEP 1161 H= 290:15 +CPU= 26.528
  18:42:55 STEP 1162 H= 290:30 +CPU= 26.129
  18:43:21 STEP 1163 H= 290:45 +CPU= 26.098
  18:43:53 STEP 1164 H= 291:00 +CPU= 32.248
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001164
  18:44:18 STEP 1165 H= 291:15 +CPU= 25.320
  18:44:45 STEP 1166 H= 291:30 +CPU= 26.344
  18:45:12 STEP 1167 H= 291:45 +CPU= 27.068
  18:45:39 STEP 1168 H= 292:00 +CPU= 27.369
  18:46:05 STEP 1169 H= 292:15 +CPU= 25.937
  18:46:32 STEP 1170 H= 292:30 +CPU= 26.577
  18:46:58 STEP 1171 H= 292:45 +CPU= 25.880
  18:47:25 STEP 1172 H= 293:00 +CPU= 26.883
  18:47:51 STEP 1173 H= 293:15 +CPU= 26.429
  18:48:18 STEP 1174 H= 293:30 +CPU= 26.339
  18:48:44 STEP 1175 H= 293:45 +CPU= 26.116
  18:49:17 STEP 1176 H= 294:00 +CPU= 32.615
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001176
  18:49:43 STEP 1177 H= 294:15 +CPU= 26.553
  18:50:09 STEP 1178 H= 294:30 +CPU= 26.096
  18:50:35 STEP 1179 H= 294:45 +CPU= 25.925
  18:51:02 STEP 1180 H= 295:00 +CPU= 26.376
  18:51:28 STEP 1181 H= 295:15 +CPU= 25.749
  18:51:54 STEP 1182 H= 295:30 +CPU= 26.200
  18:52:20 STEP 1183 H= 295:45 +CPU= 26.146
  18:52:47 STEP 1184 H= 296:00 +CPU= 27.010
  18:53:14 STEP 1185 H= 296:15 +CPU= 26.513
  18:53:39 STEP 1186 H= 296:30 +CPU= 25.615
  18:54:07 STEP 1187 H= 296:45 +CPU= 27.042
  18:54:39 STEP 1188 H= 297:00 +CPU= 32.765
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001188
  18:55:06 STEP 1189 H= 297:15 +CPU= 26.290
  18:55:33 STEP 1190 H= 297:30 +CPU= 26.965
  18:55:59 STEP 1191 H= 297:45 +CPU= 26.353
  18:56:25 STEP 1192 H= 298:00 +CPU= 25.933
  18:56:52 STEP 1193 H= 298:15 +CPU= 27.116
  18:57:20 STEP 1194 H= 298:30 +CPU= 27.249
  18:57:46 STEP 1195 H= 298:45 +CPU= 25.845
  18:58:13 STEP 1196 H= 299:00 +CPU= 27.168
  18:58:37 STEP 1197 H= 299:15 +CPU= 23.715
  18:59:00 STEP 1198 H= 299:30 +CPU= 23.314
  18:59:23 STEP 1199 H= 299:45 +CPU= 23.388
  18:59:54 STEP 1200 H= 300:00 +CPU= 30.431
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001200
  19:00:17 STEP 1201 H= 300:15 +CPU= 23.321
  19:00:42 STEP 1202 H= 300:30 +CPU= 24.398
  19:01:04 STEP 1203 H= 300:45 +CPU= 22.359
  19:01:28 STEP 1204 H= 301:00 +CPU= 23.726
  19:01:51 STEP 1205 H= 301:15 +CPU= 22.769
  19:02:14 STEP 1206 H= 301:30 +CPU= 23.807
  19:02:38 STEP 1207 H= 301:45 +CPU= 23.037
  19:03:02 STEP 1208 H= 302:00 +CPU= 24.795
  19:03:27 STEP 1209 H= 302:15 +CPU= 24.234
  19:03:51 STEP 1210 H= 302:30 +CPU= 24.076
  19:04:15 STEP 1211 H= 302:45 +CPU= 23.799
  19:04:44 STEP 1212 H= 303:00 +CPU= 29.434
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001212
  19:05:08 STEP 1213 H= 303:15 +CPU= 24.082
  19:05:32 STEP 1214 H= 303:30 +CPU= 23.376
  19:05:55 STEP 1215 H= 303:45 +CPU= 23.522
  19:06:20 STEP 1216 H= 304:00 +CPU= 24.564
  19:06:44 STEP 1217 H= 304:15 +CPU= 23.891
  19:07:07 STEP 1218 H= 304:30 +CPU= 23.531
  19:07:30 STEP 1219 H= 304:45 +CPU= 22.933
  19:07:55 STEP 1220 H= 305:00 +CPU= 24.426
  19:08:19 STEP 1221 H= 305:15 +CPU= 23.619
  19:08:43 STEP 1222 H= 305:30 +CPU= 23.881
  19:09:06 STEP 1223 H= 305:45 +CPU= 23.468
  19:09:34 STEP 1224 H= 306:00 +CPU= 28.329
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001224
  19:09:55 STEP 1225 H= 306:15 +CPU= 20.455
  19:10:15 STEP 1226 H= 306:30 +CPU= 20.455
  19:10:37 STEP 1227 H= 306:45 +CPU= 21.951
  19:10:59 STEP 1228 H= 307:00 +CPU= 21.484
  19:11:19 STEP 1229 H= 307:15 +CPU= 20.607
  19:11:39 STEP 1230 H= 307:30 +CPU= 19.270
  19:12:00 STEP 1231 H= 307:45 +CPU= 21.326
  19:12:21 STEP 1232 H= 308:00 +CPU= 20.579
  19:12:38 STEP 1233 H= 308:15 +CPU= 17.602
  19:12:57 STEP 1234 H= 308:30 +CPU= 18.813
  19:13:16 STEP 1235 H= 308:45 +CPU= 18.502
  19:13:38 STEP 1236 H= 309:00 +CPU= 22.413
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001236
  19:13:55 STEP 1237 H= 309:15 +CPU= 16.661
  19:14:13 STEP 1238 H= 309:30 +CPU= 18.308
  19:14:32 STEP 1239 H= 309:45 +CPU= 18.716
  19:14:51 STEP 1240 H= 310:00 +CPU= 19.152
  19:15:10 STEP 1241 H= 310:15 +CPU= 18.837
  19:15:28 STEP 1242 H= 310:30 +CPU= 18.272
  19:15:47 STEP 1243 H= 310:45 +CPU= 18.606
  19:16:07 STEP 1244 H= 311:00 +CPU= 20.184
  19:16:26 STEP 1245 H= 311:15 +CPU= 19.091
  19:16:45 STEP 1246 H= 311:30 +CPU= 19.038
  19:17:05 STEP 1247 H= 311:45 +CPU= 19.420
  19:17:32 STEP 1248 H= 312:00 +CPU= 26.743
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538_12.zip
Uploading the intermediate file: upload_file_12.zip
  19:17:51 STEP 1249 H= 312:15 +CPU= 18.958
  19:18:10 STEP 1250 H= 312:30 +CPU= 19.714
  19:18:30 STEP 1251 H= 312:45 +CPU= 19.985
  19:18:50 STEP 1252 H= 313:00 +CPU= 19.667
  19:19:09 STEP 1253 H= 313:15 +CPU= 18.780
  19:19:28 STEP 1254 H= 313:30 +CPU= 19.278
  19:19:48 STEP 1255 H= 313:45 +CPU= 19.443
  19:20:07 STEP 1256 H= 314:00 +CPU= 19.820
  19:20:26 STEP 1257 H= 314:15 +CPU= 18.708
  19:20:46 STEP 1258 H= 314:30 +CPU= 19.609
  19:21:05 STEP 1259 H= 314:45 +CPU= 19.156
  19:21:30 STEP 1260 H= 315:00 +CPU= 24.393
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001260
  19:21:49 STEP 1261 H= 315:15 +CPU= 19.478
  19:22:08 STEP 1262 H= 315:30 +CPU= 18.900
  19:22:28 STEP 1263 H= 315:45 +CPU= 19.599
  19:22:47 STEP 1264 H= 316:00 +CPU= 19.853
  19:23:06 STEP 1265 H= 316:15 +CPU= 18.905
  19:23:26 STEP 1266 H= 316:30 +CPU= 19.281
  19:23:45 STEP 1267 H= 316:45 +CPU= 19.435
  19:24:05 STEP 1268 H= 317:00 +CPU= 20.064
  19:24:25 STEP 1269 H= 317:15 +CPU= 19.369
  19:24:45 STEP 1270 H= 317:30 +CPU= 20.100
  19:25:04 STEP 1271 H= 317:45 +CPU= 19.314
  19:25:29 STEP 1272 H= 318:00 +CPU= 24.561
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001272
  19:25:49 STEP 1273 H= 318:15 +CPU= 19.730
  19:26:08 STEP 1274 H= 318:30 +CPU= 19.583
  19:26:28 STEP 1275 H= 318:45 +CPU= 19.630
  19:26:48 STEP 1276 H= 319:00 +CPU= 19.922
  19:27:07 STEP 1277 H= 319:15 +CPU= 19.546
  19:27:27 STEP 1278 H= 319:30 +CPU= 19.541
  19:27:46 STEP 1279 H= 319:45 +CPU= 19.238
  19:28:06 STEP 1280 H= 320:00 +CPU= 19.690
  19:28:25 STEP 1281 H= 320:15 +CPU= 19.118
  19:28:44 STEP 1282 H= 320:30 +CPU= 19.321
  19:29:04 STEP 1283 H= 320:45 +CPU= 19.326
  19:29:27 STEP 1284 H= 321:00 +CPU= 23.182
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001284
  19:29:47 STEP 1285 H= 321:15 +CPU= 19.887
  19:30:06 STEP 1286 H= 321:30 +CPU= 18.880
  19:30:25 STEP 1287 H= 321:45 +CPU= 19.528
  19:30:45 STEP 1288 H= 322:00 +CPU= 19.591
  19:31:05 STEP 1289 H= 322:15 +CPU= 19.662
  19:31:24 STEP 1290 H= 322:30 +CPU= 19.660
  19:31:44 STEP 1291 H= 322:45 +CPU= 19.332
  19:32:04 STEP 1292 H= 323:00 +CPU= 19.827
  19:32:23 STEP 1293 H= 323:15 +CPU= 19.274
  19:32:42 STEP 1294 H= 323:30 +CPU= 19.132
  19:33:01 STEP 1295 H= 323:45 +CPU= 19.291
  19:33:28 STEP 1296 H= 324:00 +CPU= 26.209
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001296
  19:33:47 STEP 1297 H= 324:15 +CPU= 19.429
  19:34:06 STEP 1298 H= 324:30 +CPU= 18.560
  19:34:25 STEP 1299 H= 324:45 +CPU= 19.026
  19:34:45 STEP 1300 H= 325:00 +CPU= 19.837
  19:35:04 STEP 1301 H= 325:15 +CPU= 19.332
  19:35:23 STEP 1302 H= 325:30 +CPU= 19.297
  19:35:43 STEP 1303 H= 325:45 +CPU= 19.693
  19:36:03 STEP 1304 H= 326:00 +CPU= 20.217
  19:36:23 STEP 1305 H= 326:15 +CPU= 19.405
  19:36:42 STEP 1306 H= 326:30 +CPU= 18.975
  19:37:01 STEP 1307 H= 326:45 +CPU= 19.245
  19:37:25 STEP 1308 H= 327:00 +CPU= 23.589
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001308
  19:37:44 STEP 1309 H= 327:15 +CPU= 19.109
  19:38:03 STEP 1310 H= 327:30 +CPU= 19.327
  19:38:22 STEP 1311 H= 327:45 +CPU= 19.204
  19:38:42 STEP 1312 H= 328:00 +CPU= 19.837
  19:39:02 STEP 1313 H= 328:15 +CPU= 19.665
  19:39:22 STEP 1314 H= 328:30 +CPU= 19.417
  19:39:41 STEP 1315 H= 328:45 +CPU= 19.164
  19:40:00 STEP 1316 H= 329:00 +CPU= 19.613
  19:40:20 STEP 1317 H= 329:15 +CPU= 19.686
  19:40:39 STEP 1318 H= 329:30 +CPU= 18.981
  19:40:58 STEP 1319 H= 329:45 +CPU= 19.398
  19:41:23 STEP 1320 H= 330:00 +CPU= 24.649
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001320
  19:41:43 STEP 1321 H= 330:15 +CPU= 19.677
  19:42:02 STEP 1322 H= 330:30 +CPU= 19.467
  19:42:22 STEP 1323 H= 330:45 +CPU= 19.672
  19:42:42 STEP 1324 H= 331:00 +CPU= 19.909
  19:43:02 STEP 1325 H= 331:15 +CPU= 19.500
  19:43:20 STEP 1326 H= 331:30 +CPU= 18.769
  19:43:40 STEP 1327 H= 331:45 +CPU= 19.435
  19:44:00 STEP 1328 H= 332:00 +CPU= 20.410
  19:44:20 STEP 1329 H= 332:15 +CPU= 19.305
  19:44:39 STEP 1330 H= 332:30 +CPU= 19.390
  19:44:58 STEP 1331 H= 332:45 +CPU= 18.633
  19:45:22 STEP 1332 H= 333:00 +CPU= 24.286
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001332
  19:45:41 STEP 1333 H= 333:15 +CPU= 19.168
  19:46:01 STEP 1334 H= 333:30 +CPU= 19.609
  19:46:21 STEP 1335 H= 333:45 +CPU= 19.969
  19:46:41 STEP 1336 H= 334:00 +CPU= 20.274
  19:47:00 STEP 1337 H= 334:15 +CPU= 18.911
  19:47:19 STEP 1338 H= 334:30 +CPU= 18.638
  19:47:38 STEP 1339 H= 334:45 +CPU= 19.373
  19:47:58 STEP 1340 H= 335:00 +CPU= 19.829
  19:48:17 STEP 1341 H= 335:15 +CPU= 18.990
  19:48:37 STEP 1342 H= 335:30 +CPU= 19.426
  19:48:56 STEP 1343 H= 335:45 +CPU= 19.035
  19:49:22 STEP 1344 H= 336:00 +CPU= 26.664
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538_13.zip
Uploading the intermediate file: upload_file_13.zip
  19:49:41 STEP 1345 H= 336:15 +CPU= 19.148
  19:50:01 STEP 1346 H= 336:30 +CPU= 19.301
  19:50:20 STEP 1347 H= 336:45 +CPU= 19.092
  19:50:40 STEP 1348 H= 337:00 +CPU= 20.290
  19:51:00 STEP 1349 H= 337:15 +CPU= 19.531
  19:51:19 STEP 1350 H= 337:30 +CPU= 19.463
  19:51:39 STEP 1351 H= 337:45 +CPU= 19.947
  19:51:59 STEP 1352 H= 338:00 +CPU= 19.756
  19:52:18 STEP 1353 H= 338:15 +CPU= 18.917
  19:52:38 STEP 1354 H= 338:30 +CPU= 19.692
  19:52:58 STEP 1355 H= 338:45 +CPU= 19.685
  19:53:23 STEP 1356 H= 339:00 +CPU= 24.864
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001356
  19:53:42 STEP 1357 H= 339:15 +CPU= 19.434
  19:54:02 STEP 1358 H= 339:30 +CPU= 19.583
  19:54:22 STEP 1359 H= 339:45 +CPU= 19.780
  19:54:42 STEP 1360 H= 340:00 +CPU= 20.536
  19:55:02 STEP 1361 H= 340:15 +CPU= 19.626
  19:55:21 STEP 1362 H= 340:30 +CPU= 18.806
  19:55:41 STEP 1363 H= 340:45 +CPU= 19.499
  19:56:01 STEP 1364 H= 341:00 +CPU= 20.172
  19:56:22 STEP 1365 H= 341:15 +CPU= 19.876
  19:56:43 STEP 1366 H= 341:30 +CPU= 19.775
  19:57:03 STEP 1367 H= 341:45 +CPU= 20.043
  19:57:28 STEP 1368 H= 342:00 +CPU= 24.708
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001368
  19:57:48 STEP 1369 H= 342:15 +CPU= 19.494
  19:58:08 STEP 1370 H= 342:30 +CPU= 19.320
  19:58:28 STEP 1371 H= 342:45 +CPU= 19.613
  19:58:48 STEP 1372 H= 343:00 +CPU= 19.871
  19:59:08 STEP 1373 H= 343:15 +CPU= 19.577
  19:59:27 STEP 1374 H= 343:30 +CPU= 19.148
  19:59:47 STEP 1375 H= 343:45 +CPU= 19.374
  20:00:07 STEP 1376 H= 344:00 +CPU= 19.910
  20:00:28 STEP 1377 H= 344:15 +CPU= 19.633
  20:00:47 STEP 1378 H= 344:30 +CPU= 19.131
  20:01:08 STEP 1379 H= 344:45 +CPU= 20.153
  20:01:34 STEP 1380 H= 345:00 +CPU= 25.455
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001380
  20:01:54 STEP 1381 H= 345:15 +CPU= 19.746
  20:02:14 STEP 1382 H= 345:30 +CPU= 19.593
  20:02:34 STEP 1383 H= 345:45 +CPU= 19.454
  20:02:54 STEP 1384 H= 346:00 +CPU= 19.634
  20:03:13 STEP 1385 H= 346:15 +CPU= 19.256
  20:03:33 STEP 1386 H= 346:30 +CPU= 19.634
  20:03:53 STEP 1387 H= 346:45 +CPU= 19.688
  20:04:14 STEP 1388 H= 347:00 +CPU= 20.206
  20:04:34 STEP 1389 H= 347:15 +CPU= 19.871
  20:04:55 STEP 1390 H= 347:30 +CPU= 19.784
  20:05:15 STEP 1391 H= 347:45 +CPU= 19.376
  20:05:43 STEP 1392 H= 348:00 +CPU= 26.784
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001392
  20:06:03 STEP 1393 H= 348:15 +CPU= 19.904
  20:06:24 STEP 1394 H= 348:30 +CPU= 19.678
  20:06:45 STEP 1395 H= 348:45 +CPU= 19.757
  20:07:06 STEP 1396 H= 349:00 +CPU= 20.292
  20:07:27 STEP 1397 H= 349:15 +CPU= 19.727
  20:07:47 STEP 1398 H= 349:30 +CPU= 19.729
  20:08:07 STEP 1399 H= 349:45 +CPU= 19.555
  20:08:26 STEP 1400 H= 350:00 +CPU= 18.514
  20:08:44 STEP 1401 H= 350:15 +CPU= 17.335
  20:09:01 STEP 1402 H= 350:30 +CPU= 16.827
  20:09:19 STEP 1403 H= 350:45 +CPU= 17.375
  20:09:42 STEP 1404 H= 351:00 +CPU= 22.636
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001404
  20:10:00 STEP 1405 H= 351:15 +CPU= 17.119
  20:10:17 STEP 1406 H= 351:30 +CPU= 16.877
  20:10:32 STEP 1407 H= 351:45 +CPU= 14.903
  20:10:48 STEP 1408 H= 352:00 +CPU= 15.546
  20:11:04 STEP 1409 H= 352:15 +CPU= 15.391
  20:11:19 STEP 1410 H= 352:30 +CPU= 15.281
  20:11:35 STEP 1411 H= 352:45 +CPU= 15.378
  20:11:51 STEP 1412 H= 353:00 +CPU= 15.924
  20:12:08 STEP 1413 H= 353:15 +CPU= 15.789
  20:12:24 STEP 1414 H= 353:30 +CPU= 15.882
  20:12:40 STEP 1415 H= 353:45 +CPU= 15.628
  20:13:01 STEP 1416 H= 354:00 +CPU= 20.134
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001416
  20:13:17 STEP 1417 H= 354:15 +CPU= 15.751
  20:13:33 STEP 1418 H= 354:30 +CPU= 15.582
  20:13:49 STEP 1419 H= 354:45 +CPU= 15.785
  20:14:06 STEP 1420 H= 355:00 +CPU= 16.848
  20:14:23 STEP 1421 H= 355:15 +CPU= 16.061
  20:14:39 STEP 1422 H= 355:30 +CPU= 16.046
  20:14:55 STEP 1423 H= 355:45 +CPU= 15.847
  20:15:12 STEP 1424 H= 356:00 +CPU= 16.728
  20:15:29 STEP 1425 H= 356:15 +CPU= 15.895
  20:15:45 STEP 1426 H= 356:30 +CPU= 15.934
  20:16:01 STEP 1427 H= 356:45 +CPU= 16.326
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a15w
(argv4) batchid: 991
(argv5) wuid: 12208295
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/13
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/13/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/13/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4faef2396f3604c899f25198800ef9a1 to: /var/lib/boinc/slots/13/oifs_43r3_bl_a15w_2016092300_15_991_12208295.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/13/oifs_43r3_bl_a15w_2016092300_15_991_12208295.zip
ic_ancil_file: ic_ancil_12208295
ifsdata_file: ifsdata_12208295
climate_data_file: clim_data_12208295
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/13/ic_ancil_12208295.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/13/ic_ancil_12208295.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c2805489b2e1a7793129430dd3071487 to: /var/lib/boinc/slots/13/ifsdata/ifsdata_12208295.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/13/ifsdata/ifsdata_12208295.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/13/159l_2/clim_data_12208295.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/13/159l_2/clim_data_12208295.zip
Checking for progress XML file: /var/lib/boinc/slots/13/progress_file_12208295.xml
Opened progress file ok : /var/lib/boinc/slots/13/progress_file_12208295.xml
-- Model is restarting --
Adjusting last_iter, 1428, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/13/progress_file_12208295.xml
last_cpu_time: 47370
upload_file_number: 14
last_iter: 1393
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538
The child process has been launched with process id: 8967
Executing the command: /var/lib/boinc/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:tweenis:1:1:8967:8967] [20230214:205300:1676407980:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  20:53:22 STEP 1392 H= 348:00 +CPU= 18.289
  20:53:38 STEP 1393 H= 348:15 +CPU= 16.362
  20:53:55 STEP 1394 H= 348:30 +CPU= 16.141
  20:54:11 STEP 1395 H= 348:45 +CPU= 16.172
  20:54:28 STEP 1396 H= 349:00 +CPU= 16.595
  20:54:45 STEP 1397 H= 349:15 +CPU= 16.185
  20:55:01 STEP 1398 H= 349:30 +CPU= 16.319
  20:55:18 STEP 1399 H= 349:45 +CPU= 16.364
  20:55:35 STEP 1400 H= 350:00 +CPU= 16.522
  20:55:51 STEP 1401 H= 350:15 +CPU= 16.082
  20:56:08 STEP 1402 H= 350:30 +CPU= 16.097
  20:56:24 STEP 1403 H= 350:45 +CPU= 16.111
  20:56:45 STEP 1404 H= 351:00 +CPU= 20.293
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001404
  20:57:02 STEP 1405 H= 351:15 +CPU= 16.559
  20:57:19 STEP 1406 H= 351:30 +CPU= 17.049
  20:57:37 STEP 1407 H= 351:45 +CPU= 16.742
  20:57:55 STEP 1408 H= 352:00 +CPU= 17.133
  20:58:11 STEP 1409 H= 352:15 +CPU= 16.447
  20:58:28 STEP 1410 H= 352:30 +CPU= 16.426
  20:58:45 STEP 1411 H= 352:45 +CPU= 16.246
  20:59:02 STEP 1412 H= 353:00 +CPU= 17.048
  20:59:19 STEP 1413 H= 353:15 +CPU= 16.300
  20:59:37 STEP 1414 H= 353:30 +CPU= 16.538
  20:59:53 STEP 1415 H= 353:45 +CPU= 16.331
  21:00:14 STEP 1416 H= 354:00 +CPU= 20.560
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001416
  21:00:30 STEP 1417 H= 354:15 +CPU= 16.085
  21:00:48 STEP 1418 H= 354:30 +CPU= 16.522
  21:01:04 STEP 1419 H= 354:45 +CPU= 15.953
  21:01:21 STEP 1420 H= 355:00 +CPU= 17.030
  21:01:38 STEP 1421 H= 355:15 +CPU= 16.228
  21:01:54 STEP 1422 H= 355:30 +CPU= 16.466
  21:02:11 STEP 1423 H= 355:45 +CPU= 16.322
  21:02:29 STEP 1424 H= 356:00 +CPU= 16.971
  21:02:46 STEP 1425 H= 356:15 +CPU= 16.787
  21:03:03 STEP 1426 H= 356:30 +CPU= 17.227
  21:03:20 STEP 1427 H= 356:45 +CPU= 17.071
  21:03:41 STEP 1428 H= 357:00 +CPU= 20.561
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001428
  21:03:57 STEP 1429 H= 357:15 +CPU= 16.387
  21:04:14 STEP 1430 H= 357:30 +CPU= 16.714
  21:04:32 STEP 1431 H= 357:45 +CPU= 17.075
  21:04:50 STEP 1432 H= 358:00 +CPU= 17.603
  21:05:07 STEP 1433 H= 358:15 +CPU= 17.070
  21:05:25 STEP 1434 H= 358:30 +CPU= 16.873
  21:05:42 STEP 1435 H= 358:45 +CPU= 16.075
  21:06:00 STEP 1436 H= 359:00 +CPU= 17.448
  21:06:17 STEP 1437 H= 359:15 +CPU= 16.664
  21:06:33 STEP 1438 H= 359:30 +CPU= 16.073
  21:06:50 STEP 1439 H= 359:45 +CPU= 16.573
  21:07:09 STEP 1440 H= 360:00 +CPU= 19.448
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.9864901074E-21      I      -0.9034479483E-06      I
ICloud fractionI      0.0000000000E+00      I       0.1250501891E+00      I
I  Relat. Hum. I      0.6105270673E+00      I       0.5404929456E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1693121259E+04      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    303951441 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0067( 4.5%)
 21:07:09 STEP 1440 H= 360:00 +CPU= 19.448
 END CNT3
 NSTEP =  1440 CNT0   000000000  14.076  14.076  14.160   0.994               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN         1418/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     28    0       34      1044068     1418 ************     2.6     1.2     1.3     1.8     2.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1  849494.1  849494.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1  846565.0  846565.0     99.66      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   53    2520.7    2520.7     15.73      0.00
   9 SPCM     - SPECTRAL COMP.                      48     420.9     420.9      2.38      0.00
  10 SCAN2M - PHYSICS                               49    8756.9    8756.9     50.51      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  4     410.5     410.5      0.19      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  3      20.8      20.8      0.01      0.00
  17 GRIDFPOS IN CNT4                                4      17.8      17.8      0.01      0.00
  27 RESTART FILES - WRITING                         1    1642.1    1642.1      0.19      0.00
  28 RESTART FILES - READING                         1    2605.6    2605.6      0.31      0.00
  29 SU4FPOS IN CNT4                                 4       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                 4    3754.7    3754.7      1.77      0.00
  31 POSDDH IN STEPO                                 1       0.6       0.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     49    3514.1    3514.1     20.27      0.00
  39 SU0YOMB                                         1     241.0     241.0      0.03      0.00
  51 SCAN2M   - SL COMM. PART 1                     49      10.1      10.1      0.06      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               48     204.5     204.5      1.16      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               48      28.9      28.9      0.16      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  3       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      108     256.2     256.2      3.26      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       100     155.6     155.6      1.83      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        100      46.7      46.7      0.55      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       108      79.9      79.9      1.02      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      26.8      26.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            108      18.6      18.6      0.24      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            100      11.6      11.6      0.14      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            108      33.7      33.7      0.43      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            100      18.5      18.5      0.22      0.00
 400 GSTATS                                       6619       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                  6113       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME      849.5 CPU TIME     831.0 VECTOR TIME      831.0


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     1530.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/13/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/13/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12208295/ICMGGh7zg+001356
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a15w_2016092300_15_991_12208295_0_r561731538_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
21:10:38 (8943): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:51:25 1531398 22308474 oifs_43r3_bl_a15w_2016092300_15_991_12208295_0 1,295,100 48,201 0.0372
09 May 2023 16:51:20 1531398 22308474 oifs_43r3_bl_a15w_2016092300_15_991_12208295_0 864,000 34,630 0.0401


©2024 cpdn.org