climateprediction.net home page
Task 22312235

Task 22312235

Name oifs_43r3_bl_a40c_2016092300_15_991_12211983_0
Workunit 12211983
Created 13 Feb 2023, 21:34:16 UTC
Sent 14 Feb 2023, 0:50:01 UTC
Report deadline 15 Apr 2023, 0:50:01 UTC
Received 14 Feb 2023, 13:26:39 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1507812
Run time 11 hours 35 min 53 sec
CPU time 10 hours 36 min 23 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 5.44 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,488.34 MB
Peak swap size 5,930.85 MB
Peak disk usage 1,684.58 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
+CPU= 29.482
  09:20:58 STEP  928 H= 232:00 +CPU= 29.695
  09:21:30 STEP  929 H= 232:15 +CPU= 29.503
  09:22:03 STEP  930 H= 232:30 +CPU= 29.714
  09:22:36 STEP  931 H= 232:45 +CPU= 29.848
  09:23:10 STEP  932 H= 233:00 +CPU= 30.934
  09:23:42 STEP  933 H= 233:15 +CPU= 29.548
  09:24:14 STEP  934 H= 233:30 +CPU= 29.819
  09:24:47 STEP  935 H= 233:45 +CPU= 29.866
  09:25:27 STEP  936 H= 234:00 +CPU= 36.598
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000936
  09:25:59 STEP  937 H= 234:15 +CPU= 29.699
  09:26:31 STEP  938 H= 234:30 +CPU= 29.197
  09:27:03 STEP  939 H= 234:45 +CPU= 29.553
  09:27:38 STEP  940 H= 235:00 +CPU= 30.698
  09:28:11 STEP  941 H= 235:15 +CPU= 29.935
  09:28:44 STEP  942 H= 235:30 +CPU= 30.283
  09:29:17 STEP  943 H= 235:45 +CPU= 30.000
  09:29:50 STEP  944 H= 236:00 +CPU= 30.454
  09:30:24 STEP  945 H= 236:15 +CPU= 30.010
  09:30:57 STEP  946 H= 236:30 +CPU= 29.976
  09:31:29 STEP  947 H= 236:45 +CPU= 29.580
  09:32:09 STEP  948 H= 237:00 +CPU= 36.575
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000948
  09:32:42 STEP  949 H= 237:15 +CPU= 29.808
  09:33:15 STEP  950 H= 237:30 +CPU= 30.062
  09:33:49 STEP  951 H= 237:45 +CPU= 30.103
  09:34:23 STEP  952 H= 238:00 +CPU= 31.406
  09:34:55 STEP  953 H= 238:15 +CPU= 29.661
  09:35:28 STEP  954 H= 238:30 +CPU= 29.967
  09:36:01 STEP  955 H= 238:45 +CPU= 30.434
  09:36:35 STEP  956 H= 239:00 +CPU= 30.512
  09:37:08 STEP  957 H= 239:15 +CPU= 29.951
  09:37:42 STEP  958 H= 239:30 +CPU= 29.995
  09:38:14 STEP  959 H= 239:45 +CPU= 29.868
  09:39:01 STEP  960 H= 240:00 +CPU= 39.708
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251_9.zip
Uploading the intermediate file: upload_file_9.zip
  09:39:34 STEP  961 H= 240:15 +CPU= 29.252
Uploading trickle at timestep: 864000
  09:40:07 STEP  962 H= 240:30 +CPU= 29.804
  09:40:40 STEP  963 H= 240:45 +CPU= 29.707
  09:41:12 STEP  964 H= 241:00 +CPU= 29.852
  09:41:46 STEP  965 H= 241:15 +CPU= 29.918
  09:42:18 STEP  966 H= 241:30 +CPU= 30.032
  09:42:51 STEP  967 H= 241:45 +CPU= 30.030
  09:43:25 STEP  968 H= 242:00 +CPU= 30.382
  09:43:59 STEP  969 H= 242:15 +CPU= 29.169
  09:44:32 STEP  970 H= 242:30 +CPU= 29.798
  09:45:05 STEP  971 H= 242:45 +CPU= 30.236
  09:45:45 STEP  972 H= 243:00 +CPU= 36.800
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000972
  09:46:18 STEP  973 H= 243:15 +CPU= 29.536
  09:46:49 STEP  974 H= 243:30 +CPU= 29.596
  09:47:23 STEP  975 H= 243:45 +CPU= 29.358
  09:47:57 STEP  976 H= 244:00 +CPU= 30.688
  09:48:29 STEP  977 H= 244:15 +CPU= 30.134
  09:49:02 STEP  978 H= 244:30 +CPU= 29.959
  09:49:35 STEP  979 H= 244:45 +CPU= 30.042
  09:50:08 STEP  980 H= 245:00 +CPU= 30.499
  09:50:42 STEP  981 H= 245:15 +CPU= 30.394
  09:51:15 STEP  982 H= 245:30 +CPU= 30.059
  09:51:47 STEP  983 H= 245:45 +CPU= 29.577
  09:52:27 STEP  984 H= 246:00 +CPU= 37.007
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000984
  09:53:00 STEP  985 H= 246:15 +CPU= 29.767
  09:53:33 STEP  986 H= 246:30 +CPU= 30.368
  09:54:06 STEP  987 H= 246:45 +CPU= 29.987
  09:54:40 STEP  988 H= 247:00 +CPU= 30.146
  09:55:14 STEP  989 H= 247:15 +CPU= 29.149
  09:55:46 STEP  990 H= 247:30 +CPU= 29.608
  09:56:19 STEP  991 H= 247:45 +CPU= 29.758
  09:56:52 STEP  992 H= 248:00 +CPU= 29.802
  09:57:24 STEP  993 H= 248:15 +CPU= 29.473
  09:57:57 STEP  994 H= 248:30 +CPU= 30.142
  09:58:30 STEP  995 H= 248:45 +CPU= 30.096
  09:59:11 STEP  996 H= 249:00 +CPU= 36.788
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000996
  09:59:44 STEP  997 H= 249:15 +CPU= 29.956
  10:00:17 STEP  998 H= 249:30 +CPU= 29.886
  10:00:50 STEP  999 H= 249:45 +CPU= 30.108
  10:01:23 STEP 1000 H= 250:00 +CPU= 30.450
  10:01:57 STEP 1001 H= 250:15 +CPU= 30.060
  10:02:29 STEP 1002 H= 250:30 +CPU= 29.947
  10:03:01 STEP 1003 H= 250:45 +CPU= 29.603
  10:03:35 STEP 1004 H= 251:00 +CPU= 30.163
  10:04:09 STEP 1005 H= 251:15 +CPU= 29.749
  10:04:41 STEP 1006 H= 251:30 +CPU= 29.558
  10:05:13 STEP 1007 H= 251:45 +CPU= 29.110
  10:06:01 STEP 1008 H= 252:00 +CPU= 40.795
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001008
  10:06:34 STEP 1009 H= 252:15 +CPU= 29.964
  10:07:07 STEP 1010 H= 252:30 +CPU= 29.839
  10:07:40 STEP 1011 H= 252:45 +CPU= 30.319
  10:08:13 STEP 1012 H= 253:00 +CPU= 29.894
  10:08:46 STEP 1013 H= 253:15 +CPU= 29.399
  10:09:19 STEP 1014 H= 253:30 +CPU= 29.801
  10:09:52 STEP 1015 H= 253:45 +CPU= 29.670
  10:10:25 STEP 1016 H= 254:00 +CPU= 29.628
  10:10:58 STEP 1017 H= 254:15 +CPU= 29.801
  10:11:31 STEP 1018 H= 254:30 +CPU= 29.766
  10:12:04 STEP 1019 H= 254:45 +CPU= 30.043
  10:12:45 STEP 1020 H= 255:00 +CPU= 37.053
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001020
  10:13:17 STEP 1021 H= 255:15 +CPU= 30.437
  10:13:51 STEP 1022 H= 255:30 +CPU= 30.190
  10:14:25 STEP 1023 H= 255:45 +CPU= 30.702
  10:14:58 STEP 1024 H= 256:00 +CPU= 30.235
  10:15:30 STEP 1025 H= 256:15 +CPU= 29.869
  10:16:03 STEP 1026 H= 256:30 +CPU= 30.101
  10:16:36 STEP 1027 H= 256:45 +CPU= 30.000
  10:17:10 STEP 1028 H= 257:00 +CPU= 30.521
  10:17:42 STEP 1029 H= 257:15 +CPU= 29.848
  10:18:16 STEP 1030 H= 257:30 +CPU= 29.930
  10:18:49 STEP 1031 H= 257:45 +CPU= 29.819
  10:19:29 STEP 1032 H= 258:00 +CPU= 37.206
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001032
  10:20:02 STEP 1033 H= 258:15 +CPU= 29.620
  10:20:34 STEP 1034 H= 258:30 +CPU= 29.884
  10:21:07 STEP 1035 H= 258:45 +CPU= 29.949
  10:21:40 STEP 1036 H= 259:00 +CPU= 30.577
  10:22:13 STEP 1037 H= 259:15 +CPU= 29.978
  10:22:47 STEP 1038 H= 259:30 +CPU= 30.210
  10:23:20 STEP 1039 H= 259:45 +CPU= 29.910
  10:23:53 STEP 1040 H= 260:00 +CPU= 30.575
  10:24:26 STEP 1041 H= 260:15 +CPU= 29.913
  10:25:00 STEP 1042 H= 260:30 +CPU= 30.167
  10:25:32 STEP 1043 H= 260:45 +CPU= 30.150
  10:26:13 STEP 1044 H= 261:00 +CPU= 37.333
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001044
  10:26:47 STEP 1045 H= 261:15 +CPU= 30.388
  10:27:20 STEP 1046 H= 261:30 +CPU= 29.891
  10:27:53 STEP 1047 H= 261:45 +CPU= 30.259
  10:28:27 STEP 1048 H= 262:00 +CPU= 29.996
  10:28:59 STEP 1049 H= 262:15 +CPU= 29.795
  10:29:33 STEP 1050 H= 262:30 +CPU= 30.094
  10:30:06 STEP 1051 H= 262:45 +CPU= 29.962
  10:30:40 STEP 1052 H= 263:00 +CPU= 30.390
  10:31:13 STEP 1053 H= 263:15 +CPU= 29.826
  10:31:46 STEP 1054 H= 263:30 +CPU= 30.314
  10:32:20 STEP 1055 H= 263:45 +CPU= 30.198
  10:33:07 STEP 1056 H= 264:00 +CPU= 40.132
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251_10.zip
Uploading the intermediate file: upload_file_10.zip
  10:33:42 STEP 1057 H= 264:15 +CPU= 29.963
  10:34:16 STEP 1058 H= 264:30 +CPU= 30.586
  10:34:49 STEP 1059 H= 264:45 +CPU= 30.020
  10:35:23 STEP 1060 H= 265:00 +CPU= 30.565
  10:35:55 STEP 1061 H= 265:15 +CPU= 30.010
  10:36:29 STEP 1062 H= 265:30 +CPU= 30.321
  10:37:02 STEP 1063 H= 265:45 +CPU= 29.696
  10:37:34 STEP 1064 H= 266:00 +CPU= 29.684
  10:38:10 STEP 1065 H= 266:15 +CPU= 28.983
  10:38:42 STEP 1066 H= 266:30 +CPU= 30.116
  10:39:15 STEP 1067 H= 266:45 +CPU= 29.916
  10:39:55 STEP 1068 H= 267:00 +CPU= 36.646
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001068
  10:40:29 STEP 1069 H= 267:15 +CPU= 30.462
  10:41:02 STEP 1070 H= 267:30 +CPU= 30.010
  10:41:36 STEP 1071 H= 267:45 +CPU= 29.663
  10:42:11 STEP 1072 H= 268:00 +CPU= 30.142
  10:42:43 STEP 1073 H= 268:15 +CPU= 29.800
  10:43:16 STEP 1074 H= 268:30 +CPU= 29.974
  10:43:49 STEP 1075 H= 268:45 +CPU= 30.172
  10:44:24 STEP 1076 H= 269:00 +CPU= 30.281
  10:44:56 STEP 1077 H= 269:15 +CPU= 29.581
  10:45:28 STEP 1078 H= 269:30 +CPU= 29.539
  10:46:00 STEP 1079 H= 269:45 +CPU= 29.691
  10:46:40 STEP 1080 H= 270:00 +CPU= 36.838
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001080
  10:47:13 STEP 1081 H= 270:15 +CPU= 29.954
  10:47:46 STEP 1082 H= 270:30 +CPU= 30.248
  10:48:19 STEP 1083 H= 270:45 +CPU= 29.250
  10:48:53 STEP 1084 H= 271:00 +CPU= 30.460
  10:49:28 STEP 1085 H= 271:15 +CPU= 29.645
  10:50:01 STEP 1086 H= 271:30 +CPU= 30.159
  10:50:35 STEP 1087 H= 271:45 +CPU= 30.274
  10:51:09 STEP 1088 H= 272:00 +CPU= 30.769
  10:51:41 STEP 1089 H= 272:15 +CPU= 29.822
  10:52:15 STEP 1090 H= 272:30 +CPU= 30.418
  10:52:49 STEP 1091 H= 272:45 +CPU= 30.063
  10:53:29 STEP 1092 H= 273:00 +CPU= 37.158
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001092
  10:54:03 STEP 1093 H= 273:15 +CPU= 30.190
  10:54:36 STEP 1094 H= 273:30 +CPU= 30.235
  10:55:09 STEP 1095 H= 273:45 +CPU= 29.972
  10:55:42 STEP 1096 H= 274:00 +CPU= 30.040
  10:56:15 STEP 1097 H= 274:15 +CPU= 29.825
  10:56:48 STEP 1098 H= 274:30 +CPU= 30.131
  10:57:21 STEP 1099 H= 274:45 +CPU= 30.066
  10:57:54 STEP 1100 H= 275:00 +CPU= 30.437
  10:58:27 STEP 1101 H= 275:15 +CPU= 29.883
  10:58:59 STEP 1102 H= 275:30 +CPU= 29.887
  10:59:32 STEP 1103 H= 275:45 +CPU= 30.200
  11:00:22 STEP 1104 H= 276:00 +CPU= 40.693
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001104
  11:00:55 STEP 1105 H= 276:15 +CPU= 30.065
  11:01:28 STEP 1106 H= 276:30 +CPU= 29.595
  11:02:01 STEP 1107 H= 276:45 +CPU= 29.622
  11:02:35 STEP 1108 H= 277:00 +CPU= 30.373
  11:03:08 STEP 1109 H= 277:15 +CPU= 29.947
  11:03:40 STEP 1110 H= 277:30 +CPU= 29.933
  11:04:12 STEP 1111 H= 277:45 +CPU= 29.684
  11:04:47 STEP 1112 H= 278:00 +CPU= 30.615
  11:05:21 STEP 1113 H= 278:15 +CPU= 30.462
  11:05:55 STEP 1114 H= 278:30 +CPU= 30.238
  11:06:27 STEP 1115 H= 278:45 +CPU= 29.981
  11:07:08 STEP 1116 H= 279:00 +CPU= 37.091
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001116
  11:07:41 STEP 1117 H= 279:15 +CPU= 30.112
  11:08:14 STEP 1118 H= 279:30 +CPU= 29.846
  11:08:48 STEP 1119 H= 279:45 +CPU= 30.053
  11:09:21 STEP 1120 H= 280:00 +CPU= 30.042
  11:09:54 STEP 1121 H= 280:15 +CPU= 29.995
  11:10:26 STEP 1122 H= 280:30 +CPU= 29.587
  11:10:59 STEP 1123 H= 280:45 +CPU= 29.861
  11:11:33 STEP 1124 H= 281:00 +CPU= 30.484
  11:12:07 STEP 1125 H= 281:15 +CPU= 30.208
  11:12:40 STEP 1126 H= 281:30 +CPU= 30.209
  11:13:12 STEP 1127 H= 281:45 +CPU= 30.099
  11:13:52 STEP 1128 H= 282:00 +CPU= 37.100
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001128
  11:14:25 STEP 1129 H= 282:15 +CPU= 30.055
  11:14:58 STEP 1130 H= 282:30 +CPU= 29.385
  11:15:30 STEP 1131 H= 282:45 +CPU= 29.696
  11:16:03 STEP 1132 H= 283:00 +CPU= 30.392
  11:16:36 STEP 1133 H= 283:15 +CPU= 30.236
  11:17:09 STEP 1134 H= 283:30 +CPU= 29.725
  11:17:43 STEP 1135 H= 283:45 +CPU= 30.341
  11:18:17 STEP 1136 H= 284:00 +CPU= 30.173
  11:18:51 STEP 1137 H= 284:15 +CPU= 30.430
  11:19:23 STEP 1138 H= 284:30 +CPU= 29.793
  11:19:58 STEP 1139 H= 284:45 +CPU= 30.339
  11:20:39 STEP 1140 H= 285:00 +CPU= 37.296
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001140
  11:21:13 STEP 1141 H= 285:15 +CPU= 30.343
  11:21:47 STEP 1142 H= 285:30 +CPU= 29.977
  11:22:20 STEP 1143 H= 285:45 +CPU= 30.516
  11:22:54 STEP 1144 H= 286:00 +CPU= 30.537
  11:23:27 STEP 1145 H= 286:15 +CPU= 30.001
  11:24:00 STEP 1146 H= 286:30 +CPU= 30.108
  11:24:33 STEP 1147 H= 286:45 +CPU= 29.909
  11:25:06 STEP 1148 H= 287:00 +CPU= 30.557
  11:25:39 STEP 1149 H= 287:15 +CPU= 30.016
  11:26:12 STEP 1150 H= 287:30 +CPU= 30.318
  11:26:45 STEP 1151 H= 287:45 +CPU= 29.740
  11:27:32 STEP 1152 H= 288:00 +CPU= 40.283
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251_11.zip
Uploading the intermediate file: upload_file_11.zip
  11:28:06 STEP 1153 H= 288:15 +CPU= 29.396
  11:28:39 STEP 1154 H= 288:30 +CPU= 30.287
  11:29:12 STEP 1155 H= 288:45 +CPU= 30.331
  11:29:46 STEP 1156 H= 289:00 +CPU= 30.629
  11:30:19 STEP 1157 H= 289:15 +CPU= 29.585
  11:30:53 STEP 1158 H= 289:30 +CPU= 30.028
  11:31:27 STEP 1159 H= 289:45 +CPU= 30.043
  11:32:01 STEP 1160 H= 290:00 +CPU= 29.463
  11:32:36 STEP 1161 H= 290:15 +CPU= 29.474
  11:33:09 STEP 1162 H= 290:30 +CPU= 29.960
  11:33:42 STEP 1163 H= 290:45 +CPU= 29.454
  11:34:21 STEP 1164 H= 291:00 +CPU= 36.669
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001164
  11:34:55 STEP 1165 H= 291:15 +CPU= 30.477
  11:35:28 STEP 1166 H= 291:30 +CPU= 29.833
  11:36:02 STEP 1167 H= 291:45 +CPU= 30.320
  11:36:37 STEP 1168 H= 292:00 +CPU= 29.774
  11:37:11 STEP 1169 H= 292:15 +CPU= 30.142
  11:37:44 STEP 1170 H= 292:30 +CPU= 30.118
  11:38:16 STEP 1171 H= 292:45 +CPU= 29.616
  11:38:50 STEP 1172 H= 293:00 +CPU= 30.335
  11:39:23 STEP 1173 H= 293:15 +CPU= 29.933
  11:39:55 STEP 1174 H= 293:30 +CPU= 29.951
  11:40:27 STEP 1175 H= 293:45 +CPU= 29.783
  11:41:09 STEP 1176 H= 294:00 +CPU= 37.426
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001176
  11:41:43 STEP 1177 H= 294:15 +CPU= 30.143
  11:42:16 STEP 1178 H= 294:30 +CPU= 30.241
  11:42:50 STEP 1179 H= 294:45 +CPU= 30.032
  11:43:24 STEP 1180 H= 295:00 +CPU= 30.676
  11:43:59 STEP 1181 H= 295:15 +CPU= 29.908
  11:44:32 STEP 1182 H= 295:30 +CPU= 30.008
  11:45:06 STEP 1183 H= 295:45 +CPU= 30.339
  11:45:40 STEP 1184 H= 296:00 +CPU= 30.689
  11:46:14 STEP 1185 H= 296:15 +CPU= 30.411
  11:46:48 STEP 1186 H= 296:30 +CPU= 30.168
  11:47:21 STEP 1187 H= 296:45 +CPU= 29.982
  11:48:02 STEP 1188 H= 297:00 +CPU= 37.542
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001188
  11:48:35 STEP 1189 H= 297:15 +CPU= 29.968
  11:49:09 STEP 1190 H= 297:30 +CPU= 30.070
  11:49:42 STEP 1191 H= 297:45 +CPU= 30.280
  11:50:16 STEP 1192 H= 298:00 +CPU= 30.577
  11:50:49 STEP 1193 H= 298:15 +CPU= 30.024
  11:51:23 STEP 1194 H= 298:30 +CPU= 30.107
  11:51:56 STEP 1195 H= 298:45 +CPU= 30.418
  11:52:29 STEP 1196 H= 299:00 +CPU= 30.786
  11:53:02 STEP 1197 H= 299:15 +CPU= 29.709
  11:53:36 STEP 1198 H= 299:30 +CPU= 30.264
  11:54:08 STEP 1199 H= 299:45 +CPU= 30.165
  11:54:57 STEP 1200 H= 300:00 +CPU= 40.369
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001200
  11:55:31 STEP 1201 H= 300:15 +CPU= 30.260
  11:56:03 STEP 1202 H= 300:30 +CPU= 29.518
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a40c
(argv4) batchid: 991
(argv5) wuid: 12211983
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/2
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_07415e0083d1f2198d9d38aa2d158e65 to: /var/lib/boinc/slots/2/oifs_43r3_bl_a40c_2016092300_15_991_12211983.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_a40c_2016092300_15_991_12211983.zip
ic_ancil_file: ic_ancil_12211983
ifsdata_file: ifsdata_12211983
climate_data_file: clim_data_12211983
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/2/ic_ancil_12211983.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/2/ic_ancil_12211983.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4738f807a8bdd18f0167b37367ce0ec3 to: /var/lib/boinc/slots/2/ifsdata/ifsdata_12211983.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/2/ifsdata/ifsdata_12211983.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/2/159l_2/clim_data_12211983.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/2/159l_2/clim_data_12211983.zip
Checking for progress XML file: /var/lib/boinc/slots/2/progress_file_12211983.xml
Opened progress file ok : /var/lib/boinc/slots/2/progress_file_12211983.xml
-- Model is restarting --
Adjusting last_iter, 1203, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/2/progress_file_12211983.xml
last_cpu_time: 32158
upload_file_number: 12
last_iter: 1201
last_upload: 1036800
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251
The child process has been launched with process id: 3488282
Executing the command: /var/lib/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:3488282:3488282] [20230214:115753:1676372273:0.002] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  11:58:22 STEP 1200 H= 300:00 +CPU= 24.326
  11:58:49 STEP 1201 H= 300:15 +CPU= 24.424
  11:59:15 STEP 1202 H= 300:30 +CPU= 23.898
  11:59:42 STEP 1203 H= 300:45 +CPU= 23.913
  12:00:08 STEP 1204 H= 301:00 +CPU= 24.138
  12:00:34 STEP 1205 H= 301:15 +CPU= 23.916
  12:01:00 STEP 1206 H= 301:30 +CPU= 24.365
  12:01:27 STEP 1207 H= 301:45 +CPU= 24.575
  12:01:55 STEP 1208 H= 302:00 +CPU= 25.123
  12:02:22 STEP 1209 H= 302:15 +CPU= 24.772
  12:02:48 STEP 1210 H= 302:30 +CPU= 24.570
  12:03:15 STEP 1211 H= 302:45 +CPU= 24.357
  12:03:47 STEP 1212 H= 303:00 +CPU= 29.882
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001212
  12:04:14 STEP 1213 H= 303:15 +CPU= 24.931
  12:04:41 STEP 1214 H= 303:30 +CPU= 24.384
  12:05:08 STEP 1215 H= 303:45 +CPU= 24.245
  12:05:34 STEP 1216 H= 304:00 +CPU= 24.955
  12:06:00 STEP 1217 H= 304:15 +CPU= 24.068
  12:06:27 STEP 1218 H= 304:30 +CPU= 24.630
  12:06:52 STEP 1219 H= 304:45 +CPU= 24.262
  12:07:20 STEP 1220 H= 305:00 +CPU= 25.247
  12:07:47 STEP 1221 H= 305:15 +CPU= 24.482
  12:08:14 STEP 1222 H= 305:30 +CPU= 24.718
  12:08:40 STEP 1223 H= 305:45 +CPU= 24.448
  12:09:13 STEP 1224 H= 306:00 +CPU= 30.202
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001224
  12:09:41 STEP 1225 H= 306:15 +CPU= 24.515
  12:10:07 STEP 1226 H= 306:30 +CPU= 24.367
  12:10:34 STEP 1227 H= 306:45 +CPU= 24.333
  12:11:02 STEP 1228 H= 307:00 +CPU= 25.209
  12:11:28 STEP 1229 H= 307:15 +CPU= 24.433
  12:11:55 STEP 1230 H= 307:30 +CPU= 24.060
  12:12:21 STEP 1231 H= 307:45 +CPU= 24.637
  12:12:48 STEP 1232 H= 308:00 +CPU= 24.908
  12:13:16 STEP 1233 H= 308:15 +CPU= 24.628
  12:13:42 STEP 1234 H= 308:30 +CPU= 24.643
  12:14:09 STEP 1235 H= 308:45 +CPU= 24.344
  12:14:41 STEP 1236 H= 309:00 +CPU= 30.152
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001236
  12:15:08 STEP 1237 H= 309:15 +CPU= 24.431
  12:15:35 STEP 1238 H= 309:30 +CPU= 24.622
  12:16:01 STEP 1239 H= 309:45 +CPU= 24.706
  12:16:28 STEP 1240 H= 310:00 +CPU= 24.702
  12:16:54 STEP 1241 H= 310:15 +CPU= 24.546
  12:17:21 STEP 1242 H= 310:30 +CPU= 24.541
  12:17:47 STEP 1243 H= 310:45 +CPU= 24.507
  12:18:15 STEP 1244 H= 311:00 +CPU= 25.230
  12:18:41 STEP 1245 H= 311:15 +CPU= 24.167
  12:19:07 STEP 1246 H= 311:30 +CPU= 24.449
  12:19:33 STEP 1247 H= 311:45 +CPU= 24.377
  12:20:10 STEP 1248 H= 312:00 +CPU= 32.984
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251_12.zip
Uploading the intermediate file: upload_file_12.zip
  12:20:38 STEP 1249 H= 312:15 +CPU= 24.286
  12:21:05 STEP 1250 H= 312:30 +CPU= 24.739
  12:21:31 STEP 1251 H= 312:45 +CPU= 24.452
  12:21:59 STEP 1252 H= 313:00 +CPU= 25.096
  12:22:25 STEP 1253 H= 313:15 +CPU= 24.475
  12:22:51 STEP 1254 H= 313:30 +CPU= 24.502
  12:23:18 STEP 1255 H= 313:45 +CPU= 24.193
  12:23:44 STEP 1256 H= 314:00 +CPU= 24.911
  12:24:10 STEP 1257 H= 314:15 +CPU= 24.429
  12:24:37 STEP 1258 H= 314:30 +CPU= 24.586
  12:25:04 STEP 1259 H= 314:45 +CPU= 24.587
  12:25:36 STEP 1260 H= 315:00 +CPU= 30.418
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001260
  12:26:03 STEP 1261 H= 315:15 +CPU= 24.515
  12:26:28 STEP 1262 H= 315:30 +CPU= 24.142
  12:26:54 STEP 1263 H= 315:45 +CPU= 24.383
  12:27:22 STEP 1264 H= 316:00 +CPU= 25.247
  12:27:48 STEP 1265 H= 316:15 +CPU= 24.455
  12:28:14 STEP 1266 H= 316:30 +CPU= 24.587
  12:28:41 STEP 1267 H= 316:45 +CPU= 24.567
  12:29:08 STEP 1268 H= 317:00 +CPU= 25.117
  12:29:35 STEP 1269 H= 317:15 +CPU= 24.805
  12:30:01 STEP 1270 H= 317:30 +CPU= 24.552
  12:30:27 STEP 1271 H= 317:45 +CPU= 24.098
  12:30:59 STEP 1272 H= 318:00 +CPU= 30.268
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001272
  12:31:26 STEP 1273 H= 318:15 +CPU= 24.795
  12:31:53 STEP 1274 H= 318:30 +CPU= 24.578
  12:32:19 STEP 1275 H= 318:45 +CPU= 24.211
  12:32:46 STEP 1276 H= 319:00 +CPU= 25.019
  12:33:13 STEP 1277 H= 319:15 +CPU= 24.540
  12:33:38 STEP 1278 H= 319:30 +CPU= 24.057
  12:34:05 STEP 1279 H= 319:45 +CPU= 24.395
  12:34:31 STEP 1280 H= 320:00 +CPU= 24.816
  12:34:57 STEP 1281 H= 320:15 +CPU= 24.267
  12:35:23 STEP 1282 H= 320:30 +CPU= 24.479
  12:35:49 STEP 1283 H= 320:45 +CPU= 24.335
  12:36:21 STEP 1284 H= 321:00 +CPU= 30.280
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001284
  12:36:47 STEP 1285 H= 321:15 +CPU= 24.404
  12:37:14 STEP 1286 H= 321:30 +CPU= 24.370
  12:37:41 STEP 1287 H= 321:45 +CPU= 24.573
  12:38:07 STEP 1288 H= 322:00 +CPU= 24.702
  12:38:33 STEP 1289 H= 322:15 +CPU= 24.240
  12:38:59 STEP 1290 H= 322:30 +CPU= 24.442
  12:39:25 STEP 1291 H= 322:45 +CPU= 24.338
  12:39:51 STEP 1292 H= 323:00 +CPU= 25.210
  12:40:17 STEP 1293 H= 323:15 +CPU= 24.461
  12:40:45 STEP 1294 H= 323:30 +CPU= 25.045
  12:41:11 STEP 1295 H= 323:45 +CPU= 24.412
  12:41:50 STEP 1296 H= 324:00 +CPU= 32.579
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001296
  12:42:19 STEP 1297 H= 324:15 +CPU= 24.339
  12:42:46 STEP 1298 H= 324:30 +CPU= 24.620
  12:43:13 STEP 1299 H= 324:45 +CPU= 24.660
  12:43:40 STEP 1300 H= 325:00 +CPU= 25.138
  12:44:07 STEP 1301 H= 325:15 +CPU= 24.485
  12:44:34 STEP 1302 H= 325:30 +CPU= 24.627
  12:44:59 STEP 1303 H= 325:45 +CPU= 24.290
  12:45:26 STEP 1304 H= 326:00 +CPU= 24.883
  12:45:52 STEP 1305 H= 326:15 +CPU= 24.153
  12:46:18 STEP 1306 H= 326:30 +CPU= 24.606
  12:46:44 STEP 1307 H= 326:45 +CPU= 24.474
  12:47:16 STEP 1308 H= 327:00 +CPU= 30.399
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001308
  12:47:43 STEP 1309 H= 327:15 +CPU= 24.402
  12:48:10 STEP 1310 H= 327:30 +CPU= 24.674
  12:48:36 STEP 1311 H= 327:45 +CPU= 24.373
  12:49:03 STEP 1312 H= 328:00 +CPU= 24.884
  12:49:30 STEP 1313 H= 328:15 +CPU= 24.503
  12:49:56 STEP 1314 H= 328:30 +CPU= 24.466
  12:50:22 STEP 1315 H= 328:45 +CPU= 24.545
  12:50:49 STEP 1316 H= 329:00 +CPU= 24.956
  12:51:15 STEP 1317 H= 329:15 +CPU= 24.531
  12:51:42 STEP 1318 H= 329:30 +CPU= 24.275
  12:52:09 STEP 1319 H= 329:45 +CPU= 24.845
  12:52:42 STEP 1320 H= 330:00 +CPU= 30.414
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001320
  12:53:08 STEP 1321 H= 330:15 +CPU= 24.538
  12:53:35 STEP 1322 H= 330:30 +CPU= 24.310
  12:54:01 STEP 1323 H= 330:45 +CPU= 24.518
  12:54:28 STEP 1324 H= 331:00 +CPU= 24.957
  12:54:54 STEP 1325 H= 331:15 +CPU= 24.268
  12:55:20 STEP 1326 H= 331:30 +CPU= 24.321
  12:55:47 STEP 1327 H= 331:45 +CPU= 25.253
  12:56:14 STEP 1328 H= 332:00 +CPU= 24.938
  12:56:40 STEP 1329 H= 332:15 +CPU= 24.492
  12:57:07 STEP 1330 H= 332:30 +CPU= 24.626
  12:57:33 STEP 1331 H= 332:45 +CPU= 24.386
  12:58:06 STEP 1332 H= 333:00 +CPU= 30.070
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001332
  12:58:32 STEP 1333 H= 333:15 +CPU= 24.411
  12:58:58 STEP 1334 H= 333:30 +CPU= 24.375
  12:59:24 STEP 1335 H= 333:45 +CPU= 24.470
  12:59:51 STEP 1336 H= 334:00 +CPU= 25.001
  13:00:17 STEP 1337 H= 334:15 +CPU= 24.320
  13:00:43 STEP 1338 H= 334:30 +CPU= 24.341
  13:01:09 STEP 1339 H= 334:45 +CPU= 24.473
  13:01:37 STEP 1340 H= 335:00 +CPU= 25.197
  13:02:03 STEP 1341 H= 335:15 +CPU= 24.712
  13:02:29 STEP 1342 H= 335:30 +CPU= 24.017
  13:02:57 STEP 1343 H= 335:45 +CPU= 24.580
  13:03:36 STEP 1344 H= 336:00 +CPU= 32.227
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251_13.zip
Uploading the intermediate file: upload_file_13.zip
  13:04:03 STEP 1345 H= 336:15 +CPU= 24.210
  13:04:29 STEP 1346 H= 336:30 +CPU= 24.233
  13:04:55 STEP 1347 H= 336:45 +CPU= 24.377
  13:05:22 STEP 1348 H= 337:00 +CPU= 24.985
  13:05:49 STEP 1349 H= 337:15 +CPU= 24.740
  13:06:14 STEP 1350 H= 337:30 +CPU= 24.060
  13:06:40 STEP 1351 H= 337:45 +CPU= 24.174
  13:07:07 STEP 1352 H= 338:00 +CPU= 25.023
  13:07:34 STEP 1353 H= 338:15 +CPU= 24.677
  13:08:01 STEP 1354 H= 338:30 +CPU= 24.469
  13:08:27 STEP 1355 H= 338:45 +CPU= 24.207
  13:08:58 STEP 1356 H= 339:00 +CPU= 29.664
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001356
  13:09:24 STEP 1357 H= 339:15 +CPU= 24.394
  13:09:51 STEP 1358 H= 339:30 +CPU= 24.349
  13:10:17 STEP 1359 H= 339:45 +CPU= 24.507
  13:10:44 STEP 1360 H= 340:00 +CPU= 24.828
  13:11:10 STEP 1361 H= 340:15 +CPU= 24.497
  13:11:36 STEP 1362 H= 340:30 +CPU= 24.410
  13:12:03 STEP 1363 H= 340:45 +CPU= 24.475
  13:12:30 STEP 1364 H= 341:00 +CPU= 24.992
  13:12:56 STEP 1365 H= 341:15 +CPU= 24.305
  13:13:22 STEP 1366 H= 341:30 +CPU= 24.945
  13:13:49 STEP 1367 H= 341:45 +CPU= 24.688
  13:14:22 STEP 1368 H= 342:00 +CPU= 30.470
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001368
  13:14:51 STEP 1369 H= 342:15 +CPU= 25.025
  13:15:17 STEP 1370 H= 342:30 +CPU= 24.561
  13:15:44 STEP 1371 H= 342:45 +CPU= 24.670
  13:16:11 STEP 1372 H= 343:00 +CPU= 25.075
  13:16:37 STEP 1373 H= 343:15 +CPU= 24.482
  13:17:03 STEP 1374 H= 343:30 +CPU= 24.448
  13:17:29 STEP 1375 H= 343:45 +CPU= 24.634
  13:17:57 STEP 1376 H= 344:00 +CPU= 25.271
  13:18:23 STEP 1377 H= 344:15 +CPU= 24.369
  13:18:48 STEP 1378 H= 344:30 +CPU= 24.060
  13:19:15 STEP 1379 H= 344:45 +CPU= 24.279
  13:19:48 STEP 1380 H= 345:00 +CPU= 30.495
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001380
  13:20:14 STEP 1381 H= 345:15 +CPU= 24.324
  13:20:40 STEP 1382 H= 345:30 +CPU= 24.282
  13:21:06 STEP 1383 H= 345:45 +CPU= 24.635
  13:21:33 STEP 1384 H= 346:00 +CPU= 25.016
  13:22:00 STEP 1385 H= 346:15 +CPU= 24.977
  13:22:26 STEP 1386 H= 346:30 +CPU= 24.850
  13:22:52 STEP 1387 H= 346:45 +CPU= 24.411
  13:23:19 STEP 1388 H= 347:00 +CPU= 25.059
  13:23:45 STEP 1389 H= 347:15 +CPU= 24.210
  13:24:12 STEP 1390 H= 347:30 +CPU= 24.358
  13:24:35 STEP 1391 H= 347:45 +CPU= 22.302
  13:25:06 STEP 1392 H= 348:00 +CPU= 28.703
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001392
  13:25:31 STEP 1393 H= 348:15 +CPU= 22.025
  13:25:55 STEP 1394 H= 348:30 +CPU= 21.978
  13:26:17 STEP 1395 H= 348:45 +CPU= 21.212
  13:26:40 STEP 1396 H= 349:00 +CPU= 21.728
  13:27:02 STEP 1397 H= 349:15 +CPU= 21.337
  13:27:24 STEP 1398 H= 349:30 +CPU= 21.611
  13:27:46 STEP 1399 H= 349:45 +CPU= 20.960
  13:28:09 STEP 1400 H= 350:00 +CPU= 21.945
  13:28:35 STEP 1401 H= 350:15 +CPU= 23.883
  13:29:00 STEP 1402 H= 350:30 +CPU= 23.852
  13:29:26 STEP 1403 H= 350:45 +CPU= 23.773
  13:30:00 STEP 1404 H= 351:00 +CPU= 30.733
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001404
  13:30:26 STEP 1405 H= 351:15 +CPU= 24.019
  13:30:53 STEP 1406 H= 351:30 +CPU= 23.953
  13:31:19 STEP 1407 H= 351:45 +CPU= 24.370
  13:31:46 STEP 1408 H= 352:00 +CPU= 24.343
  13:32:12 STEP 1409 H= 352:15 +CPU= 23.712
  13:32:37 STEP 1410 H= 352:30 +CPU= 23.570
  13:33:04 STEP 1411 H= 352:45 +CPU= 24.666
  13:33:30 STEP 1412 H= 353:00 +CPU= 24.669
  13:33:57 STEP 1413 H= 353:15 +CPU= 24.444
  13:34:22 STEP 1414 H= 353:30 +CPU= 23.582
  13:34:49 STEP 1415 H= 353:45 +CPU= 24.756
  13:35:21 STEP 1416 H= 354:00 +CPU= 30.341
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001416
  13:35:48 STEP 1417 H= 354:15 +CPU= 24.896
  13:36:15 STEP 1418 H= 354:30 +CPU= 24.456
  13:36:41 STEP 1419 H= 354:45 +CPU= 24.773
  13:37:08 STEP 1420 H= 355:00 +CPU= 24.719
  13:37:34 STEP 1421 H= 355:15 +CPU= 24.095
  13:38:00 STEP 1422 H= 355:30 +CPU= 24.351
  13:38:27 STEP 1423 H= 355:45 +CPU= 24.355
  13:38:54 STEP 1424 H= 356:00 +CPU= 24.956
  13:39:19 STEP 1425 H= 356:15 +CPU= 23.602
  13:39:45 STEP 1426 H= 356:30 +CPU= 24.248
  13:40:11 STEP 1427 H= 356:45 +CPU= 23.930
  13:40:43 STEP 1428 H= 357:00 +CPU= 30.178
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001428
  13:41:09 STEP 1429 H= 357:15 +CPU= 23.849
  13:41:36 STEP 1430 H= 357:30 +CPU= 24.544
  13:42:03 STEP 1431 H= 357:45 +CPU= 24.739
  13:42:30 STEP 1432 H= 358:00 +CPU= 25.163
  13:42:58 STEP 1433 H= 358:15 +CPU= 26.039
  13:43:27 STEP 1434 H= 358:30 +CPU= 26.894
  13:43:56 STEP 1435 H= 358:45 +CPU= 26.778
  13:44:26 STEP 1436 H= 359:00 +CPU= 27.405
  13:44:54 STEP 1437 H= 359:15 +CPU= 26.681
  13:45:23 STEP 1438 H= 359:30 +CPU= 26.814
  13:45:52 STEP 1439 H= 359:45 +CPU= 26.858
  13:46:31 STEP 1440 H= 360:00 +CPU= 34.264
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.6907680574E-21      I       0.1264860791E-06      I
ICloud fractionI      0.0000000000E+00      I       0.1815571652E-01      I
I  Relat. Hum. I      0.4565137900E+00      I       0.4834441216E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6415354880E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    323344993 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS808362.5 Mbytes/s, TOTAL TIME=    0.0223( 1.8%)
 13:46:31 STEP 1440 H= 360:00 +CPU= 34.264
 END CNT3
 NSTEP =  1440 CNT0   000000000  25.469  25.469  28.251   0.902               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN         7090/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    128    0       34      1044068     7090 ************    11.0     1.5     9.5     1.8    23.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 6517783.5 6517783.5    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 6515038.5 6515038.5     99.96      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  261    4645.8    4645.8     18.60      0.00
   9 SPCM     - SPECTRAL COMP.                     240     767.2     767.2      2.82      0.00
  10 SCAN2M - PHYSICS                              241   12114.3   12114.3     44.79      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 20    1004.5    1004.5      0.31      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 11      27.0      27.0      0.00      0.00
  17 GRIDFPOS IN CNT4                               20      33.2      33.2      0.01      0.00
  27 RESTART FILES - WRITING                         5    4017.6    4017.6      0.31      0.00
  28 RESTART FILES - READING                         1    2216.0    2216.0      0.03      0.00
  29 SU4FPOS IN CNT4                                20       0.3       0.3      0.00      0.00
  30 DYNFPOS IN CNT4                                20    5695.1    5695.1      1.75      0.00
  31 POSDDH IN STEPO                                 2      22.9      22.9      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    241    5831.3    5831.3     21.56      0.00
  39 SU0YOMB                                         1     279.8     279.8      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    241      26.2      26.2      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              240     413.1     413.1      1.52      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              240      77.0      77.0      0.28      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 11       0.8       0.8      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      524     417.6     417.6      3.36      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       500     269.0     269.0      2.06      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        500      86.2      86.2      0.66      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       524     138.7     138.7      1.11      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      32.9      32.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            524      48.1      48.1      0.39      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            500      25.7      25.7      0.20      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            524      52.6      52.6      0.42      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            500      21.2      21.2      0.16      0.00
 400 GSTATS                                      28768       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 26718       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     6517.8 CPU TIME    6025.4 VECTOR TIME     6025.4


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      198.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/2/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/2/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12211983/ICMUAh7zg+001440
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a40c_2016092300_15_991_12211983_0_r2037453251_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
13:50:05 (3484464): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:51:13 1507812 22312235 oifs_43r3_bl_a40c_2016092300_15_991_12211983_0 1,295,100 38,183 0.0295
09 May 2023 16:51:07 1507812 22312235 oifs_43r3_bl_a40c_2016092300_15_991_12211983_0 864,000 24,715 0.0286


©2024 cpdn.org