climateprediction.net home page
Task 22448985

Task 22448985

Name oifs_43r3_bl_a205_2016092300_20_1018_12290924_2
Workunit 12290924
Created 15 Jun 2024, 4:14:29 UTC
Sent 15 Jun 2024, 4:14:33 UTC
Report deadline 14 Aug 2024, 4:14:33 UTC
Received 15 Jun 2024, 16:16:09 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1533065
Run time 11 hours 23 min 11 sec
CPU time 11 hours 18 min 11 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 5.37 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,023.57 MB
Peak swap size 5,981.11 MB
Peak disk usage 1,602.37 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
h7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000804
  13:32:36 STEP  805 H= 201:15 +CPU= 38.988
  13:33:14 STEP  806 H= 201:30 +CPU= 37.983
  13:33:52 STEP  807 H= 201:45 +CPU= 37.304
  13:34:29 STEP  808 H= 202:00 +CPU= 37.522
  13:35:06 STEP  809 H= 202:15 +CPU= 36.878
  13:35:42 STEP  810 H= 202:30 +CPU= 35.104
  13:36:15 STEP  811 H= 202:45 +CPU= 33.445
  13:36:50 STEP  812 H= 203:00 +CPU= 35.281
  13:37:24 STEP  813 H= 203:15 +CPU= 33.544
  13:37:57 STEP  814 H= 203:30 +CPU= 33.282
  13:38:30 STEP  815 H= 203:45 +CPU= 32.983
  13:39:14 STEP  816 H= 204:00 +CPU= 43.766
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000816
  13:39:50 STEP  817 H= 204:15 +CPU= 35.666
  13:40:27 STEP  818 H= 204:30 +CPU= 36.814
  13:41:03 STEP  819 H= 204:45 +CPU= 36.520
  13:41:41 STEP  820 H= 205:00 +CPU= 37.844
  13:42:17 STEP  821 H= 205:15 +CPU= 36.195
  13:42:54 STEP  822 H= 205:30 +CPU= 37.120
  13:43:34 STEP  823 H= 205:45 +CPU= 39.114
  13:44:13 STEP  824 H= 206:00 +CPU= 39.506
  13:44:51 STEP  825 H= 206:15 +CPU= 37.680
  13:45:24 STEP  826 H= 206:30 +CPU= 33.104
  13:45:57 STEP  827 H= 206:45 +CPU= 32.591
  13:46:39 STEP  828 H= 207:00 +CPU= 41.639
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000828
  13:47:12 STEP  829 H= 207:15 +CPU= 33.334
  13:47:45 STEP  830 H= 207:30 +CPU= 32.476
  13:48:19 STEP  831 H= 207:45 +CPU= 34.316
  13:48:57 STEP  832 H= 208:00 +CPU= 37.603
  13:49:34 STEP  833 H= 208:15 +CPU= 37.228
  13:50:12 STEP  834 H= 208:30 +CPU= 37.467
  13:50:49 STEP  835 H= 208:45 +CPU= 37.596
  13:51:26 STEP  836 H= 209:00 +CPU= 36.912
  13:52:03 STEP  837 H= 209:15 +CPU= 36.134
  13:52:40 STEP  838 H= 209:30 +CPU= 36.963
  13:53:17 STEP  839 H= 209:45 +CPU= 36.692
  13:54:02 STEP  840 H= 210:00 +CPU= 45.645
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000840
  13:54:36 STEP  841 H= 210:15 +CPU= 33.359
  13:55:08 STEP  842 H= 210:30 +CPU= 31.873
  13:55:40 STEP  843 H= 210:45 +CPU= 31.929
  13:56:13 STEP  844 H= 211:00 +CPU= 33.004
  13:56:46 STEP  845 H= 211:15 +CPU= 33.168
  13:57:19 STEP  846 H= 211:30 +CPU= 32.993
  13:57:53 STEP  847 H= 211:45 +CPU= 33.410
  13:58:30 STEP  848 H= 212:00 +CPU= 37.846
  13:59:07 STEP  849 H= 212:15 +CPU= 36.080
  13:59:43 STEP  850 H= 212:30 +CPU= 36.037
  14:00:19 STEP  851 H= 212:45 +CPU= 36.034
  14:01:03 STEP  852 H= 213:00 +CPU= 44.611
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000852
  14:01:40 STEP  853 H= 213:15 +CPU= 36.227
  14:02:16 STEP  854 H= 213:30 +CPU= 36.223
  14:02:52 STEP  855 H= 213:45 +CPU= 36.306
  14:03:29 STEP  856 H= 214:00 +CPU= 36.894
  14:04:05 STEP  857 H= 214:15 +CPU= 35.826
  14:04:38 STEP  858 H= 214:30 +CPU= 33.202
  14:05:11 STEP  859 H= 214:45 +CPU= 32.707
  14:05:45 STEP  860 H= 215:00 +CPU= 33.834
  14:06:18 STEP  861 H= 215:15 +CPU= 32.659
  14:06:50 STEP  862 H= 215:30 +CPU= 32.780
  14:07:23 STEP  863 H= 215:45 +CPU= 32.622
  14:08:08 STEP  864 H= 216:00 +CPU= 44.711
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a205_2016092300_20_1018_12290924_2_r1741806571_8.zip
Uploading the intermediate file: upload_file_8.zip
  14:08:47 STEP  865 H= 216:15 +CPU= 36.890
double free or corruption (!prev)
  14:09:25 STEP  866 H= 216:30 +CPU= 37.917
  14:10:02 STEP  867 H= 216:45 +CPU= 36.996
  14:10:39 STEP  868 H= 217:00 +CPU= 37.212
  14:11:17 STEP  869 H= 217:15 +CPU= 37.500
  14:11:53 STEP  870 H= 217:30 +CPU= 36.664
  14:12:31 STEP  871 H= 217:45 +CPU= 37.775
  14:13:09 STEP  872 H= 218:00 +CPU= 38.048
  14:13:46 STEP  873 H= 218:15 +CPU= 36.039
  14:14:18 STEP  874 H= 218:30 +CPU= 32.795
  14:14:51 STEP  875 H= 218:45 +CPU= 32.855
  14:15:32 STEP  876 H= 219:00 +CPU= 40.720
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000876
  14:16:05 STEP  877 H= 219:15 +CPU= 33.236
  14:16:38 STEP  878 H= 219:30 +CPU= 32.623
  14:17:12 STEP  879 H= 219:45 +CPU= 33.436
  14:17:46 STEP  880 H= 220:00 +CPU= 34.323
  14:18:24 STEP  881 H= 220:15 +CPU= 37.516
  14:19:02 STEP  882 H= 220:30 +CPU= 38.421
  14:19:41 STEP  883 H= 220:45 +CPU= 38.673
  14:20:21 STEP  884 H= 221:00 +CPU= 39.653
  14:21:00 STEP  885 H= 221:15 +CPU= 38.852
  14:21:38 STEP  886 H= 221:30 +CPU= 38.848
  14:22:17 STEP  887 H= 221:45 +CPU= 38.426
  14:23:03 STEP  888 H= 222:00 +CPU= 46.204
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000888
  14:23:37 STEP  889 H= 222:15 +CPU= 34.038
  14:24:11 STEP  890 H= 222:30 +CPU= 33.375
  14:24:45 STEP  891 H= 222:45 +CPU= 33.567
  14:25:18 STEP  892 H= 223:00 +CPU= 33.406
  14:25:51 STEP  893 H= 223:15 +CPU= 32.644
  14:26:24 STEP  894 H= 223:30 +CPU= 32.836
  14:26:57 STEP  895 H= 223:45 +CPU= 33.334
  14:27:35 STEP  896 H= 224:00 +CPU= 37.285
  14:28:13 STEP  897 H= 224:15 +CPU= 36.944
  14:28:52 STEP  898 H= 224:30 +CPU= 38.906
  14:29:30 STEP  899 H= 224:45 +CPU= 38.479
  14:30:18 STEP  900 H= 225:00 +CPU= 47.380
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000900
  14:30:56 STEP  901 H= 225:15 +CPU= 37.753
  14:31:35 STEP  902 H= 225:30 +CPU= 39.079
  14:32:14 STEP  903 H= 225:45 +CPU= 38.865
  14:32:51 STEP  904 H= 226:00 +CPU= 37.243
  14:33:24 STEP  905 H= 226:15 +CPU= 32.756
  14:33:57 STEP  906 H= 226:30 +CPU= 32.894
  14:34:30 STEP  907 H= 226:45 +CPU= 32.432
  14:35:03 STEP  908 H= 227:00 +CPU= 33.053
  14:35:36 STEP  909 H= 227:15 +CPU= 32.829
  14:36:08 STEP  910 H= 227:30 +CPU= 32.438
  14:36:41 STEP  911 H= 227:45 +CPU= 32.498
  14:37:25 STEP  912 H= 228:00 +CPU= 44.177
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000912
  14:37:58 STEP  913 H= 228:15 +CPU= 32.521
  14:38:30 STEP  914 H= 228:30 +CPU= 32.451
  14:39:03 STEP  915 H= 228:45 +CPU= 32.911
  14:39:38 STEP  916 H= 229:00 +CPU= 34.598
  14:40:13 STEP  917 H= 229:15 +CPU= 34.404
  14:40:47 STEP  918 H= 229:30 +CPU= 34.189
  14:41:21 STEP  919 H= 229:45 +CPU= 34.426
  14:41:57 STEP  920 H= 230:00 +CPU= 35.179
  14:42:31 STEP  921 H= 230:15 +CPU= 34.490
  14:43:05 STEP  922 H= 230:30 +CPU= 33.596
  14:43:37 STEP  923 H= 230:45 +CPU= 32.569
  14:44:18 STEP  924 H= 231:00 +CPU= 40.230
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000924
  14:44:52 STEP  925 H= 231:15 +CPU= 34.022
  14:45:26 STEP  926 H= 231:30 +CPU= 33.981
  14:46:00 STEP  927 H= 231:45 +CPU= 33.945
  14:46:35 STEP  928 H= 232:00 +CPU= 34.747
  14:47:08 STEP  929 H= 232:15 +CPU= 33.649
  14:47:42 STEP  930 H= 232:30 +CPU= 33.831
  14:48:16 STEP  931 H= 232:45 +CPU= 33.800
  14:48:51 STEP  932 H= 233:00 +CPU= 34.449
  14:49:24 STEP  933 H= 233:15 +CPU= 33.631
  14:49:58 STEP  934 H= 233:30 +CPU= 33.617
  14:50:31 STEP  935 H= 233:45 +CPU= 32.742
  14:51:11 STEP  936 H= 234:00 +CPU= 40.382
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000936
  14:51:44 STEP  937 H= 234:15 +CPU= 32.676
  14:52:17 STEP  938 H= 234:30 +CPU= 32.473
  14:52:49 STEP  939 H= 234:45 +CPU= 32.623
  14:53:23 STEP  940 H= 235:00 +CPU= 33.181
  14:53:55 STEP  941 H= 235:15 +CPU= 32.320
  14:54:27 STEP  942 H= 235:30 +CPU= 32.442
  14:55:02 STEP  943 H= 235:45 +CPU= 34.110
  14:55:37 STEP  944 H= 236:00 +CPU= 35.066
  14:56:10 STEP  945 H= 236:15 +CPU= 32.688
  14:56:42 STEP  946 H= 236:30 +CPU= 32.481
  14:57:15 STEP  947 H= 236:45 +CPU= 32.708
  14:57:55 STEP  948 H= 237:00 +CPU= 40.323
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000948
  14:58:28 STEP  949 H= 237:15 +CPU= 32.696
  14:59:01 STEP  950 H= 237:30 +CPU= 33.357
  14:59:35 STEP  951 H= 237:45 +CPU= 33.191
  15:00:09 STEP  952 H= 238:00 +CPU= 34.305
  15:00:42 STEP  953 H= 238:15 +CPU= 33.451
  15:01:16 STEP  954 H= 238:30 +CPU= 33.397
  15:01:49 STEP  955 H= 238:45 +CPU= 33.290
  15:02:23 STEP  956 H= 239:00 +CPU= 33.839
  15:02:56 STEP  957 H= 239:15 +CPU= 33.240
  15:03:30 STEP  958 H= 239:30 +CPU= 33.232
  15:04:04 STEP  959 H= 239:45 +CPU= 34.063
  15:04:49 STEP  960 H= 240:00 +CPU= 45.010
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a205_2016092300_20_1018_12290924_2_r1741806571_9.zip
Uploading the intermediate file: upload_file_9.zip
  15:05:25 STEP  961 H= 240:15 +CPU= 33.772
  15:05:58 STEP  962 H= 240:30 +CPU= 33.761
  15:06:32 STEP  963 H= 240:45 +CPU= 33.776
  15:07:07 STEP  964 H= 241:00 +CPU= 34.220
  15:07:40 STEP  965 H= 241:15 +CPU= 33.414
  15:08:14 STEP  966 H= 241:30 +CPU= 33.589
  15:08:47 STEP  967 H= 241:45 +CPU= 33.253
  15:09:20 STEP  968 H= 242:00 +CPU= 33.381
  15:09:53 STEP  969 H= 242:15 +CPU= 32.856
  15:10:26 STEP  970 H= 242:30 +CPU= 32.898
  15:10:59 STEP  971 H= 242:45 +CPU= 32.918
  15:11:41 STEP  972 H= 243:00 +CPU= 41.414
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000972
  15:12:15 STEP  973 H= 243:15 +CPU= 34.547
  15:12:49 STEP  974 H= 243:30 +CPU= 34.000
  15:13:23 STEP  975 H= 243:45 +CPU= 33.865
  15:13:57 STEP  976 H= 244:00 +CPU= 34.265
  15:14:31 STEP  977 H= 244:15 +CPU= 33.946
  15:15:05 STEP  978 H= 244:30 +CPU= 33.910
  15:15:40 STEP  979 H= 244:45 +CPU= 34.687
  15:16:14 STEP  980 H= 245:00 +CPU= 34.222
  15:16:47 STEP  981 H= 245:15 +CPU= 32.732
  15:17:20 STEP  982 H= 245:30 +CPU= 33.058
  15:17:54 STEP  983 H= 245:45 +CPU= 33.321
  15:18:34 STEP  984 H= 246:00 +CPU= 40.593
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000984
  15:19:07 STEP  985 H= 246:15 +CPU= 32.646
  15:19:41 STEP  986 H= 246:30 +CPU= 34.136
  15:20:15 STEP  987 H= 246:45 +CPU= 34.234
  15:20:51 STEP  988 H= 247:00 +CPU= 35.121
  15:21:26 STEP  989 H= 247:15 +CPU= 34.146
  15:22:00 STEP  990 H= 247:30 +CPU= 34.247
  15:22:34 STEP  991 H= 247:45 +CPU= 33.718
  15:23:09 STEP  992 H= 248:00 +CPU= 35.234
  15:23:43 STEP  993 H= 248:15 +CPU= 34.541
  15:24:18 STEP  994 H= 248:30 +CPU= 34.079
  15:24:52 STEP  995 H= 248:45 +CPU= 34.462
  15:25:36 STEP  996 H= 249:00 +CPU= 43.646
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+000996
  15:26:11 STEP  997 H= 249:15 +CPU= 35.347
  15:26:46 STEP  998 H= 249:30 +CPU= 34.722
  15:27:21 STEP  999 H= 249:45 +CPU= 34.752
  15:27:56 STEP 1000 H= 250:00 +CPU= 35.306
  15:28:31 STEP 1001 H= 250:15 +CPU= 34.663
  15:29:05 STEP 1002 H= 250:30 +CPU= 34.577
  15:29:39 STEP 1003 H= 250:45 +CPU= 33.713
  15:30:14 STEP 1004 H= 251:00 +CPU= 34.334
  15:30:47 STEP 1005 H= 251:15 +CPU= 33.664
  15:31:21 STEP 1006 H= 251:30 +CPU= 33.646
  15:31:55 STEP 1007 H= 251:45 +CPU= 33.726
  15:32:39 STEP 1008 H= 252:00 +CPU= 44.356
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001008
  15:33:13 STEP 1009 H= 252:15 +CPU= 33.470
  15:33:46 STEP 1010 H= 252:30 +CPU= 33.659
  15:34:21 STEP 1011 H= 252:45 +CPU= 33.998
  15:34:55 STEP 1012 H= 253:00 +CPU= 34.241
  15:35:30 STEP 1013 H= 253:15 +CPU= 34.512
  15:36:04 STEP 1014 H= 253:30 +CPU= 33.671
  15:36:37 STEP 1015 H= 253:45 +CPU= 32.902
  15:37:11 STEP 1016 H= 254:00 +CPU= 33.934
  15:37:45 STEP 1017 H= 254:15 +CPU= 34.073
  15:38:19 STEP 1018 H= 254:30 +CPU= 34.217
  15:38:53 STEP 1019 H= 254:45 +CPU= 33.706
  15:39:35 STEP 1020 H= 255:00 +CPU= 41.746
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001020
  15:40:10 STEP 1021 H= 255:15 +CPU= 34.892
  15:40:43 STEP 1022 H= 255:30 +CPU= 33.811
  15:41:17 STEP 1023 H= 255:45 +CPU= 33.603
  15:41:52 STEP 1024 H= 256:00 +CPU= 35.074
  15:42:27 STEP 1025 H= 256:15 +CPU= 34.847
  15:43:00 STEP 1026 H= 256:30 +CPU= 33.197
  15:43:34 STEP 1027 H= 256:45 +CPU= 33.662
  15:44:09 STEP 1028 H= 257:00 +CPU= 34.861
  15:44:43 STEP 1029 H= 257:15 +CPU= 33.665
  15:45:16 STEP 1030 H= 257:30 +CPU= 33.060
  15:45:49 STEP 1031 H= 257:45 +CPU= 33.807
  15:46:32 STEP 1032 H= 258:00 +CPU= 42.238
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001032
  15:47:06 STEP 1033 H= 258:15 +CPU= 34.334
  15:47:40 STEP 1034 H= 258:30 +CPU= 33.774
  15:48:15 STEP 1035 H= 258:45 +CPU= 34.956
  15:48:50 STEP 1036 H= 259:00 +CPU= 34.174
  15:49:23 STEP 1037 H= 259:15 +CPU= 33.449
  15:49:56 STEP 1038 H= 259:30 +CPU= 33.178
  15:50:30 STEP 1039 H= 259:45 +CPU= 33.385
  15:51:04 STEP 1040 H= 260:00 +CPU= 34.164
  15:51:37 STEP 1041 H= 260:15 +CPU= 32.764
  15:52:09 STEP 1042 H= 260:30 +CPU= 32.521
  15:52:42 STEP 1043 H= 260:45 +CPU= 32.816
  15:53:23 STEP 1044 H= 261:00 +CPU= 40.704
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001044
  15:53:57 STEP 1045 H= 261:15 +CPU= 33.755
  15:54:31 STEP 1046 H= 261:30 +CPU= 33.778
  15:55:05 STEP 1047 H= 261:45 +CPU= 33.916
  15:55:39 STEP 1048 H= 262:00 +CPU= 34.700
  15:56:14 STEP 1049 H= 262:15 +CPU= 34.566
  15:56:49 STEP 1050 H= 262:30 +CPU= 34.531
  15:57:23 STEP 1051 H= 262:45 +CPU= 34.463
  15:57:58 STEP 1052 H= 263:00 +CPU= 35.141
  15:58:32 STEP 1053 H= 263:15 +CPU= 33.599
  15:59:05 STEP 1054 H= 263:30 +CPU= 33.307
  15:59:39 STEP 1055 H= 263:45 +CPU= 33.421
  16:00:24 STEP 1056 H= 264:00 +CPU= 44.742
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a205_2016092300_20_1018_12290924_2_r1741806571_10.zip
Uploading the intermediate file: upload_file_10.zip
  16:00:59 STEP 1057 H= 264:15 +CPU= 33.601
  16:01:32 STEP 1058 H= 264:30 +CPU= 33.260
  16:02:07 STEP 1059 H= 264:45 +CPU= 34.620
  16:02:41 STEP 1060 H= 265:00 +CPU= 34.403
  16:03:15 STEP 1061 H= 265:15 +CPU= 33.787
  16:03:49 STEP 1062 H= 265:30 +CPU= 33.952
  16:04:24 STEP 1063 H= 265:45 +CPU= 34.450
  16:04:59 STEP 1064 H= 266:00 +CPU= 35.138
  16:05:33 STEP 1065 H= 266:15 +CPU= 33.795
  16:06:06 STEP 1066 H= 266:30 +CPU= 33.340
  16:06:40 STEP 1067 H= 266:45 +CPU= 34.453
  16:07:23 STEP 1068 H= 267:00 +CPU= 42.773
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001068
  16:07:57 STEP 1069 H= 267:15 +CPU= 33.424
  16:08:31 STEP 1070 H= 267:30 +CPU= 34.059
  16:09:05 STEP 1071 H= 267:45 +CPU= 33.741
  16:09:40 STEP 1072 H= 268:00 +CPU= 34.919
  16:10:14 STEP 1073 H= 268:15 +CPU= 34.055
  16:10:48 STEP 1074 H= 268:30 +CPU= 34.117
  16:11:22 STEP 1075 H= 268:45 +CPU= 33.996
  16:11:56 STEP 1076 H= 269:00 +CPU= 34.357
  16:12:31 STEP 1077 H= 269:15 +CPU= 34.330
  16:13:05 STEP 1078 H= 269:30 +CPU= 34.007
  16:13:39 STEP 1079 H= 269:45 +CPU= 34.642
  16:14:22 STEP 1080 H= 270:00 +CPU= 42.388
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001080
  16:14:56 STEP 1081 H= 270:15 +CPU= 33.035
  16:15:30 STEP 1082 H= 270:30 +CPU= 33.786
  16:16:04 STEP 1083 H= 270:45 +CPU= 33.811
  16:16:39 STEP 1084 H= 271:00 +CPU= 34.614
  16:17:13 STEP 1085 H= 271:15 +CPU= 34.082
  16:17:47 STEP 1086 H= 271:30 +CPU= 33.653
  16:18:21 STEP 1087 H= 271:45 +CPU= 33.868
  16:18:56 STEP 1088 H= 272:00 +CPU= 34.852
  16:19:29 STEP 1089 H= 272:15 +CPU= 33.339
  16:20:02 STEP 1090 H= 272:30 +CPU= 33.159
  16:20:36 STEP 1091 H= 272:45 +CPU= 33.449
  16:21:18 STEP 1092 H= 273:00 +CPU= 42.270
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001092
  16:21:53 STEP 1093 H= 273:15 +CPU= 34.510
  16:22:28 STEP 1094 H= 273:30 +CPU= 34.543
  16:23:03 STEP 1095 H= 273:45 +CPU= 34.311
  16:23:37 STEP 1096 H= 274:00 +CPU= 34.347
  16:24:11 STEP 1097 H= 274:15 +CPU= 33.819
  16:24:46 STEP 1098 H= 274:30 +CPU= 34.518
  16:25:21 STEP 1099 H= 274:45 +CPU= 35.051
  16:25:57 STEP 1100 H= 275:00 +CPU= 35.374
  16:26:32 STEP 1101 H= 275:15 +CPU= 34.798
  16:27:07 STEP 1102 H= 275:30 +CPU= 34.782
  16:27:42 STEP 1103 H= 275:45 +CPU= 34.698
  16:28:28 STEP 1104 H= 276:00 +CPU= 45.691
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001104
  16:29:02 STEP 1105 H= 276:15 +CPU= 34.152
  16:29:36 STEP 1106 H= 276:30 +CPU= 34.097
  16:30:10 STEP 1107 H= 276:45 +CPU= 33.996
  16:30:45 STEP 1108 H= 277:00 +CPU= 34.379
  16:31:19 STEP 1109 H= 277:15 +CPU= 33.741
  16:31:52 STEP 1110 H= 277:30 +CPU= 33.694
  16:32:26 STEP 1111 H= 277:45 +CPU= 33.843
  16:33:01 STEP 1112 H= 278:00 +CPU= 34.742
  16:33:35 STEP 1113 H= 278:15 +CPU= 34.346
  16:34:10 STEP 1114 H= 278:30 +CPU= 34.247
  16:34:43 STEP 1115 H= 278:45 +CPU= 33.006
  16:35:24 STEP 1116 H= 279:00 +CPU= 40.805
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001116
  16:35:57 STEP 1117 H= 279:15 +CPU= 33.278
  16:36:30 STEP 1118 H= 279:30 +CPU= 33.026
  16:37:03 STEP 1119 H= 279:45 +CPU= 33.271
  16:37:37 STEP 1120 H= 280:00 +CPU= 33.982
  16:38:11 STEP 1121 H= 280:15 +CPU= 33.620
  16:38:44 STEP 1122 H= 280:30 +CPU= 33.393
  16:39:18 STEP 1123 H= 280:45 +CPU= 33.390
  16:39:52 STEP 1124 H= 281:00 +CPU= 34.221
  16:40:25 STEP 1125 H= 281:15 +CPU= 33.332
  16:41:00 STEP 1126 H= 281:30 +CPU= 34.124
  16:41:37 STEP 1127 H= 281:45 +CPU= 34.232
  16:42:20 STEP 1128 H= 282:00 +CPU= 41.691
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001128
  16:42:55 STEP 1129 H= 282:15 +CPU= 34.305
  16:43:29 STEP 1130 H= 282:30 +CPU= 34.141
  16:44:04 STEP 1131 H= 282:45 +CPU= 34.799
  16:44:39 STEP 1132 H= 283:00 +CPU= 34.557
  16:45:14 STEP 1133 H= 283:15 +CPU= 34.358
  16:45:49 STEP 1134 H= 283:30 +CPU= 33.991
  16:46:24 STEP 1135 H= 283:45 +CPU= 34.061
  16:46:59 STEP 1136 H= 284:00 +CPU= 34.563
  16:47:33 STEP 1137 H= 284:15 +CPU= 34.759
  16:48:08 STEP 1138 H= 284:30 +CPU= 34.148
  16:48:43 STEP 1139 H= 284:45 +CPU= 33.951
  16:49:25 STEP 1140 H= 285:00 +CPU= 41.546
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001140
  16:50:00 STEP 1141 H= 285:15 +CPU= 34.380
  16:50:34 STEP 1142 H= 285:30 +CPU= 33.522
  16:51:07 STEP 1143 H= 285:45 +CPU= 33.322
  16:51:43 STEP 1144 H= 286:00 +CPU= 35.409
  16:52:18 STEP 1145 H= 286:15 +CPU= 34.374
  16:52:53 STEP 1146 H= 286:30 +CPU= 33.999
  16:53:28 STEP 1147 H= 286:45 +CPU= 34.098
  16:54:04 STEP 1148 H= 287:00 +CPU= 35.203
  16:54:38 STEP 1149 H= 287:15 +CPU= 34.041
  16:55:13 STEP 1150 H= 287:30 +CPU= 34.115
  16:55:49 STEP 1151 H= 287:45 +CPU= 34.245
  16:56:36 STEP 1152 H= 288:00 +CPU= 45.538
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290924/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a205_2016092300_20_1018_12290924_2_r1741806571_11.zip
Uploading the intermediate file: upload_file_11.zip
  16:57:11 STEP 1153 H= 288:15 +CPU= 33.816
  16:57:45 STEP 1154 H= 288:30 +CPU= 33.536
  16:58:20 STEP 1155 H= 288:45 +CPU= 33.508
  16:58:55 STEP 1156 H= 289:00 +CPU= 34.586
  16:59:30 STEP 1157 H= 289:15 +CPU= 34.663
  17:00:05 STEP 1158 H= 289:30 +CPU= 34.009
  17:00:40 STEP 1159 H= 289:45 +CPU= 34.734
  17:01:16 STEP 1160 H= 290:00 +CPU= 34.946
  17:01:52 STEP 1161 H= 290:15 +CPU= 34.615
  17:02:27 STEP 1162 H= 290:30 +CPU= 34.562
  17:03:02 STEP 1163 H= 290:45 +CPU= 33.997
  17:03:44 STEP 1164 H= 291:00 +CPU= 41.908
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001164
  17:04:19 STEP 1165 H= 291:15 +CPU= 34.283
  17:04:54 STEP 1166 H= 291:30 +CPU= 33.627
  17:05:29 STEP 1167 H= 291:45 +CPU= 34.588
  17:06:04 STEP 1168 H= 292:00 +CPU= 35.186
  17:06:39 STEP 1169 H= 292:15 +CPU= 33.844
  17:07:13 STEP 1170 H= 292:30 +CPU= 33.432
  17:07:48 STEP 1171 H= 292:45 +CPU= 34.877
  17:08:24 STEP 1172 H= 293:00 +CPU= 34.908
  17:09:00 STEP 1173 H= 293:15 +CPU= 34.652
  17:09:35 STEP 1174 H= 293:30 +CPU= 34.354
  17:10:10 STEP 1175 H= 293:45 +CPU= 35.048
  17:10:54 STEP 1176 H= 294:00 +CPU= 42.870
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001176
  17:11:29 STEP 1177 H= 294:15 +CPU= 34.156
  17:12:04 STEP 1178 H= 294:30 +CPU= 34.339
  17:12:39 STEP 1179 H= 294:45 +CPU= 34.375
  17:13:15 STEP 1180 H= 295:00 +CPU= 35.325
  17:13:49 STEP 1181 H= 295:15 +CPU= 34.276
  17:14:25 STEP 1182 H= 295:30 +CPU= 35.483
  17:14:59 STEP 1183 H= 295:45 +CPU= 33.507
  17:15:35 STEP 1184 H= 296:00 +CPU= 35.675
  17:16:10 STEP 1185 H= 296:15 +CPU= 34.548
  17:16:44 STEP 1186 H= 296:30 +CPU= 34.009
  17:17:19 STEP 1187 H= 296:45 +CPU= 34.134
  17:18:02 STEP 1188 H= 297:00 +CPU= 42.578
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001188
  17:18:37 STEP 1189 H= 297:15 +CPU= 34.604
  17:19:13 STEP 1190 H= 297:30 +CPU= 34.769
  17:19:47 STEP 1191 H= 297:45 +CPU= 33.566
  17:20:22 STEP 1192 H= 298:00 +CPU= 34.904
  17:20:57 STEP 1193 H= 298:15 +CPU= 34.269
  17:21:32 STEP 1194 H= 298:30 +CPU= 34.447
  17:22:07 STEP 1195 H= 298:45 +CPU= 34.131
  17:22:43 STEP 1196 H= 299:00 +CPU= 35.786
  17:23:19 STEP 1197 H= 299:15 +CPU= 34.789
  17:23:53 STEP 1198 H= 299:30 +CPU= 33.658
  17:24:27 STEP 1199 H= 299:45 +CPU= 33.753
  17:25:15 STEP 1200 H= 300:00 +CPU= 46.270
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001200
  17:25:49 STEP 1201 H= 300:15 +CPU= 33.788
  17:26:25 STEP 1202 H= 300:30 +CPU= 35.138
  17:26:59 STEP 1203 H= 300:45 +CPU= 33.118
  17:27:34 STEP 1204 H= 301:00 +CPU= 34.097
  17:28:09 STEP 1205 H= 301:15 +CPU= 34.417
  17:28:44 STEP 1206 H= 301:30 +CPU= 34.641
  17:29:18 STEP 1207 H= 301:45 +CPU= 33.935
  17:29:53 STEP 1208 H= 302:00 +CPU= 33.737
  17:30:28 STEP 1209 H= 302:15 +CPU= 35.037
  17:31:04 STEP 1210 H= 302:30 +CPU= 35.174
  17:31:38 STEP 1211 H= 302:45 +CPU= 33.572
  17:32:21 STEP 1212 H= 303:00 +CPU= 41.964
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001212
  17:32:56 STEP 1213 H= 303:15 +CPU= 34.013
  17:33:30 STEP 1214 H= 303:30 +CPU= 33.491
  17:34:05 STEP 1215 H= 303:45 +CPU= 34.631
  17:34:41 STEP 1216 H= 304:00 +CPU= 35.670
  17:35:17 STEP 1217 H= 304:15 +CPU= 34.800
  17:35:51 STEP 1218 H= 304:30 +CPU= 34.570
  17:36:27 STEP 1219 H= 304:45 +CPU= 34.841
  17:37:02 STEP 1220 H= 305:00 +CPU= 35.038
  17:37:37 STEP 1221 H= 305:15 +CPU= 33.975
  17:38:11 STEP 1222 H= 305:30 +CPU= 34.465
  17:38:46 STEP 1223 H= 305:45 +CPU= 34.802
  17:39:29 STEP 1224 H= 306:00 +CPU= 42.062
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001224
  17:40:04 STEP 1225 H= 306:15 +CPU= 33.400
  17:40:38 STEP 1226 H= 306:30 +CPU= 33.758
  17:41:14 STEP 1227 H= 306:45 +CPU= 34.975
  17:41:49 STEP 1228 H= 307:00 +CPU= 34.985
  17:42:24 STEP 1229 H= 307:15 +CPU= 34.709
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.045] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 3958MB (heap), 4790MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.067] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.067] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.067] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.067] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.068] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3958 MB (maxheap), 4790 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.068] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174248:1718466168:32507.071] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/16/oifs_43r3_model.exe' : sigcontextptr=0x7ffddf9dd310
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/16/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.071] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/16/oifs_43r3_model.exe' : sigcontextptr=0x7ffddf9dd220
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/16/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.316] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:2733:2733] [20240615:174249:1718466169:32508.316] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.277571900231948E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.269966203343464E-05 0.000000000000000E+00 0.973189227581211E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.193350425104555E-04 0.000000000000000E+00 0.136001891675077E-01
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.277221862126827E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.936784412618245E-05 0.000000000000000E+00 0.279052984077892E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.998639405117107E-05 0.000000000000000E+00 0.660220573312028E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.339176922699769E-05 0.000000000000000E+00 0.124926434271307E-01
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.122176675557748E-05 0.000000000000000E+00 0.870128790501024E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.246266831493042E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  1224 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1224 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001224 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+001224
 MPI-TASK:   1 -    295101236 BYTES IN      6 RECORDS TRANSFERRED IN    0.0004 SECONDS737753.1 Mbytes/s, TOTAL TIME=    0.0125( 3.2%)
 NSTEP =  1224 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+001224 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+001224
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0091 SECONDS 77736.2 Mbytes/s, TOTAL TIME=    0.3356( 2.7%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001224 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+001224
 MPI-TASK:   1 -   4721619776 BYTES IN    125 RECORDS TRANSFERRED IN    0.0045 SECONDS******** Mbytes/s, TOTAL TIME=    0.2533( 1.8%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001224 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001224
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS     0.1 Mbytes/s, TOTAL TIME=    0.0038( 2.6%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001224 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001224
 MPI-TASK:   1 -    286521507 BYTES IN      4 RECORDS TRANSFERRED IN    0.0003 SECONDS955071.7 Mbytes/s, TOTAL TIME=    0.0092( 3.3%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =  1224 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1224 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001224 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001224
 MPI-TASK:   1 -    295101236 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS737753.1 Mbytes/s, TOTAL TIME=    0.0134( 3.0%)
 17:39:29 STEP 1224 H= 306:00 +CPU= 42.062
ISTEP=    1225 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=    1225 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =  1225 STEPO   0AAA00AAA
 17:40:04 STEP 1225 H= 306:15 +CPU= 33.400
 NSTEP =  1226 STEPO   0AAA00AAA
 17:40:38 STEP 1226 H= 306:30 +CPU= 33.758
 NSTEP =  1227 STEPO   0AAA00AAA
 17:41:14 STEP 1227 H= 306:45 +CPU= 34.975
 NSTEP =  1228 STEPO   0AAA00AAA
 17:41:49 STEP 1228 H= 307:00 +CPU= 34.985
 NSTEP =  1229 STEPO   0AAA00AAA
 17:42:24 STEP 1229 H= 307:15 +CPU= 34.709
ISTEP=    1230 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=    1230 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =  1230 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 17:38:54 0AAA00AAA STEPO     1224     0.570    0.570    0.573    534:39    537:53 0.32631905097247E-04       2GB       0MB
 17:39:29 0AAA00AAA STEPO     1225    34.701   34.701   35.391    535:14    538:28 0.32631905097247E-04       2GB       0MB
 17:40:04 0AAA00AAA STEPO     1226    33.400   33.400   34.338    535:47    539:02 0.32631905097247E-04       2GB       0MB
 17:40:38 0AAA00AAA STEPO     1227    33.758   33.758   33.784    536:21    539:36 0.32631905097247E-04       2GB       0MB
 17:41:14 0AAA00AAA STEPO     1228    34.975   34.975   35.937    536:56    540:12 0.32631905097247E-04       2GB       0MB
 17:41:49 0AAA00AAA STEPO     1229    34.985   34.985   35.482    537:31    540:48 0.32631905097247E-04       2GB       0MB
 17:42:24 0AAA00AAA STEPO     1230    34.709   34.709   35.320    538:06    541:23 0.32631905097247E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1200",
 CTIME   = "00121200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,        1188, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/16/progress_file_12290924.xml

<running_values>
  <last_cpu_time>40690.360000</last_cpu_time>
  <upload_file_number>12</upload_file_number>
  <last_iter>1230</last_iter>
  <last_upload>1036800</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org