climateprediction.net home page
Task 22309643

Task 22309643

Name oifs_43r3_bl_a21u_2016092300_15_991_12209445_0
Workunit 12209445
Created 13 Feb 2023, 16:54:21 UTC
Sent 13 Feb 2023, 17:06:36 UTC
Report deadline 14 Apr 2023, 17:06:36 UTC
Received 14 Feb 2023, 5:14:45 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1534452
Run time 10 hours 15 min 10 sec
CPU time 10 hours 15 min 10 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.57 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 4,880.34 MB
Peak swap size 5,930.78 MB
Peak disk usage 1,692.35 MB
Stderr
<core_client_version>7.9.3</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000744
  00:53:22 STEP  745 H= 186:15 +CPU= 30.346
  00:53:53 STEP  746 H= 186:30 +CPU= 30.567
  00:54:23 STEP  747 H= 186:45 +CPU= 30.128
  00:54:54 STEP  748 H= 187:00 +CPU= 31.131
  00:55:25 STEP  749 H= 187:15 +CPU= 30.589
  00:55:55 STEP  750 H= 187:30 +CPU= 30.724
  00:56:26 STEP  751 H= 187:45 +CPU= 30.479
  00:56:57 STEP  752 H= 188:00 +CPU= 31.161
  00:57:28 STEP  753 H= 188:15 +CPU= 30.847
  00:57:59 STEP  754 H= 188:30 +CPU= 30.717
  00:58:29 STEP  755 H= 188:45 +CPU= 30.705
  00:59:07 STEP  756 H= 189:00 +CPU= 37.801
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000756
  00:59:37 STEP  757 H= 189:15 +CPU= 30.120
  01:00:07 STEP  758 H= 189:30 +CPU= 29.998
  01:00:37 STEP  759 H= 189:45 +CPU= 29.891
  01:01:08 STEP  760 H= 190:00 +CPU= 30.888
  01:01:38 STEP  761 H= 190:15 +CPU= 30.230
  01:02:09 STEP  762 H= 190:30 +CPU= 30.383
  01:02:39 STEP  763 H= 190:45 +CPU= 30.393
  01:03:10 STEP  764 H= 191:00 +CPU= 30.897
  01:03:40 STEP  765 H= 191:15 +CPU= 30.221
  01:04:11 STEP  766 H= 191:30 +CPU= 30.401
  01:04:41 STEP  767 H= 191:45 +CPU= 30.441
  01:05:22 STEP  768 H= 192:00 +CPU= 40.925
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a21u_2016092300_15_991_12209445_0_r1583579673_7.zip
Uploading the intermediate file: upload_file_7.zip
  01:05:53 STEP  769 H= 192:15 +CPU= 29.730
double free or corruption (!prev)
  01:06:23 STEP  770 H= 192:30 +CPU= 29.581
  01:06:53 STEP  771 H= 192:45 +CPU= 29.769
  01:07:23 STEP  772 H= 193:00 +CPU= 30.119
  01:07:53 STEP  773 H= 193:15 +CPU= 29.627
  01:08:23 STEP  774 H= 193:30 +CPU= 29.873
  01:08:52 STEP  775 H= 193:45 +CPU= 29.697
  01:09:23 STEP  776 H= 194:00 +CPU= 30.376
  01:09:53 STEP  777 H= 194:15 +CPU= 29.841
  01:10:22 STEP  778 H= 194:30 +CPU= 29.633
  01:10:52 STEP  779 H= 194:45 +CPU= 29.453
  01:11:29 STEP  780 H= 195:00 +CPU= 36.770
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000780
  01:11:58 STEP  781 H= 195:15 +CPU= 29.456
  01:12:28 STEP  782 H= 195:30 +CPU= 29.477
  01:12:57 STEP  783 H= 195:45 +CPU= 29.167
  01:13:28 STEP  784 H= 196:00 +CPU= 30.032
  01:13:59 STEP  785 H= 196:15 +CPU= 30.437
  01:14:32 STEP  786 H= 196:30 +CPU= 32.503
  01:15:05 STEP  787 H= 196:45 +CPU= 32.513
  01:15:38 STEP  788 H= 197:00 +CPU= 33.134
  01:16:10 STEP  789 H= 197:15 +CPU= 32.497
  01:16:43 STEP  790 H= 197:30 +CPU= 32.302
  01:17:15 STEP  791 H= 197:45 +CPU= 32.461
  01:17:55 STEP  792 H= 198:00 +CPU= 39.829
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000792
  01:18:27 STEP  793 H= 198:15 +CPU= 32.235
  01:18:59 STEP  794 H= 198:30 +CPU= 31.862
  01:19:31 STEP  795 H= 198:45 +CPU= 32.232
  01:20:04 STEP  796 H= 199:00 +CPU= 32.904
  01:20:37 STEP  797 H= 199:15 +CPU= 32.331
  01:21:09 STEP  798 H= 199:30 +CPU= 32.340
  01:21:41 STEP  799 H= 199:45 +CPU= 31.995
  01:22:14 STEP  800 H= 200:00 +CPU= 32.591
  01:22:46 STEP  801 H= 200:15 +CPU= 31.911
  01:23:18 STEP  802 H= 200:30 +CPU= 31.968
  01:23:50 STEP  803 H= 200:45 +CPU= 32.052
  01:24:29 STEP  804 H= 201:00 +CPU= 39.533
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000804
  01:25:02 STEP  805 H= 201:15 +CPU= 32.211
  01:25:34 STEP  806 H= 201:30 +CPU= 32.220
  01:26:06 STEP  807 H= 201:45 +CPU= 32.422
  01:26:40 STEP  808 H= 202:00 +CPU= 33.024
  01:27:13 STEP  809 H= 202:15 +CPU= 32.269
  01:27:46 STEP  810 H= 202:30 +CPU= 32.288
  01:28:19 STEP  811 H= 202:45 +CPU= 32.580
  01:28:56 STEP  812 H= 203:00 +CPU= 33.384
  01:29:33 STEP  813 H= 203:15 +CPU= 32.821
  01:30:10 STEP  814 H= 203:30 +CPU= 32.765
  01:30:44 STEP  815 H= 203:45 +CPU= 32.136
  01:31:29 STEP  816 H= 204:00 +CPU= 43.874
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000816
  01:32:02 STEP  817 H= 204:15 +CPU= 32.287
  01:32:36 STEP  818 H= 204:30 +CPU= 32.149
  01:33:11 STEP  819 H= 204:45 +CPU= 32.361
  01:33:45 STEP  820 H= 205:00 +CPU= 32.898
  01:34:18 STEP  821 H= 205:15 +CPU= 31.716
  01:34:51 STEP  822 H= 205:30 +CPU= 31.792
  01:35:26 STEP  823 H= 205:45 +CPU= 31.952
  01:36:03 STEP  824 H= 206:00 +CPU= 32.703
  01:36:35 STEP  825 H= 206:15 +CPU= 31.954
  01:37:07 STEP  826 H= 206:30 +CPU= 31.865
  01:37:39 STEP  827 H= 206:45 +CPU= 32.113
  01:38:19 STEP  828 H= 207:00 +CPU= 39.430
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000828
  01:38:50 STEP  829 H= 207:15 +CPU= 31.668
  01:39:22 STEP  830 H= 207:30 +CPU= 31.598
  01:39:54 STEP  831 H= 207:45 +CPU= 31.799
  01:40:26 STEP  832 H= 208:00 +CPU= 32.407
  01:40:58 STEP  833 H= 208:15 +CPU= 31.917
  01:41:30 STEP  834 H= 208:30 +CPU= 31.618
  01:42:02 STEP  835 H= 208:45 +CPU= 31.625
  01:42:34 STEP  836 H= 209:00 +CPU= 32.426
  01:43:06 STEP  837 H= 209:15 +CPU= 31.796
  01:43:38 STEP  838 H= 209:30 +CPU= 31.721
  01:44:09 STEP  839 H= 209:45 +CPU= 31.759
  01:44:49 STEP  840 H= 210:00 +CPU= 39.452
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000840
  01:45:21 STEP  841 H= 210:15 +CPU= 31.985
  01:45:53 STEP  842 H= 210:30 +CPU= 31.654
  01:46:24 STEP  843 H= 210:45 +CPU= 31.507
  01:46:57 STEP  844 H= 211:00 +CPU= 32.392
  01:47:28 STEP  845 H= 211:15 +CPU= 31.781
  01:48:00 STEP  846 H= 211:30 +CPU= 31.746
  01:48:33 STEP  847 H= 211:45 +CPU= 31.986
  01:49:07 STEP  848 H= 212:00 +CPU= 32.877
  01:49:40 STEP  849 H= 212:15 +CPU= 32.258
  01:50:15 STEP  850 H= 212:30 +CPU= 32.062
  01:50:47 STEP  851 H= 212:45 +CPU= 32.077
  01:51:34 STEP  852 H= 213:00 +CPU= 40.724
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000852
  01:52:10 STEP  853 H= 213:15 +CPU= 32.399
  01:52:42 STEP  854 H= 213:30 +CPU= 30.713
  01:53:12 STEP  855 H= 213:45 +CPU= 29.932
  01:53:42 STEP  856 H= 214:00 +CPU= 30.580
  01:54:13 STEP  857 H= 214:15 +CPU= 30.007
  01:54:43 STEP  858 H= 214:30 +CPU= 29.986
  01:55:13 STEP  859 H= 214:45 +CPU= 30.030
  01:55:43 STEP  860 H= 215:00 +CPU= 30.687
  01:56:13 STEP  861 H= 215:15 +CPU= 30.146
  01:56:44 STEP  862 H= 215:30 +CPU= 30.018
  01:57:14 STEP  863 H= 215:45 +CPU= 30.097
  01:57:57 STEP  864 H= 216:00 +CPU= 42.851
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a21u_2016092300_15_991_12209445_0_r1583579673_8.zip
Uploading the intermediate file: upload_file_8.zip
  01:58:31 STEP  865 H= 216:15 +CPU= 32.109
  01:59:04 STEP  866 H= 216:30 +CPU= 32.499
  01:59:36 STEP  867 H= 216:45 +CPU= 32.439
  02:00:09 STEP  868 H= 217:00 +CPU= 32.892
  02:00:41 STEP  869 H= 217:15 +CPU= 32.285
  02:01:14 STEP  870 H= 217:30 +CPU= 32.315
  02:01:46 STEP  871 H= 217:45 +CPU= 32.462
  02:02:19 STEP  872 H= 218:00 +CPU= 33.120
  02:02:52 STEP  873 H= 218:15 +CPU= 32.573
  02:03:24 STEP  874 H= 218:30 +CPU= 32.516
  02:03:57 STEP  875 H= 218:45 +CPU= 32.501
  02:04:37 STEP  876 H= 219:00 +CPU= 40.223
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000876
  02:05:08 STEP  877 H= 219:15 +CPU= 30.404
  02:05:41 STEP  878 H= 219:30 +CPU= 30.309
  02:06:14 STEP  879 H= 219:45 +CPU= 31.374
  02:06:47 STEP  880 H= 220:00 +CPU= 33.065
  02:07:19 STEP  881 H= 220:15 +CPU= 32.329
  02:07:52 STEP  882 H= 220:30 +CPU= 32.462
  02:08:25 STEP  883 H= 220:45 +CPU= 32.491
  02:08:58 STEP  884 H= 221:00 +CPU= 32.900
  02:09:31 STEP  885 H= 221:15 +CPU= 32.392
  02:10:04 STEP  886 H= 221:30 +CPU= 32.772
  02:10:37 STEP  887 H= 221:45 +CPU= 32.578
  02:11:18 STEP  888 H= 222:00 +CPU= 40.279
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000888
  02:11:51 STEP  889 H= 222:15 +CPU= 32.475
  02:12:23 STEP  890 H= 222:30 +CPU= 32.577
  02:12:55 STEP  891 H= 222:45 +CPU= 32.265
  02:13:29 STEP  892 H= 223:00 +CPU= 32.989
  02:14:01 STEP  893 H= 223:15 +CPU= 32.411
  02:14:34 STEP  894 H= 223:30 +CPU= 32.422
  02:15:06 STEP  895 H= 223:45 +CPU= 32.426
  02:15:39 STEP  896 H= 224:00 +CPU= 33.083
  02:16:12 STEP  897 H= 224:15 +CPU= 32.498
  02:16:44 STEP  898 H= 224:30 +CPU= 32.395
  02:17:17 STEP  899 H= 224:45 +CPU= 32.489
  02:17:57 STEP  900 H= 225:00 +CPU= 40.117
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000900
  02:18:29 STEP  901 H= 225:15 +CPU= 32.314
  02:19:02 STEP  902 H= 225:30 +CPU= 32.384
  02:19:34 STEP  903 H= 225:45 +CPU= 32.379
  02:20:07 STEP  904 H= 226:00 +CPU= 33.136
  02:20:40 STEP  905 H= 226:15 +CPU= 32.389
  02:21:13 STEP  906 H= 226:30 +CPU= 32.579
  02:21:45 STEP  907 H= 226:45 +CPU= 32.468
  02:22:18 STEP  908 H= 227:00 +CPU= 32.963
  02:22:51 STEP  909 H= 227:15 +CPU= 32.397
  02:23:23 STEP  910 H= 227:30 +CPU= 32.305
  02:23:56 STEP  911 H= 227:45 +CPU= 32.625
  02:24:38 STEP  912 H= 228:00 +CPU= 40.612
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000912
  02:25:08 STEP  913 H= 228:15 +CPU= 29.127
  02:25:37 STEP  914 H= 228:30 +CPU= 29.127
  02:26:06 STEP  915 H= 228:45 +CPU= 29.121
  02:26:36 STEP  916 H= 229:00 +CPU= 29.918
  02:27:05 STEP  917 H= 229:15 +CPU= 29.268
  02:27:35 STEP  918 H= 229:30 +CPU= 29.336
  02:28:04 STEP  919 H= 229:45 +CPU= 29.400
  02:28:34 STEP  920 H= 230:00 +CPU= 29.996
  02:29:04 STEP  921 H= 230:15 +CPU= 29.591
  02:29:34 STEP  922 H= 230:30 +CPU= 29.963
  02:30:04 STEP  923 H= 230:45 +CPU= 30.505
  02:30:42 STEP  924 H= 231:00 +CPU= 37.855
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000924
  02:31:13 STEP  925 H= 231:15 +CPU= 30.536
  02:31:43 STEP  926 H= 231:30 +CPU= 30.588
  02:32:14 STEP  927 H= 231:45 +CPU= 30.602
  02:32:45 STEP  928 H= 232:00 +CPU= 31.165
  02:33:16 STEP  929 H= 232:15 +CPU= 30.539
  02:33:47 STEP  930 H= 232:30 +CPU= 30.594
  02:34:17 STEP  931 H= 232:45 +CPU= 30.643
  02:34:48 STEP  932 H= 233:00 +CPU= 31.179
  02:35:19 STEP  933 H= 233:15 +CPU= 30.718
  02:35:51 STEP  934 H= 233:30 +CPU= 30.907
  02:36:21 STEP  935 H= 233:45 +CPU= 30.532
  02:37:00 STEP  936 H= 234:00 +CPU= 38.834
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000936
  02:37:31 STEP  937 H= 234:15 +CPU= 31.069
  02:38:02 STEP  938 H= 234:30 +CPU= 30.862
  02:38:33 STEP  939 H= 234:45 +CPU= 30.937
  02:39:05 STEP  940 H= 235:00 +CPU= 31.284
  02:39:35 STEP  941 H= 235:15 +CPU= 30.795
  02:40:06 STEP  942 H= 235:30 +CPU= 30.554
  02:40:37 STEP  943 H= 235:45 +CPU= 30.585
  02:41:08 STEP  944 H= 236:00 +CPU= 31.074
  02:41:38 STEP  945 H= 236:15 +CPU= 30.371
  02:42:08 STEP  946 H= 236:30 +CPU= 29.992
  02:42:39 STEP  947 H= 236:45 +CPU= 30.436
  02:43:18 STEP  948 H= 237:00 +CPU= 37.937
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000948
  02:43:48 STEP  949 H= 237:15 +CPU= 30.389
  02:44:22 STEP  950 H= 237:30 +CPU= 30.876
  02:44:53 STEP  951 H= 237:45 +CPU= 30.845
  02:45:24 STEP  952 H= 238:00 +CPU= 31.490
  02:45:56 STEP  953 H= 238:15 +CPU= 31.284
  02:46:27 STEP  954 H= 238:30 +CPU= 31.245
  02:46:58 STEP  955 H= 238:45 +CPU= 31.051
  02:47:29 STEP  956 H= 239:00 +CPU= 31.505
  02:48:00 STEP  957 H= 239:15 +CPU= 30.639
  02:48:30 STEP  958 H= 239:30 +CPU= 30.180
  02:49:01 STEP  959 H= 239:45 +CPU= 30.749
  02:49:45 STEP  960 H= 240:00 +CPU= 41.248
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a21u_2016092300_15_991_12209445_0_r1583579673_9.zip
Uploading the intermediate file: upload_file_9.zip
  02:50:16 STEP  961 H= 240:15 +CPU= 29.625
  02:50:46 STEP  962 H= 240:30 +CPU= 30.071
  02:51:16 STEP  963 H= 240:45 +CPU= 30.124
  02:51:47 STEP  964 H= 241:00 +CPU= 30.507
  02:52:17 STEP  965 H= 241:15 +CPU= 30.199
  02:52:47 STEP  966 H= 241:30 +CPU= 30.042
  02:53:17 STEP  967 H= 241:45 +CPU= 29.883
  02:53:48 STEP  968 H= 242:00 +CPU= 30.772
  02:54:18 STEP  969 H= 242:15 +CPU= 30.137
  02:54:48 STEP  970 H= 242:30 +CPU= 30.388
  02:55:19 STEP  971 H= 242:45 +CPU= 29.614
  02:55:57 STEP  972 H= 243:00 +CPU= 36.865
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000972
  02:56:27 STEP  973 H= 243:15 +CPU= 29.350
  02:56:57 STEP  974 H= 243:30 +CPU= 29.726
  02:57:26 STEP  975 H= 243:45 +CPU= 29.685
  02:57:56 STEP  976 H= 244:00 +CPU= 29.976
  02:58:27 STEP  977 H= 244:15 +CPU= 29.735
  02:59:00 STEP  978 H= 244:30 +CPU= 29.788
  02:59:32 STEP  979 H= 244:45 +CPU= 29.765
  03:00:02 STEP  980 H= 245:00 +CPU= 30.361
  03:00:33 STEP  981 H= 245:15 +CPU= 29.948
  03:01:03 STEP  982 H= 245:30 +CPU= 29.559
  03:01:36 STEP  983 H= 245:45 +CPU= 29.353
  03:02:14 STEP  984 H= 246:00 +CPU= 37.440
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000984
  03:02:44 STEP  985 H= 246:15 +CPU= 29.935
  03:03:15 STEP  986 H= 246:30 +CPU= 29.895
  03:03:45 STEP  987 H= 246:45 +CPU= 29.938
  03:04:15 STEP  988 H= 247:00 +CPU= 30.264
  03:04:45 STEP  989 H= 247:15 +CPU= 29.689
  03:05:15 STEP  990 H= 247:30 +CPU= 29.779
  03:05:47 STEP  991 H= 247:45 +CPU= 32.252
  03:06:21 STEP  992 H= 248:00 +CPU= 33.393
  03:06:53 STEP  993 H= 248:15 +CPU= 32.591
  03:07:26 STEP  994 H= 248:30 +CPU= 32.774
  03:07:59 STEP  995 H= 248:45 +CPU= 32.805
  03:08:44 STEP  996 H= 249:00 +CPU= 40.434
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000996
  03:09:16 STEP  997 H= 249:15 +CPU= 32.104
  03:09:47 STEP  998 H= 249:30 +CPU= 30.372
  03:10:20 STEP  999 H= 249:45 +CPU= 30.437
  03:10:55 STEP 1000 H= 250:00 +CPU= 31.173
  03:11:26 STEP 1001 H= 250:15 +CPU= 30.633
  03:11:56 STEP 1002 H= 250:30 +CPU= 30.818
  03:12:28 STEP 1003 H= 250:45 +CPU= 31.124
  03:12:59 STEP 1004 H= 251:00 +CPU= 30.979
  03:13:29 STEP 1005 H= 251:15 +CPU= 30.666
  03:14:00 STEP 1006 H= 251:30 +CPU= 30.763
  03:14:31 STEP 1007 H= 251:45 +CPU= 30.528
  03:15:12 STEP 1008 H= 252:00 +CPU= 41.659
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001008
  03:15:43 STEP 1009 H= 252:15 +CPU= 30.239
  03:16:13 STEP 1010 H= 252:30 +CPU= 30.339
  03:16:43 STEP 1011 H= 252:45 +CPU= 30.359
  03:17:14 STEP 1012 H= 253:00 +CPU= 30.732
  03:17:45 STEP 1013 H= 253:15 +CPU= 30.444
  03:18:17 STEP 1014 H= 253:30 +CPU= 29.911
  03:18:47 STEP 1015 H= 253:45 +CPU= 29.424
  03:19:20 STEP 1016 H= 254:00 +CPU= 30.593
  03:19:52 STEP 1017 H= 254:15 +CPU= 32.393
  03:20:25 STEP 1018 H= 254:30 +CPU= 32.488
  03:20:58 STEP 1019 H= 254:45 +CPU= 32.567
  03:21:38 STEP 1020 H= 255:00 +CPU= 40.301
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001020
  03:22:11 STEP 1021 H= 255:15 +CPU= 32.614
  03:22:44 STEP 1022 H= 255:30 +CPU= 32.448
  03:23:16 STEP 1023 H= 255:45 +CPU= 32.253
  03:23:49 STEP 1024 H= 256:00 +CPU= 33.006
  03:24:21 STEP 1025 H= 256:15 +CPU= 32.495
  03:24:54 STEP 1026 H= 256:30 +CPU= 32.437
  03:25:26 STEP 1027 H= 256:45 +CPU= 32.393
  03:25:59 STEP 1028 H= 257:00 +CPU= 33.103
  03:26:32 STEP 1029 H= 257:15 +CPU= 32.609
  03:27:05 STEP 1030 H= 257:30 +CPU= 32.636
  03:27:37 STEP 1031 H= 257:45 +CPU= 32.507
  03:28:17 STEP 1032 H= 258:00 +CPU= 40.078
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001032
  03:28:50 STEP 1033 H= 258:15 +CPU= 32.314
  03:29:22 STEP 1034 H= 258:30 +CPU= 32.519
  03:29:55 STEP 1035 H= 258:45 +CPU= 32.326
  03:30:28 STEP 1036 H= 259:00 +CPU= 33.045
  03:31:00 STEP 1037 H= 259:15 +CPU= 32.345
  03:31:32 STEP 1038 H= 259:30 +CPU= 32.446
  03:32:05 STEP 1039 H= 259:45 +CPU= 32.383
  03:32:38 STEP 1040 H= 260:00 +CPU= 32.993
  03:33:10 STEP 1041 H= 260:15 +CPU= 32.280
  03:33:42 STEP 1042 H= 260:30 +CPU= 32.166
  03:34:15 STEP 1043 H= 260:45 +CPU= 32.214
  03:34:55 STEP 1044 H= 261:00 +CPU= 39.941
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001044
  03:35:27 STEP 1045 H= 261:15 +CPU= 32.569
  03:36:00 STEP 1046 H= 261:30 +CPU= 32.534
  03:36:34 STEP 1047 H= 261:45 +CPU= 32.308
  03:37:12 STEP 1048 H= 262:00 +CPU= 33.167
  03:37:49 STEP 1049 H= 262:15 +CPU= 32.505
  03:38:25 STEP 1050 H= 262:30 +CPU= 32.618
  03:39:01 STEP 1051 H= 262:45 +CPU= 32.808
  03:39:34 STEP 1052 H= 263:00 +CPU= 32.834
  03:40:07 STEP 1053 H= 263:15 +CPU= 32.124
  03:40:41 STEP 1054 H= 263:30 +CPU= 32.520
  03:41:12 STEP 1055 H= 263:45 +CPU= 31.258
  03:41:53 STEP 1056 H= 264:00 +CPU= 40.325
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a21u_2016092300_15_991_12209445_0_r1583579673_10.zip
Uploading the intermediate file: upload_file_10.zip
  03:42:23 STEP 1057 H= 264:15 +CPU= 29.710
  03:42:57 STEP 1058 H= 264:30 +CPU= 32.262
  03:43:30 STEP 1059 H= 264:45 +CPU= 31.769
  03:44:00 STEP 1060 H= 265:00 +CPU= 30.466
  03:44:30 STEP 1061 H= 265:15 +CPU= 29.871
  03:45:00 STEP 1062 H= 265:30 +CPU= 29.462
  03:45:30 STEP 1063 H= 265:45 +CPU= 29.815
  03:46:00 STEP 1064 H= 266:00 +CPU= 30.617
  03:46:30 STEP 1065 H= 266:15 +CPU= 29.703
  03:47:00 STEP 1066 H= 266:30 +CPU= 29.962
  03:47:30 STEP 1067 H= 266:45 +CPU= 29.896
  03:48:07 STEP 1068 H= 267:00 +CPU= 36.722
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001068
  03:48:36 STEP 1069 H= 267:15 +CPU= 29.930
  03:49:06 STEP 1070 H= 267:30 +CPU= 29.815
  03:49:36 STEP 1071 H= 267:45 +CPU= 30.005
  03:50:07 STEP 1072 H= 268:00 +CPU= 30.451
  03:50:37 STEP 1073 H= 268:15 +CPU= 29.919
  03:51:07 STEP 1074 H= 268:30 +CPU= 29.938
  03:51:37 STEP 1075 H= 268:45 +CPU= 29.787
  03:52:07 STEP 1076 H= 269:00 +CPU= 30.297
  03:52:37 STEP 1077 H= 269:15 +CPU= 29.867
  03:53:06 STEP 1078 H= 269:30 +CPU= 29.621
  03:53:36 STEP 1079 H= 269:45 +CPU= 29.870
  03:54:13 STEP 1080 H= 270:00 +CPU= 36.932
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001080
  03:54:43 STEP 1081 H= 270:15 +CPU= 29.784
  03:55:13 STEP 1082 H= 270:30 +CPU= 29.657
  03:55:44 STEP 1083 H= 270:45 +CPU= 30.442
  03:56:21 STEP 1084 H= 271:00 +CPU= 33.129
  03:56:56 STEP 1085 H= 271:15 +CPU= 31.888
  03:57:32 STEP 1086 H= 271:30 +CPU= 32.226
  03:58:04 STEP 1087 H= 271:45 +CPU= 32.345
  03:58:37 STEP 1088 H= 272:00 +CPU= 32.893
  03:59:09 STEP 1089 H= 272:15 +CPU= 32.034
  03:59:41 STEP 1090 H= 272:30 +CPU= 30.686
  04:00:11 STEP 1091 H= 272:45 +CPU= 30.115
  04:00:50 STEP 1092 H= 273:00 +CPU= 37.682
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001092
  04:01:20 STEP 1093 H= 273:15 +CPU= 29.762
  04:01:50 STEP 1094 H= 273:30 +CPU= 29.966
  04:02:22 STEP 1095 H= 273:45 +CPU= 29.817
  04:02:53 STEP 1096 H= 274:00 +CPU= 30.572
  04:03:23 STEP 1097 H= 274:15 +CPU= 29.899
  04:03:52 STEP 1098 H= 274:30 +CPU= 29.919
  04:04:22 STEP 1099 H= 274:45 +CPU= 29.950
  04:04:53 STEP 1100 H= 275:00 +CPU= 30.431
  04:05:23 STEP 1101 H= 275:15 +CPU= 29.998
  04:05:55 STEP 1102 H= 275:30 +CPU= 32.414
  04:06:28 STEP 1103 H= 275:45 +CPU= 32.593
  04:07:15 STEP 1104 H= 276:00 +CPU= 39.794
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001104
  04:07:48 STEP 1105 H= 276:15 +CPU= 30.248
  04:08:22 STEP 1106 H= 276:30 +CPU= 30.285
  04:08:52 STEP 1107 H= 276:45 +CPU= 29.961
  04:09:23 STEP 1108 H= 277:00 +CPU= 30.825
  04:09:53 STEP 1109 H= 277:15 +CPU= 30.043
  04:10:23 STEP 1110 H= 277:30 +CPU= 30.154
  04:10:53 STEP 1111 H= 277:45 +CPU= 30.312
  04:11:24 STEP 1112 H= 278:00 +CPU= 30.427
  04:11:54 STEP 1113 H= 278:15 +CPU= 30.187
  04:12:24 STEP 1114 H= 278:30 +CPU= 29.964
  04:12:54 STEP 1115 H= 278:45 +CPU= 29.887
  04:13:31 STEP 1116 H= 279:00 +CPU= 36.640
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001116
  04:14:00 STEP 1117 H= 279:15 +CPU= 29.867
  04:14:31 STEP 1118 H= 279:30 +CPU= 30.103
  04:15:01 STEP 1119 H= 279:45 +CPU= 30.130
  04:15:31 STEP 1120 H= 280:00 +CPU= 30.723
  04:16:01 STEP 1121 H= 280:15 +CPU= 29.904
  04:16:32 STEP 1122 H= 280:30 +CPU= 30.200
  04:17:02 STEP 1123 H= 280:45 +CPU= 30.328
  04:17:33 STEP 1124 H= 281:00 +CPU= 30.800
  04:18:03 STEP 1125 H= 281:15 +CPU= 30.001
  04:18:33 STEP 1126 H= 281:30 +CPU= 30.078
  04:19:03 STEP 1127 H= 281:45 +CPU= 30.159
  04:19:40 STEP 1128 H= 282:00 +CPU= 37.147
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001128
  04:20:12 STEP 1129 H= 282:15 +CPU= 30.248
  04:20:42 STEP 1130 H= 282:30 +CPU= 30.209
  04:21:12 STEP 1131 H= 282:45 +CPU= 30.110
  04:21:43 STEP 1132 H= 283:00 +CPU= 30.698
  04:22:13 STEP 1133 H= 283:15 +CPU= 30.186
  04:22:43 STEP 1134 H= 283:30 +CPU= 30.089
  04:23:13 STEP 1135 H= 283:45 +CPU= 29.939
  04:23:44 STEP 1136 H= 284:00 +CPU= 30.467
  04:24:14 STEP 1137 H= 284:15 +CPU= 29.960
  04:24:44 STEP 1138 H= 284:30 +CPU= 29.766
  04:25:13 STEP 1139 H= 284:45 +CPU= 29.828
  04:25:51 STEP 1140 H= 285:00 +CPU= 37.477
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001140
  04:26:21 STEP 1141 H= 285:15 +CPU= 29.996
  04:26:51 STEP 1142 H= 285:30 +CPU= 29.838
  04:27:21 STEP 1143 H= 285:45 +CPU= 29.791
  04:27:51 STEP 1144 H= 286:00 +CPU= 30.392
  04:28:21 STEP 1145 H= 286:15 +CPU= 29.970
  04:28:51 STEP 1146 H= 286:30 +CPU= 30.071
  04:29:21 STEP 1147 H= 286:45 +CPU= 29.950
  04:29:52 STEP 1148 H= 287:00 +CPU= 30.574
  04:30:22 STEP 1149 H= 287:15 +CPU= 29.964
  04:30:52 STEP 1150 H= 287:30 +CPU= 30.018
  04:31:22 STEP 1151 H= 287:45 +CPU= 30.060
  04:32:02 STEP 1152 H= 288:00 +CPU= 40.526
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a21u_2016092300_15_991_12209445_0_r1583579673_11.zip
Uploading the intermediate file: upload_file_11.zip
  04:32:33 STEP 1153 H= 288:15 +CPU= 29.683
  04:33:04 STEP 1154 H= 288:30 +CPU= 29.566
  04:33:34 STEP 1155 H= 288:45 +CPU= 30.503
  04:34:06 STEP 1156 H= 289:00 +CPU= 31.803
  04:34:43 STEP 1157 H= 289:15 +CPU= 32.739
  04:35:18 STEP 1158 H= 289:30 +CPU= 32.784
  04:35:51 STEP 1159 H= 289:45 +CPU= 32.747
  04:36:24 STEP 1160 H= 290:00 +CPU= 33.338
  04:36:57 STEP 1161 H= 290:15 +CPU= 32.880
  04:37:30 STEP 1162 H= 290:30 +CPU= 33.006
  04:38:03 STEP 1163 H= 290:45 +CPU= 33.169
  04:38:44 STEP 1164 H= 291:00 +CPU= 40.835
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001164
  04:39:17 STEP 1165 H= 291:15 +CPU= 33.147
  04:39:51 STEP 1166 H= 291:30 +CPU= 33.523
  04:40:24 STEP 1167 H= 291:45 +CPU= 33.456
  04:40:58 STEP 1168 H= 292:00 +CPU= 33.611
  04:41:33 STEP 1169 H= 292:15 +CPU= 33.336
  04:42:10 STEP 1170 H= 292:30 +CPU= 35.260
  04:42:45 STEP 1171 H= 292:45 +CPU= 34.816
  04:43:19 STEP 1172 H= 293:00 +CPU= 34.565
  04:43:53 STEP 1173 H= 293:15 +CPU= 32.471
  04:44:26 STEP 1174 H= 293:30 +CPU= 32.376
  04:45:28 STEP 1175 H= 293:45 +CPU= 31.425
  04:46:11 STEP 1176 H= 294:00 +CPU= 42.106
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001176
  04:46:44 STEP 1177 H= 294:15 +CPU= 32.908
  04:47:18 STEP 1178 H= 294:30 +CPU= 33.094
  04:47:51 STEP 1179 H= 294:45 +CPU= 33.545
  04:48:26 STEP 1180 H= 295:00 +CPU= 34.023
  04:49:01 STEP 1181 H= 295:15 +CPU= 33.855
  04:49:35 STEP 1182 H= 295:30 +CPU= 34.106
  04:50:09 STEP 1183 H= 295:45 +CPU= 33.964
  04:50:43 STEP 1184 H= 296:00 +CPU= 34.398
  04:51:18 STEP 1185 H= 296:15 +CPU= 34.225
  04:51:52 STEP 1186 H= 296:30 +CPU= 34.512
  04:52:27 STEP 1187 H= 296:45 +CPU= 34.779
  04:53:10 STEP 1188 H= 297:00 +CPU= 42.618
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001188
  04:53:44 STEP 1189 H= 297:15 +CPU= 33.517
  04:54:17 STEP 1190 H= 297:30 +CPU= 33.289
  04:54:50 STEP 1191 H= 297:45 +CPU= 33.191
  04:55:32 STEP 1192 H= 298:00 +CPU= 33.616
  04:56:07 STEP 1193 H= 298:15 +CPU= 33.324
  04:56:40 STEP 1194 H= 298:30 +CPU= 32.851
  04:57:21 STEP 1195 H= 298:45 +CPU= 36.273
  04:57:56 STEP 1196 H= 299:00 +CPU= 33.368
  04:58:16 STEP 1197 H= 299:15 +CPU= 20.438
  04:58:37 STEP 1198 H= 299:30 +CPU= 20.331
  04:58:57 STEP 1199 H= 299:45 +CPU= 20.198
  04:59:52 STEP 1200 H= 300:00 +CPU= 29.878
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001200
  05:00:16 STEP 1201 H= 300:15 +CPU= 23.633
  05:00:37 STEP 1202 H= 300:30 +CPU= 20.703
  05:00:57 STEP 1203 H= 300:45 +CPU= 20.780
  05:01:22 STEP 1204 H= 301:00 +CPU= 24.293
  05:01:46 STEP 1205 H= 301:15 +CPU= 24.253
  05:02:06 STEP 1206 H= 301:30 +CPU= 19.698
  05:02:24 STEP 1207 H= 301:45 +CPU= 18.200
  05:02:43 STEP 1208 H= 302:00 +CPU= 18.927
  05:03:01 STEP 1209 H= 302:15 +CPU= 18.645
  05:03:20 STEP 1210 H= 302:30 +CPU= 18.824
  05:03:39 STEP 1211 H= 302:45 +CPU= 18.344
  05:04:01 STEP 1212 H= 303:00 +CPU= 22.143
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001212
  05:04:19 STEP 1213 H= 303:15 +CPU= 17.983
  05:04:39 STEP 1214 H= 303:30 +CPU= 20.056
  05:04:59 STEP 1215 H= 303:45 +CPU= 20.300
  05:05:21 STEP 1216 H= 304:00 +CPU= 21.338
  05:05:43 STEP 1217 H= 304:15 +CPU= 21.713
  05:06:04 STEP 1218 H= 304:30 +CPU= 20.798
  05:06:30 STEP 1219 H= 304:45 +CPU= 26.355
  05:06:52 STEP 1220 H= 305:00 +CPU= 21.818
  05:07:13 STEP 1221 H= 305:15 +CPU= 21.330
  05:07:34 STEP 1222 H= 305:30 +CPU= 21.405
  05:07:58 STEP 1223 H= 305:45 +CPU= 23.912
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a21u
(argv4) batchid: 991
(argv5) wuid: 12209445
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209445
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_29537c9a486c488fa06230e55665be86 to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a21u_2016092300_15_991_12209445.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a21u_2016092300_15_991_12209445.zip
ic_ancil_file: ic_ancil_12209445
ifsdata_file: ifsdata_12209445
climate_data_file: clim_data_12209445
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/8/ic_ancil_12209445.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12209445.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_36d049545f33b19490c63dd816ad497f to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12209445.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12209445.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12209445.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12209445.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12209445.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12209445.xml
-- Model is restarting --
Adjusting last_iter, 1223, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12209445.xml
last_cpu_time: 36733
upload_file_number: 12
last_iter: 1201
last_upload: 1036800
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a21u_2016092300_15_991_12209445_0_r1583579673
The child process has been launched with process id: 17253
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:17253:17253] [20230214:050903:1676347743:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:10:28 STEP 1200 H= 300:00 +CPU= 20.844
  05:10:49 STEP 1201 H= 300:15 +CPU= 21.026
  05:11:13 STEP 1202 H= 300:30 +CPU= 21.069
  05:11:33 STEP 1203 H= 300:45 +CPU= 20.577
  05:11:54 STEP 1204 H= 301:00 +CPU= 21.230
  05:12:19 STEP 1205 H= 301:15 +CPU= 22.943
  05:12:40 STEP 1206 H= 301:30 +CPU= 21.086
  05:13:02 STEP 1207 H= 301:45 +CPU= 21.481
forrtl: severe (41): insufficient virtual memory
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001081813  call_sl_heap_              92  call_sl_heap.F90
oifs_43r3_model.e  0000000000E67648  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
..The child process terminated with status: 41
>>> Printing last 70 lines from file: NODE.001_01

I     I Cond. Ice       II      0.10271 I              I     -0.43852 I              I      0.43852  I
I     I Snow            II              I              I              I              I     -0.03624  I
I     I Sens. heat rain II      0.00000 I              I              I              I               I
I     I Sens. heat snow II      0.00000 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I     I Cond. Water     II      4.98775 I              I     -1.25085 I      1.25085 I               I
I STRA- Rain            II              I              I              I     -1.99446 I               I
I     I Cond. Ice       II      4.72529 I              I     -0.51776 I              I      0.51776  I
I     I Snow            II              I              I              I              I     -1.66706  I
I     I Sens. heat rain II      0.00000 I              I              I              I               I
I     I Sens. heat snow II      0.00000 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I PREC- Pot. and kin. E.II      0.00000 I      0.00000 I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I   Orographic drag     II      0.00000 I      0.00000 I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I      I Balance        II     -0.00016 I              I              I              I               I
I ADIAB- of which       II              I              I              I              I               I
I      I Hor. Flux of PEII     -0.00000 I              I              I              I               I
I      I Vert.Flux of PEII      0.00000 I              I              I              I               I
I      I Work Pres.G.F  II              I      1.10029 I              I              I               I
I      I Conv. Ad.-Int. II     -1.10044 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I DYN- Horizont. Flux   II      0.17582 I      0.20658 I      1.68941 I     -0.00548 I     -0.04228  I
I    I Vertical Flux    II      0.00000 I      0.00000 I      0.00000 I      0.00000 I      0.00000  I
DDH--------------------------------------------------------------------------------------------------I
  
DDH----------------------------------------------------------------------------------------------------------------------------------
  I   - Mass balance    (10**(-6)  KG/M**2) 
  
  I INITIAL      I HORIZ. FLUX  I DYN. FLUX    I DYN. FLUX    I PHYS. FLUX   I PHYS. FLUX   I CURRENT      I I I RESIDUAL     I
  I MASS         I              I TOA          I SURFACE      I TOA          I SURFACE      I MASS         I I I              I
  ---------------------------------------------------------------------------------------------------------- I ----------------
  -              -              -              -              -              -              -              - I -              -
  - 10206523039. I           3. .           0. .           0. .           0. .           0. I 10207553515. - I -     1030473. -
  -              -              -              -              -              -              -              - I -              -
  ---------------------------------------------------------------------------------------------------------- I ----------------
 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  300.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1315988919E-21      I       0.2175810078E-04      I
ICloud fractionI      0.0000000000E+00      I       0.5028621537E-01      I
I  Relat. Hum. I      0.5328327139E+00      I       0.5239150657E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6992134771E+03      I
DDH------------------------------------------------------------------------
  
 05:10:28 STEP 1200 H= 300:00 +CPU= 20.844
 NSTEP =  1201 STEPO   0AAA00AAA
 05:10:49 STEP 1201 H= 300:15 +CPU= 21.026
 NSTEP =  1202 STEPO   0AAA00AAA
 05:11:13 STEP 1202 H= 300:30 +CPU= 21.069
 NSTEP =  1203 STEPO   0AAA00AAA
 05:11:33 STEP 1203 H= 300:45 +CPU= 20.577
 NSTEP =  1204 STEPO   0AAA00AAA
 05:11:54 STEP 1204 H= 301:00 +CPU= 21.230
ISTEP=    1205 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=    1205 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =  1205 STEPO   0AAA00AAA
 05:12:19 STEP 1205 H= 301:15 +CPU= 22.943
 NSTEP =  1206 STEPO   0AAA00AAA
 05:12:40 STEP 1206 H= 301:30 +CPU= 21.086
 NSTEP =  1207 STEPO   0AAA00AAA
 05:13:02 STEP 1207 H= 301:45 +CPU= 21.481
 NSTEP =  1208 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 05:10:51 0AAA00AAA STEPO     1202    21.026   21.026   23.635      0:44      1:48 0.49248746457764E-05       2GB       0MB
 05:11:13 0AAA00AAA STEPO     1203    21.069   21.069   21.070      1:05      2:09 0.49248746457764E-05       2GB       0MB
 05:11:33 0AAA00AAA STEPO     1204    20.577   20.577   20.576      1:26      2:30 0.49248746457764E-05       2GB       0MB
 05:11:56 0AAA00AAA STEPO     1205    21.231   21.231   22.905      1:47      2:53 0.49248746457764E-05       2GB       0MB
 05:12:19 0AAA00AAA STEPO     1206    22.943   22.943   22.954      2:10      3:16 0.49248746457764E-05       2GB       0MB
 05:12:40 0AAA00AAA STEPO     1207    21.086   21.086   21.086      2:31      3:37 0.49248746457764E-05       2GB       0MB
 05:13:02 0AAA00AAA STEPO     1208    21.481   21.481   21.480      2:53      3:58 0.49248746457764E-05       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1200",
 CTIME   = "00121200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,        1188, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/8/progress_file_12209445.xml

<running_values>
  <last_cpu_time>36906.700000</last_cpu_time>
  <upload_file_number>12</upload_file_number>
  <last_iter>1208</last_iter>
  <last_upload>1036800</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org