Name | oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0 |
Workunit | 12208976 |
Created | 13 Feb 2023, 16:02:29 UTC |
Sent | 13 Feb 2023, 16:10:29 UTC |
Report deadline | 14 Apr 2023, 16:10:29 UTC |
Received | 14 Feb 2023, 6:47:10 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1536520 |
Run time | 13 hours 48 min 59 sec |
CPU time | 13 hours 30 min 4 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 7.36 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.11 x86_64-pc-linux-gnu |
Peak working set size | 5,058.16 MB |
Peak swap size | 5,930.79 MB |
Peak disk usage | 1,370.25 MB |
Stderr | <core_client_version>7.22.0</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> 0213:101052:1676304652:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil)) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil)) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil)) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil)) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at 0x1) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil)) [EC_DRHOOK:steve-System-Product-Name:1:1:167:167] [20230213:101052:1676304652:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 10:12:08 STEP 0 H= 0:00 +CPU= 71.340 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000000 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000000 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000000 10:13:28 STEP 1 H= 0:15 +CPU= 77.132 10:14:41 STEP 2 H= 0:30 +CPU= 71.959 10:15:51 STEP 3 H= 0:45 +CPU= 69.270 10:17:08 STEP 4 H= 1:00 +CPU= 75.700 10:18:24 STEP 5 H= 1:15 +CPU= 75.152 10:19:38 STEP 6 H= 1:30 +CPU= 72.133 10:20:56 STEP 7 H= 1:45 +CPU= 77.607 10:22:14 STEP 8 H= 2:00 +CPU= 76.201 10:23:29 STEP 9 H= 2:15 +CPU= 73.527 10:24:50 STEP 10 H= 2:30 +CPU= 79.892 10:26:08 STEP 11 H= 2:45 +CPU= 76.979 10:27:49 STEP 12 H= 3:00 +CPU= 99.294 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000012 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000012 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000012 10:29:13 STEP 13 H= 3:15 +CPU= 81.000 10:30:35 STEP 14 H= 3:30 +CPU= 80.662 10:32:02 STEP 15 H= 3:45 +CPU= 85.318 10:33:28 STEP 16 H= 4:00 +CPU= 84.859 10:34:56 STEP 17 H= 4:15 +CPU= 87.054 10:36:23 STEP 18 H= 4:30 +CPU= 85.522 10:37:55 STEP 19 H= 4:45 +CPU= 91.346 10:39:19 STEP 20 H= 5:00 +CPU= 83.372 10:40:46 STEP 21 H= 5:15 +CPU= 85.665 10:42:18 STEP 22 H= 5:30 +CPU= 90.680 10:43:39 STEP 23 H= 5:45 +CPU= 80.097 10:45:27 STEP 24 H= 6:00 +CPU=105.925 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000024 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000024 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000024 10:47:00 STEP 25 H= 6:15 +CPU= 91.984 10:48:21 STEP 26 H= 6:30 +CPU= 79.716 10:49:44 STEP 27 H= 6:45 +CPU= 81.715 10:51:08 STEP 28 H= 7:00 +CPU= 82.656 10:52:33 STEP 29 H= 7:15 +CPU= 83.923 10:53:59 STEP 30 H= 7:30 +CPU= 84.748 10:55:24 STEP 31 H= 7:45 +CPU= 84.152 10:56:52 STEP 32 H= 8:00 +CPU= 86.569 10:58:23 STEP 33 H= 8:15 +CPU= 89.963 10:59:50 STEP 34 H= 8:30 +CPU= 84.817 11:01:21 STEP 35 H= 8:45 +CPU= 90.285 11:03:15 STEP 36 H= 9:00 +CPU=112.017 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000036 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000036 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000036 11:04:34 STEP 37 H= 9:15 +CPU= 78.427 11:06:00 STEP 38 H= 9:30 +CPU= 84.150 11:07:30 STEP 39 H= 9:45 +CPU= 89.554 11:08:59 STEP 40 H= 10:00 +CPU= 87.278 11:10:26 STEP 41 H= 10:15 +CPU= 85.937 11:11:54 STEP 42 H= 10:30 +CPU= 86.318 11:13:18 STEP 43 H= 10:45 +CPU= 83.165 11:14:46 STEP 44 H= 11:00 +CPU= 86.627 11:16:14 STEP 45 H= 11:15 +CPU= 87.656 11:17:40 STEP 46 H= 11:30 +CPU= 84.652 11:19:08 STEP 47 H= 11:45 +CPU= 86.203 11:21:06 STEP 48 H= 12:00 +CPU=115.104 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000048 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000048 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000048 11:22:37 STEP 49 H= 12:15 +CPU= 89.218 11:24:02 STEP 50 H= 12:30 +CPU= 83.534 11:25:32 STEP 51 H= 12:45 +CPU= 88.779 11:27:04 STEP 52 H= 13:00 +CPU= 91.108 11:28:24 STEP 53 H= 13:15 +CPU= 77.835 11:29:53 STEP 54 H= 13:30 +CPU= 87.688 11:31:16 STEP 55 H= 13:45 +CPU= 82.110 11:32:43 STEP 56 H= 14:00 +CPU= 84.923 11:34:10 STEP 57 H= 14:15 +CPU= 86.092 11:35:36 STEP 58 H= 14:30 +CPU= 84.979 11:37:02 STEP 59 H= 14:45 +CPU= 84.842 11:38:42 STEP 60 H= 15:00 +CPU= 98.194 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000060 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000060 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000060 11:40:18 STEP 61 H= 15:15 +CPU= 93.986 11:41:42 STEP 62 H= 15:30 +CPU= 82.596 11:43:10 STEP 63 H= 15:45 +CPU= 86.614 11:44:33 STEP 64 H= 16:00 +CPU= 81.320 11:45:56 STEP 65 H= 16:15 +CPU= 81.530 11:47:25 STEP 66 H= 16:30 +CPU= 87.172 11:48:51 STEP 67 H= 16:45 +CPU= 85.086 11:50:16 STEP 68 H= 17:00 +CPU= 83.792 11:51:45 STEP 69 H= 17:15 +CPU= 87.490 11:53:08 STEP 70 H= 17:30 +CPU= 81.788 11:54:35 STEP 71 H= 17:45 +CPU= 84.939 11:56:30 STEP 72 H= 18:00 +CPU=113.220 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000072 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000072 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000072 11:57:54 STEP 73 H= 18:15 +CPU= 82.447 11:59:26 STEP 74 H= 18:30 +CPU= 91.029 12:00:54 STEP 75 H= 18:45 +CPU= 87.020 12:02:18 STEP 76 H= 19:00 +CPU= 82.730 12:03:51 STEP 77 H= 19:15 +CPU= 90.749 12:05:15 STEP 78 H= 19:30 +CPU= 83.167 12:06:44 STEP 79 H= 19:45 +CPU= 87.338 12:08:13 STEP 80 H= 20:00 +CPU= 87.450 12:09:41 STEP 81 H= 20:15 +CPU= 86.286 12:11:08 STEP 82 H= 20:30 +CPU= 86.592 12:12:34 STEP 83 H= 20:45 +CPU= 84.343 12:14:12 STEP 84 H= 21:00 +CPU= 96.418 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000084 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000084 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000084 12:15:36 STEP 85 H= 21:15 +CPU= 82.161 12:17:08 STEP 86 H= 21:30 +CPU= 90.908 12:18:31 STEP 87 H= 21:45 +CPU= 81.271 12:20:02 STEP 88 H= 22:00 +CPU= 89.538 12:21:27 STEP 89 H= 22:15 +CPU= 83.234 12:22:55 STEP 90 H= 22:30 +CPU= 86.729 12:24:26 STEP 91 H= 22:45 +CPU= 88.937 12:25:53 STEP 92 H= 23:00 +CPU= 85.748 12:27:20 STEP 93 H= 23:15 +CPU= 84.617 12:28:49 STEP 94 H= 23:30 +CPU= 86.252 12:30:11 STEP 95 H= 23:45 +CPU= 79.029 12:32:18 STEP 96 H= 24:00 +CPU=123.141 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000096 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000096 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000096 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000000 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000000 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000000 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000012 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000012 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000012 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000024 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000024 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000024 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000036 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000036 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000036 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000048 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000048 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000048 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000060 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000060 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000060 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000072 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000072 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000072 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000084 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000084 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000084 Zipping up the intermediate file: /home/steve/projects/climateprediction.net/oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0_r669128754_0.zip Uploading the intermediate file: upload_file_0.zip 12:33:43 STEP 97 H= 24:15 +CPU= 80.546 12:35:12 STEP 98 H= 24:30 +CPU= 88.007 12:36:40 STEP 99 H= 24:45 +CPU= 86.265 12:38:03 STEP 100 H= 25:00 +CPU= 81.447 12:39:31 STEP 101 H= 25:15 +CPU= 86.985 12:41:04 STEP 102 H= 25:30 +CPU= 90.922 12:42:25 STEP 103 H= 25:45 +CPU= 79.835 12:43:58 STEP 104 H= 26:00 +CPU= 88.322 12:45:29 STEP 105 H= 26:15 +CPU= 89.312 12:46:52 STEP 106 H= 26:30 +CPU= 81.685 12:48:20 STEP 107 H= 26:45 +CPU= 86.078 12:50:08 STEP 108 H= 27:00 +CPU=107.493 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000108 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000108 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000108 12:51:36 STEP 109 H= 27:15 +CPU= 85.696 12:53:04 STEP 110 H= 27:30 +CPU= 86.642 12:54:28 STEP 111 H= 27:45 +CPU= 83.507 12:55:57 STEP 112 H= 28:00 +CPU= 87.922 12:57:26 STEP 113 H= 28:15 +CPU= 87.321 12:58:49 STEP 114 H= 28:30 +CPU= 81.571 13:00:17 STEP 115 H= 28:45 +CPU= 87.353 13:01:42 STEP 116 H= 29:00 +CPU= 83.373 13:03:06 STEP 117 H= 29:15 +CPU= 83.094 13:04:33 STEP 118 H= 29:30 +CPU= 85.815 13:05:59 STEP 119 H= 29:45 +CPU= 84.447 13:07:51 STEP 120 H= 30:00 +CPU=109.812 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000120 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000120 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000120 13:09:21 STEP 121 H= 30:15 +CPU= 89.383 13:10:45 STEP 122 H= 30:30 +CPU= 82.184 13:12:15 STEP 123 H= 30:45 +CPU= 88.425 13:13:44 STEP 124 H= 31:00 +CPU= 88.125 13:15:17 STEP 125 H= 31:15 +CPU= 91.332 13:16:51 STEP 126 H= 31:30 +CPU= 93.285 13:18:16 STEP 127 H= 31:45 +CPU= 83.018 13:19:46 STEP 128 H= 32:00 +CPU= 88.499 13:21:18 STEP 129 H= 32:15 +CPU= 91.199 13:22:42 STEP 130 H= 32:30 +CPU= 82.734 13:24:16 STEP 131 H= 32:45 +CPU= 92.757 13:26:00 STEP 132 H= 33:00 +CPU=101.951 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000132 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000132 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000132 13:27:27 STEP 133 H= 33:15 +CPU= 85.855 13:28:56 STEP 134 H= 33:30 +CPU= 87.367 13:30:24 STEP 135 H= 33:45 +CPU= 86.680 13:31:54 STEP 136 H= 34:00 +CPU= 88.859 13:33:24 STEP 137 H= 34:15 +CPU= 88.088 13:34:51 STEP 138 H= 34:30 +CPU= 85.723 13:36:17 STEP 139 H= 34:45 +CPU= 85.499 13:37:47 STEP 140 H= 35:00 +CPU= 87.540 13:39:13 STEP 141 H= 35:15 +CPU= 84.018 13:40:46 STEP 142 H= 35:30 +CPU= 91.419 13:42:16 STEP 143 H= 35:45 +CPU= 88.949 13:44:16 STEP 144 H= 36:00 +CPU=116.912 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000144 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000144 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000144 13:45:41 STEP 145 H= 36:15 +CPU= 84.023 13:47:06 STEP 146 H= 36:30 +CPU= 83.252 13:48:33 STEP 147 H= 36:45 +CPU= 85.259 13:49:57 STEP 148 H= 37:00 +CPU= 83.041 13:51:19 STEP 149 H= 37:15 +CPU= 80.503 13:52:51 STEP 150 H= 37:30 +CPU= 90.153 13:54:17 STEP 151 H= 37:45 +CPU= 84.647 13:55:43 STEP 152 H= 38:00 +CPU= 85.546 13:57:11 STEP 153 H= 38:15 +CPU= 86.650 13:58:42 STEP 154 H= 38:30 +CPU= 89.729 14:00:05 STEP 155 H= 38:45 +CPU= 80.975 14:01:50 STEP 156 H= 39:00 +CPU=103.467 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000156 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000156 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000156 14:03:19 STEP 157 H= 39:15 +CPU= 87.806 14:04:50 STEP 158 H= 39:30 +CPU= 89.796 14:06:18 STEP 159 H= 39:45 +CPU= 86.077 14:07:43 STEP 160 H= 40:00 +CPU= 83.740 14:09:10 STEP 161 H= 40:15 +CPU= 85.266 14:10:37 STEP 162 H= 40:30 +CPU= 85.794 14:12:06 STEP 163 H= 40:45 +CPU= 87.949 14:13:32 STEP 164 H= 41:00 +CPU= 84.713 14:15:00 STEP 165 H= 41:15 +CPU= 87.124 14:16:30 STEP 166 H= 41:30 +CPU= 88.461 14:17:50 STEP 167 H= 41:45 +CPU= 78.815 14:19:40 STEP 168 H= 42:00 +CPU=108.310 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000168 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000168 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000168 14:21:10 STEP 169 H= 42:15 +CPU= 88.917 14:22:37 STEP 170 H= 42:30 +CPU= 86.039 14:23:59 STEP 171 H= 42:45 +CPU= 79.911 14:25:34 STEP 172 H= 43:00 +CPU= 94.054 14:26:59 STEP 173 H= 43:15 +CPU= 84.276 14:28:24 STEP 174 H= 43:30 +CPU= 84.008 14:29:52 STEP 175 H= 43:45 +CPU= 86.453 14:31:24 STEP 176 H= 44:00 +CPU= 90.928 14:32:45 STEP 177 H= 44:15 +CPU= 79.746 14:34:10 STEP 178 H= 44:30 +CPU= 84.324 14:35:41 STEP 179 H= 44:45 +CPU= 88.944 14:37:21 STEP 180 H= 45:00 +CPU= 98.871 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000180 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000180 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000180 14:38:51 STEP 181 H= 45:15 +CPU= 88.508 14:40:11 STEP 182 H= 45:30 +CPU= 78.902 14:41:37 STEP 183 H= 45:45 +CPU= 84.473 14:43:07 STEP 184 H= 46:00 +CPU= 88.961 14:44:31 STEP 185 H= 46:15 +CPU= 82.899 14:45:55 STEP 186 H= 46:30 +CPU= 81.856 14:47:23 STEP 187 H= 46:45 +CPU= 83.591 14:48:50 STEP 188 H= 47:00 +CPU= 84.272 14:50:13 STEP 189 H= 47:15 +CPU= 81.742 14:51:40 STEP 190 H= 47:30 +CPU= 85.409 14:53:06 STEP 191 H= 47:45 +CPU= 85.191 14:55:05 STEP 192 H= 48:00 +CPU=114.125 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000192 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000192 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000192 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000096 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000096 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000096 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000108 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000108 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000108 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000120 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000120 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000120 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000132 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000132 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000132 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000144 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000144 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000144 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000156 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000156 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000156 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000168 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000168 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000168 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000180 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000180 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000180 Zipping up the intermediate file: /home/steve/projects/climateprediction.net/oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0_r669128754_1.zip Uploading the intermediate file: upload_file_1.zip 14:56:32 STEP 193 H= 48:15 +CPU= 84.774 14:58:01 STEP 194 H= 48:30 +CPU= 87.383 14:59:37 STEP 195 H= 48:45 +CPU= 93.112 15:00:58 STEP 196 H= 49:00 +CPU= 79.666 15:02:28 STEP 197 H= 49:15 +CPU= 88.660 15:03:59 STEP 198 H= 49:30 +CPU= 89.449 15:05:26 STEP 199 H= 49:45 +CPU= 85.532 15:07:03 STEP 200 H= 50:00 +CPU= 95.950 15:08:33 STEP 201 H= 50:15 +CPU= 88.727 15:10:02 STEP 202 H= 50:30 +CPU= 87.058 15:11:31 STEP 203 H= 50:45 +CPU= 87.208 15:13:13 STEP 204 H= 51:00 +CPU=100.650 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000204 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000204 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000204 15:14:43 STEP 205 H= 51:15 +CPU= 88.219 15:16:18 STEP 206 H= 51:30 +CPU= 93.623 15:17:42 STEP 207 H= 51:45 +CPU= 83.372 15:19:16 STEP 208 H= 52:00 +CPU= 93.169 15:20:40 STEP 209 H= 52:15 +CPU= 82.114 15:22:08 STEP 210 H= 52:30 +CPU= 86.768 15:23:43 STEP 211 H= 52:45 +CPU= 93.339 15:25:13 STEP 212 H= 53:00 +CPU= 88.250 15:26:44 STEP 213 H= 53:15 +CPU= 89.742 15:28:10 STEP 214 H= 53:30 +CPU= 84.809 15:29:30 STEP 215 H= 53:45 +CPU= 78.968 15:31:17 STEP 216 H= 54:00 +CPU=105.050 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000216 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000216 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000216 15:32:42 STEP 217 H= 54:15 +CPU= 84.597 15:34:09 STEP 218 H= 54:30 +CPU= 84.568 15:35:38 STEP 219 H= 54:45 +CPU= 75.688 15:36:59 STEP 220 H= 55:00 +CPU= 72.720 15:38:23 STEP 221 H= 55:15 +CPU= 76.384 15:39:51 STEP 222 H= 55:30 +CPU= 86.784 15:41:09 STEP 223 H= 55:45 +CPU= 76.522 15:42:35 STEP 224 H= 56:00 +CPU= 84.890 15:44:01 STEP 225 H= 56:15 +CPU= 84.953 15:45:19 STEP 226 H= 56:30 +CPU= 76.371 15:46:39 STEP 227 H= 56:45 +CPU= 78.645 15:48:25 STEP 228 H= 57:00 +CPU=103.921 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000228 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000228 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000228 15:49:46 STEP 229 H= 57:15 +CPU= 79.475 15:51:14 STEP 230 H= 57:30 +CPU= 86.662 15:52:33 STEP 231 H= 57:45 +CPU= 77.424 15:53:50 STEP 232 H= 58:00 +CPU= 75.882 15:55:12 STEP 233 H= 58:15 +CPU= 80.817 15:56:32 STEP 234 H= 58:30 +CPU= 78.284 15:57:54 STEP 235 H= 58:45 +CPU= 80.803 15:59:20 STEP 236 H= 59:00 +CPU= 84.604 16:00:41 STEP 237 H= 59:15 +CPU= 79.681 16:02:04 STEP 238 H= 59:30 +CPU= 80.787 16:03:26 STEP 239 H= 59:45 +CPU= 80.912 16:05:12 STEP 240 H= 60:00 +CPU=103.809 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000240 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000240 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000240 16:06:37 STEP 241 H= 60:15 +CPU= 83.520 16:07:59 STEP 242 H= 60:30 +CPU= 80.567 16:09:22 STEP 243 H= 60:45 +CPU= 81.021 16:10:47 STEP 244 H= 61:00 +CPU= 83.556 16:12:07 STEP 245 H= 61:15 +CPU= 78.572 16:13:37 STEP 246 H= 61:30 +CPU= 87.784 16:14:56 STEP 247 H= 61:45 +CPU= 77.913 16:16:17 STEP 248 H= 62:00 +CPU= 79.266 16:17:42 STEP 249 H= 62:15 +CPU= 84.285 16:18:59 STEP 250 H= 62:30 +CPU= 76.015 16:20:14 STEP 251 H= 62:45 +CPU= 73.942 16:22:01 STEP 252 H= 63:00 +CPU=105.400 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000252 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000252 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000252 16:23:24 STEP 253 H= 63:15 +CPU= 81.747 16:24:47 STEP 254 H= 63:30 +CPU= 81.625 16:26:08 STEP 255 H= 63:45 +CPU= 80.177 16:27:28 STEP 256 H= 64:00 +CPU= 78.728 16:28:51 STEP 257 H= 64:15 +CPU= 81.309 16:30:11 STEP 258 H= 64:30 +CPU= 79.209 16:31:27 STEP 259 H= 64:45 +CPU= 75.042 16:32:52 STEP 260 H= 65:00 +CPU= 83.714 16:34:11 STEP 261 H= 65:15 +CPU= 78.047 16:35:27 STEP 262 H= 65:30 +CPU= 74.962 16:36:51 STEP 263 H= 65:45 +CPU= 82.800 16:38:34 STEP 264 H= 66:00 +CPU=100.688 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000264 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000264 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000264 16:39:54 STEP 265 H= 66:15 +CPU= 76.882 16:41:21 STEP 266 H= 66:30 +CPU= 78.391 16:42:41 STEP 267 H= 66:45 +CPU= 78.149 16:43:59 STEP 268 H= 67:00 +CPU= 77.568 16:45:23 STEP 269 H= 67:15 +CPU= 82.783 16:46:41 STEP 270 H= 67:30 +CPU= 76.617 16:48:06 STEP 271 H= 67:45 +CPU= 84.712 16:49:26 STEP 272 H= 68:00 +CPU= 78.695 16:50:45 STEP 273 H= 68:15 +CPU= 77.710 16:52:07 STEP 274 H= 68:30 +CPU= 81.117 16:53:31 STEP 275 H= 68:45 +CPU= 82.389 16:55:05 STEP 276 H= 69:00 +CPU= 92.840 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000276 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000276 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000276 16:56:23 STEP 277 H= 69:15 +CPU= 76.974 16:57:45 STEP 278 H= 69:30 +CPU= 80.237 16:59:01 STEP 279 H= 69:45 +CPU= 72.247 17:00:28 STEP 280 H= 70:00 +CPU= 85.267 17:01:51 STEP 281 H= 70:15 +CPU= 81.081 17:03:11 STEP 282 H= 70:30 +CPU= 77.988 17:04:33 STEP 283 H= 70:45 +CPU= 81.005 17:05:59 STEP 284 H= 71:00 +CPU= 84.853 17:07:18 STEP 285 H= 71:15 +CPU= 77.907 17:08:40 STEP 286 H= 71:30 +CPU= 80.531 17:09:59 STEP 287 H= 71:45 +CPU= 76.582 17:11:52 STEP 288 H= 72:00 +CPU=109.935 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000288 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000288 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000288 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000192 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000192 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000192 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000204 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000204 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000204 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000216 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000216 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000216 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000228 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000228 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000228 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000240 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000240 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000240 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000252 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000252 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000252 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000264 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000264 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000264 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000276 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000276 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000276 Zipping up the intermediate file: /home/steve/projects/climateprediction.net/oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0_r669128754_2.zip Uploading the intermediate file: upload_file_2.zip 17:13:15 STEP 289 H= 72:15 +CPU= 79.973 17:14:36 STEP 290 H= 72:30 +CPU= 79.577 17:15:54 STEP 291 H= 72:45 +CPU= 77.108 17:17:14 STEP 292 H= 73:00 +CPU= 78.721 17:18:39 STEP 293 H= 73:15 +CPU= 82.158 17:20:06 STEP 294 H= 73:30 +CPU= 85.133 17:21:28 STEP 295 H= 73:45 +CPU= 80.781 17:22:54 STEP 296 H= 74:00 +CPU= 84.634 17:24:17 STEP 297 H= 74:15 +CPU= 80.851 17:25:40 STEP 298 H= 74:30 +CPU= 82.157 17:26:59 STEP 299 H= 74:45 +CPU= 77.520 17:28:43 STEP 300 H= 75:00 +CPU=101.742 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000300 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000300 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000300 17:30:01 STEP 301 H= 75:15 +CPU= 77.642 17:31:26 STEP 302 H= 75:30 +CPU= 83.308 17:32:49 STEP 303 H= 75:45 +CPU= 81.600 17:34:08 STEP 304 H= 76:00 +CPU= 78.283 17:35:34 STEP 305 H= 76:15 +CPU= 84.556 17:36:57 STEP 306 H= 76:30 +CPU= 81.678 17:38:14 STEP 307 H= 76:45 +CPU= 75.735 17:39:37 STEP 308 H= 77:00 +CPU= 81.712 17:40:59 STEP 309 H= 77:15 +CPU= 80.960 17:42:21 STEP 310 H= 77:30 +CPU= 81.140 17:43:45 STEP 311 H= 77:45 +CPU= 82.915 17:45:25 STEP 312 H= 78:00 +CPU= 97.650 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000312 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000312 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000312 17:46:46 STEP 313 H= 78:15 +CPU= 80.660 17:48:12 STEP 314 H= 78:30 +CPU= 85.061 17:49:32 STEP 315 H= 78:45 +CPU= 78.879 17:50:55 STEP 316 H= 79:00 +CPU= 81.872 17:52:17 STEP 317 H= 79:15 +CPU= 80.824 17:53:40 STEP 318 H= 79:30 +CPU= 81.788 17:55:03 STEP 319 H= 79:45 +CPU= 82.164 17:56:31 STEP 320 H= 80:00 +CPU= 86.935 17:57:56 STEP 321 H= 80:15 +CPU= 83.838 17:59:14 STEP 322 H= 80:30 +CPU= 76.516 18:00:35 STEP 323 H= 80:45 +CPU= 79.419 18:02:11 STEP 324 H= 81:00 +CPU= 94.191 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000324 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000324 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000324 18:03:34 STEP 325 H= 81:15 +CPU= 81.870 18:04:48 STEP 326 H= 81:30 +CPU= 73.140 18:06:12 STEP 327 H= 81:45 +CPU= 81.989 18:07:38 STEP 328 H= 82:00 +CPU= 85.651 18:08:55 STEP 329 H= 82:15 +CPU= 75.693 18:10:19 STEP 330 H= 82:30 +CPU= 82.621 18:11:41 STEP 331 H= 82:45 +CPU= 80.141 18:13:03 STEP 332 H= 83:00 +CPU= 80.513 18:14:24 STEP 333 H= 83:15 +CPU= 79.619 18:15:44 STEP 334 H= 83:30 +CPU= 78.666 18:17:07 STEP 335 H= 83:45 +CPU= 81.845 18:18:53 STEP 336 H= 84:00 +CPU=105.191 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000336 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000336 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000336 18:20:17 STEP 337 H= 84:15 +CPU= 82.151 18:21:38 STEP 338 H= 84:30 +CPU= 80.075 18:22:57 STEP 339 H= 84:45 +CPU= 78.207 18:24:19 STEP 340 H= 85:00 +CPU= 80.695 18:25:45 STEP 341 H= 85:15 +CPU= 84.218 18:27:07 STEP 342 H= 85:30 +CPU= 81.395 18:28:24 STEP 343 H= 85:45 +CPU= 75.979 18:29:41 STEP 344 H= 86:00 +CPU= 75.497 18:31:00 STEP 345 H= 86:15 +CPU= 78.251 18:32:14 STEP 346 H= 86:30 +CPU= 72.438 18:33:27 STEP 347 H= 86:45 +CPU= 72.686 18:35:00 STEP 348 H= 87:00 +CPU= 91.491 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000348 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000348 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000348 18:36:18 STEP 349 H= 87:15 +CPU= 77.286 18:37:33 STEP 350 H= 87:30 +CPU= 74.645 18:38:50 STEP 351 H= 87:45 +CPU= 75.195 18:40:05 STEP 352 H= 88:00 +CPU= 71.113 18:41:24 STEP 353 H= 88:15 +CPU= 75.457 18:42:40 STEP 354 H= 88:30 +CPU= 75.097 18:43:55 STEP 355 H= 88:45 +CPU= 72.599 18:45:16 STEP 356 H= 89:00 +CPU= 76.962 18:46:34 STEP 357 H= 89:15 +CPU= 75.193 18:47:47 STEP 358 H= 89:30 +CPU= 71.187 18:49:00 STEP 359 H= 89:45 +CPU= 72.602 18:50:33 STEP 360 H= 90:00 +CPU= 91.386 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000360 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000360 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000360 18:51:48 STEP 361 H= 90:15 +CPU= 73.889 18:53:06 STEP 362 H= 90:30 +CPU= 75.461 18:54:20 STEP 363 H= 90:45 +CPU= 72.269 18:55:41 STEP 364 H= 91:00 +CPU= 79.045 18:56:58 STEP 365 H= 91:15 +CPU= 76.123 18:58:15 STEP 366 H= 91:30 +CPU= 74.632 18:59:33 STEP 367 H= 91:45 +CPU= 76.729 19:00:54 STEP 368 H= 92:00 +CPU= 79.810 19:02:14 STEP 369 H= 92:15 +CPU= 78.064 19:03:30 STEP 370 H= 92:30 +CPU= 74.997 19:04:48 STEP 371 H= 92:45 +CPU= 75.479 19:06:15 STEP 372 H= 93:00 +CPU= 83.901 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000372 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000372 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000372 19:07:26 STEP 373 H= 93:15 +CPU= 67.202 19:08:37 STEP 374 H= 93:30 +CPU= 67.920 19:09:45 STEP 375 H= 93:45 +CPU= 65.947 19:10:57 STEP 376 H= 94:00 +CPU= 69.694 19:12:10 STEP 377 H= 94:15 +CPU= 66.163 19:13:23 STEP 378 H= 94:30 +CPU= 66.746 19:14:37 STEP 379 H= 94:45 +CPU= 68.511 19:15:51 STEP 380 H= 95:00 +CPU= 66.357 19:16:59 STEP 381 H= 95:15 +CPU= 62.023 19:18:10 STEP 382 H= 95:30 +CPU= 62.354 19:19:25 STEP 383 H= 95:45 +CPU= 66.359 19:21:05 STEP 384 H= 96:00 +CPU= 92.164 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000384 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000384 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000384 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000288 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000288 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000288 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000300 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000300 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000300 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000312 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000312 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000312 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000324 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000324 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000324 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000336 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000336 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000336 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000348 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000348 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000348 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000360 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000360 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000360 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000372 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000372 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000372 Zipping up the intermediate file: /home/steve/projects/climateprediction.net/oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0_r669128754_3.zip Uploading the intermediate file: upload_file_3.zip 19:22:16 STEP 385 H= 96:15 +CPU= 65.109 19:23:29 STEP 386 H= 96:30 +CPU= 66.385 19:24:45 STEP 387 H= 96:45 +CPU= 68.245 19:25:56 STEP 388 H= 97:00 +CPU= 66.666 19:27:08 STEP 389 H= 97:15 +CPU= 66.405 19:28:24 STEP 390 H= 97:30 +CPU= 69.444 19:29:45 STEP 391 H= 97:45 +CPU= 70.878 19:31:08 STEP 392 H= 98:00 +CPU= 75.397 19:32:25 STEP 393 H= 98:15 +CPU= 72.463 19:33:43 STEP 394 H= 98:30 +CPU= 71.480 19:35:05 STEP 395 H= 98:45 +CPU= 78.556 19:36:37 STEP 396 H= 99:00 +CPU= 89.379 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000396 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000396 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000396 19:37:53 STEP 397 H= 99:15 +CPU= 74.887 19:39:12 STEP 398 H= 99:30 +CPU= 77.384 19:40:30 STEP 399 H= 99:45 +CPU= 76.557 19:41:49 STEP 400 H= 100:00 +CPU= 77.493 19:43:04 STEP 401 H= 100:15 +CPU= 72.576 19:44:20 STEP 402 H= 100:30 +CPU= 73.877 19:45:40 STEP 403 H= 100:45 +CPU= 77.661 19:47:00 STEP 404 H= 101:00 +CPU= 76.950 19:48:11 STEP 405 H= 101:15 +CPU= 69.881 19:49:31 STEP 406 H= 101:30 +CPU= 75.683 19:50:51 STEP 407 H= 101:45 +CPU= 77.698 19:52:24 STEP 408 H= 102:00 +CPU= 90.370 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000408 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000408 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000408 19:53:45 STEP 409 H= 102:15 +CPU= 78.655 19:55:03 STEP 410 H= 102:30 +CPU= 76.446 19:56:21 STEP 411 H= 102:45 +CPU= 74.342 19:57:39 STEP 412 H= 103:00 +CPU= 75.595 19:58:56 STEP 413 H= 103:15 +CPU= 73.180 20:00:11 STEP 414 H= 103:30 +CPU= 72.289 20:01:21 STEP 415 H= 103:45 +CPU= 67.336 20:02:32 STEP 416 H= 104:00 +CPU= 68.935 20:03:46 STEP 417 H= 104:15 +CPU= 72.031 20:04:56 STEP 418 H= 104:30 +CPU= 68.942 20:06:07 STEP 419 H= 104:45 +CPU= 68.658 20:07:34 STEP 420 H= 105:00 +CPU= 85.687 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000420 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000420 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000420 20:08:46 STEP 421 H= 105:15 +CPU= 70.197 20:09:56 STEP 422 H= 105:30 +CPU= 67.795 20:11:07 STEP 423 H= 105:45 +CPU= 69.376 20:12:21 STEP 424 H= 106:00 +CPU= 71.688 20:13:33 STEP 425 H= 106:15 +CPU= 70.009 20:14:42 STEP 426 H= 106:30 +CPU= 68.233 20:15:58 STEP 427 H= 106:45 +CPU= 73.311 20:17:10 STEP 428 H= 107:00 +CPU= 70.108 20:18:20 STEP 429 H= 107:15 +CPU= 68.522 20:19:31 STEP 430 H= 107:30 +CPU= 68.886 20:20:40 STEP 431 H= 107:45 +CPU= 67.424 20:22:15 STEP 432 H= 108:00 +CPU= 92.504 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000432 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000432 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000432 20:23:24 STEP 433 H= 108:15 +CPU= 67.677 20:24:34 STEP 434 H= 108:30 +CPU= 68.410 20:25:44 STEP 435 H= 108:45 +CPU= 67.829 20:26:58 STEP 436 H= 109:00 +CPU= 71.516 20:28:07 STEP 437 H= 109:15 +CPU= 67.705 20:29:19 STEP 438 H= 109:30 +CPU= 69.907 20:30:30 STEP 439 H= 109:45 +CPU= 68.491 20:31:41 STEP 440 H= 110:00 +CPU= 69.771 20:32:54 STEP 441 H= 110:15 +CPU= 70.481 20:34:05 STEP 442 H= 110:30 +CPU= 68.697 20:35:15 STEP 443 H= 110:45 +CPU= 69.104 20:36:44 STEP 444 H= 111:00 +CPU= 86.723 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000444 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000444 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000444 20:37:53 STEP 445 H= 111:15 +CPU= 67.013 20:39:05 STEP 446 H= 111:30 +CPU= 69.990 20:40:18 STEP 447 H= 111:45 +CPU= 71.233 20:41:28 STEP 448 H= 112:00 +CPU= 67.494 20:42:38 STEP 449 H= 112:15 +CPU= 69.106 20:43:50 STEP 450 H= 112:30 +CPU= 69.457 20:44:55 STEP 451 H= 112:45 +CPU= 64.116 20:46:07 STEP 452 H= 113:00 +CPU= 69.520 20:47:18 STEP 453 H= 113:15 +CPU= 69.378 20:48:28 STEP 454 H= 113:30 +CPU= 67.982 20:49:40 STEP 455 H= 113:45 +CPU= 70.417 20:51:07 STEP 456 H= 114:00 +CPU= 84.436 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000456 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000456 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000456 20:52:18 STEP 457 H= 114:15 +CPU= 69.179 20:53:27 STEP 458 H= 114:30 +CPU= 67.212 20:54:39 STEP 459 H= 114:45 +CPU= 70.244 20:55:50 STEP 460 H= 115:00 +CPU= 69.535 20:57:03 STEP 461 H= 115:15 +CPU= 70.939 20:58:13 STEP 462 H= 115:30 +CPU= 68.484 20:59:24 STEP 463 H= 115:45 +CPU= 68.390 21:00:39 STEP 464 H= 116:00 +CPU= 70.468 21:01:47 STEP 465 H= 116:15 +CPU= 67.003 21:02:59 STEP 466 H= 116:30 +CPU= 69.911 21:04:11 STEP 467 H= 116:45 +CPU= 69.980 21:05:36 STEP 468 H= 117:00 +CPU= 83.937 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000468 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000468 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000468 21:06:51 STEP 469 H= 117:15 +CPU= 73.653 21:08:02 STEP 470 H= 117:30 +CPU= 70.597 21:09:13 STEP 471 H= 117:45 +CPU= 70.199 21:10:27 STEP 472 H= 118:00 +CPU= 72.360 21:11:36 STEP 473 H= 118:15 +CPU= 67.421 21:12:49 STEP 474 H= 118:30 +CPU= 71.839 21:14:04 STEP 475 H= 118:45 +CPU= 74.291 21:15:14 STEP 476 H= 119:00 +CPU= 69.077 21:16:23 STEP 477 H= 119:15 +CPU= 66.348 21:17:36 STEP 478 H= 119:30 +CPU= 71.388 21:18:42 STEP 479 H= 119:45 +CPU= 65.464 21:20:15 STEP 480 H= 120:00 +CPU= 91.013 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000480 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000480 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000480 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000384 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000384 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000384 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000396 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000396 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000396 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000408 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000408 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000408 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000420 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000420 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000420 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000432 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000432 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000432 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000444 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000444 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000444 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000456 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000456 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000456 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000468 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000468 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000468 Zipping up the intermediate file: /home/steve/projects/climateprediction.net/oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0_r669128754_4.zip Uploading the intermediate file: upload_file_4.zip 21:21:26 STEP 481 H= 120:15 +CPU= 69.343 21:22:37 STEP 482 H= 120:30 +CPU= 70.020 21:23:48 STEP 483 H= 120:45 +CPU= 69.972 21:25:03 STEP 484 H= 121:00 +CPU= 73.773 21:26:11 STEP 485 H= 121:15 +CPU= 67.238 21:27:27 STEP 486 H= 121:30 +CPU= 75.979 21:28:46 STEP 487 H= 121:45 +CPU= 77.985 21:29:55 STEP 488 H= 122:00 +CPU= 68.233 21:31:07 STEP 489 H= 122:15 +CPU= 68.979 21:32:16 STEP 490 H= 122:30 +CPU= 68.097 21:33:22 STEP 491 H= 122:45 +CPU= 64.461 21:34:43 STEP 492 H= 123:00 +CPU= 80.268 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000492 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000492 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000492 21:35:54 STEP 493 H= 123:15 +CPU= 69.977 21:37:01 STEP 494 H= 123:30 +CPU= 66.664 21:38:13 STEP 495 H= 123:45 +CPU= 70.731 21:39:24 STEP 496 H= 124:00 +CPU= 70.090 21:40:34 STEP 497 H= 124:15 +CPU= 68.831 21:41:41 STEP 498 H= 124:30 +CPU= 67.243 21:42:51 STEP 499 H= 124:45 +CPU= 68.710 21:44:05 STEP 500 H= 125:00 +CPU= 73.234 21:45:19 STEP 501 H= 125:15 +CPU= 73.204 21:46:29 STEP 502 H= 125:30 +CPU= 68.878 21:47:36 STEP 503 H= 125:45 +CPU= 66.972 21:49:04 STEP 504 H= 126:00 +CPU= 86.751 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000504 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000504 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000504 21:50:15 STEP 505 H= 126:15 +CPU= 69.998 21:51:28 STEP 506 H= 126:30 +CPU= 72.803 21:52:37 STEP 507 H= 126:45 +CPU= 67.744 21:53:48 STEP 508 H= 127:00 +CPU= 70.515 21:54:56 STEP 509 H= 127:15 +CPU= 66.852 21:56:03 STEP 510 H= 127:30 +CPU= 67.095 21:57:15 STEP 511 H= 127:45 +CPU= 70.832 21:58:28 STEP 512 H= 128:00 +CPU= 71.296 21:59:41 STEP 513 H= 128:15 +CPU= 73.065 22:00:54 STEP 514 H= 128:30 +CPU= 71.899 22:02:06 STEP 515 H= 128:45 +CPU= 70.503 22:03:26 STEP 516 H= 129:00 +CPU= 79.051 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000516 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000516 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000516 22:04:36 STEP 517 H= 129:15 +CPU= 69.101 22:05:45 STEP 518 H= 129:30 +CPU= 68.874 22:06:54 STEP 519 H= 129:45 +CPU= 67.641 22:08:19 STEP 520 H= 130:00 +CPU= 60.436 22:09:09 STEP 521 H= 130:15 +CPU= 48.649 22:10:06 STEP 522 H= 130:30 +CPU= 56.761 22:11:05 STEP 523 H= 130:45 +CPU= 57.030 22:12:07 STEP 524 H= 131:00 +CPU= 61.850 22:13:08 STEP 525 H= 131:15 +CPU= 59.747 22:14:09 STEP 526 H= 131:30 +CPU= 59.337 22:15:19 STEP 527 H= 131:45 +CPU= 67.159 22:16:50 STEP 528 H= 132:00 +CPU= 87.967 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000528 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000528 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000528 22:18:00 STEP 529 H= 132:15 +CPU= 67.077 22:19:13 STEP 530 H= 132:30 +CPU= 69.508 22:20:19 STEP 531 H= 132:45 +CPU= 63.548 22:21:25 STEP 532 H= 133:00 +CPU= 64.338 22:22:35 STEP 533 H= 133:15 +CPU= 67.921 22:23:42 STEP 534 H= 133:30 +CPU= 64.356 22:24:51 STEP 535 H= 133:45 +CPU= 66.752 22:26:01 STEP 536 H= 134:00 +CPU= 67.571 22:27:08 STEP 537 H= 134:15 +CPU= 63.167 22:28:23 STEP 538 H= 134:30 +CPU= 62.565 22:29:35 STEP 539 H= 134:45 +CPU= 67.553 22:30:57 STEP 540 H= 135:00 +CPU= 82.086 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000540 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000540 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000540 22:32:05 STEP 541 H= 135:15 +CPU= 66.994 22:33:16 STEP 542 H= 135:30 +CPU= 69.975 22:34:25 STEP 543 H= 135:45 +CPU= 67.900 22:35:34 STEP 544 H= 136:00 +CPU= 67.972 22:36:41 STEP 545 H= 136:15 +CPU= 66.507 22:37:52 STEP 546 H= 136:30 +CPU= 70.344 22:39:05 STEP 547 H= 136:45 +CPU= 71.690 22:40:13 STEP 548 H= 137:00 +CPU= 66.641 22:41:22 STEP 549 H= 137:15 +CPU= 68.919 22:42:33 STEP 550 H= 137:30 +CPU= 69.943 22:43:41 STEP 551 H= 137:45 +CPU= 66.299 22:45:07 STEP 552 H= 138:00 +CPU= 85.495 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000552 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000552 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000552 22:46:13 STEP 553 H= 138:15 +CPU= 65.508 22:47:26 STEP 554 H= 138:30 +CPU= 71.728 22:48:38 STEP 555 H= 138:45 +CPU= 70.576 22:49:49 STEP 556 H= 139:00 +CPU= 70.034 22:50:59 STEP 557 H= 139:15 +CPU= 68.904 22:52:10 STEP 558 H= 139:30 +CPU= 69.035 22:53:17 STEP 559 H= 139:45 +CPU= 66.748 22:54:30 STEP 560 H= 140:00 +CPU= 71.506 22:55:37 STEP 561 H= 140:15 +CPU= 66.911 22:56:44 STEP 562 H= 140:30 +CPU= 65.834 22:57:49 STEP 563 H= 140:45 +CPU= 64.193 22:59:18 STEP 564 H= 141:00 +CPU= 88.268 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000564 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000564 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000564 23:00:26 STEP 565 H= 141:15 +CPU= 67.430 23:01:35 STEP 566 H= 141:30 +CPU= 68.031 23:02:44 STEP 567 H= 141:45 +CPU= 67.939 23:03:50 STEP 568 H= 142:00 +CPU= 65.538 23:05:03 STEP 569 H= 142:15 +CPU= 71.902 23:06:15 STEP 570 H= 142:30 +CPU= 70.127 23:07:25 STEP 571 H= 142:45 +CPU= 68.660 23:08:40 STEP 572 H= 143:00 +CPU= 74.131 23:09:52 STEP 573 H= 143:15 +CPU= 70.702 23:11:07 STEP 574 H= 143:30 +CPU= 72.605 23:12:20 STEP 575 H= 143:45 +CPU= 70.212 23:14:00 STEP 576 H= 144:00 +CPU= 98.391 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000576 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000576 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000576 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000480 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000480 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000480 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000492 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000492 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000492 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000504 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000504 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000504 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000516 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000516 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000516 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000528 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000528 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000528 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000540 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000540 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000540 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000552 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000552 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000552 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMGGh7zg+000564 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMSHh7zg+000564 Adding to the zip: /home/steve/projects/climateprediction.net/oifs_43r3_bl_12208976/ICMUAh7zg+000564 Zipping up the intermediate file: /home/steve/projects/climateprediction.net/oifs_43r3_bl_a1ot_2016092300_15_991_12208976_0_r669128754_5.zip Uploading the intermediate file: upload_file_5.zip 23:15:16 STEP 577 H= 144:15 +CPU= 73.324 23:16:33 STEP 578 H= 144:30 +CPU= 75.508 23:17:47 STEP 579 H= 144:45 +CPU= 73.231 23:19:01 STEP 580 H= 145:00 +CPU= 73.419 23:20:14 STEP 581 H= 145:15 +CPU= 71.544 23:21:28 STEP 582 H= 145:30 +CPU= 72.549 23:22:44 STEP 583 H= 145:45 +CPU= 75.364 23:23:58 STEP 584 H= 146:00 +CPU= 72.226 23:25:08 STEP 585 H= 146:15 +CPU= 69.405 23:26:24 STEP 586 H= 146:30 +CPU= 74.654 23:27:37 STEP 587 H= 146:45 +CPU= 72.174 23:29:04 STEP 588 H= 147:00 +CPU= 85.998 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000588 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000588 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000588 23:30:19 STEP 589 H= 147:15 +CPU= 73.904 23:31:31 STEP 590 H= 147:30 +CPU= 71.243 23:32:48 STEP 591 H= 147:45 +CPU= 76.697 23:34:02 STEP 592 H= 148:00 +CPU= 72.589 23:35:16 STEP 593 H= 148:15 +CPU= 73.586 23:36:27 STEP 594 H= 148:30 +CPU= 69.731 23:37:42 STEP 595 H= 148:45 +CPU= 73.316 23:38:56 STEP 596 H= 149:00 +CPU= 73.197 23:40:12 STEP 597 H= 149:15 +CPU= 75.543 23:41:28 STEP 598 H= 149:30 +CPU= 74.313 23:42:40 STEP 599 H= 149:45 +CPU= 71.357 23:44:13 STEP 600 H= 150:00 +CPU= 91.500 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000600 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000600 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000600 23:45:27 STEP 601 H= 150:15 +CPU= 71.086 23:46:36 STEP 602 H= 150:30 +CPU= 68.588 23:47:51 STEP 603 H= 150:45 +CPU= 73.203 23:49:03 STEP 604 H= 151:00 +CPU= 71.645 23:50:13 STEP 605 H= 151:15 +CPU= 68.774 23:51:26 STEP 606 H= 151:30 +CPU= 72.207 23:52:28 STEP 607 H= 151:45 +CPU= 61.411 23:53:34 STEP 608 H= 152:00 +CPU= 65.605 23:54:42 STEP 609 H= 152:15 +CPU= 67.300 23:55:51 STEP 610 H= 152:30 +CPU= 67.745 23:57:00 STEP 611 H= 152:45 +CPU= 68.075 23:58:28 STEP 612 H= 153:00 +CPU= 87.153 Moving to projects directory: /home/steve/slots/18/ICMGGh7zg+000612 Moving to projects directory: /home/steve/slots/18/ICMSHh7zg+000612 Moving to projects directory: /home/steve/slots/18/ICMUAh7zg+000612 ..The child process has been killed with signal: 9 >>> Printing last 70 lines from file: NODE.001_01 UPDT RAESHDU = 2000.0 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT) 20160923 630 9.999999999999998E-005 GPNORM HUMIDITY AVERAGE MINIMUM MAXIMUM AVE 0.112942252861804E-02 0.999999999999998E-08 0.106111919444244E-01 GPNORM LIQUID WATER AVERAGE MINIMUM MAXIMUM AVE 0.532926271906845E-06 0.000000000000000E+00 0.235769542003753E-03 GPNORM ICE WATER AVERAGE MINIMUM MAXIMUM AVE 0.549663327914405E-07 0.000000000000000E+00 0.409782644545962E-04 GPNORM SNOW AVERAGE MINIMUM MAXIMUM AVE 0.248638711031289E-07 0.000000000000000E+00 0.941478063433477E-04 GPNORM RAIN AVERAGE MINIMUM MAXIMUM AVE 0.486997462584753E-08 0.000000000000000E+00 0.940165502749611E-05 GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM AVE 0.145973252922963E+00 0.000000000000000E+00 0.100000000000000E+01 GPNORM CLW AVERAGE MINIMUM MAXIMUM AVE 0.533569231861014E-06 0.000000000000000E+00 0.237865717027892E-03 GPNORM CIW AVERAGE MINIMUM MAXIMUM AVE 0.549441750215672E-07 0.000000000000000E+00 0.516641724698275E-04 GPNORM CC AVERAGE MINIMUM MAXIMUM AVE 0.145933977632731E+00 0.000000000000000E+00 0.100000000000000E+01 GPNORM RFL AVERAGE MINIMUM MAXIMUM AVE 0.221093216148668E-07 0.000000000000000E+00 0.449008648426110E-04 GPNORM SFL AVERAGE MINIMUM MAXIMUM AVE 0.240706343219099E-07 0.000000000000000E+00 0.114979320887227E-03 GPNORM CRFL AVERAGE MINIMUM MAXIMUM AVE 0.289671091951212E-06 0.000000000000000E+00 0.200787912621304E-03 GPNORM CSFL AVERAGE MINIMUM MAXIMUM AVE 0.112942220196761E-06 0.000000000000000E+00 0.194465629814909E-03 GPNORM PFRC AVERAGE MINIMUM MAXIMUM AVE 0.340015110248924E-01 0.000000000000000E+00 0.100000000000000E+01 NSTEP = 612 STEPO A00000000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 612 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+000612 MODE=w IO-STREAM CLOSED - ICMGGh7zg+000612 MPI-TASK: 1 - 303292266 BYTES IN 6 RECORDS TRANSFERRED IN 0.0018 SECONDS168495.7 Mbytes/s, TOTAL TIME= 0.0466( 3.9%) NSTEP = 612 STEPO 0AA000000 IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMSHh7zg+000612 MODE=a IO-STREAM CLOSED - ./ICMSHh7zg+000612 MPI-TASK: 1 - 707765250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0297 SECONDS 23830.5 Mbytes/s, TOTAL TIME= 0.7483( 4.0%) IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000612 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000612 MPI-TASK: 1 - 4852676247 BYTES IN 125 RECORDS TRANSFERRED IN 0.0172 SECONDS282132.3 Mbytes/s, TOTAL TIME= 0.4891( 3.5%) STEPO_FPOS WAS : B00000F00E IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000612 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000612 MPI-TASK: 1 - 294718633 BYTES IN 2 RECORDS TRANSFERRED IN 0.0003 SECONDS982395.5 Mbytes/s, TOTAL TIME= 0.0113( 2.7%) STEPO_FPOS WAS : V00000000Y IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000612 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000612 MPI-TASK: 1 - 303292266 BYTES IN 4 RECORDS TRANSFERRED IN 0.0006 SECONDS505487.1 Mbytes/s, TOTAL TIME= 0.0182( 3.3%) STEPO_FPOS WAS : T00000000M NSTEP = 612 STEPO 0AAA00AAA MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 612 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+000612 MODE=a IO-STREAM CLOSED - ICMGGh7zg+000612 MPI-TASK: 1 - 303292257 BYTES IN 7 RECORDS TRANSFERRED IN 0.0007 SECONDS433274.7 Mbytes/s, TOTAL TIME= 0.0183( 3.8%) 23:58:28 STEP 612 H= 153:00 +CPU= 87.153 NSTEP = 613 STEPO 0AAA00AAA ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 23:57:00 A00000000 STEPO 612 68.352 68.352 69.582 808:10 -2752:** 0.28482202874385E-05 2GB 0MB 23:57:00 0AA000000 STEPO 612 0.087 0.087 0.099 808:10 -2752:** 0.28482202874385E-05 2GB 0MB 23:57:06 FULLPOS-B DYNFPOS 612 5.261 5.261 5.334 808:15 -2752:** 0.28482202874385E-05 2GB 0MB 23:57:15 FULLPOS-V DYNFPOS 612 9.473 9.473 9.602 808:24 -2752:** 0.28482202874385E-05 2GB 0MB 23:57:17 FULLPOS-T DYNFPOS 612 1.195 1.195 1.199 808:26 -2752:** 0.28482202874385E-05 2GB 0MB 23:57:17 0AAA00AAA STEPO 612 0.834 0.834 0.836 808:26 -2752:** 0.28482202874385E-05 2GB 0MB 23:58:28 0AAA00AAA STEPO 613 70.029 70.029 70.995 809:36 -2751:** 0.28482202874385E-05 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 576", CTIME = "00060000 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 564, 2*0, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 0.000000000000000E+000, GMASSI = 0.000000000000000E+000 / ------------------------------------------------ >>> Printing last 8 lines from file: /home/steve/slots/18/progress_file_12208976.xml <running_values> <last_cpu_time>48603.050000</last_cpu_time> <upload_file_number>6</upload_file_number> <last_iter>613</last_iter> <last_upload>518400</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully </stderr_txt> ]]> |
No trickles! |
---|
©2024 cpdn.org