Name | oifs_43r3_ps_0330_1997050100_123_966_12182974_0 |
Workunit | 12182974 |
Created | 22 Dec 2022, 13:19:04 UTC |
Sent | 3 Jan 2023, 19:56:39 UTC |
Report deadline | 2 Feb 2023, 19:56:39 UTC |
Received | 5 Jan 2023, 7:33:20 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1519251 |
Run time | 13 hours 25 min 9 sec |
CPU time | 11 hours 48 min 44 sec |
Validate state | Invalid |
Credit | 2,419.20 |
Device peak FLOPS | 4.94 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,415.61 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,769.62 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> ok.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1187440:1187440] [20230105:111715:1672885035:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 11:17:56 STEP 1320 H=1320:00 +CPU= 36.550 11:18:17 STEP 1321 H=1321:00 +CPU= 20.311 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:19:29 STEP 1322 H=1322:00 +CPU= 21.287 11:20:01 STEP 1323 H=1323:00 +CPU= 31.977 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:21:24 STEP 1324 H=1324:00 +CPU= 21.304 11:21:45 STEP 1325 H=1325:00 +CPU= 21.100 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:22:59 STEP 1326 H=1326:00 +CPU= 33.124 11:23:20 STEP 1327 H=1327:00 +CPU= 20.709 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:24:43 STEP 1328 H=1328:00 +CPU= 21.746 11:25:16 STEP 1329 H=1329:00 +CPU= 32.686 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:26:29 STEP 1330 H=1330:00 +CPU= 20.678 11:26:50 STEP 1331 H=1331:00 +CPU= 21.399 11:27:26 STEP 1332 H=1332:00 +CPU= 35.679 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001332 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001332 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001332 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:28:47 STEP 1333 H=1333:00 +CPU= 20.292 11:29:08 STEP 1334 H=1334:00 +CPU= 21.540 11:29:41 STEP 1335 H=1335:00 +CPU= 32.624 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:31:02 STEP 1336 H=1336:00 +CPU= 20.907 11:31:22 STEP 1337 H=1337:00 +CPU= 20.241 11:31:56 STEP 1338 H=1338:00 +CPU= 33.410 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:33:28 STEP 1339 H=1339:00 +CPU= 20.289 11:33:50 STEP 1340 H=1340:00 +CPU= 21.107 11:34:22 STEP 1341 H=1341:00 +CPU= 32.357 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:35:55 STEP 1342 H=1342:00 +CPU= 22.032 11:36:16 STEP 1343 H=1343:00 +CPU= 20.833 11:36:54 STEP 1344 H=1344:00 +CPU= 37.487 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001344 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001344 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001332 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001332 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001332 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_55.zip Uploading the intermediate file: upload_file_55.zip 11:37:13 STEP 1345 H=1345:00 +CPU= 18.774 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:38:32 STEP 1346 H=1346:00 +CPU= 21.110 11:39:04 STEP 1347 H=1347:00 +CPU= 31.698 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:40:16 STEP 1348 H=1348:00 +CPU= 20.740 11:40:36 STEP 1349 H=1349:00 +CPU= 20.511 11:41:09 STEP 1350 H=1350:00 +CPU= 32.643 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:42:41 STEP 1351 H=1351:00 +CPU= 20.420 11:43:02 STEP 1352 H=1352:00 +CPU= 20.830 11:43:34 STEP 1353 H=1353:00 +CPU= 32.117 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:44:57 STEP 1354 H=1354:00 +CPU= 21.539 11:45:17 STEP 1355 H=1355:00 +CPU= 20.588 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:46:44 STEP 1356 H=1356:00 +CPU= 36.545 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001356 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001356 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001356 11:47:05 STEP 1357 H=1357:00 +CPU= 20.968 11:47:26 STEP 1358 H=1358:00 +CPU= 20.425 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:49:10 STEP 1359 H=1359:00 +CPU= 33.138 11:49:31 STEP 1360 H=1360:00 +CPU= 20.503 11:49:51 STEP 1361 H=1361:00 +CPU= 20.198 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:51:17 STEP 1362 H=1362:00 +CPU= 33.907 11:51:37 STEP 1363 H=1363:00 +CPU= 20.250 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:52:50 STEP 1364 H=1364:00 +CPU= 20.876 11:53:23 STEP 1365 H=1365:00 +CPU= 32.236 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:54:34 STEP 1366 H=1366:00 +CPU= 21.151 11:54:54 STEP 1367 H=1367:00 +CPU= 20.385 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:56:35 STEP 1368 H=1368:00 +CPU= 38.552 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001368 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001368 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001356 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001356 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001356 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_56.zip Uploading the intermediate file: upload_file_56.zip 11:56:57 STEP 1369 H=1369:00 +CPU= 21.001 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:57:59 STEP 1370 H=1370:00 +CPU= 21.277 11:58:32 STEP 1371 H=1371:00 +CPU= 32.550 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:59:54 STEP 1372 H=1372:00 +CPU= 21.239 12:00:14 STEP 1373 H=1373:00 +CPU= 20.060 12:00:48 STEP 1374 H=1374:00 +CPU= 33.140 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:02:20 STEP 1375 H=1375:00 +CPU= 20.545 12:02:41 STEP 1376 H=1376:00 +CPU= 20.748 12:03:14 STEP 1377 H=1377:00 +CPU= 32.378 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:04:16 STEP 1378 H=1378:00 +CPU= 21.555 12:04:37 STEP 1379 H=1379:00 +CPU= 20.157 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:06:15 STEP 1380 H=1380:00 +CPU= 37.078 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001380 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001380 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001380 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:07:26 STEP 1381 H=1381:00 +CPU= 20.450 12:07:48 STEP 1382 H=1382:00 +CPU= 21.387 12:08:21 STEP 1383 H=1383:00 +CPU= 32.468 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:09:42 STEP 1384 H=1384:00 +CPU= 20.774 12:10:03 STEP 1385 H=1385:00 +CPU= 20.380 12:10:37 STEP 1386 H=1386:00 +CPU= 33.452 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:11:58 STEP 1387 H=1387:00 +CPU= 20.661 12:12:19 STEP 1388 H=1388:00 +CPU= 20.985 12:12:52 STEP 1389 H=1389:00 +CPU= 32.103 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:14:15 STEP 1390 H=1390:00 +CPU= 21.791 12:14:36 STEP 1391 H=1391:00 +CPU= 21.257 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:16:16 STEP 1392 H=1392:00 +CPU= 38.770 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001392 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001392 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001380 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001380 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001380 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_57.zip Uploading the intermediate file: upload_file_57.zip 12:16:36 STEP 1393 H=1393:00 +CPU= 20.068 12:16:57 STEP 1394 H=1394:00 +CPU= 20.817 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:18:41 STEP 1395 H=1395:00 +CPU= 32.812 12:19:02 STEP 1396 H=1396:00 +CPU= 21.174 12:19:22 STEP 1397 H=1397:00 +CPU= 19.924 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:20:48 STEP 1398 H=1398:00 +CPU= 33.609 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:21:41 STEP 1399 H=1399:00 +CPU= 20.207 12:22:02 STEP 1400 H=1400:00 +CPU= 21.401 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:23:36 STEP 1401 H=1401:00 +CPU= 32.353 12:23:56 STEP 1402 H=1402:00 +CPU= 20.631 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:25:09 STEP 1403 H=1403:00 +CPU= 20.585 12:25:45 STEP 1404 H=1404:00 +CPU= 35.878 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001404 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001404 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001404 12:26:05 STEP 1405 H=1405:00 +CPU= 19.941 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:27:28 STEP 1406 H=1406:00 +CPU= 21.840 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:28:54 STEP 1407 H=1407:00 +CPU= 33.417 12:29:15 STEP 1408 H=1408:00 +CPU= 21.204 12:29:35 STEP 1409 H=1409:00 +CPU= 20.018 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:30:59 STEP 1410 H=1410:00 +CPU= 33.534 12:31:19 STEP 1411 H=1411:00 +CPU= 20.243 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:32:41 STEP 1412 H=1412:00 +CPU= 20.666 12:33:14 STEP 1413 H=1413:00 +CPU= 33.136 12:33:35 STEP 1414 H=1414:00 +CPU= 20.567 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:34:58 STEP 1415 H=1415:00 +CPU= 20.818 12:35:37 STEP 1416 H=1416:00 +CPU= 39.206 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001416 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001416 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001404 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001404 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001404 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_58.zip Uploading the intermediate file: upload_file_58.zip 12:36:59 STEP 1417 H=1417:00 +CPU= 20.898 12:37:20 STEP 1418 H=1418:00 +CPU= 20.629 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:39:05 STEP 1419 H=1419:00 +CPU= 32.713 12:39:26 STEP 1420 H=1420:00 +CPU= 21.225 12:39:47 STEP 1421 H=1421:00 +CPU= 20.241 12:40:20 STEP 1422 H=1422:00 +CPU= 33.162 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:41:52 STEP 1423 H=1423:00 +CPU= 21.191 12:42:14 STEP 1424 H=1424:00 +CPU= 21.133 12:42:46 STEP 1425 H=1425:00 +CPU= 32.233 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:44:19 STEP 1426 H=1426:00 +CPU= 21.364 12:44:40 STEP 1427 H=1427:00 +CPU= 20.558 12:45:16 STEP 1428 H=1428:00 +CPU= 35.978 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001428 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001428 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001428 12:46:28 STEP 1429 H=1429:00 +CPU= 21.107 12:46:49 STEP 1430 H=1430:00 +CPU= 20.950 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:48:25 STEP 1431 H=1431:00 +CPU= 32.851 12:48:46 STEP 1432 H=1432:00 +CPU= 21.262 12:49:06 STEP 1433 H=1433:00 +CPU= 20.091 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:50:40 STEP 1434 H=1434:00 +CPU= 33.517 12:51:00 STEP 1435 H=1435:00 +CPU= 20.314 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:52:22 STEP 1436 H=1436:00 +CPU= 21.353 12:52:54 STEP 1437 H=1437:00 +CPU= 32.048 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:54:06 STEP 1438 H=1438:00 +CPU= 20.420 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:55:10 STEP 1439 H=1439:00 +CPU= 22.531 12:55:48 STEP 1440 H=1440:00 +CPU= 38.080 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001440 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001440 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001428 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001428 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001428 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_59.zip Uploading the intermediate file: upload_file_59.zip 12:56:09 STEP 1441 H=1441:00 +CPU= 20.459 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:57:21 STEP 1442 H=1442:00 +CPU= 21.204 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:59:46 STEP 1443 H=1443:00 +CPU= 33.506 13:00:13 STEP 1444 H=1444:00 +CPU= 25.565 13:00:37 STEP 1445 H=1445:00 +CPU= 24.253 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:02:46 STEP 1446 H=1446:00 +CPU= 37.638 13:03:09 STEP 1447 H=1447:00 +CPU= 22.546 13:03:31 STEP 1448 H=1448:00 +CPU= 21.891 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:05:36 STEP 1449 H=1449:00 +CPU= 34.566 13:05:59 STEP 1450 H=1450:00 +CPU= 22.639 13:06:22 STEP 1451 H=1451:00 +CPU= 22.482 Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1997050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0330 (argv4) batchid: 966 (argv5) wuid: 12182974 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_987e57a9ceb96035b02b9af7e354e042 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0330_1997050100_123_966_12182974.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0330_1997050100_123_966_12182974.zip ic_ancil_file: ic_ancil_12182974 ifsdata_file: ifsdata_12182974 climate_data_file: clim_data_12182974 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8a2bcbaa7adb2422ec8b59d850d21b5e to: /var/lib/boinc-client/slots/0/ic_ancil_12182974.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12182974.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_d7f9b4096455331a68677c534044aad5 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12182974.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12182974.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12182974.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12182974.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12182974.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12182974.xml -- Model is restarting -- Adjusting last_iter, 1452, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12182974.xml last_cpu_time: 38535 upload_file_number: 60 last_iter: 1439 last_upload: 5184000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787 The child process has been launched with process id: 1207111 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1207111:1207111] [20230105:131924:1672892364:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:21:41 STEP 1440 H=1440:00 +CPU= 40.772 13:22:04 STEP 1441 H=1441:00 +CPU= 22.653 13:22:28 STEP 1442 H=1442:00 +CPU= 23.315 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:24:16 STEP 1443 H=1443:00 +CPU= 35.860 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:27:02 STEP 1444 H=1444:00 +CPU= 24.624 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:39:19 STEP 1445 H=1445:00 +CPU= 23.728 13:39:57 STEP 1446 H=1446:00 +CPU= 37.449 13:40:20 STEP 1447 H=1447:00 +CPU= 21.990 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:43:35 STEP 1448 H=1448:00 +CPU= 23.465 13:44:12 STEP 1449 H=1449:00 +CPU= 36.827 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:49:48 STEP 1450 H=1450:00 +CPU= 23.900 13:50:11 STEP 1451 H=1451:00 +CPU= 22.457 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:53:03 STEP 1452 H=1452:00 +CPU= 39.780 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001452 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001452 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001452 13:53:25 STEP 1453 H=1453:00 +CPU= 22.123 13:53:45 STEP 1454 H=1454:00 +CPU= 20.089 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:56:19 STEP 1455 H=1455:00 +CPU= 33.024 13:56:41 STEP 1456 H=1456:00 +CPU= 21.152 13:57:00 STEP 1457 H=1457:00 +CPU= 19.526 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:58:55 STEP 1458 H=1458:00 +CPU= 33.283 13:59:15 STEP 1459 H=1459:00 +CPU= 20.051 13:59:36 STEP 1460 H=1460:00 +CPU= 20.577 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:00:59 STEP 1461 H=1461:00 +CPU= 32.630 14:01:20 STEP 1462 H=1462:00 +CPU= 20.705 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:02:43 STEP 1463 H=1463:00 +CPU= 20.763 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:04:04 STEP 1464 H=1464:00 +CPU= 38.695 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001464 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001464 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001452 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001452 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001452 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_60.zip Uploading the intermediate file: upload_file_60.zip 14:04:24 STEP 1465 H=1465:00 +CPU= 20.351 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:05:37 STEP 1466 H=1466:00 +CPU= 20.542 14:06:10 STEP 1467 H=1467:00 +CPU= 32.976 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:07:42 STEP 1468 H=1468:00 +CPU= 20.529 14:08:03 STEP 1469 H=1469:00 +CPU= 21.143 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:10:09 STEP 1470 H=1470:00 +CPU= 33.730 14:10:29 STEP 1471 H=1471:00 +CPU= 20.242 14:10:50 STEP 1472 H=1472:00 +CPU= 20.613 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:12:24 STEP 1473 H=1473:00 +CPU= 32.960 14:12:44 STEP 1474 H=1474:00 +CPU= 20.138 14:13:04 STEP 1475 H=1475:00 +CPU= 19.922 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:16:04 STEP 1476 H=1476:00 +CPU= 36.527 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001476 14:16:25 STEP 1477 H=1477:00 +CPU= 20.637 14:16:47 STEP 1478 H=1478:00 +CPU= 21.506 14:17:18 STEP 1479 H=1479:00 +CPU= 31.505 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:19:00 STEP 1480 H=1480:00 +CPU= 20.454 14:19:20 STEP 1481 H=1481:00 +CPU= 20.210 14:19:53 STEP 1482 H=1482:00 +CPU= 32.870 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:21:24 STEP 1483 H=1483:00 +CPU= 20.281 14:21:46 STEP 1484 H=1484:00 +CPU= 21.376 14:22:18 STEP 1485 H=1485:00 +CPU= 32.206 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:23:50 STEP 1486 H=1486:00 +CPU= 21.152 14:24:10 STEP 1487 H=1487:00 +CPU= 20.462 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:25:59 STEP 1488 H=1488:00 +CPU= 37.692 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001488 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001488 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001476 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001476 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001476 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_61.zip Uploading the intermediate file: upload_file_61.zip 14:26:19 STEP 1489 H=1489:00 +CPU= 19.672 14:26:40 STEP 1490 H=1490:00 +CPU= 20.993 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:28:35 STEP 1491 H=1491:00 +CPU= 33.131 14:28:56 STEP 1492 H=1492:00 +CPU= 20.529 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:30:18 STEP 1493 H=1493:00 +CPU= 20.791 14:30:51 STEP 1494 H=1494:00 +CPU= 32.744 14:31:10 STEP 1495 H=1495:00 +CPU= 19.705 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:32:42 STEP 1496 H=1496:00 +CPU= 21.081 14:33:14 STEP 1497 H=1497:00 +CPU= 32.708 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:34:46 STEP 1498 H=1498:00 +CPU= 20.197 14:35:07 STEP 1499 H=1499:00 +CPU= 21.212 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:36:24 STEP 1500 H=1500:00 +CPU= 35.603 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001500 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001500 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001500 14:36:44 STEP 1501 H=1501:00 +CPU= 20.461 14:37:05 STEP 1502 H=1502:00 +CPU= 20.532 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:38:39 STEP 1503 H=1503:00 +CPU= 32.986 14:39:00 STEP 1504 H=1504:00 +CPU= 20.610 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:40:22 STEP 1505 H=1505:00 +CPU= 21.014 14:40:55 STEP 1506 H=1506:00 +CPU= 33.138 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:42:27 STEP 1507 H=1507:00 +CPU= 20.361 14:42:48 STEP 1508 H=1508:00 +CPU= 21.086 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:44:12 STEP 1509 H=1509:00 +CPU= 32.749 14:44:32 STEP 1510 H=1510:00 +CPU= 20.387 14:44:52 STEP 1511 H=1511:00 +CPU= 20.222 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:46:32 STEP 1512 H=1512:00 +CPU= 38.768 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001512 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001512 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001512 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001500 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001500 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001500 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_62.zip Uploading the intermediate file: upload_file_62.zip 14:46:53 STEP 1513 H=1513:00 +CPU= 20.259 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:48:06 STEP 1514 H=1514:00 +CPU= 20.763 14:48:38 STEP 1515 H=1515:00 +CPU= 32.483 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:49:51 STEP 1516 H=1516:00 +CPU= 20.798 14:50:12 STEP 1517 H=1517:00 +CPU= 20.539 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:51:47 STEP 1518 H=1518:00 +CPU= 33.119 14:52:07 STEP 1519 H=1519:00 +CPU= 19.978 14:52:28 STEP 1520 H=1520:00 +CPU= 20.896 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:54:01 STEP 1521 H=1521:00 +CPU= 32.728 14:54:22 STEP 1522 H=1522:00 +CPU= 20.550 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:55:42 STEP 1523 H=1523:00 +CPU= 20.363 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:57:21 STEP 1524 H=1524:00 +CPU= 36.160 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001524 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001524 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001524 14:57:41 STEP 1525 H=1525:00 +CPU= 20.551 14:58:02 STEP 1526 H=1526:00 +CPU= 20.718 Suspend request received from the BOINC client, suspending the child process Resuming the child process 14:59:35 STEP 1527 H=1527:00 +CPU= 32.946 14:59:56 STEP 1528 H=1528:00 +CPU= 20.653 15:00:16 STEP 1529 H=1529:00 +CPU= 19.738 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:02:10 STEP 1530 H=1530:00 +CPU= 32.852 15:02:29 STEP 1531 H=1531:00 +CPU= 19.649 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:03:42 STEP 1532 H=1532:00 +CPU= 21.032 15:04:15 STEP 1533 H=1533:00 +CPU= 33.069 15:04:35 STEP 1534 H=1534:00 +CPU= 20.109 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:06:08 STEP 1535 H=1535:00 +CPU= 21.458 15:06:45 STEP 1536 H=1536:00 +CPU= 37.192 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001536 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001536 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001512 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001512 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001512 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001524 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001524 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001524 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_63.zip Uploading the intermediate file: upload_file_63.zip 15:07:04 STEP 1537 H=1537:00 +CPU= 18.862 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:08:16 STEP 1538 H=1538:00 +CPU= 19.111 15:08:48 STEP 1539 H=1539:00 +CPU= 32.267 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:10:09 STEP 1540 H=1540:00 +CPU= 20.360 15:10:30 STEP 1541 H=1541:00 +CPU= 21.134 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:12:05 STEP 1542 H=1542:00 +CPU= 33.587 15:12:26 STEP 1543 H=1543:00 +CPU= 21.008 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:13:49 STEP 1544 H=1544:00 +CPU= 20.998 15:14:22 STEP 1545 H=1545:00 +CPU= 32.287 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:15:33 STEP 1546 H=1546:00 +CPU= 20.922 15:15:53 STEP 1547 H=1547:00 +CPU= 20.498 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:18:23 STEP 1548 H=1548:00 +CPU= 36.592 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001548 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001548 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001548 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:19:25 STEP 1549 H=1549:00 +CPU= 20.435 15:19:46 STEP 1550 H=1550:00 +CPU= 20.970 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1997050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0330 (argv4) batchid: 966 (argv5) wuid: 12182974 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_987e57a9ceb96035b02b9af7e354e042 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0330_1997050100_123_966_12182974.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0330_1997050100_123_966_12182974.zip ic_ancil_file: ic_ancil_12182974 ifsdata_file: ifsdata_12182974 climate_data_file: clim_data_12182974 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8a2bcbaa7adb2422ec8b59d850d21b5e to: /var/lib/boinc-client/slots/0/ic_ancil_12182974.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12182974.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_d7f9b4096455331a68677c534044aad5 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12182974.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12182974.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12182974.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12182974.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12182974.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12182974.xml -- Model is restarting -- Adjusting last_iter, 1550, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12182974.xml last_cpu_time: 41352 upload_file_number: 64 last_iter: 1535 last_upload: 5529600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787 The child process has been launched with process id: 1225350 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:1225350:1225350] [20230105:152101:1672899661:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 15:21:40 STEP 1536 H=1536:00 +CPU= 36.301 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:23:10 STEP 1537 H=1537:00 +CPU= 19.596 15:23:32 STEP 1538 H=1538:00 +CPU= 21.815 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:25:25 STEP 1539 H=1539:00 +CPU= 31.954 15:25:47 STEP 1540 H=1540:00 +CPU= 21.805 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:27:09 STEP 1541 H=1541:00 +CPU= 20.538 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:28:44 STEP 1542 H=1542:00 +CPU= 33.244 15:29:05 STEP 1543 H=1543:00 +CPU= 20.644 15:29:26 STEP 1544 H=1544:00 +CPU= 20.877 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:31:09 STEP 1545 H=1545:00 +CPU= 32.177 15:31:30 STEP 1546 H=1546:00 +CPU= 21.224 15:31:50 STEP 1547 H=1547:00 +CPU= 19.780 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:33:27 STEP 1548 H=1548:00 +CPU= 35.325 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001548 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001548 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001548 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:34:50 STEP 1549 H=1549:00 +CPU= 21.028 15:35:11 STEP 1550 H=1550:00 +CPU= 21.061 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:36:54 STEP 1551 H=1551:00 +CPU= 31.891 15:37:15 STEP 1552 H=1552:00 +CPU= 20.942 15:37:36 STEP 1553 H=1553:00 +CPU= 20.445 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:39:00 STEP 1554 H=1554:00 +CPU= 33.284 15:39:20 STEP 1555 H=1555:00 +CPU= 19.808 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:40:40 STEP 1556 H=1556:00 +CPU= 20.322 15:41:13 STEP 1557 H=1557:00 +CPU= 32.067 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:42:54 STEP 1558 H=1558:00 +CPU= 20.561 15:43:15 STEP 1559 H=1559:00 +CPU= 20.965 15:43:53 STEP 1560 H=1560:00 +CPU= 37.547 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001560 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001560 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMGGhq0f+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMSHhq0f+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182974/ICMUAhq0f+001548 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0330_1997050100_123_966_12182974_0_r1282578787_64.zip Uploading the intermediate file: upload_file_64.zip 15:44:13 STEP 1561 H=1561:00 +CPU= 19.492 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:45:37 STEP 1562 H=1562:00 +CPU= 19.744 15:46:09 STEP 1563 H=1563:00 +CPU= 31.620 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:47:51 STEP 1564 H=1564:00 +CPU= 20.668 15:48:11 STEP 1565 H=1565:00 +CPU= 20.192 15:48:45 STEP 1566 H=1566:00 +CPU= 33.148 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:50:17 STEP 1567 H=1567:00 +CPU= 21.210 15:50:38 STEP 1568 H=1568:00 +CPU= 20.647 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:52:32 STEP 1569 H=1569:00 +CPU= 32.083 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:53:45 STEP 1570 H=1570:00 +CPU= 20.755 15:54:07 STEP 1571 H=1571:00 +CPU= 21.568 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:55:43 STEP 1572 H=1572:00 +CPU= 36.057 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001572 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001572 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001572 15:56:03 STEP 1573 H=1573:00 +CPU= 20.136 15:56:24 STEP 1574 H=1574:00 +CPU= 20.844 Suspend request received from the BOINC client, suspending the child process Resuming the child process 15:58:17 STEP 1575 H=1575:00 +CPU= 32.504 15:58:38 STEP 1576 H=1576:00 +CPU= 20.989 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:00:21 STEP 1577 H=1577:00 +CPU= 21.037 16:00:53 STEP 1578 H=1578:00 +CPU= 32.479 16:01:13 STEP 1579 H=1579:00 +CPU= 19.733 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:02:36 STEP 1580 H=1580:00 +CPU= 21.580 16:03:08 STEP 1581 H=1581:00 +CPU= 31.646 Suspend request received from the BOINC client, suspending the child process Resuming the child process Abort request received from BOINC client, ending the child process </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:07:55 | 1519251 | 22268122 | oifs_43r3_ps_0330_1997050100_123_966_12182974_0 | 1,728,000 | 11,893 | 0.0069 |
09 May 2023 16:07:55 | 1519251 | 22268122 | oifs_43r3_ps_0330_1997050100_123_966_12182974_0 | 1,728,000 | 11,893 | 0.0069 |
09 May 2023 16:07:49 | 1519251 | 22268122 | oifs_43r3_ps_0330_1997050100_123_966_12182974_0 | 864,000 | 6,110 | 0.0071 |
©2024 climateprediction.net