Name | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 |
Workunit | 12176662 |
Created | 21 Dec 2022, 20:42:16 UTC |
Sent | 21 Dec 2022, 20:48:34 UTC |
Report deadline | 20 Jan 2023, 20:48:34 UTC |
Received | 26 Dec 2022, 16:18:05 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 148 (0x00000094) Unknown error code |
Computer ID | 1534746 |
Run time | 14 hours 30 min 7 sec |
CPU time | 14 hours 30 min 7 sec |
Validate state | Invalid |
Credit | 12,096.00 |
Device peak FLOPS | 5.61 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,795.16 MB |
Peak swap size | 4,974.32 MB |
Peak disk usage | 1,265.37 MB |
Stderr | <core_client_version>7.20.5</core_client_version> <![CDATA[ <message> process exited with code 148 (0x94, -108)</message> <stderr_txt> _ps_12176662/ICMGGhq0f+002424 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002424 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002424 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002436 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002436 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002436 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422_101.zip Uploading the intermediate file: upload_file_101.zip 17:21:29 STEP 2449 H=2449:00 +CPU= 17.959 17:21:46 STEP 2450 H=2450:00 +CPU= 17.456 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:22:18 STEP 2451 H=2451:00 +CPU= 28.850 17:22:37 STEP 2452 H=2452:00 +CPU= 18.705 17:22:55 STEP 2453 H=2453:00 +CPU= 17.205 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:23:26 STEP 2454 H=2454:00 +CPU= 29.118 17:23:43 STEP 2455 H=2455:00 +CPU= 16.782 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:24:04 STEP 2456 H=2456:00 +CPU= 19.227 17:24:33 STEP 2457 H=2457:00 +CPU= 28.482 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:24:53 STEP 2458 H=2458:00 +CPU= 18.151 17:25:13 STEP 2459 H=2459:00 +CPU= 18.438 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:25:47 STEP 2460 H=2460:00 +CPU= 32.421 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002460 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002460 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002460 17:26:04 STEP 2461 H=2461:00 +CPU= 17.032 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:26:24 STEP 2462 H=2462:00 +CPU= 17.013 17:26:53 STEP 2463 H=2463:00 +CPU= 28.758 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:27:14 STEP 2464 H=2464:00 +CPU= 19.321 17:27:33 STEP 2465 H=2465:00 +CPU= 18.675 17:28:04 STEP 2466 H=2466:00 +CPU= 29.756 17:28:22 STEP 2467 H=2467:00 +CPU= 17.581 17:28:41 STEP 2468 H=2468:00 +CPU= 18.799 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:29:14 STEP 2469 H=2469:00 +CPU= 29.203 17:29:33 STEP 2470 H=2470:00 +CPU= 19.043 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:29:57 STEP 2471 H=2471:00 +CPU= 21.041 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:30:33 STEP 2472 H=2472:00 +CPU= 33.857 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002472 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002472 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002472 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002448 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002448 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002448 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002460 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002460 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002460 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422_102.zip Uploading the intermediate file: upload_file_102.zip 17:30:51 STEP 2473 H=2473:00 +CPU= 17.157 17:31:10 STEP 2474 H=2474:00 +CPU= 18.867 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:31:42 STEP 2475 H=2475:00 +CPU= 30.203 17:32:01 STEP 2476 H=2476:00 +CPU= 18.663 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:32:22 STEP 2477 H=2477:00 +CPU= 18.794 17:32:53 STEP 2478 H=2478:00 +CPU= 30.466 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:33:15 STEP 2479 H=2479:00 +CPU= 20.413 17:33:35 STEP 2480 H=2480:00 +CPU= 19.840 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:34:07 STEP 2481 H=2481:00 +CPU= 29.608 17:34:26 STEP 2482 H=2482:00 +CPU= 18.606 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:34:48 STEP 2483 H=2483:00 +CPU= 19.471 17:35:21 STEP 2484 H=2484:00 +CPU= 33.275 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002484 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002484 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002484 17:35:42 STEP 2485 H=2485:00 +CPU= 19.286 17:36:02 STEP 2486 H=2486:00 +CPU= 20.145 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:36:36 STEP 2487 H=2487:00 +CPU= 31.178 17:36:55 STEP 2488 H=2488:00 +CPU= 18.648 17:37:14 STEP 2489 H=2489:00 +CPU= 19.221 17:37:49 STEP 2490 H=2490:00 +CPU= 30.488 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:38:11 STEP 2491 H=2491:00 +CPU= 20.282 17:38:32 STEP 2492 H=2492:00 +CPU= 20.701 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:39:06 STEP 2493 H=2493:00 +CPU= 31.596 17:39:26 STEP 2494 H=2494:00 +CPU= 19.859 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:39:49 STEP 2495 H=2495:00 +CPU= 20.829 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:40:28 STEP 2496 H=2496:00 +CPU= 35.840 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002496 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002496 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002496 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002472 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002472 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002472 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002484 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002484 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002484 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422_103.zip Uploading the intermediate file: upload_file_103.zip 17:40:47 STEP 2497 H=2497:00 +CPU= 18.551 17:41:06 STEP 2498 H=2498:00 +CPU= 18.921 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:41:40 STEP 2499 H=2499:00 +CPU= 32.116 17:42:00 STEP 2500 H=2500:00 +CPU= 19.848 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:42:22 STEP 2501 H=2501:00 +CPU= 19.188 17:42:52 STEP 2502 H=2502:00 +CPU= 29.717 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:43:14 STEP 2503 H=2503:00 +CPU= 20.446 17:43:35 STEP 2504 H=2504:00 +CPU= 20.005 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:44:06 STEP 2505 H=2505:00 +CPU= 29.796 17:44:26 STEP 2506 H=2506:00 +CPU= 18.958 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:44:47 STEP 2507 H=2507:00 +CPU= 19.501 17:45:20 STEP 2508 H=2508:00 +CPU= 31.901 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002508 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002508 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002508 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:45:41 STEP 2509 H=2509:00 +CPU= 19.387 17:46:00 STEP 2510 H=2510:00 +CPU= 18.938 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:46:33 STEP 2511 H=2511:00 +CPU= 31.259 17:46:53 STEP 2512 H=2512:00 +CPU= 19.772 17:47:12 STEP 2513 H=2513:00 +CPU= 18.655 17:47:42 STEP 2514 H=2514:00 +CPU= 30.347 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:48:03 STEP 2515 H=2515:00 +CPU= 18.898 17:48:22 STEP 2516 H=2516:00 +CPU= 18.947 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:48:56 STEP 2517 H=2517:00 +CPU= 31.439 17:49:14 STEP 2518 H=2518:00 +CPU= 18.537 17:49:33 STEP 2519 H=2519:00 +CPU= 18.535 17:50:09 STEP 2520 H=2520:00 +CPU= 35.955 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002520 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002520 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002520 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002496 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002496 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002496 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002508 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002508 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002508 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422_104.zip Uploading the intermediate file: upload_file_104.zip 17:50:29 STEP 2521 H=2521:00 +CPU= 19.740 17:50:48 STEP 2522 H=2522:00 +CPU= 19.792 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:51:22 STEP 2523 H=2523:00 +CPU= 31.583 17:51:43 STEP 2524 H=2524:00 +CPU= 20.871 17:52:02 STEP 2525 H=2525:00 +CPU= 19.030 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:52:34 STEP 2526 H=2526:00 +CPU= 30.360 17:52:53 STEP 2527 H=2527:00 +CPU= 18.391 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:53:15 STEP 2528 H=2528:00 +CPU= 19.832 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:53:48 STEP 2529 H=2529:00 +CPU= 31.390 17:54:08 STEP 2530 H=2530:00 +CPU= 20.290 17:54:27 STEP 2531 H=2531:00 +CPU= 18.364 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:55:02 STEP 2532 H=2532:00 +CPU= 32.390 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002532 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002532 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002532 17:55:21 STEP 2533 H=2533:00 +CPU= 18.939 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:55:42 STEP 2534 H=2534:00 +CPU= 18.668 17:56:10 STEP 2535 H=2535:00 +CPU= 28.375 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:56:34 STEP 2536 H=2536:00 +CPU= 20.440 17:56:53 STEP 2537 H=2537:00 +CPU= 19.716 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:57:27 STEP 2538 H=2538:00 +CPU= 31.347 17:57:46 STEP 2539 H=2539:00 +CPU= 18.769 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:58:08 STEP 2540 H=2540:00 +CPU= 19.825 17:58:38 STEP 2541 H=2541:00 +CPU= 30.702 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:59:00 STEP 2542 H=2542:00 +CPU= 20.155 17:59:19 STEP 2543 H=2543:00 +CPU= 18.518 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:59:57 STEP 2544 H=2544:00 +CPU= 35.025 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002544 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002544 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002544 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002520 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002520 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002520 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMGGhq0f+002532 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMSHhq0f+002532 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662/ICMUAhq0f+002532 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422_105.zip Uploading the intermediate file: upload_file_105.zip 18:00:15 STEP 2545 H=2545:00 +CPU= 18.763 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:00:37 STEP 2546 H=2546:00 +CPU= 19.729 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:01:21 STEP 2547 H=2547:00 +CPU= 30.311 18:01:42 STEP 2548 H=2548:00 +CPU= 20.904 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:07:19 STEP 2549 H=2549:00 +CPU= 22.816 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:07:59 STEP 2550 H=2550:00 +CPU= 33.353 18:08:19 STEP 2551 H=2551:00 +CPU= 19.606 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:08:41 STEP 2552 H=2552:00 +CPU= 19.492 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:09:13 STEP 2553 H=2553:00 +CPU= 30.183 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:13:03 STEP 2554 H=2554:00 +CPU= 23.702 18:13:22 STEP 2555 H=2555:00 +CPU= 18.417 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:13:58 STEP 2556 H=2556:00 +CPU= 33.863 Moving to projects directory: /var/lib/boinc/slots/19/ICMGGhq0f+002556 Moving to projects directory: /var/lib/boinc/slots/19/ICMSHhq0f+002556 Moving to projects directory: /var/lib/boinc/slots/19/ICMUAhq0f+002556 Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:14:21 STEP 2557 H=2557:00 +CPU= 19.825 (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_d087200953d9223950268b726b59e7f1 to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/19/ic_ancil_12176662.zip 18:46:28 STEP 2558 H=2558:00 +CPU= 29.603 ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8d4d53f4eb835dbb5d4ebb1da099633e to: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip Unzipping the ifsdata_zip file: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Unzipping the climate data zip file: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Checking for progress XML file: /var/lib/boinc/slots/19/progress_file_12176662.xml Opened progress file ok : /var/lib/boinc/slots/19/progress_file_12176662.xml -- Model is restarting -- Adjusting last_iter, 2557, to previous model restart step. Creating progress file: /var/lib/boinc/slots/19/progress_file_12176662.xml last_cpu_time: 48947 upload_file_number: 106 last_iter: 2543 last_upload: 9158400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422 The child process has been launched with process id: 14902 Executing the command: /var/lib/boinc/slots/19/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.034] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:14902:14902] [20221222:184656:1671709616:0.043] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 18:47:08 STEP 2559 H=2559:00 +CPU= 33.784 Suspend request received from the BOINC client, suspending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_d087200953d9223950268b726b59e7f1 to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/19/ic_ancil_12176662.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8d4d53f4eb835dbb5d4ebb1da099633e to: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip Unzipping the ifsdata_zip file: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Unzipping the climate data zip file: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Checking for progress XML file: /var/lib/boinc/slots/19/progress_file_12176662.xml Opened progress file ok : /var/lib/boinc/slots/19/progress_file_12176662.xml -- Model is restarting -- Adjusting last_iter, 2560, to previous model restart step. Creating progress file: /var/lib/boinc/slots/19/progress_file_12176662.xml last_cpu_time: 50389 upload_file_number: 106 last_iter: 2543 last_upload: 9158400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422 The child process has been launched with process id: 20027 Executing the command: /var/lib/boinc/slots/19/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.103] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:20027:20027] [20221225:142436:1671953076:0.145] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_d087200953d9223950268b726b59e7f1 to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/19/ic_ancil_12176662.zip 05:15:00 STEP 2544 H=2544:00 +CPU=547.790 ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8d4d53f4eb835dbb5d4ebb1da099633e to: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip Unzipping the ifsdata_zip file: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Unzipping the climate data zip file: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Checking for progress XML file: /var/lib/boinc/slots/19/progress_file_12176662.xml Opened progress file ok : /var/lib/boinc/slots/19/progress_file_12176662.xml -- Model is restarting -- Adjusting last_iter, -999, to previous model restart step. Creating progress file: /var/lib/boinc/slots/19/progress_file_12176662.xml last_cpu_time: 51838 upload_file_number: 106 last_iter: -985 last_upload: 9158400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422 The child process has been launched with process id: 21892 Executing the command: /var/lib/boinc/slots/19/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.008] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.008] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.008] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.008] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.012] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.016] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.016] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.016] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.016] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.016] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:21892:21892] [20221226:051529:1672006529:0.016] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 05:15:29 STEP 2545 H=2545:00 +CPU= 24.915 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip 07:33:13 STEP 2546 H=2546:00 +CPU=797.220 Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_d087200953d9223950268b726b59e7f1 to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/19/ic_ancil_12176662.zip 07:36:05 STEP 2547 H=2547:00 +CPU= 49.191 ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8d4d53f4eb835dbb5d4ebb1da099633e to: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip Unzipping the ifsdata_zip file: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip 20:12:28 STEP 2548 H=2548:00 +CPU=600.435 Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_d087200953d9223950268b726b59e7f1 to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/19/ic_ancil_12176662.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8d4d53f4eb835dbb5d4ebb1da099633e to: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip Unzipping the ifsdata_zip file: /var/lib/boinc/slots/19/ifsdata/ifsdata_12176662.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Unzipping the climate data zip file: /var/lib/boinc/slots/19/159l_2/clim_data_12176662.zip Checking for progress XML file: /var/lib/boinc/slots/19/progress_file_12176662.xml Opened progress file ok : /var/lib/boinc/slots/19/progress_file_12176662.xml -- Model is restarting -- Adjusting last_iter, 2544, to previous model restart step. Creating progress file: /var/lib/boinc/slots/19/progress_file_12176662.xml last_cpu_time: 52136 upload_file_number: 106 last_iter: 2543 last_upload: 9158400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0018_1991050100_123_960_12176662_0_r1818012422 The child process has been launched with process id: 23079 Executing the command: /var/lib/boinc/slots/19/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.038] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:vm9:1:1:23079:23079] [20221226:201257:1672060377:0.039] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 20:12:58 STEP 2549 H=2549:00 +CPU= 25.263 Suspend request received from the BOINC client, suspending the child process Resuming the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_d087200953d9223950268b726b59e7f1 to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/19/ic_ancil_12176662.zip 22:03:52 STEP 2550 H=2550:00 +CPU=292.260 (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1991050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0018 (argv4) batchid: 960 (argv5) wuid: 12176662 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/19 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12176662 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_13c6ad5f83f02eafb87924fb75d7b434 to: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip Unzipping the namelist zip file: /var/lib/boinc/slots/19/oifs_43r3_ps_0018_1991050100_123_960_12176662.zip ic_ancil_file: ic_ancil_12176662 ifsdata_file: ifsdata_12176662 climate_data_file: clim_data_12176662 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: to: /var/lib/boinc/slots/19/ic_ancil_12176662.zip ..Copying the IC ancils to the working directory failed 22:15:10 STEP 2551 H=2551:00 +CPU=166.449 </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 15:47:49 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 8,640,000 | 45,406 | 0.0053 |
09 May 2023 15:47:39 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 7,776,000 | 40,357 | 0.0052 |
09 May 2023 15:47:33 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 6,912,000 | 35,619 | 0.0052 |
09 May 2023 15:47:22 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 6,048,000 | 31,007 | 0.0051 |
09 May 2023 15:47:16 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 5,184,000 | 26,451 | 0.0051 |
09 May 2023 15:47:11 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 4,320,000 | 21,878 | 0.0051 |
09 May 2023 15:47:04 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 3,456,000 | 17,275 | 0.0050 |
09 May 2023 15:46:54 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 2,592,000 | 12,878 | 0.0050 |
09 May 2023 15:46:48 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 1,728,000 | 8,579 | 0.0050 |
09 May 2023 15:46:43 | 1534746 | 22261421 | oifs_43r3_ps_0018_1991050100_123_960_12176662_0 | 864,000 | 4,298 | 0.0050 |
©2024 cpdn.org