climateprediction.net home page
Task 22444884

Task 22444884

Name oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0
Workunit 12291285
Created 13 Jun 2024, 0:19:17 UTC
Sent 14 Jun 2024, 0:14:03 UTC
Report deadline 13 Aug 2024, 0:14:03 UTC
Received 15 Jun 2024, 2:15:26 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1533065
Run time 22 hours 16 min 27 sec
CPU time 22 hours 16 min 27 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.37 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 4,796.21 MB
Peak swap size 5,981.12 MB
Peak disk usage 1,974.08 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001536
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077_15.zip
Uploading the intermediate file: upload_file_15.zip
  23:50:18 STEP 1538 H= 384:30 +CPU= 38.655
  23:51:05 STEP 1539 H= 384:45 +CPU= 46.988
  23:51:54 STEP 1540 H= 385:00 +CPU= 48.315
  23:52:33 STEP 1541 H= 385:15 +CPU= 38.288
  23:53:02 STEP 1542 H= 385:30 +CPU= 29.622
  23:53:23 STEP 1543 H= 385:45 +CPU= 20.560
  23:53:44 STEP 1544 H= 386:00 +CPU= 20.888
  23:54:09 STEP 1545 H= 386:15 +CPU= 24.707
  23:54:43 STEP 1546 H= 386:30 +CPU= 33.845
  23:55:11 STEP 1547 H= 386:45 +CPU= 28.413
  23:55:40 STEP 1548 H= 387:00 +CPU= 29.319
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001548
  23:56:13 STEP 1549 H= 387:15 +CPU= 32.546
  23:56:57 STEP 1550 H= 387:30 +CPU= 43.767
  23:57:50 STEP 1551 H= 387:45 +CPU= 51.911
  23:58:29 STEP 1552 H= 388:00 +CPU= 38.206
  23:58:59 STEP 1553 H= 388:15 +CPU= 29.889
  23:59:20 STEP 1554 H= 388:30 +CPU= 21.028
  23:59:41 STEP 1555 H= 388:45 +CPU= 20.923
  00:00:10 STEP 1556 H= 389:00 +CPU= 29.487
  00:00:49 STEP 1557 H= 389:15 +CPU= 37.758
  00:01:18 STEP 1558 H= 389:30 +CPU= 28.308
  00:01:48 STEP 1559 H= 389:45 +CPU= 30.272
  00:02:26 STEP 1560 H= 390:00 +CPU= 37.976
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001560
  00:02:50 STEP 1561 H= 390:15 +CPU= 23.470
  00:03:33 STEP 1562 H= 390:30 +CPU= 43.320
  00:04:21 STEP 1563 H= 390:45 +CPU= 47.458
  00:05:07 STEP 1564 H= 391:00 +CPU= 45.218
  00:05:47 STEP 1565 H= 391:15 +CPU= 39.747
  00:06:19 STEP 1566 H= 391:30 +CPU= 31.400
  00:07:02 STEP 1567 H= 391:45 +CPU= 42.543
  00:07:50 STEP 1568 H= 392:00 +CPU= 46.639
  00:08:23 STEP 1569 H= 392:15 +CPU= 32.288
  00:08:46 STEP 1570 H= 392:30 +CPU= 23.473
  00:09:10 STEP 1571 H= 392:45 +CPU= 23.249
  00:10:01 STEP 1572 H= 393:00 +CPU= 51.047
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001572
  00:10:51 STEP 1573 H= 393:15 +CPU= 49.585
  00:11:33 STEP 1574 H= 393:30 +CPU= 41.207
  00:11:57 STEP 1575 H= 393:45 +CPU= 23.545
  00:12:20 STEP 1576 H= 394:00 +CPU= 23.564
  00:12:51 STEP 1577 H= 394:15 +CPU= 30.443
  00:13:29 STEP 1578 H= 394:30 +CPU= 37.073
  00:14:03 STEP 1579 H= 394:45 +CPU= 34.306
  00:14:36 STEP 1580 H= 395:00 +CPU= 32.337
  00:15:04 STEP 1581 H= 395:15 +CPU= 28.636
  00:15:35 STEP 1582 H= 395:30 +CPU= 30.383
  00:16:15 STEP 1583 H= 395:45 +CPU= 39.913
  00:17:21 STEP 1584 H= 396:00 +CPU= 63.779
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001584
  00:17:56 STEP 1585 H= 396:15 +CPU= 34.646
  00:18:26 STEP 1586 H= 396:30 +CPU= 29.797
  00:19:12 STEP 1587 H= 396:45 +CPU= 46.053
  00:20:00 STEP 1588 H= 397:00 +CPU= 46.960
  00:20:34 STEP 1589 H= 397:15 +CPU= 34.515
  00:21:09 STEP 1590 H= 397:30 +CPU= 34.281
  00:21:44 STEP 1591 H= 397:45 +CPU= 34.824
  00:22:11 STEP 1592 H= 398:00 +CPU= 27.148
  00:22:42 STEP 1593 H= 398:15 +CPU= 30.817
  00:23:26 STEP 1594 H= 398:30 +CPU= 44.193
  00:24:17 STEP 1595 H= 398:45 +CPU= 50.649
  00:25:05 STEP 1596 H= 399:00 +CPU= 47.163
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001596
  00:25:39 STEP 1597 H= 399:15 +CPU= 34.599
  00:26:20 STEP 1598 H= 399:30 +CPU= 40.686
  00:27:05 STEP 1599 H= 399:45 +CPU= 44.805
  00:28:07 STEP 1600 H= 400:00 +CPU= 50.611
  00:28:54 STEP 1601 H= 400:15 +CPU= 46.035
  00:29:43 STEP 1602 H= 400:30 +CPU= 48.144
  00:30:17 STEP 1603 H= 400:45 +CPU= 34.105
  00:30:51 STEP 1604 H= 401:00 +CPU= 34.343
  00:31:28 STEP 1605 H= 401:15 +CPU= 36.511
  00:32:00 STEP 1606 H= 401:30 +CPU= 32.596
  00:32:34 STEP 1607 H= 401:45 +CPU= 33.538
  00:33:11 STEP 1608 H= 402:00 +CPU= 36.200
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001608
  00:33:53 STEP 1609 H= 402:15 +CPU= 41.696
  00:34:41 STEP 1610 H= 402:30 +CPU= 47.668
  00:35:18 STEP 1611 H= 402:45 +CPU= 36.665
  00:35:47 STEP 1612 H= 403:00 +CPU= 28.987
  00:36:23 STEP 1613 H= 403:15 +CPU= 35.583
  00:36:57 STEP 1614 H= 403:30 +CPU= 34.171
  00:37:43 STEP 1615 H= 403:45 +CPU= 45.834
  00:38:29 STEP 1616 H= 404:00 +CPU= 45.146
  00:39:03 STEP 1617 H= 404:15 +CPU= 34.602
  00:39:42 STEP 1618 H= 404:30 +CPU= 37.986
  00:40:19 STEP 1619 H= 404:45 +CPU= 37.092
  00:41:06 STEP 1620 H= 405:00 +CPU= 45.980
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001620
  00:41:38 STEP 1621 H= 405:15 +CPU= 32.753
  00:42:14 STEP 1622 H= 405:30 +CPU= 35.321
  00:42:49 STEP 1623 H= 405:45 +CPU= 34.324
  00:43:40 STEP 1624 H= 406:00 +CPU= 50.934
  00:44:26 STEP 1625 H= 406:15 +CPU= 45.761
  00:45:13 STEP 1626 H= 406:30 +CPU= 46.149
  00:45:48 STEP 1627 H= 406:45 +CPU= 34.895
  00:46:19 STEP 1628 H= 407:00 +CPU= 30.447
  00:46:44 STEP 1629 H= 407:15 +CPU= 24.761
  00:47:08 STEP 1630 H= 407:30 +CPU= 24.078
  00:47:32 STEP 1631 H= 407:45 +CPU= 23.776
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2a6
(argv4) batchid: 1018
(argv5) wuid: 12291285
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f835385c2128fc9962d8d8dd6c2c3ac7 to: /var/lib/boinc-client/slots/12/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285.zip
ic_ancil_file: ic_ancil_12291285
ifsdata_file: ifsdata_12291285
climate_data_file: clim_data_12291285
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/12/ic_ancil_12291285.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12291285.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_676b4f4e4f7977e75e4c6f0897747a4e to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12291285.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12291285.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12291285.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12291285.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12291285.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12291285.xml
-- Model is restarting --
Adjusting last_iter, 1632, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12291285.xml
last_cpu_time: 70386
upload_file_number: 16
last_iter: 1633
last_upload: 1383300
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077
The child process has been launched with process id: 6955
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:6955:6955] [20240615:004908:1718405348:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077_16.zip
Uploading the intermediate file: upload_file_16.zip
  00:49:54 STEP 1632 H= 408:00 +CPU= 39.447
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001632
  00:50:27 STEP 1633 H= 408:15 +CPU= 32.869
  00:50:57 STEP 1634 H= 408:30 +CPU= 30.590
  00:51:27 STEP 1635 H= 408:45 +CPU= 29.838
  00:52:10 STEP 1636 H= 409:00 +CPU= 42.721
  00:52:33 STEP 1637 H= 409:15 +CPU= 22.960
  00:52:57 STEP 1638 H= 409:30 +CPU= 23.541
  00:53:18 STEP 1639 H= 409:45 +CPU= 21.248
  00:53:52 STEP 1640 H= 410:00 +CPU= 33.861
  00:54:40 STEP 1641 H= 410:15 +CPU= 47.926
  00:55:27 STEP 1642 H= 410:30 +CPU= 45.704
  00:55:54 STEP 1643 H= 410:45 +CPU= 26.810
  00:56:34 STEP 1644 H= 411:00 +CPU= 40.659
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001644
  00:57:08 STEP 1645 H= 411:15 +CPU= 33.302
  00:57:39 STEP 1646 H= 411:30 +CPU= 30.781
  00:58:06 STEP 1647 H= 411:45 +CPU= 27.690
  00:58:44 STEP 1648 H= 412:00 +CPU= 37.113
  00:59:31 STEP 1649 H= 412:15 +CPU= 46.721
  01:00:04 STEP 1650 H= 412:30 +CPU= 32.864
  01:00:28 STEP 1651 H= 412:45 +CPU= 23.628
  01:00:52 STEP 1652 H= 413:00 +CPU= 24.029
  01:01:16 STEP 1653 H= 413:15 +CPU= 24.496
  01:01:40 STEP 1654 H= 413:30 +CPU= 23.435
  01:02:04 STEP 1655 H= 413:45 +CPU= 23.715
  01:02:39 STEP 1656 H= 414:00 +CPU= 35.852
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001656
  01:03:03 STEP 1657 H= 414:15 +CPU= 23.645
  01:03:26 STEP 1658 H= 414:30 +CPU= 23.084
  01:04:04 STEP 1659 H= 414:45 +CPU= 37.099
  01:04:45 STEP 1660 H= 415:00 +CPU= 41.535
  01:05:07 STEP 1661 H= 415:15 +CPU= 21.391
  01:05:48 STEP 1662 H= 415:30 +CPU= 41.307
  01:06:35 STEP 1663 H= 415:45 +CPU= 46.699
  01:07:20 STEP 1664 H= 416:00 +CPU= 44.636
  01:07:54 STEP 1665 H= 416:15 +CPU= 34.330
  01:08:19 STEP 1666 H= 416:30 +CPU= 24.945
  01:08:41 STEP 1667 H= 416:45 +CPU= 21.251
  01:09:06 STEP 1668 H= 417:00 +CPU= 25.247
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001668
  01:09:38 STEP 1669 H= 417:15 +CPU= 31.361
  01:10:10 STEP 1670 H= 417:30 +CPU= 31.814
  01:10:50 STEP 1671 H= 417:45 +CPU= 39.540
  01:11:27 STEP 1672 H= 418:00 +CPU= 36.175
  01:11:56 STEP 1673 H= 418:15 +CPU= 28.813
  01:12:17 STEP 1674 H= 418:30 +CPU= 21.380
  01:12:38 STEP 1675 H= 418:45 +CPU= 21.146
  01:13:00 STEP 1676 H= 419:00 +CPU= 21.649
  01:13:21 STEP 1677 H= 419:15 +CPU= 20.653
  01:13:42 STEP 1678 H= 419:30 +CPU= 21.416
  01:14:13 STEP 1679 H= 419:45 +CPU= 30.498
  01:15:05 STEP 1680 H= 420:00 +CPU= 50.506
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001680
  01:15:40 STEP 1681 H= 420:15 +CPU= 34.369
  01:16:12 STEP 1682 H= 420:30 +CPU= 32.207
  01:16:45 STEP 1683 H= 420:45 +CPU= 32.566
  01:17:09 STEP 1684 H= 421:00 +CPU= 24.260
  01:17:33 STEP 1685 H= 421:15 +CPU= 23.947
  01:18:07 STEP 1686 H= 421:30 +CPU= 33.556
  01:18:39 STEP 1687 H= 421:45 +CPU= 31.615
  01:19:27 STEP 1688 H= 422:00 +CPU= 48.136
  01:20:15 STEP 1689 H= 422:15 +CPU= 48.241
  01:20:52 STEP 1690 H= 422:30 +CPU= 36.001
  01:21:13 STEP 1691 H= 422:45 +CPU= 21.069
  01:21:39 STEP 1692 H= 423:00 +CPU= 25.614
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001692
  01:22:11 STEP 1693 H= 423:15 +CPU= 32.077
  01:22:49 STEP 1694 H= 423:30 +CPU= 37.392
  01:23:22 STEP 1695 H= 423:45 +CPU= 33.213
  01:23:55 STEP 1696 H= 424:00 +CPU= 32.649
  01:24:24 STEP 1697 H= 424:15 +CPU= 29.000
  01:24:49 STEP 1698 H= 424:30 +CPU= 25.080
  01:25:12 STEP 1699 H= 424:45 +CPU= 23.508
  01:25:36 STEP 1700 H= 425:00 +CPU= 24.036
  01:26:06 STEP 1701 H= 425:15 +CPU= 29.663
  01:26:34 STEP 1702 H= 425:30 +CPU= 27.752
  01:27:03 STEP 1703 H= 425:45 +CPU= 28.777
  01:27:37 STEP 1704 H= 426:00 +CPU= 33.813
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001704
  01:28:11 STEP 1705 H= 426:15 +CPU= 33.412
  01:28:34 STEP 1706 H= 426:30 +CPU= 23.401
  01:28:57 STEP 1707 H= 426:45 +CPU= 23.475
  01:29:28 STEP 1708 H= 427:00 +CPU= 30.868
  01:29:56 STEP 1709 H= 427:15 +CPU= 27.708
  01:30:26 STEP 1710 H= 427:30 +CPU= 29.429
  01:31:06 STEP 1711 H= 427:45 +CPU= 39.687
  01:31:35 STEP 1712 H= 428:00 +CPU= 28.931
  01:31:56 STEP 1713 H= 428:15 +CPU= 21.085
  01:32:17 STEP 1714 H= 428:30 +CPU= 20.931
  01:32:40 STEP 1715 H= 428:45 +CPU= 22.673
  01:33:12 STEP 1716 H= 429:00 +CPU= 32.361
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001716
  01:33:47 STEP 1717 H= 429:15 +CPU= 34.348
  01:34:25 STEP 1718 H= 429:30 +CPU= 37.828
  01:34:52 STEP 1719 H= 429:45 +CPU= 26.697
  01:35:22 STEP 1720 H= 430:00 +CPU= 29.574
  01:35:55 STEP 1721 H= 430:15 +CPU= 33.471
  01:36:27 STEP 1722 H= 430:30 +CPU= 31.217
  01:36:54 STEP 1723 H= 430:45 +CPU= 27.140
  01:37:29 STEP 1724 H= 431:00 +CPU= 34.594
  01:38:14 STEP 1725 H= 431:15 +CPU= 45.053
  01:39:01 STEP 1726 H= 431:30 +CPU= 46.780
  01:39:31 STEP 1727 H= 431:45 +CPU= 29.242
  01:40:03 STEP 1728 H= 432:00 +CPU= 30.805
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001728
  01:40:28 STEP 1729 H= 432:15 +CPU= 24.760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001728
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077_17.zip
Uploading the intermediate file: upload_file_17.zip
  01:40:58 STEP 1730 H= 432:30 +CPU= 29.276
  01:41:30 STEP 1731 H= 432:45 +CPU= 32.513
  01:42:18 STEP 1732 H= 433:00 +CPU= 46.960
  01:43:07 STEP 1733 H= 433:15 +CPU= 49.123
  01:43:42 STEP 1734 H= 433:30 +CPU= 34.930
  01:44:12 STEP 1735 H= 433:45 +CPU= 29.258
  01:44:43 STEP 1736 H= 434:00 +CPU= 31.555
  01:45:24 STEP 1737 H= 434:15 +CPU= 40.463
  01:45:53 STEP 1738 H= 434:30 +CPU= 29.089
  01:46:29 STEP 1739 H= 434:45 +CPU= 35.396
  01:47:25 STEP 1740 H= 435:00 +CPU= 56.291
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001740
  01:48:09 STEP 1741 H= 435:15 +CPU= 42.722
  01:48:36 STEP 1742 H= 435:30 +CPU= 27.210
  01:49:05 STEP 1743 H= 435:45 +CPU= 28.666
  01:49:39 STEP 1744 H= 436:00 +CPU= 34.454
  01:50:11 STEP 1745 H= 436:15 +CPU= 31.640
  01:50:39 STEP 1746 H= 436:30 +CPU= 27.850
  01:51:04 STEP 1747 H= 436:45 +CPU= 25.297
  01:51:26 STEP 1748 H= 437:00 +CPU= 22.125
  01:51:47 STEP 1749 H= 437:15 +CPU= 20.830
  01:52:08 STEP 1750 H= 437:30 +CPU= 20.934
  01:52:41 STEP 1751 H= 437:45 +CPU= 32.958
  01:53:37 STEP 1752 H= 438:00 +CPU= 55.168
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001752
  01:54:20 STEP 1753 H= 438:15 +CPU= 42.521
  01:54:47 STEP 1754 H= 438:30 +CPU= 27.350
  01:55:17 STEP 1755 H= 438:45 +CPU= 29.378
  01:56:12 STEP 1756 H= 439:00 +CPU= 42.559
  01:56:57 STEP 1757 H= 439:15 +CPU= 44.608
  01:57:41 STEP 1758 H= 439:30 +CPU= 43.849
  01:58:15 STEP 1759 H= 439:45 +CPU= 34.188
  01:58:51 STEP 1760 H= 440:00 +CPU= 35.386
  01:59:24 STEP 1761 H= 440:15 +CPU= 33.563
  01:59:55 STEP 1762 H= 440:30 +CPU= 31.194
  02:00:28 STEP 1763 H= 440:45 +CPU= 32.020
  02:01:03 STEP 1764 H= 441:00 +CPU= 34.441
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001764
  02:01:44 STEP 1765 H= 441:15 +CPU= 40.699
  02:02:34 STEP 1766 H= 441:30 +CPU= 50.298
  02:03:26 STEP 1767 H= 441:45 +CPU= 50.859
  02:04:08 STEP 1768 H= 442:00 +CPU= 41.141
  02:04:40 STEP 1769 H= 442:15 +CPU= 32.090
  02:05:18 STEP 1770 H= 442:30 +CPU= 38.098
  02:06:04 STEP 1771 H= 442:45 +CPU= 45.682
  02:06:46 STEP 1772 H= 443:00 +CPU= 41.792
  02:07:29 STEP 1773 H= 443:15 +CPU= 42.562
  02:08:13 STEP 1774 H= 443:30 +CPU= 44.170
  02:08:57 STEP 1775 H= 443:45 +CPU= 43.752
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2a6
(argv4) batchid: 1018
(argv5) wuid: 12291285
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f835385c2128fc9962d8d8dd6c2c3ac7 to: /var/lib/boinc-client/slots/12/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285.zip
ic_ancil_file: ic_ancil_12291285
ifsdata_file: ifsdata_12291285
climate_data_file: clim_data_12291285
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/12/ic_ancil_12291285.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12291285.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_676b4f4e4f7977e75e4c6f0897747a4e to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12291285.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12291285.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12291285.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12291285.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12291285.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12291285.xml
-- Model is restarting --
Adjusting last_iter, 1776, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12291285.xml
last_cpu_time: 75138
upload_file_number: 18
last_iter: 1777
last_upload: 1556100
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077
The child process has been launched with process id: 7599
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL01:1:1:7599:7599] [20240615:021011:1718410211:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:11:04 STEP 1776 H= 444:00 +CPU= 46.535
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001776
  02:11:46 STEP 1777 H= 444:15 +CPU= 42.164
  02:12:28 STEP 1778 H= 444:30 +CPU= 41.245
  02:13:08 STEP 1779 H= 444:45 +CPU= 40.239
  02:13:57 STEP 1780 H= 445:00 +CPU= 47.477
  02:14:43 STEP 1781 H= 445:15 +CPU= 45.175
  02:15:29 STEP 1782 H= 445:30 +CPU= 45.260
  02:16:18 STEP 1783 H= 445:45 +CPU= 48.572
  02:17:09 STEP 1784 H= 446:00 +CPU= 49.884
  02:17:59 STEP 1785 H= 446:15 +CPU= 48.340
  02:18:49 STEP 1786 H= 446:30 +CPU= 48.731
  02:19:39 STEP 1787 H= 446:45 +CPU= 50.010
  02:20:36 STEP 1788 H= 447:00 +CPU= 55.239
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001788
  02:21:24 STEP 1789 H= 447:15 +CPU= 47.507
  02:22:12 STEP 1790 H= 447:30 +CPU= 47.131
  02:22:58 STEP 1791 H= 447:45 +CPU= 45.224
  02:23:39 STEP 1792 H= 448:00 +CPU= 40.156
  02:24:29 STEP 1793 H= 448:15 +CPU= 47.870
  02:25:14 STEP 1794 H= 448:30 +CPU= 44.098
  02:25:54 STEP 1795 H= 448:45 +CPU= 39.297
  02:26:35 STEP 1796 H= 449:00 +CPU= 40.622
  02:27:18 STEP 1797 H= 449:15 +CPU= 42.250
  02:28:02 STEP 1798 H= 449:30 +CPU= 41.887
  02:28:48 STEP 1799 H= 449:45 +CPU= 45.397
  02:29:41 STEP 1800 H= 450:00 +CPU= 52.022
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001800
  02:30:23 STEP 1801 H= 450:15 +CPU= 41.320
  02:31:09 STEP 1802 H= 450:30 +CPU= 44.573
  02:31:53 STEP 1803 H= 450:45 +CPU= 41.061
  02:32:35 STEP 1804 H= 451:00 +CPU= 41.297
  02:33:16 STEP 1805 H= 451:15 +CPU= 40.311
  02:33:58 STEP 1806 H= 451:30 +CPU= 41.737
  02:34:41 STEP 1807 H= 451:45 +CPU= 42.303
  02:35:21 STEP 1808 H= 452:00 +CPU= 39.623
  02:35:59 STEP 1809 H= 452:15 +CPU= 37.210
  02:36:37 STEP 1810 H= 452:30 +CPU= 37.356
  02:37:14 STEP 1811 H= 452:45 +CPU= 36.060
  02:38:01 STEP 1812 H= 453:00 +CPU= 46.746
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001812
  02:38:36 STEP 1813 H= 453:15 +CPU= 34.276
  02:39:11 STEP 1814 H= 453:30 +CPU= 34.482
  02:39:46 STEP 1815 H= 453:45 +CPU= 33.886
  02:40:22 STEP 1816 H= 454:00 +CPU= 35.543
  02:40:58 STEP 1817 H= 454:15 +CPU= 34.915
  02:41:33 STEP 1818 H= 454:30 +CPU= 34.045
  02:42:09 STEP 1819 H= 454:45 +CPU= 34.508
  02:42:45 STEP 1820 H= 455:00 +CPU= 36.074
  02:43:20 STEP 1821 H= 455:15 +CPU= 34.830
  02:43:54 STEP 1822 H= 455:30 +CPU= 33.281
  02:44:30 STEP 1823 H= 455:45 +CPU= 34.825
  02:45:17 STEP 1824 H= 456:00 +CPU= 46.371
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001824
  02:45:52 STEP 1825 H= 456:15 +CPU= 34.231
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077_18.zip
Uploading the intermediate file: upload_file_18.zip
  02:46:28 STEP 1826 H= 456:30 +CPU= 33.990
  02:47:03 STEP 1827 H= 456:45 +CPU= 34.333
  02:47:37 STEP 1828 H= 457:00 +CPU= 33.020
  02:48:09 STEP 1829 H= 457:15 +CPU= 31.800
  02:48:41 STEP 1830 H= 457:30 +CPU= 31.788
  02:49:13 STEP 1831 H= 457:45 +CPU= 31.920
  02:49:45 STEP 1832 H= 458:00 +CPU= 32.438
  02:50:18 STEP 1833 H= 458:15 +CPU= 32.257
  02:50:50 STEP 1834 H= 458:30 +CPU= 31.887
  02:51:23 STEP 1835 H= 458:45 +CPU= 31.989
  02:52:02 STEP 1836 H= 459:00 +CPU= 38.015
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001836
  02:52:34 STEP 1837 H= 459:15 +CPU= 31.438
  02:53:07 STEP 1838 H= 459:30 +CPU= 32.982
  02:53:41 STEP 1839 H= 459:45 +CPU= 32.193
  02:54:14 STEP 1840 H= 460:00 +CPU= 32.154
  02:54:47 STEP 1841 H= 460:15 +CPU= 32.351
  02:55:20 STEP 1842 H= 460:30 +CPU= 32.664
  02:55:53 STEP 1843 H= 460:45 +CPU= 32.447
  02:56:26 STEP 1844 H= 461:00 +CPU= 32.394
  02:56:58 STEP 1845 H= 461:15 +CPU= 31.742
  02:57:32 STEP 1846 H= 461:30 +CPU= 32.454
  02:58:04 STEP 1847 H= 461:45 +CPU= 32.096
  02:58:44 STEP 1848 H= 462:00 +CPU= 38.831
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001848
  02:59:17 STEP 1849 H= 462:15 +CPU= 32.071
  02:59:50 STEP 1850 H= 462:30 +CPU= 32.947
  03:00:22 STEP 1851 H= 462:45 +CPU= 31.926
  03:00:55 STEP 1852 H= 463:00 +CPU= 32.303
  03:01:29 STEP 1853 H= 463:15 +CPU= 32.646
  03:02:00 STEP 1854 H= 463:30 +CPU= 31.416
  03:02:32 STEP 1855 H= 463:45 +CPU= 30.981
  03:03:03 STEP 1856 H= 464:00 +CPU= 31.013
  03:03:34 STEP 1857 H= 464:15 +CPU= 30.757
  03:04:06 STEP 1858 H= 464:30 +CPU= 30.770
  03:04:37 STEP 1859 H= 464:45 +CPU= 30.745
  03:05:15 STEP 1860 H= 465:00 +CPU= 36.988
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001860
  03:05:45 STEP 1861 H= 465:15 +CPU= 29.528
  03:06:14 STEP 1862 H= 465:30 +CPU= 29.317
  03:06:44 STEP 1863 H= 465:45 +CPU= 29.404
  03:07:14 STEP 1864 H= 466:00 +CPU= 30.443
  03:07:44 STEP 1865 H= 466:15 +CPU= 29.979
  03:08:14 STEP 1866 H= 466:30 +CPU= 29.362
  03:08:44 STEP 1867 H= 466:45 +CPU= 29.362
  03:09:15 STEP 1868 H= 467:00 +CPU= 29.981
  03:09:45 STEP 1869 H= 467:15 +CPU= 29.766
  03:10:15 STEP 1870 H= 467:30 +CPU= 30.253
  03:10:45 STEP 1871 H= 467:45 +CPU= 28.582
  03:11:22 STEP 1872 H= 468:00 +CPU= 36.887
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001872
  03:11:51 STEP 1873 H= 468:15 +CPU= 27.883
  03:12:20 STEP 1874 H= 468:30 +CPU= 29.678
  03:12:50 STEP 1875 H= 468:45 +CPU= 29.466
  03:13:21 STEP 1876 H= 469:00 +CPU= 30.275
  03:13:51 STEP 1877 H= 469:15 +CPU= 29.731
  03:14:20 STEP 1878 H= 469:30 +CPU= 29.291
  03:14:49 STEP 1879 H= 469:45 +CPU= 28.910
  03:15:19 STEP 1880 H= 470:00 +CPU= 29.582
  03:15:49 STEP 1881 H= 470:15 +CPU= 29.382
  03:16:18 STEP 1882 H= 470:30 +CPU= 29.265
  03:16:49 STEP 1883 H= 470:45 +CPU= 29.396
  03:17:25 STEP 1884 H= 471:00 +CPU= 36.124
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001884
  03:17:55 STEP 1885 H= 471:15 +CPU= 29.395
  03:18:25 STEP 1886 H= 471:30 +CPU= 29.540
  03:18:56 STEP 1887 H= 471:45 +CPU= 30.203
  03:19:26 STEP 1888 H= 472:00 +CPU= 29.694
  03:19:55 STEP 1889 H= 472:15 +CPU= 29.326
  03:20:26 STEP 1890 H= 472:30 +CPU= 29.458
  03:20:55 STEP 1891 H= 472:45 +CPU= 29.508
  03:21:26 STEP 1892 H= 473:00 +CPU= 30.289
  03:21:57 STEP 1893 H= 473:15 +CPU= 29.993
  03:22:27 STEP 1894 H= 473:30 +CPU= 29.379
  03:22:56 STEP 1895 H= 473:45 +CPU= 29.087
  03:23:33 STEP 1896 H= 474:00 +CPU= 36.594
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001896
  03:24:03 STEP 1897 H= 474:15 +CPU= 29.623
  03:24:33 STEP 1898 H= 474:30 +CPU= 29.289
  03:25:03 STEP 1899 H= 474:45 +CPU= 29.387
  03:25:32 STEP 1900 H= 475:00 +CPU= 29.710
  03:26:01 STEP 1901 H= 475:15 +CPU= 29.002
  03:26:32 STEP 1902 H= 475:30 +CPU= 29.783
  03:27:02 STEP 1903 H= 475:45 +CPU= 29.354
  03:27:32 STEP 1904 H= 476:00 +CPU= 30.104
  03:28:03 STEP 1905 H= 476:15 +CPU= 29.802
  03:28:33 STEP 1906 H= 476:30 +CPU= 29.131
  03:29:04 STEP 1907 H= 476:45 +CPU= 30.687
  03:29:42 STEP 1908 H= 477:00 +CPU= 37.727
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001908
  03:30:12 STEP 1909 H= 477:15 +CPU= 29.452
  03:30:43 STEP 1910 H= 477:30 +CPU= 29.650
  03:31:13 STEP 1911 H= 477:45 +CPU= 29.332
  03:31:44 STEP 1912 H= 478:00 +CPU= 29.960
  03:32:13 STEP 1913 H= 478:15 +CPU= 29.411
  03:32:42 STEP 1914 H= 478:30 +CPU= 29.087
  03:33:12 STEP 1915 H= 478:45 +CPU= 29.358
  03:33:43 STEP 1916 H= 479:00 +CPU= 30.008
  03:34:13 STEP 1917 H= 479:15 +CPU= 29.358
  03:34:43 STEP 1918 H= 479:30 +CPU= 29.709
  03:35:13 STEP 1919 H= 479:45 +CPU= 29.487
  03:35:50 STEP 1920 H= 480:00 +CPU= 36.852
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

  ---------------------------------------------------------------------------------------------------------- I ----------------
 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1006498952E-20      I       0.1701378277E-04      I
ICloud fractionI      0.0000000000E+00      I       0.1015575544E+00      I
I  Relat. Hum. I      0.8103834968E-01      I       0.3668523320E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1296385987E+04      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    325886772 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS814717.0 Mbytes/s, TOTAL TIME=    0.0115( 3.5%)
 03:35:50 STEP 1920 H= 480:00 +CPU= 36.852
 END CNT3
 NSTEP =  1920 CNT0   000000000  27.998  27.998  28.598   0.979               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         4290/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       46      1044068     4290 ************     8.7     1.9     6.7     3.0    16.3
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 *********       0.0    100.00   -100.00
   1 CNT4     - FORWARD INTEGRATION                  1 *********       0.0    100.00   -100.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    5200.5    5200.5      0.00      0.00
   9 SPCM     - SPECTRAL COMP.                     144    1055.0    1055.0      0.00      0.00
  10 SCAN2M - PHYSICS                              145   17651.4   17651.4      0.00      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12     780.0     780.0      0.00      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      68.7      68.7      0.00      0.00
  17 GRIDFPOS IN CNT4                               12      31.7      31.7      0.00      0.00
  27 RESTART FILES - WRITING                         3    3119.9    3119.9      0.00      0.00
  28 RESTART FILES - READING                         1    4313.6    4313.6      0.00      0.00
  29 SU4FPOS IN CNT4                                12       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                12    6738.0    6738.0      0.00      0.00
  31 POSDDH IN STEPO                                 1       0.9       0.9      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145 *********       0.0    101.96   -101.96
  39 SU0YOMB                                         1     381.7     381.7      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      37.8      37.8      0.00      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     440.9     440.9      0.00      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144      85.4      85.4      0.00      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316     506.5     506.5      0.00      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     301.8     301.8      0.00      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300     100.9     100.9      0.00      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     169.7     169.7      0.00      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      59.2      59.2      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316      52.2      52.2      0.00      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      30.0      30.0      0.00      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316     109.2     109.2      0.00      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300      32.0      32.0      0.00      0.00
 400 GSTATS                                      17763       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME  -209608.9 CPU TIME    5049.4 VECTOR TIME     5049.4


===-=== END   OF TIMING STATISTICS ===-===
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/12/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/12/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291285/ICMUAh7zg+001884
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0_r1062165077_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
03:39:25 (7596): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 02:16:13 1533065 22444884 oifs_43r3_bl_a2a6_2016092300_20_1018_12291285_0 1,727,100 80,187 0.0464


©2024 cpdn.org