climateprediction.net (CPDN) home page
Thread 'Connection and Download issues Oct24'

Thread 'Connection and Download issues Oct24'

Message boards : Number crunching : Connection and Download issues Oct24
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 71699 - Posted: 29 Oct 2024, 21:10:50 UTC - in response to Message 71695.  

Boinc trying to connect to cpdn gives "request failed HTTP internal server error"
ID: 71699 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 71700 - Posted: 29 Oct 2024, 21:44:38 UTC - in response to Message 71699.  

Boinc trying to connect to cpdn gives "request failed HTTP internal server error"
I am just getting the normal "No tasks available" message.
ID: 71700 · Report as offensive     Reply Quote
ProfileBill F

Send message
Joined: 17 Jan 09
Posts: 124
Credit: 2,053,559
RAC: 2,682
Message 71701 - Posted: 29 Oct 2024, 22:03:25 UTC - in response to Message 71700.  

Boinc trying to connect to cpdn gives "request failed HTTP internal server error"
I am just getting the normal "No tasks available" message.


Same "no Tasks available"
ID: 71701 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 71702 - Posted: 30 Oct 2024, 0:09:59 UTC - in response to Message 71700.  

Weird. All my PCs give the same message, both Linux and Windows. No problems connecting to and requesting work from the development site (although none available of course).

As little as 10 days ago, there were no errors when requesting work from cpdn.
ID: 71702 · Report as offensive     Reply Quote
[AF] Kalianthys

Send message
Joined: 20 Dec 20
Posts: 13
Credit: 40,069,885
RAC: 6,860
Message 71703 - Posted: 30 Oct 2024, 6:39:01 UTC - in response to Message 71699.  
Last modified: 30 Oct 2024, 6:39:27 UTC

Boinc trying to connect to cpdn gives "request failed HTTP internal server error"


I have the same error when validating completed tasks.
No credit validated for 4 days.

Kali.
ID: 71703 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 71704 - Posted: 30 Oct 2024, 7:42:07 UTC - in response to Message 71702.  
Last modified: 30 Oct 2024, 7:48:21 UTC

Weird. All my PCs give the same message, both Linux and Windows. No problems connecting to and requesting work from the development site (although none available of course).

As little as 10 days ago, there were no errors when requesting work from cpdn.
Just fired up the VM. I am getting the normal "No tasks available" message on Windows as well as on Linux.
Edit:Boinc8.0.4 Windows, 8.1.0 Linux.
ID: 71704 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,575,067
RAC: 15,735
Message 71705 - Posted: 30 Oct 2024, 9:24:09 UTC

hmm... this thread seems to have drifted off the 'new work announcements' thread.

There have been significant internet outage recently in the UK in multiple regions (including Oxford). It's ploughing season... there's probably info online.

As soon as Sarah finds the XML I'm sending the NZ25 batch out.
---
CPDN Visiting Scientist
ID: 71705 · Report as offensive     Reply Quote
TLD

Send message
Joined: 11 Dec 05
Posts: 14
Credit: 2,252,550
RAC: 6,681
Message 71722 - Posted: 30 Oct 2024, 23:35:18 UTC

I'm getting a download failed with these WUs. Windows 11 Boinc 8.0.2

15714 climateprediction.net 10/30/2024 3:41:53 PM Scheduler request completed: got 4 new tasks
15715 climateprediction.net 10/30/2024 3:41:53 PM Project requested delay of 3636 seconds
15716 climateprediction.net 10/30/2024 3:41:55 PM Started download of wah2_8.24_windows_intelx86.exe
15717 climateprediction.net 10/30/2024 3:41:55 PM Started download of wah2_data_8.24_windows_intelx86.zip
15718 climateprediction.net 10/30/2024 3:41:57 PM Giving up on download of wah2_8.24_windows_intelx86.exe: permanent HTTP error
15719 climateprediction.net 10/30/2024 3:41:57 PM Giving up on download of wah2_data_8.24_windows_intelx86.zip: permanent HTTP error
15720 climateprediction.net 10/30/2024 3:41:57 PM Started download of wah2_se_8.24_windows_intelx86.zip
15721 climateprediction.net 10/30/2024 3:41:57 PM Started download of wah2am3m2_um_8.24_windows_intelx86.zip
15722 climateprediction.net 10/30/2024 3:41:58 PM Giving up on download of wah2_se_8.24_windows_intelx86.zip: permanent HTTP error
15723 climateprediction.net 10/30/2024 3:41:58 PM Giving up on download of wah2am3m2_um_8.24_windows_intelx86.zip: permanent HTTP error
15724 climateprediction.net 10/30/2024 3:41:58 PM Started download of wah2rm3m2t_um_8.24_windows_intelx86.zip
15725 climateprediction.net 10/30/2024 3:41:58 PM Started download of wah2_nz25_10er_209105_25_1028_012344458.zip
15726 climateprediction.net 10/30/2024 3:41:59 PM Giving up on download of wah2rm3m2t_um_8.24_windows_intelx86.zip: permanent HTTP error
ID: 71722 · Report as offensive     Reply Quote
ProfilePDW

Send message
Joined: 29 Nov 17
Posts: 82
Credit: 15,207,444
RAC: 81,474
Message 71723 - Posted: 30 Oct 2024, 23:49:33 UTC - in response to Message 71722.  

Same here.
ID: 71723 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,575,067
RAC: 15,735
Message 71724 - Posted: 31 Oct 2024, 9:06:24 UTC - in response to Message 71722.  

I'm getting a download failed with these WUs. Windows 11 Boinc 8.0.2

15714 climateprediction.net 10/30/2024 3:41:53 PM Scheduler request completed: got 4 new tasks
15715 climateprediction.net 10/30/2024 3:41:53 PM Project requested delay of 3636 seconds
15716 climateprediction.net 10/30/2024 3:41:55 PM Started download of wah2_8.24_windows_intelx86.exe
15717 climateprediction.net 10/30/2024 3:41:55 PM Started download of wah2_data_8.24_windows_intelx86.zip
15718 climateprediction.net 10/30/2024 3:41:57 PM Giving up on download of wah2_8.24_windows_intelx86.exe: permanent HTTP error
15719 climateprediction.net 10/30/2024 3:41:57 PM Giving up on download of wah2_data_8.24_windows_intelx86.zip: permanent HTTP error

Yes, I saw a lot of download errors in the logs late last night. I have just connected a new 8.0.2 client and successfully downloaded a task.

Are you still having download problems? Might just have been temporary.
---
CPDN Visiting Scientist
ID: 71724 · Report as offensive     Reply Quote
ProfilePDW

Send message
Joined: 29 Nov 17
Posts: 82
Credit: 15,207,444
RAC: 81,474
Message 71725 - Posted: 31 Oct 2024, 9:09:45 UTC - in response to Message 71723.  

I was just checking and yes some tasks have downloaded successfully.
At least 1 client has completed its quota of 2 tasks today so can't get more tasks to work on yet !
ID: 71725 · Report as offensive     Reply Quote
Albert H.

Send message
Joined: 18 Feb 06
Posts: 73
Credit: 62,033,339
RAC: 47,463
Message 71726 - Posted: 31 Oct 2024, 9:27:35 UTC - in response to Message 71725.  

Here in Luxembourg, all OK

31/10/2024 10:07:16 | climateprediction.net | Scheduler request completed: got 6 new tasks
31/10/2024 10:07:16 | climateprediction.net | Project requested delay of 3636 seconds
31/10/2024 10:07:18 | climateprediction.net | Started download of wah2_nz25_11bf_209405_25_1028_012345634.zip
31/10/2024 10:07:18 | climateprediction.net | Started download of generic_phase1_spinup_nz25_global_faaaca_0.gz
31/10/2024 10:07:19 | climateprediction.net | Finished download of wah2_nz25_11bf_209405_25_1028_012345634.zip (17124 bytes)
31/10/2024 10:07:19 | climateprediction.net | Started download of generic_phase1_spinup_nz25_regional_faaaca_0.gz
31/10/2024 10:07:20 | climateprediction.net | Finished download of generic_phase1_spinup_nz25_global_faaaca_0.gz (29926082 bytes)
31/10/2024 10:07:20 | climateprediction.net | Started download of ic19610311_14_N96.gz
31/10/2024 10:07:21 | climateprediction.net | Finished download of generic_phase1_spinup_nz25_regional_faaaca_0.gz (45316285 bytes)
31/10/2024 10:07:21 | climateprediction.net | Finished download of ic19610311_14_N96.gz (1314153 bytes)
31/10/2024 10:07:21 | climateprediction.net | Started download of HAPPI15C_ancil_146months_OSTIA_sst_MMM_2088-12-01_2101-01-30.gz
31/10/2024 10:07:21 | climateprediction.net | Started download of HAPPI_15k_SIC_208912_210111.gz
31/10/2024 10:07:22 | climateprediction.net | Finished download of HAPPI_15k_SIC_208912_210111.gz (15403151 bytes)
31/10/2024 10:07:22 | climateprediction.net | Started download of so2dms_rcp26_N96_2089_2100.gz
31/10/2024 10:07:24 | climateprediction.net | Finished download of HAPPI15C_ancil_146months_OSTIA_sst_MMM_2088-12-01_2101-01-30.gz (48005111 bytes)
31/10/2024 10:07:24 | climateprediction.net | Finished download of so2dms_rcp26_N96_2089_2100.gz (26235530 bytes)
31/10/2024 10:07:24 | climateprediction.net | Started download of oxi.addfa.gz
31/10/2024 10:07:24 | climateprediction.net | Started download of ozone_rcp26_N96_2089_2100v2.gz
31/10/2024 10:07:25 | climateprediction.net | Finished download of ozone_rcp26_N96_2089_2100v2.gz (1415721 bytes)
31/10/2024 10:07:25 | climateprediction.net | Started download of wah2_nz25_10qy_209205_25_1028_012344897.zip
31/10/2024 10:07:27 | climateprediction.net | Finished download of wah2_nz25_10qy_209205_25_1028_012344897.zip (17094 bytes)
31/10/2024 10:07:27 | climateprediction.net | Started download of ic19610627_12_N96.gz
31/10/2024 10:07:28 | climateprediction.net | Finished download of ic19610627_12_N96.gz (1312403 bytes)
31/10/2024 10:07:28 | climateprediction.net | Started download of wah2_nz25_104l_209005_25_1028_012344092.zip
31/10/2024 10:07:29 | climateprediction.net | Finished download of oxi.addfa.gz (82651774 bytes)
31/10/2024 10:07:29 | climateprediction.net | Finished download of wah2_nz25_104l_209005_25_1028_012344092.zip (17501 bytes)
31/10/2024 10:07:29 | climateprediction.net | Started download of ic19610820_14_N96.gz
31/10/2024 10:07:29 | climateprediction.net | Started download of wah2_nz25_101d_209005_25_1028_012343976.zip
31/10/2024 10:07:30 | climateprediction.net | Finished download of ic19610820_14_N96.gz (1313070 bytes)
31/10/2024 10:07:30 | climateprediction.net | Finished download of wah2_nz25_101d_209005_25_1028_012343976.zip (17501 bytes)
31/10/2024 10:07:30 | climateprediction.net | Started download of ic19611104_16_N96.gz
31/10/2024 10:07:30 | climateprediction.net | Started download of wah2_nz25_122z_209705_25_1028_012346626.zip
31/10/2024 10:07:30 | climateprediction.net | Starting task wah2_nz25_10qy_209205_25_1028_012344897_1
31/10/2024 10:07:30 | climateprediction.net | Starting task wah2_nz25_11bf_209405_25_1028_012345634_1
31/10/2024 10:07:30 | climateprediction.net | Starting task wah2_nz25_104l_209005_25_1028_012344092_1
31/10/2024 10:07:31 | climateprediction.net | Finished download of ic19611104_16_N96.gz (1309372 bytes)
31/10/2024 10:07:31 | climateprediction.net | Finished download of wah2_nz25_122z_209705_25_1028_012346626.zip (17108 bytes)
31/10/2024 10:07:31 | climateprediction.net | Started download of ic19610108_12_N96.gz
31/10/2024 10:07:31 | climateprediction.net | Started download of wah2_nz25_11md_209605_25_1028_012346028.zip
31/10/2024 10:07:32 | climateprediction.net | Starting task wah2_nz25_101d_209005_25_1028_012343976_1
31/10/2024 10:07:33 | climateprediction.net | Finished download of ic19610108_12_N96.gz (1312640 bytes)
31/10/2024 10:07:33 | climateprediction.net | Finished download of wah2_nz25_11md_209605_25_1028_012346028.zip (17115 bytes)
31/10/2024 10:07:33 | climateprediction.net | Started download of ic19611205_16_N96.gz
31/10/2024 10:07:33 | climateprediction.net | Starting task wah2_nz25_122z_209705_25_1028_012346626_1
31/10/2024 10:07:34 | climateprediction.net | Finished download of ic19611205_16_N96.gz (1310676 bytes)
31/10/2024 10:07:34 | climateprediction.net | Starting task wah2_nz25_11md_209605_25_1028_012346028_1
ID: 71726 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,742,261
RAC: 6,307
Message 71727 - Posted: 31 Oct 2024, 9:55:32 UTC - in response to Message 71724.  

Yes, I saw a lot of download errors in the logs late last night. I have just connected a new 8.0.2 client and successfully downloaded a task.
This is going to be a complicated one, and I may be barking up the wrong tree - handle with caution - bit here goes.

The download errors are all for application files, not model data. Applications are only downloaded once, when a new version is released.

Yesterday, I noticed the word 'cloudflare' in Glenn's opening log post in the 'clients can't connect' thread. They both occur just after a HTTP/2 error code - 308 in the first case, 500 in the second.

As I understand it, cloudflare provides a transparent caching service, which helps projects' bandwidth greatly when multiple copies of the same file have to be downloaded in a short time period - exactly the situation with a new application release. Could the two symptoms be related?

Things are getting messy here: some machines are running dev test tasks, some have picked up batch 2020 timeout/resends, some are locked on a daily quota of 1, and some have downloaded batch 2028 data and apps and are working on them. But I'll try and find a machine to reproduce the app download error on, and generate a debug log.
ID: 71727 · Report as offensive     Reply Quote
ProfilePDW

Send message
Joined: 29 Nov 17
Posts: 82
Credit: 15,207,444
RAC: 81,474
Message 71728 - Posted: 31 Oct 2024, 10:10:11 UTC - in response to Message 71727.  

There are no new application versions, the hosts already had the existing applications, there was no need to download them !

How long have they been using cloudfare ?
ID: 71728 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,742,261
RAC: 6,307
Message 71729 - Posted: 31 Oct 2024, 10:27:46 UTC - in response to Message 71728.  

Anyone who has attached a new machine recently, or who got caught up in the master file address switcheroo, will need a fresh copy of this particular app version.
ID: 71729 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,575,067
RAC: 15,735
Message 71731 - Posted: 31 Oct 2024, 10:44:00 UTC - in response to Message 71727.  

The download errors are all for application files, not model data.

Richard, that's not correct. If you refer back to TLD's earlier post the log shows that it had problems downloading the wah2_*_data*.zip file too. This batch uses the 8.24 WaH app, which has not been changed for years. A URL change, or disconnect/reconnect to CPDN would trigger the app download.

Whatever the problem was it seems to have gone away. I've been looking at the database this morning. Of 3150 workunits sent out, we've had 48 tasks fail, 90% of those due to download issues. There are currently 2400 running workunits.
---
CPDN Visiting Scientist
ID: 71731 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,742,261
RAC: 6,307
Message 71734 - Posted: 31 Oct 2024, 11:14:42 UTC - in response to Message 71731.  

It hasn't completely gone. I've just fired up the little Windows 10 Celeron, and deliberately downgraded it to BOINC v7.16.20

The debug log ends:

31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Info:  We are completely uploaded and fine
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: HTTP/1.1 500 Internal Server Error
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: Date: Thu, 31 Oct 2024 11:02:26 GMT
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: Content-Length: 0
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: Connection: keep-alive
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: CF-Ray: 8db3271fcdd7bd7f-LHR
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: CF-Cache-Status: DYNAMIC
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: x-powered-by: ASP.NET
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: Vary: Accept-Encoding
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: Server: cloudflare
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server: alt-svc: h3=":443"; ma=86400
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Received header from server:
31/10/2024 11:02:09 | climateprediction.net | [http] [ID#1] Info:  Connection #13 to host cpdn.org left intact
31/10/2024 11:02:10 | climateprediction.net | Scheduler request failed: HTTP internal server error
I'll upgrade it incrementally, to try and work out when it starts working again.
ID: 71734 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 71735 - Posted: 31 Oct 2024, 11:27:30 UTC

It hasn't completely gone. I've just fired up the little Windows 10 Celeron, and deliberately downgraded it to BOINC v7.16.20
And with 8.0.4 I just had four tasks fail because some of the files didn't download. When project backoff allows if there are any left, I will try again with http-debug enabled.
ID: 71735 · Report as offensive     Reply Quote
ProfilePDW

Send message
Joined: 29 Nov 17
Posts: 82
Credit: 15,207,444
RAC: 81,474
Message 71737 - Posted: 31 Oct 2024, 11:31:57 UTC - in response to Message 71734.  

I'll upgrade it incrementally, to try and work out when it starts working again.
7.22.2 (x86) fails to connect.
Do we want to move it out of this thread ?
ID: 71737 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 71738 - Posted: 31 Oct 2024, 11:35:08 UTC

New thread to clear the announcement thread.
ID: 71738 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : Number crunching : Connection and Download issues Oct24

©2024 cpdn.org