climateprediction.net home page
Problems with uploads of zip files :- Backing off ... on upload of...
Problems with uploads of zip files :- Backing off ... on upload of...
log in

Advanced search

Message boards : Number crunching : Problems with uploads of zip files :- Backing off ... on upload of...

1 · 2 · Next
Author Message
Dave Roberts
Send message
Joined: 15 Jan 11
Posts: 147
Credit: 4,056,217
RAC: 5,460
Message 56801 - Posted: 8 Sep 2017, 11:22:26 UTC

Task 20652406 Windows 7

I have 2 tasks that have been trying to upload zip files for the last 5 days -
getting mesages :- eg

Started upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_11.zip 07/09/2017 21:07:13 | climateprediction.net |
Backing off 00:02:00 on upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_11.zip
08/09/2017 02:17:41 | climateprediction.net |
Started upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_6.zip
08/09/2017 02:17:41 | climateprediction.net |
Started upload of wah2_afr50_k736_201512_13_641_011202321_0_r1780376226_10.zip
08/09/2017 02:18:06 | climateprediction.net |
Backing off 04:34:31 on upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_6.zip
etc etc

Can't see any problems with the servers.
Any ideas anyone?

Dave Roberts
Send message
Joined: 15 Jan 11
Posts: 147
Credit: 4,056,217
RAC: 5,460
Message 56802 - Posted: 8 Sep 2017, 11:24:30 UTC - in response to Message 56801.

PS Internet connection is OK, no time restrictions.

WB8ILI
Send message
Joined: 1 Sep 04
Posts: 122
Credit: 45,061,497
RAC: 23,433
Message 56804 - Posted: 8 Sep 2017, 12:32:08 UTC

Not an expert, but I think I remember reading that all uploads don't go to Oxford. As I remember, sometime back there was an issue with some upload server in Mexico that prevented uploads from working. If your uploads issues are with just one batch number, this might be a clue.
____________

Profile JIM
Send message
Joined: 31 Dec 07
Posts: 1026
Credit: 16,941,149
RAC: 9,750
Message 56805 - Posted: 8 Sep 2017, 13:16:08 UTC - in response to Message 56801.

All the regional models, the zips go to the Universities that contracted for them except of the last zip. If they are having trouble receiving these zips files it would not show on the Server Status page. The last zip, the restart dump used to create the next segment of the model, still goes to Oxford.
____________

Dave Roberts
Send message
Joined: 15 Jan 11
Posts: 147
Credit: 4,056,217
RAC: 5,460
Message 56809 - Posted: 9 Sep 2017, 16:13:46 UTC - in response to Message 56805.

Thanks for your replies. FYI, I'm getting the same problem with restart zips??

08/09/2017 19:29:11 | climateprediction.net Started upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_9.zip
08/09/2017 19:29:11 | climateprediction.net | Started upload of wah2_afr50_k736_201512_13_641_011202321_0_r1780376226_restart.zip
08/09/2017 19:29:34 | climateprediction.net | Backing off 00:06:43 on upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_9.zip
08/09/2017 19:29:34 | climateprediction.net | Backing off 00:07:42 on upload of wah2_afr50_k736_201512_13_641_011202321_0_r1780376226_restart.zip

Dave Roberts
Send message
Joined: 15 Jan 11
Posts: 147
Credit: 4,056,217
RAC: 5,460
Message 56819 - Posted: 11 Sep 2017, 10:02:21 UTC - in response to Message 56809.

Well, everything got uploaded late last night, after 8 days.

I'd previously checked all the forums to see if anyone had seen anything like this in the past. Various combinations of 'project' 'backup', 'pending', 'upload' etc.. with no luck. (But I think the searches only look at titles)

It was the restsrt zips that threw me, since, if they all went to Oxford, and no server problems had been reported there, was there another problem?

Anyway, looks like patience is the key.

Profile geophi
Volunteer moderator
Send message
Joined: 7 Aug 04
Posts: 1718
Credit: 33,727,870
RAC: 19,632
Message 56821 - Posted: 11 Sep 2017, 15:24:56 UTC

I e-mailed them last night about the problem and someone must have taken care of it. I didn't hear back what the issue was.

When you were searching, did you do an advanced search? The default search limit is 30 days, but you can change that to longer when searching with advanced options.

Dave Roberts
Send message
Joined: 15 Jan 11
Posts: 147
Credit: 4,056,217
RAC: 5,460
Message 56822 - Posted: 11 Sep 2017, 16:41:41 UTC

Thanks for the info geophi, I'd used an advanced search but didn't spot anything like my tasks - upload problems after successful completion - that had such a long delay before uploading properly.

There was a similar problem back in 2005, but that had a "Temporarily failed upload" message whilst my error was "Backing off... "'.

It would be interesting to know the reason for the problem.
Thanks again.
Dave

KANE47
Send message
Joined: 11 Sep 12
Posts: 2
Credit: 2,188,559
RAC: 0
Message 57440 - Posted: 6 Dec 2017, 3:43:48 UTC - in response to Message 56822.

Is it safe to say this one (see log below) will get resolved with some patience?
Seems like I can download work with no probs ever.

But uploading finished work? forget about it.

Other projects via BOINC transfer up and down with no problem; just this one seems to be odd.


12/5/2017 21:25:00 | | Project communication failed: attempting access to reference site
12/5/2017 21:25:01 | | Internet access OK - project servers may be temporarily down.
12/5/2017 21:30:14 | climateprediction.net | Started upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_restart.zip
12/5/2017 21:30:14 | climateprediction.net | Started upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_out.zip
12/5/2017 21:30:17 | climateprediction.net | Temporarily failed upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_restart.zip: connect() failed
12/5/2017 21:30:17 | climateprediction.net | Backing off 05:58:54 on upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_restart.zip
12/5/2017 21:30:17 | climateprediction.net | Temporarily failed upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_out.zip: connect() failed
12/5/2017 21:30:17 | climateprediction.net | Backing off 05:47:08 on upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_out.zip
12/5/2017 21:30:18 | | Project communication failed: attempting access to reference site
12/5/2017 21:30:19 | | Internet access OK - project servers may be temporarily down.

Profile geophi
Volunteer moderator
Send message
Joined: 7 Aug 04
Posts: 1718
Credit: 33,727,870
RAC: 19,632
Message 57441 - Posted: 6 Dec 2017, 5:52:27 UTC - in response to Message 57440.

The server that accepts the EAS model files has been having problems for the last week or two. The computer staff is looking into it/working on it. No ETA for a fix at this time.

Henk Haneveld
Send message
Joined: 9 Dec 17
Posts: 2
Credit: 86,830
RAC: 25
Message 57465 - Posted: 12 Dec 2017, 8:08:12 UTC - in response to Message 57441.

After a long time away I returned to this project only to find my old acount had been deleted (not very happy about that) and to run straight in this upload problem.

Any progress on fixing this?

Profile Dave Jackson
Send message
Joined: 15 May 09
Posts: 2098
Credit: 2,804,645
RAC: 1,647
Message 57468 - Posted: 12 Dec 2017, 8:43:50 UTC - in response to Message 57465.
Last modified: 12 Dec 2017, 8:51:35 UTC

Hi Henk, still no news on this, I know one of the key people at Oxford is away for a few weeks so those covering may not be as familiar with what needs to be done making it take longer.

KANE47
Send message
Joined: 11 Sep 12
Posts: 2
Credit: 2,188,559
RAC: 0
Message 57484 - Posted: 13 Dec 2017, 23:37:43 UTC

geo and Dave -


Thanks much!

-K

Alex Plantema
Send message
Joined: 3 Sep 04
Posts: 109
Credit: 19,113,602
RAC: 32,499
Message 57488 - Posted: 15 Dec 2017, 22:59:06 UTC - in response to Message 57440.

I also have a task from batch 686 that won't upload at all. Trickles 1 to 12, restart and out are still in the upload queue.

Eirik Redd
Send message
Joined: 31 Aug 04
Posts: 348
Credit: 86,425,571
RAC: 99,913
Message 57489 - Posted: 16 Dec 2017, 6:50:49 UTC
Last modified: 16 Dec 2017, 7:16:40 UTC

batches 686 and 690 eas50
Yeah, these batches do, and have for at least a month, possibly once accepted uploads.
But mostly never.
Trickles, yes.

What I do, is , if the sponsor of the batch doesn't respond in a few weeks to an upload problem -- either they don't know or don't care, and if the growing backlog of unwanted uploads starts to delay subprojects that do care, or fill my disk,
it's "cancel another few "eas50 wu's that were misconfigured" no biggie.
It is annoying that I have to also cancel the worthless uploads of "eas50 - 1-12 singly or one at a time even after the tasks have completed (according to BOINC)

Short form -- eas50 workunits -- whatever batch -- let them finish, but if you have a lot of unaccepted uploads for a week or two -- CANCEL the tasks and the uploads - we aren't paying for upstream misconfigs. NOT cpdn project's fault.
____________

Sergey Lovtsov
Send message
Joined: 13 Nov 06
Posts: 3
Credit: 577,278
RAC: 1
Message 57502 - Posted: 20 Dec 2017, 11:58:03 UTC

Hello!

So, I can't upload 14 files. For example:

20.12.2017 14:56:25 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_1.zip
20.12.2017 14:56:25 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_2.zip
20.12.2017 14:56:28 | climateprediction.net | Backing off 05:16:10 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_1.zip
20.12.2017 14:56:28 | climateprediction.net | Backing off 03:14:22 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_2.zip
20.12.2017 14:56:29 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_3.zip
20.12.2017 14:56:29 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_4.zip
20.12.2017 14:56:32 | climateprediction.net | Backing off 03:03:50 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_3.zip
20.12.2017 14:56:32 | climateprediction.net | Backing off 04:07:17 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_4.zip

Alex Plantema
Send message
Joined: 3 Sep 04
Posts: 109
Credit: 19,113,602
RAC: 32,499
Message 57508 - Posted: 20 Dec 2017, 21:42:44 UTC

It's strange that the 12 trickles are both on the result page and in the upload queue.

Sergey Lovtsov
Send message
Joined: 13 Nov 06
Posts: 3
Credit: 577,278
RAC: 1
Message 57511 - Posted: 21 Dec 2017, 8:08:10 UTC

I have 6 projects, but I have problems only with this one.

Profile Dave Jackson
Send message
Joined: 15 May 09
Posts: 2098
Credit: 2,804,645
RAC: 1,647
Message 57512 - Posted: 21 Dec 2017, 8:39:14 UTC

I have 6 projects, but I have problems only with this one.


In many ways it is not surprising that this project has more problems than many others. The program files are much longer - something like a million lines of Fortran altogether. The tasks are put together by a lot of different people so more chance of getting something wrong and it isn't just the computers at Oxford that can mess things up as the zips often go to computers at the institution where the scientists who get Oxford to send the work out on their behalf are based.

In an ideal world the code which is used on license from the met office here in UK would I guess be rewritten from scratch. Certainly some bits of the Linux code would obviating the need to install 32bit libraries. However, I suspect the project will never have the resources to make that possible.

Profile Dave Jackson
Send message
Joined: 15 May 09
Posts: 2098
Credit: 2,804,645
RAC: 1,647
Message 57513 - Posted: 21 Dec 2017, 8:45:20 UTC - in response to Message 57508.

It's strange that the 12 trickles are both on the result page and in the upload queue.


The trickles are what appear on the result page. They are different from the zip files. It is just that with recent model types they are produced at the same time. On older model types I think that there were only zips produced at the end but the trickles were still produced and allowed people to gain credit for a task they crunched for 9 months or more which still crashed before finishing.

That is the route cause of most of the problems there have been with the credit system over the years.

1 · 2 · Next

Message boards : Number crunching : Problems with uploads of zip files :- Backing off ... on upload of...


Main page · Your account · Message boards


Copyright © 2018 climateprediction.net