climateprediction.net home page
Tasks in progress
Tasks in progress
log in

Advanced search

Message boards : Cafe CPDN : Tasks in progress

1 · 2 · Next
Author Message
Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6408
Credit: 16,839,542
RAC: 21,887
Message 53191 - Posted: 5 Jan 2016, 6:00:19 UTC

At the time that the project ran out of new work units just before Christmas, the number in progress was just over 92,000. It's now just under 82,000.
So the stock piles are being eaten up, which is a good thing for researchers who want their data.

bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 252
Credit: 5,903,045
RAC: 23
Message 53192 - Posted: 5 Jan 2016, 13:10:41 UTC - in response to Message 53191.

Hi Les,
I was wondering why the return pace is so slow? I was expecting after 10-12 days after the last batch was sent most computers to finish with all CPDN work, hence reduce the number in progress fast. But I guess most crunchers allocate some time to CPDN tasks and they will end much later than the longest model run at 24/7.

Profile Dave Jackson
Send message
Joined: 15 May 09
Posts: 1793
Credit: 2,671,578
RAC: 898
Message 53193 - Posted: 5 Jan 2016, 14:16:17 UTC - in response to Message 53192.

Many have slow computers - it needs a new hd at the moment but I have completed tasks on a very slow netbook. Also many do not crunch 24/7. I have one quad core that does but my desktop machine is switched off when not in use. The two tasks it is running are currently 1/3 of the way through. Tasks were sent on 4th December. Sadly my computer that is running 24/7 is out of work at the moment. I spotted a task on service status the other day and did a project update but got a permanent http error on it. So I am now waiting to see whether windows tasks to run under WINE or Linux tasks arrive first.

Profile JIM
Send message
Joined: 31 Dec 07
Posts: 983
Credit: 14,320,108
RAC: 19,627
Message 53194 - Posted: 5 Jan 2016, 14:39:13 UTC
Last modified: 5 Jan 2016, 14:40:48 UTC

Profile Dave Jackson
Send message
Joined: 15 May 09
Posts: 1793
Credit: 2,671,578
RAC: 898
Message 53195 - Posted: 5 Jan 2016, 14:59:32 UTC - in response to Message 53194.

bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 252
Credit: 5,903,045
RAC: 23
Message 53196 - Posted: 5 Jan 2016, 15:28:21 UTC - in response to Message 53193.

So my guess was almost correct. Not running 24/7 and shared project resources might be the leading reason though as most of my machines are Core2Duo's and some even run at 1.6GHz, but they run 24/7 exclusively for CPDN. I got recently a machine with i5 2520M, so nowhere near 3-4 GHz. A Devil's Canyon is still a dream.

Happy new year fellow crunchers

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6408
Credit: 16,839,542
RAC: 21,887
Message 53197 - Posted: 5 Jan 2016, 19:44:28 UTC

Hi Bernard

This thread expanded a bit while I was sleeping. :)

My guess for the long return times is that a lot of people still treat the BOINC deadline as the time when the project expects to get results back, and just take their time.
Plus I've seen a few computers with lots more tasks "in progress" than the number of cpu cores. (Hence my use of the term "stock piles".)

bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 252
Credit: 5,903,045
RAC: 23
Message 53198 - Posted: 5 Jan 2016, 19:56:55 UTC - in response to Message 53197.

Hi Les,
in fact I also have 3 tasks in progress that has been aborted or otherwise cancelled and no longer crunched, but will die out in 2016 and one in 2023. As for more tasks than CPU cores I guess this is a BOINC setting (or project specific) that allows users to stock pile few tasks not to get starved, isn't it?

noderaser
Avatar
Send message
Joined: 17 Nov 04
Posts: 11
Credit: 913,996
RAC: 981
Message 53199 - Posted: 6 Jan 2016, 6:28:40 UTC - in response to Message 53198.

in fact I also have 3 tasks in progress that has been aborted or otherwise cancelled and no longer crunched, but will die out in 2016 and one in 2023.

Is that a typo? I thought the deadlines for the longest tasks were one year out...?

I must admit, when I can get CPDN units on my less-often-used computers, I eat up a few units there because they're not on frequently enough to get tasks back to projects that have 1-2 week deadlines. But, a year is usually more than enough :P Hope I'm not causing too much trouble there.
____________

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6408
Credit: 16,839,542
RAC: 21,887
Message 53200 - Posted: 6 Jan 2016, 7:06:56 UTC - in response to Message 53199.

I think that the very long deadline was an attempt to get around a BOINC problem.
I don't think that they should actually run.

1 year is not the "results wanted by" date. It's a BOINC deadline that has to be worked around, for people running cpdn stuff very slowly.

At present, if a result that's of particular interest hasn't been returned in a timely fashion, a new task can be created, same data set, different name. Then the original task will be ignored.

But I've been thinking about this ...


bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 252
Credit: 5,903,045
RAC: 23
Message 53202 - Posted: 6 Jan 2016, 8:50:02 UTC - in response to Message 53199.

Is that a typo? I thought the deadlines for the longest tasks were one year out...? .


Nope, here is the task until 2023. The other two are as Les said within a year.

So noderaser how do you manage to get more tasks than CPU cores? Where can one set such a preference?

Kevin
Send message
Joined: 5 Jul 09
Posts: 60
Credit: 4,641,723
RAC: 2,126
Message 53203 - Posted: 6 Jan 2016, 8:57:39 UTC - in response to Message 53200.

I think that the very long deadline was an attempt to get around a BOINC problem.
I don't think that they should actually run.




A longer deadline is handy, I only run CPND as a secondary project on my CPU, as it is only a secondary project I normally try to only allocate 1 or 2 cores to CPND, unfortunately sometimes I have to pause these for several days at a time.

Sometimes I find that I get clobbered when work appears here, nothing for a long while then crash more work than I need. So sometimes some work units may not be processed immediately, they may be subject to some delay.

I am not asking for a full year but 3 to 4 months would a comfortable deadline.




____________
Kevin

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6408
Credit: 16,839,542
RAC: 21,887
Message 53204 - Posted: 6 Jan 2016, 9:28:21 UTC - in response to Message 53203.

I am not asking for a full year but 3 to 4 months would a comfortable deadline.


Kevin

That's my thinking too.
The 1 year was from 9-10 years ago, when the models took 3-4 months to run. Now they take 1-3 weeks. With at least one batch that ran for 1 day. But that may have been beta.

Perhaps in 2 stages - first to 6 months for a while, to get peoples attention.
Then to 3-4 months, as you said. Although some of the models are getting bigger.
It can always be relaxed a bit if this tend continues.

Profile Alan K
Send message
Joined: 22 Feb 06
Posts: 203
Credit: 10,727,650
RAC: 8,191
Message 53206 - Posted: 6 Jan 2016, 11:24:51 UTC - in response to Message 53204.

It's almost a shame that there is no easy way to diferentiate between "short" and "long" models so that the short ones can be processed by the slower computers leaving the faster machines to cope with the longer - and possibly more demanding - longer models. Mearly a thought.

Profile Dave Jackson
Send message
Joined: 15 May 09
Posts: 1793
Credit: 2,671,578
RAC: 898
Message 53207 - Posted: 6 Jan 2016, 12:20:15 UTC

It's almost a shame that there is no easy way to diferentiate between "short" and "long" models so that the short ones can be processed by the slower computers leaving the faster machines to cope with the longer - and possibly more demanding - longer models.


Agreed, though, at the moment I will take any tasks I can get long or short!

Profile JIM
Send message
Joined: 31 Dec 07
Posts: 983
Credit: 14,320,108
RAC: 19,627
Message 53209 - Posted: 6 Jan 2016, 16:48:49 UTC

One problem is that when Boinc tries to run short deadline projects along side CPDN Boinc has the tendency to pause all CPDN work and run the other projects first to avoid missing there deadlines. This means that CPDN tasks sit and wait despite the fact that CPDN has a 600 share and other projects have a 100 share.

____________

Jim1348
Send message
Joined: 15 Jan 06
Posts: 174
Credit: 7,286,458
RAC: 8,182
Message 53210 - Posted: 6 Jan 2016, 17:39:41 UTC - in response to Message 53209.
Last modified: 6 Jan 2016, 17:46:18 UTC

It helps to keep a short buffer. If you load up on projects, BOINC has no choice but to try and get them done on time. It has no other way to prioritize them.

Also, I set "Switch between tasks" to 1440 minutes (24 hours), and it should perhaps be longer. How much good it does may depend on the projects running, and it won't extend the due dates in any case.

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6408
Credit: 16,839,542
RAC: 21,887
Message 53211 - Posted: 6 Jan 2016, 21:35:26 UTC - in response to Message 53209.

Jim

That IS the reason for the year long deadline, but there's too many people now that are taking advantage of it.
If I ordered an item for delivery in the next few days, I wouldn't be happy if it took a year to arrive. And now that our "customers" are professional climate physicists, I feel that they should get a better deal. Even if it means losing a lot of slow crunchers, with slow (relative), computers. :(

And thanks everyone for all of the feed back. I didn't expect it, but it's welcome. :(


Kevin
Send message
Joined: 5 Jul 09
Posts: 60
Credit: 4,641,723
RAC: 2,126
Message 53212 - Posted: 7 Jan 2016, 5:32:04 UTC - in response to Message 53210.

It helps to keep a short buffer. If you load up on projects, BOINC has no choice but to try and get them done on time. It has no other way to prioritize them.


Sometimes buffer length settings are dictated by another project and when you are not expecting to get anything from CPND and in a single request you get 4 tasks (_1 or_2's) as happened on the 4th of this month to me, yes they will all be processed, but it may take a month to do so.

It might be an idea to ask if buffer settings per project could be set in Boinc, but that would have to be requested elsewere. This would also be very useful when getting work from Einstein which has short deadline work. I know that we can set NNT per project but sometimes I do not remember untill it is too late.

One thing that has got to be taken into account, not all people that are doing CPND work are doing it as their main project, some of us have other projects, mine are SETI and Einstein.


____________
Kevin

noderaser
Avatar
Send message
Joined: 17 Nov 04
Posts: 11
Credit: 913,996
RAC: 981
Message 53213 - Posted: 7 Jan 2016, 7:12:59 UTC - in response to Message 53202.

So noderaser how do you manage to get more tasks than CPU cores? Where can one set such a preference?

I don't, I just try to get on task per core on those machines which are used less often. Unfortunately, that can be pretty hard due to the nature of the project and the low flow of tasks.

I know the long deadlines must be rough for the researchers as well, it's certainly a long waiting game for results. I think a deadline of even six months would work for my purposes, if that ended up being the new deadline.
____________

1 · 2 · Next

Message boards : Cafe CPDN : Tasks in progress


Main page · Your account · Message boards


Copyright © 2017 climateprediction.net