As title.

It's almost 2025. Why doesn’t this project support multithreading?

The WUs are extremely long.

There are multi-threaded tasks on the way. They will be the OIFS code from ECMWF. Glenn posted a link to the program for Linux that you can run in a terminal on Linux recently and play around with the file that determines how many cores it uses. Most of the work however is still the Met Office code which is all 32bit which says something about its age in itself. Initially at least the multithreaded apps will like the other OIFS work all be Linux only.

The answer to your query however is that the project hasn't had a scientist with the required programming skills to write multi-threaded apps till Glenn came on board as a volunteer scientist. I don't know whether Andy who does the sysadmin work has the skills but his other work for the project is full time and would not allow him the space to develop the multithreaded apps.

This is the link to the thread with discussion about the multi-core app and the link to try it if you have access to a Linux box.

Interestingly, I would say that less than a quarter of the projects I have looked at have multithreaded apps and even those that do, don't allow it on all task types.

Edit: If I have missed anything important I am sure another moderator or Glenn will add to this but I can't imagine it is straightforward to modify the Met Office code to make it multithreaded even if that is possible and that is the code most of the scientists around the world are still using for their tasks.

As title.
It's almost 2025. Why doesn’t this project support multithreading?
The WUs are extremely long.

Depends what you mean by 'extremely long'. With a recent processor and keeping the machine on 24/7, most workunits finish in about 2 days. I switch mine off overnight so it'll take 10 days or so.

But if you prefer seeing tasks finish after a hour or two, CPDN might not be the project for you. When we introduce multithreaded apps, it will be to allow for more complex configurations to ensure they finish in a reasonable time not make the existing configurations complete quicker. Running single threaded is still the most efficient way of running the models.

All the meteorological codes that CPDN run are capable of multiprocessing, even the older ones. We disable it because it reduces the memory footprint of the models and makes their implementation in boinc simpler.

---

---
CPDN Visiting Scientist

All the meteorological codes that CPDN run are capable of multiprocessing, even the older ones. We disable it because it reduces the memory footprint of the models and makes their implementation in boinc simpler.

Interesting. I always thought it's the additional work in code base being the problem. If multi-thread mode is validated anyway, I'd love to have the option through cc_config.xml to pass in the right arguments. (Or is it possible already, just not advertised so that we don't know?) That way, the project doesn't need to onboard any new complexity from server side, but the volunteers who have the need or preference can customize on their end. It can be helpful for shortening the runtime, or better utilize the cores when the host is memory bound but not running other projects.

All the meteorological codes that CPDN run are capable of multiprocessing, even the older ones.

Of course they are capable of multiprocessing. They were written to run on supercomputers. I blame my last post on having a head stuffed full of cold at the moment!

I can see that total throughput of tasks might be more without multithreading. However when there are relatively small batches and the first few hundred computers grab them all, then multithreading would spread the tasks out between more computers and would get them returned more quickly.

Sorry, enabling & testing the multiprocessing in the older models is not something I'm going to spend time doing. They work fine as they are. I have more pressing things to do.

---

---
CPDN Visiting Scientist

Sorry, enabling & testing the multiprocessing in the older models is not something I'm going to spend time doing. They work fine as they are. I have more pressing things to do.

Fair enough. There are always going to be priorities that we won't know anything about. If climate science had the funding and resources it deserves....