Gaia 5

Message boards : Number crunching : Gaia 5
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
[AF>Le_Pommier] Jerome_C2005

Send message
Joined: 28 Sep 20
Posts: 13
Credit: 647,614
RAC: 4,943
Message 492 - Posted: 24 Oct 2024, 8:44:41 UTC - in response to Message 491.  

All the 4_G tasks I got errored with the same problem, but early this morning I got two 4_G and three 5_G tasks that did succeed (!) and then again I started to receive only 4_G that all fail for the same reason.

Note that yesterday morning I had also received one single 5_G that had failed with time limit (after only 6mn of calculation). It was the first and only 5_G until the 3 I got this morning.

I have suspended gaia for the moment until the (nice) admin/does does his maths again.
ID: 492 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Conan
Avatar

Send message
Joined: 27 Apr 20
Posts: 40
Credit: 3,261,559
RAC: 40,914
Message 494 - Posted: 24 Oct 2024, 11:18:55 UTC
Last modified: 24 Oct 2024, 11:33:23 UTC

Sorry zupa, as I am watching all my 5_Gaia work units are going into aborted run time exceeded, at about 2 hours 33 minutes and 50 seconds.

It was looking so good too.

They got to 100% at about 1 hour 20 minutes and then stayed there.

Conan

PS: :-- I just had one complete normally after 2 hours 23 minutes.:
ID: 494 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Freewill

Send message
Joined: 21 Oct 24
Posts: 9
Credit: 3,265,800
RAC: 130,186
Message 495 - Posted: 24 Oct 2024, 18:50:35 UTC - in response to Message 494.  

Can someone report if the timing out of tasks is fixed yet?
ID: 495 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zupa

Send message
Joined: 21 Aug 19
Posts: 115
Credit: 888,695
RAC: 1,654
Message 497 - Posted: 24 Oct 2024, 19:58:20 UTC - in response to Message 495.  

I recalculate the rsc_fpops parameters and I am waiting for new results ...
ID: 497 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Conan
Avatar

Send message
Joined: 27 Apr 20
Posts: 40
Credit: 3,261,559
RAC: 40,914
Message 498 - Posted: 24 Oct 2024, 20:41:28 UTC
Last modified: 24 Oct 2024, 20:44:48 UTC

My first batch of 31 tasks had 3 successful results, they all ran for less than 2 hours 33 minutes.

The other 28 tasks ran for 2 hours 33 minutes 50 seconds each.

I have downloaded over 200 new tasks on a different host nut none have finished yet but have already gone past 3 hours run time so looks promising.

Conan
ID: 498 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Freewill

Send message
Joined: 21 Oct 24
Posts: 9
Credit: 3,265,800
RAC: 130,186
Message 499 - Posted: 24 Oct 2024, 21:36:27 UTC - in response to Message 497.  

I recalculate the rsc_fpops parameters and I am waiting for new results ...

Do the Gaia 4s still have the timeout problem? All 4s and a few of the 5s have ridiculously long time remaining estimates.
I have had some 5s finish without problems, but they had ~10 hr estimates, not the 29-76 day estimates. Do those long estimates go along with short timeout errors?
ID: 499 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Conan
Avatar

Send message
Joined: 27 Apr 20
Posts: 40
Credit: 3,261,559
RAC: 40,914
Message 500 - Posted: 24 Oct 2024, 22:36:56 UTC - in response to Message 497.  

I recalculate the rsc_fpops parameters and I am waiting for new results ...


I have had some successes with 3,600 Gflops but most didn't work.

The new 1,264,000 Gflop estimate is working well and I have a page of successful results so far.

Also with the new rsc_fpops estimate I have a 4_Gaia be successful even though it failed 3 times before.

Conan
ID: 500 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Freewill

Send message
Joined: 21 Oct 24
Posts: 9
Credit: 3,265,800
RAC: 130,186
Message 501 - Posted: 24 Oct 2024, 23:30:51 UTC

I now have a bunch of Gaia 5 tasks failing with time limit exceeded around 15min31sec. Here's an example:

https://gaiaathome.eu/gaiaathome/result.php?resultid=144313

Roger
ID: 501 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Conan
Avatar

Send message
Joined: 27 Apr 20
Posts: 40
Credit: 3,261,559
RAC: 40,914
Message 502 - Posted: 25 Oct 2024, 10:28:17 UTC - in response to Message 501.  
Last modified: 25 Oct 2024, 10:34:33 UTC

I now have a bunch of Gaia 5 tasks failing with time limit exceeded around 15min31sec. Here's an example:

https://gaiaathome.eu/gaiaathome/result.php?resultid=144313

Roger


Yes I just had a whole batch (100) of these as well. They have the updated rsc_fpops but this made no difference. Mine were 5_31xxx

See https://gaiaathome.eu/gaiaathome/result.php?resultid=146041

Conan
ID: 502 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Drago75

Send message
Joined: 11 Jul 22
Posts: 3
Credit: 1,022,119
RAC: 20,666
Message 503 - Posted: 25 Oct 2024, 11:58:39 UTC - in response to Message 502.  

Zupa, can't you just remove the time limits? At least for a while until you see more clearly how long people actually need to crunch those wus.
ID: 503 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
boboviz

Send message
Joined: 19 May 20
Posts: 38
Credit: 44,565
RAC: 739
Message 504 - Posted: 25 Oct 2024, 14:48:25 UTC

At the end 1 wu (5_Gaia) correctly chrunced!! (3,5 hrs and 200 points)
ID: 504 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 22 Oct 24
Posts: 27
Credit: 570,400
RAC: 16,950
Message 505 - Posted: 25 Oct 2024, 17:51:24 UTC

@zupa Did you actually set up the wrapper to just crunch for 3.5 hours elapsed time on the tasks no matter their completion percentage? LIke how Rosetta does it for their tasks?
ID: 505 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Freewill

Send message
Joined: 21 Oct 24
Posts: 9
Credit: 3,265,800
RAC: 130,186
Message 506 - Posted: 25 Oct 2024, 17:56:39 UTC - in response to Message 505.  

@Keith, I doubt it because I'm still getting Gaia 5 tasks failing at 15m30sec. Haven't seen where the researchers have tried anything new today.
ID: 506 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 22 Oct 24
Posts: 27
Credit: 570,400
RAC: 16,950
Message 507 - Posted: 25 Oct 2024, 18:00:03 UTC
Last modified: 25 Oct 2024, 18:45:43 UTC

I looked further and had some fail at the 15 minute 30 second point too. But the majority successfully validate at around 3.5 hours. So maybe just a bad series you got and other series are good.
ID: 507 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Matthias Lehmkuhl

Send message
Joined: 12 Oct 20
Posts: 12
Credit: 1,654,706
RAC: 4,121
Message 508 - Posted: 25 Oct 2024, 19:36:47 UTC

For all my Gaia@home tasks send >= 25 Oct 2024, 15:37:09 UTC I get End status "197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED"
https://gaiaathome.eu/gaiaathome/result.php?resultid=167683
<message>
exceeded elapsed time limit 7791.99 (102526.13G/13.69G)</message>
https://gaiaathome.eu/gaiaathome/result.php?resultid=168058
<message>
exceeded elapsed time limit 8656.73 (102526.13G/11.84G)</message>
https://gaiaathome.eu/gaiaathome/result.php?resultid=168224
<message>
exceeded elapsed time limit 7487.89 (102526.13G/13.69G)</message>
https://gaiaathome.eu/gaiaathome/result.php?resultid=168398
<message>
exceeded elapsed time limit 7082.87 (102526.13G/14.48G)</message>
https://gaiaathome.eu/gaiaathome/result.php?resultid=168546
<message>
exceeded elapsed time limit 6571.83 (102526.13G/15.60G)</message>
Matthias
ID: 508 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 22 Oct 24
Posts: 27
Credit: 570,400
RAC: 16,950
Message 509 - Posted: 25 Oct 2024, 22:33:04 UTC
Last modified: 25 Oct 2024, 23:30:37 UTC

I see the latest tasks have had their rsc_fpops_est values cut by 10X again. Now just 126,400 GFLOPS and much closer to the original 3,600 GFLOPS that was working for mostly everyone when the project first came back.
Hope to see this trend continue when we can expect all tasks be immune from the exceeded time limit errors.
[Edit] Still getting timeout errors on my slower Epyc's. They need to cut the GFLOPS values by another 10X.
ID: 509 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Matthias Lehmkuhl

Send message
Joined: 12 Oct 20
Posts: 12
Credit: 1,654,706
RAC: 4,121
Message 511 - Posted: 26 Oct 2024, 11:18:30 UTC

Looks like it's working now
new tasks send from 26 Oct 2024, 7:00:18 UTC are now finishing successful
https://gaiaathome.eu/gaiaathome/result.php?resultid=176802
<stderr_txt>
09:00:23 (180420): wrapper (7.15.26016): starting
09:00:23 (180420): wrapper (7.15.26016): starting
09:00:23 (180420): wrapper: running ../../projects/gaiaathome.eu_gaiaathome/5_gaia@home[20241023.2232]_x86_64-pc-linux-gnu ()
12:23:23 (180420): 5_gaia@home[20241023.2232] exited; CPU time 10981.851425
12:23:23 (180420): called boinc_finish(0)

</stderr_txt>
Matthias
ID: 511 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 22 Oct 24
Posts: 27
Credit: 570,400
RAC: 16,950
Message 512 - Posted: 26 Oct 2024, 15:24:09 UTC
Last modified: 26 Oct 2024, 15:26:20 UTC

Looks like the latest cutdown in the GFLOPS value for the tasks is working. Too bad it took them several days to properly configure their work generator.

Now out of work. Shame all those prior tasks were wasted.
ID: 512 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
entity

Send message
Joined: 23 May 21
Posts: 5
Credit: 9,118,722
RAC: 259,134
Message 513 - Posted: 26 Oct 2024, 15:58:19 UTC - in response to Message 512.  

It looks like Gaia tasks don't honor BOINC suspension. Boinc Manager says the tasks are suspended but the OS still shows running processes. This is under Linux
ID: 513 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 22 Oct 24
Posts: 27
Credit: 570,400
RAC: 16,950
Message 514 - Posted: 26 Oct 2024, 18:05:15 UTC - in response to Message 513.  

It looks like Gaia tasks don't honor BOINC suspension. Boinc Manager says the tasks are suspended but the OS still shows running processes. This is under Linux

Are you sure?? Suspending tasks won't suspend the wrapper app which is just sleeping during suspend and actually during most of the crunching. The main app is suspended though and that is the one that actually does the crunching.
The wrapper app is the process with big 'G' in the name. The process with the little 'g' in the name is the actual science app.
ID: 514 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Gaia 5

©2024 GAVIP-GC