Jump to content
Sign in to follow this  
  • entries
    4
  • comment
    1
  • views
    51643

SshResourceManager - Running DesignEngines away from home.... and more

Those of you that attended last year's user's meeting in Berlin already got a glimpse of some pretty exciting new functionality. For those of you that couldn't make it and because it has progressed into an actual releasable feature (once it has passed the testing phase, that is), let me give you a short introduction.

 

Ok, I am sure all of you know the SshResourceManager, right?! ... NO?! Hmpf, OK. The SshResourceManager (SSHRM) is a light-weight grid-engine shipped with CAESES that allows you to run your external (CFD) solvers on remote computers (sorry, it is not usable with the CAESES free).

So, for example, your solver only runs on Linux but you prefer working on Windows with your workstation? No problem, use the SSHRM to act as a bridge to that Linux machine.

Your external code should run on that super powerful 128 core machine in your basement? Use the SSHRM to start it there.

You have a bunch of computers standing around in your office and all those CPUs/cores inside of them are bored all the time? Use the SSHRM to put those CPUs to good use.

The only requirement is, that the computer(s)s you want to run your external code on is (are) accessible through SSH (hence the name SSHResourceManager). Don't worry, there are SSH-servers for Windows, too...

The SSHRM is shipped with a detailed admin guide, so the setup should be pretty straight-forward. Don't be scared (it's no Voodoo, really - and we are always willing to help).

 

So, now that we got that out of the way, let's get to the point of this entry:

 

The SSHRM is great to distribute the workload of your external solvers to the resources you have available in your network. It does not only keep track of the CPUs that are currently in use, but also helps you to make sure those expensive licenses for your CFD programs do not go idle (but are not overused at the same time). The SSHRM may even be helpful if you do not want to use remote computers at all, but just need to make sure that your floating licenses users do not block each other. Even if the actual program runs locally on the engineer's machine.

I'm drifting off my actual target here, but I just have the feeling too few people know about these possibilities or are afraid of setting up this system. If the latter applies to you: DON'T BE AFRAID! It's really not that complicated and, after all, the good guys at FRIENDSHIP SYSTEMS are always there to assist you. (Did I mention, that the admin guide that is shipped with the SSHRM is really good and thorough?)

 

OK, so where were we, before I went off to tell you how great the SSHRM is? Right... the new stuff! In the next version (4.0 - yeah, baby!) it will be possible to not only run the external code on the remote computer, but run your whole optimization there!

 

Wait! ...

 

What? ...

 

Yeah, you read correctly!

 

... I know... how cool is that!? Well, of course at least one of your remote computers needs to be running CAESES to enable this option, so license costs may apply...

So, the workflow breaks down like this:

  1. Start a DesignEngine on your workstation
  2. Pause it
  3. Close the project
  4. If you have CAESES as an application in the SSHRM, you will be asked whether you want to send the project to the SSHRM and have the DesignEngine continue there.
  5. If you choose to do so, the SSHRM will select one computer (with CAESES installed) to run your DesignEngine and send your project there.
  6. Keep on working on a different project (or a copy of the same one or just do something completely different... I heard that "youtube" is supposed to be pretty good ;) ) on your workstation, your optimization will run in your personal "cloud" controlled by the SSHRM.
  7. Every once in a while, check in with the SSHRM through the webinterface in your browser to see the current state of your DesignEngine (basically the Design-Results-Table delivered to your web browser) ... dang.. I didn't even mention that nice new feature in 4.0, yet. I'll get back to that later,
  8. Once the DesignEngine has finished (or you decide that it should not run anymore), open the local copy of the project. This will automatically request the project's current state of the project from the SSHRM (of course, you have the freedom of choice. You can just continue the project where you left off). Yes.. this part may take some time, since all that result data needs to be copied back to your machine.
  9. If the DesignEngine finished remotely, you can now start post-processing. If it did not finish, you can keep it running locally, abort it, or even send it to the SSHRM again.

Number 6. seems to be interesting (imho)... So, in the next version, it will be possible to publish the current state of a running DesignEngine to the network. This means that <note: some technical stuff ahead> CAESES/FFW will start a webserver on a (configurable) port </end technical stuff> that allows to view the current state of the DesignEngine through a web-browser. This is independent of whether you use the SSHRM or not. As long as you are able to access the workstation where the optimization runs through a network, you will be able to monitor your DesignEngine in a browser (even with a dedicated mobile interface if you happen to use your smart-phone or tablet).

Even when the DesignEngine runs on your personal workstation, it is possible to "publish" it to the SshResourceManager. In that case the SSHRM will serve as a proxy to the computer running the DesignEngine, so all you need to do is access the web-interface of the SSHRM and it will give you the details of the optimization. Of course, it is possible to protect that web-access with a username and a password.

 

All of this will happen pretty much automagically!

Yes...

 

but ....

 

there are some things to keep in mind, when the DesignEngine utilizes external solvers (which it most likely will).

  • If those external processes run remotely through a SSHRM, the remote execution can only succeed if the remote computer running the DesignEngine has the same SSHRM configured as the originating workstation (this is actually the easy case. If this is given, it is possible to pause, close and send a project while a remote process on the SSHRM is running!)
  • If those external processes run locally, the remote computer needs to have the needed LocalApplications configured (preferably to run the desired program). Note that in this case any running processes need to finish before a) sending the project to the SSHRM and B), when requesting the project from a remote executor before the DesignEngine has finished, the system will wait for all local processes to finish before "giving the project back" (step 8).
  • In both cases, during sending, it is possible to run a check on the remote computers before sending the project to one of them (if multiple computers are configured to run CAESES/FFW only those will be considered that passed this check)
  • .... but... yes.... there is some human knowledge required in order to make sure that this works! We tried to automatize as much as possible, but only the user can tell if the external programs that will be launched are actually those that are meant.

As a side note (for those of you that are not even closely interested in running anything on any remote computers): In CAESES 4.0 it will be possible to really pause a DesignEngine, close the Project (or the whole program if desired), and open the project later with the DesignEngine continuing to run where you left off (the restrictions regarding external processes mentioned above still apply, though).

 

 

I know... (again) this was a pretty long read... I apologize! But, somehow I just can't keep it short if there is so much cool new exciting functionality to be introduced...

 

 

Please let me know, what you think of this, point out stuff that I got you confused with, or (I'd love this the most) give new ideas on taking this even further!

 

Thanks for reading, until next time!

 

Arne

 

P.S.: You made it... have a cookie!



0 Comments


Recommended Comments

There are no comments to display.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...