Mr. Arne Bergmann

Wow, has it been that long since my last entry? Well, I have a very short one with a big attachment. I always wanted to write something about "best practices" for Feature programming. That "something" became larger and larger and it became pretty clear that it's too large for a "regular" blog post. That's why I decided to create a PDF file, instead. In the end it turns out to be a 37 page booklet instead of an article. I compiled some techniques that I consider to be the most effective for creating Feature definitions with good performance. So, please take a look at "Effective FPL" and let me know what you think about it in the comments. Merry Christmas and a good start into 2016!

Those of you that attended last year's user's meeting in Berlin already got a glimpse of some pretty exciting new functionality. For those of you that couldn't make it and because it has progressed into an actual releasable feature (once it has passed the testing phase, that is), let me give you a short introduction. Ok, I am sure all of you know the SshResourceManager, right?! ... NO?! Hmpf, OK. The SshResourceManager (SSHRM) is a light-weight grid-engine shipped with CAESES that allows you to run your external (CFD) solvers on remote computers (sorry, it is not usable with the CAESES free). So, for example, your solver only runs on Linux but you prefer working on Windows with your workstation? No problem, use the SSHRM to act as a bridge to that Linux machine. Your external code should run on that super powerful 128 core machine in your basement? Use the SSHRM to start it there. You have a bunch of computers standing around in your office and all those CPUs/cores inside of them are bored all the time? Use the SSHRM to put those CPUs to good use. The only requirement is, that the computer(s)s you want to run your external code on is (are) accessible through SSH (hence the name SSHResourceManager). Don't worry, there are SSH-servers for Windows, too... The SSHRM is shipped with a detailed admin guide, so the setup should be pretty straight-forward. Don't be scared (it's no Voodoo, really - and we are always willing to help). So, now that we got that out of the way, let's get to the point of this entry: The SSHRM is great to distribute the workload of your external solvers to the resources you have available in your network. It does not only keep track of the CPUs that are currently in use, but also helps you to make sure those expensive licenses for your CFD programs do not go idle (but are not overused at the same time). The SSHRM may even be helpful if you do not want to use remote computers at all, but just need to make sure that your floating licenses users do not block each other. Even if the actual program runs locally on the engineer's machine. I'm drifting off my actual target here, but I just have the feeling too few people know about these possibilities or are afraid of setting up this system. If the latter applies to you: DON'T BE AFRAID! It's really not that complicated and, after all, the good guys at FRIENDSHIP SYSTEMS are always there to assist you. (Did I mention, that the admin guide that is shipped with the SSHRM is really good and thorough?) OK, so where were we, before I went off to tell you how great the SSHRM is? Right... the new stuff! In the next version (4.0 - yeah, baby!) it will be possible to not only run the external code on the remote computer, but run your whole optimization there! Wait! ... What? ... Yeah, you read correctly! ... I know... how cool is that!? Well, of course at least one of your remote computers needs to be running CAESES to enable this option, so license costs may apply... So, the workflow breaks down like this: Start a DesignEngine on your workstation Pause it Close the project If you have CAESES as an application in the SSHRM, you will be asked whether you want to send the project to the SSHRM and have the DesignEngine continue there. If you choose to do so, the SSHRM will select one computer (with CAESES installed) to run your DesignEngine and send your project there. Keep on working on a different project (or a copy of the same one or just do something completely different... I heard that "youtube" is supposed to be pretty good ;) ) on your workstation, your optimization will run in your personal "cloud" controlled by the SSHRM. Every once in a while, check in with the SSHRM through the webinterface in your browser to see the current state of your DesignEngine (basically the Design-Results-Table delivered to your web browser) ... dang.. I didn't even mention that nice new feature in 4.0, yet. I'll get back to that later, Once the DesignEngine has finished (or you decide that it should not run anymore), open the local copy of the project. This will automatically request the project's current state of the project from the SSHRM (of course, you have the freedom of choice. You can just continue the project where you left off). Yes.. this part may take some time, since all that result data needs to be copied back to your machine. If the DesignEngine finished remotely, you can now start post-processing. If it did not finish, you can keep it running locally, abort it, or even send it to the SSHRM again. Number 6. seems to be interesting (imho)... So, in the next version, it will be possible to publish the current state of a running DesignEngine to the network. This means that <note: some technical stuff ahead> CAESES/FFW will start a webserver on a (configurable) port </end technical stuff> that allows to view the current state of the DesignEngine through a web-browser. This is independent of whether you use the SSHRM or not. As long as you are able to access the workstation where the optimization runs through a network, you will be able to monitor your DesignEngine in a browser (even with a dedicated mobile interface if you happen to use your smart-phone or tablet). Even when the DesignEngine runs on your personal workstation, it is possible to "publish" it to the SshResourceManager. In that case the SSHRM will serve as a proxy to the computer running the DesignEngine, so all you need to do is access the web-interface of the SSHRM and it will give you the details of the optimization. Of course, it is possible to protect that web-access with a username and a password. All of this will happen pretty much automagically! Yes... but .... there are some things to keep in mind, when the DesignEngine utilizes external solvers (which it most likely will). If those external processes run remotely through a SSHRM, the remote execution can only succeed if the remote computer running the DesignEngine has the same SSHRM configured as the originating workstation (this is actually the easy case. If this is given, it is possible to pause, close and send a project while a remote process on the SSHRM is running!) If those external processes run locally, the remote computer needs to have the needed LocalApplications configured (preferably to run the desired program). Note that in this case any running processes need to finish before a) sending the project to the SSHRM and B), when requesting the project from a remote executor before the DesignEngine has finished, the system will wait for all local processes to finish before "giving the project back" (step 8). In both cases, during sending, it is possible to run a check on the remote computers before sending the project to one of them (if multiple computers are configured to run CAESES/FFW only those will be considered that passed this check) .... but... yes.... there is some human knowledge required in order to make sure that this works! We tried to automatize as much as possible, but only the user can tell if the external programs that will be launched are actually those that are meant. As a side note (for those of you that are not even closely interested in running anything on any remote computers): In CAESES 4.0 it will be possible to really pause a DesignEngine, close the Project (or the whole program if desired), and open the project later with the DesignEngine continuing to run where you left off (the restrictions regarding external processes mentioned above still apply, though). I know... (again) this was a pretty long read... I apologize! But, somehow I just can't keep it short if there is so much cool new exciting functionality to be introduced... Please let me know, what you think of this, point out stuff that I got you confused with, or (I'd love this the most) give new ideas on taking this even further! Thanks for reading, until next time! Arne P.S.: You made it... have a cookie!

It can be quite frustrating... You have an unbelievable powerful computer with the latest CPU. However, when using CAESES, you need to wait for your model to update (or your project to load) while you see that only one of those eight CPU-cores is actually busy and the others are playing a round of poker?! If you know this feeling, this post is for you! (If you do not know this feeling... read the post anyway... big things are coming our way ;) ) Those that attended last year's European User's Meeting already got a short glimpse into CAESES' future from Stefan Harries' talk. So, I figured, I could elaborate a little... So, deep down in our "labs" we are working on the "next generation" of CAESES (I will, from now on, refer to it as ngCAESES, in lack of a better name just to make it easier for me - no, I will not throw any versions number out there). While this "next generation" does not include awesome new features (well, of course it does, but we don't need a "next generation" for that, we just throw those great new features out there as they come along ;) ), there are a lot of changes on the inside - the heart of CAESES. However, it is not only different, but also better. The main benifit for you - the user - is, that everything will be a lot faster (provided that you have a multicore CPU). Before going into some (boring - and maybe also painful) details, I want to give you some numbers... You probably know the "Bulk Carrier 55000 Tdw" sample that is shipped with CAESES (if you don't, you are probably not a naval architect, but even then you should check it out; it's a great model built by one of our most talented engineers). So, in the current release (3.0.17 Windows 64 Bit) the model takes about 14 seconds to load on my machine. When I open that project in the ngCAESES I am up and running after 2.5 seconds! (Disclaimer: The mentioned version is in "hot" development, i.e. a lot is happening and the current state is very temporary. The mentioned timing will most probably change for the actual release - hopefully it will go down even further). Of course, this does not only affect project loading but also model update times and overall responsiveness, but project loading is the easiest thing to measure.... So what do we do to achieve this? Does this mean the ogCAESES ("old generation") did bad things and we finally fixed that? No! It wasn't bad, it's just that now it's better ;). Ok, so we actually took several steps (and are still taking more) to achieve this. The main idea is to utilize the available hardware as much as possible. Watch out! Geek-Speak ahead! The thing about that hardware is, that modern CPUs are not very fast considering clock rate. Well... of course they are blazingly fast compared to old 486 CPUs, for example, but compared to, for example, the Pentium 4 Prescott CPU from !2004! which had clock rates from up to 3.8 GHz, the clock rate did not go up at all in the last 10 years (I know, today there are more instructions per cycle, higher FSB, yadda, yadda, yadda... still, the point is valid). There are several reasons why the clock rate could not be increased anymore. The two main points were (and are) thermal problems and energy consumption (of course, the two go hand-in-hand). So, instead, CPU manufacturers figured it would be a better idea, to put several processors on one chip. That way, they cannot process a single task faster, but perform multiple tasks at the same time at the same speed as the single task. For us software developers this was "black friday". Afterall, it means that we do not get speed improvements for free anymore, but we actually need to work for it, because in order for the CPU to execute our software faster, it needs to be split into several individual task that can be run in parallel. End Geek-Speak (for now... maybe more later ;)) The following picture (which was also shown at last year's User's Meeting in Berlin) tries to illustrate the difference: So, what we can see is, in the current version (up top in the picture) objects are updated serially - ordered by their dependency. Some objects already make use of the multiple cores the CPU provides, as they calculate some stuff in parallel (the last box), but that is rare. NgCAESES (bottom - labeled as CAESES 4, but I, personally, don't want to throw any version numbers around) builds a graph of dependent objects, and different branches of that graph are then updated in parallel. Additionally, a lot more objects have their internal algorithms parallized. This - as you can clearly see in the picture - leaves you - the engineer - a lot more time to travel, relax, and party ;) With great power comes great responsibility Yep... that's true. It cannot be all gold and shiny. But, the nice thing is, the drawbacks will only affect those people who were already told by us that what they are doing is "evil" or "out of the specification" in the past - I am pretty sure that (if they are reading this) they will know that I am addressing them... Ok, so we needed to limit the freedom and flexibility the user had so far a little. You cannot use features to "mess everything up anymore" ;). The major change for the user (which "average Joe" will not even notice) is that we introduced the notion of "constness". Those of you that know C++ (or C#, or Ada, or D, or one of the other programming languages that make an explicit distinction between constant and mutable expressions) will know that this can be quite painful, but also very good for guaranteeing that nothing bad is happening in your code. So, what did we actually change for the user? What is not possible anymore? You cannot call a command that modifies the object it is called on anymore, if that object is labeled as "const" and the command isn't. You cannot pass an object that is labeled as "const" as an argument to a command that takes a non-const argument. That's it..... I can literally see your face right now: :blink: :blink: :wacko: :unsure: :wacko: :blink: :blink:. So. please let me explain: When is an object const? I think that is best explained through an example: You can ask a F3DPoint for it's global vector by using the command FVector3 FPoint.getGlobalVector() The return value of that command is (now) const. Why? Easy: The command returns a value that is a derivative of the point's input values (the x, y, and z coordinate). Changing that return value would not result in a change of those values. They are what define the point and also the global vector. To make this clear, the return value is now marked as const and will result in an error when trying to modify it. So in the past it was actually possible to write nonsense like this (provided there is a F3DPoint called p1): p1.getGlobalVector().setX(8) It was valid, but didn't do anything. Of course, the "didn't do anything"-part means it didn't do any harm, but if the user (i.e. YOU) would execute something like this and not get an error message, the understandable assumption would be that it DOES do something, is quite understandable. But, not only is the return value of "getGlobalVector" const, but the command itself is also const, because it does not modify the object it is called on (i.e. the point). So in ngCAESES the complete signature for that command is const FVector3 FPoint.getGlobalVector() const The first "const" refers to the return type. The returned object cannot be modified. The "const" at the end refers to the command in respect to it's receiver (i.e. the point it is called on). As stated, that example is harmless, but there are several not so harmless cases, that lead to undesired behavior. One more example: Let's assume you have a point p1. Let's further assume you have a parameter pm1 that defines the x-coordinate of that point. Now, for whatever reason, the user enters the command "p1:x = 9" in the console which is supposed to set the x-coordinate to the value 9. Well... it does set the x-coordinate to the value 9 (i.e. the point will actually be there), but it will leave you with the following undesired state: So what do we have here? The parameter pm1 has the value 1, the x-coordinate of the point is still set to be said parameter, but the actual value is 9?! That can't be right! So what happened? Well, without going into too much detail: p1:x returns a FDouble, which is then assigned to using the '=' operator. Nothing bad about that. BUT! p1:x does not really have a double as a value, but an FParameter. That FParameter is (automagically) converted into an FDouble (while the conversion is pretty straight forward in this case, it's still neccessary). So the p1:x actually returns a temporary FDouble that is the result of the conversion of the parameter pm1 to a (temporary) FDouble. And now, the operator '=' assigns the new value '9' to exactly that temporary FDouble. In turn, this means that the wrong state is resolved after changing the parameter (because the temporary FDouble will be replaced by a new temporary FDouble) or after re-opening the project, but it's still bad that this state can occur. Ok, to be honest, I probably lost most of my readers (if you even got this far) with that last paragraph... Read it again... It might just make sense eventually... So, the main point is: we are trying to protect you from experiencing something unexpected when using our software! We are well aware of the pitfalls, but we are also aware of the great power and flexibility we offer. So, we want to keep the flexibility and power while reducing the possible pitfalls (now that's something the C++ commity should have decided at least 10 years ago). That's why in ngCAESES stuff like assigning a value to p1:globalX will not be possible anymore. How does this affect me? Short answer: It doesn't (or it shouldn't). If you do get an error because of this (and you should not), it means that in the past you were doing something that had (in the best case) no effect (like the .getGlobalVector().setX(9) case) or really, really bad effects (like the p1:x = 9 case). With regards to Features there are some things, however, that used to work and will no longer work.To be honest, those things are things, that we never promoted and even told those that used them that they shouldn't. I am kind of hesitating to include them in this post, since I do not want to give those people that didn't do this or didn't know that they could do this any bad ideas... Ok, I am still gonna do it... First of all, it will no longer be possible to change (i.e. call non-const commands on) arguments of a Feature within the Feature. Why? Watch out! Geek-Speak ahead! (but this time all CAESES users should understand it...) Well, technically, the Feature is a client of the argument. When the argument changes its value, the Feature needs to update. So, changing the argument value within the Feature actually creates an untracked recursion. The only reason it worked until now was that there was a safety-net in the code that made sure that objects cannot go out of date while they are updating. But... it also left the whole model in an undefined state. Some other objects that were clients of the object that the feature changed may have already been up to date, and they never noticed that the object has already changed its state... This is not good! Kind-of done with geek-speak, but the following may still be kind of tech-heavy! So... we are totally disallowing to change arguments of a feature within a feature. This will not be the case in the upcoming 3.1 release, but I hope, it will be in the one after that! Ok, so those Features that I know that did change their arguments did this for one - and only one - reason: display the user some additional info. Since we know and understand, that that is a good thing, we have introduced a new thing (not yet, but in the release that will include all that "const" stuff): You can now tell a feature argument that it can act as a label. The "advanced options" of feature arguments have 5 more fields (names are subject to change - so don't quote me here): - "Display as label command": Here you can specify under which circumstances the given argument is supposed to display a value and nothing but a value... The value displayed has nothing to do with the value of the argument. If it's supposed to be a value only, you would set this to "true" or if it depends on the state of the model or other input of the feature you can use the usual command syntax. If this command evaluates to "true", the editor for this argument will be hidden and a label will be displayed instead. - "Label Content Command". If the above determines that this argument is displayed as a label... here you can say what the label is supposed to display. This can be just a string, or any other object. If it's anything but a string, the usual display as a string will be used that applies to the type (e.g. if it's a fvector3, it will be displayed as [1,2,3] if it has x = 1, y = 2, z = 3). This label is very flexible! It can even display HTML! So, go crazy with it! - "Keep Label UpToDate" If this is set to true every object that is needed to determine the value of the previous two commands is automatically updated as soon as their value changes (same as automatic update for parameters/constraints/features: I do not recommend to use this option - or better: use at your own risk!) - "Hide Check Box for Label". If the attribute has a "Display as label Command" (see above) and that command evaluates to "true", the checkbox (see the documentation about the advanced options of feature arguments in versoin 3.1) will be hidden. - "Hide Buttons For Label" If the attribute has a "Display as label Command" (see above) and that command evaluates to "true", all buttons configured for the attribute (see documentation about the advanced options of feature arguments in version 3.1) will be hidden. Wow... this is a terribly wrong post.... To all of you that are still here... HAVE A COOKIE!!! OR TWO!!! I just wanted to share some of the new development with you - that I am personally very excited about. Also, I wanted to warn some of you Feature-Artists that some of the freedom is coming to an end ;)

When using an external program with CAESES, there is one major requirement to that program: Once the executable is finished, all result files need to exist. This is true no matter whether the external software is called directly as a local application or through the SshResourceManager as a remote application. Why is that so? CAESES needs to know when the external program is done writing its results, in order to be able to read in those results. So, what it does is to assume that once the executable that was called returns, all results are available. What does that mean for me? In general you can safely assume that a program's executable will not return until the actual process is finished. However, there are some exceptions to that rule. One example for such a behavior (although it is highly unlikely that it will be used from within CAESES) is Firefox. Only the first instance of Firefox will behave in the required way. When starting a second instance of Firefox, the process will return right away, as it uses the original Firefox process to run. Another example that is a lot more likely to be used from within CAESES is when trying to execute an external program through a grid engine, for example the Oracle Grid Engine (formerly Sun Grid Engine). In general, submitting jobs (i.e. requests to run a certain program) to a grid engine through a program like qsub is a non-blocking process. The job is added to the waiting queue of the grid engine and the submitting program will return right away. Additional tools (e.g. qstat) can then be used to monitor the state of the job. However, when using qsub or a similar program as the executable in CAESES, it will be recognized that the executable has returned and assumed that all result files are present, which is not the case. In the example of the Oracle Grid Engine there is a way to tell qsub that it should not return until the job has finished. That execution mode is done by providing the "sync y" option to qsub (either on the command line or in the configuration file). Note that the Sun Grid Engine must be configured to allow blocking jobs (i.e. must allow to use the "sync y" option). How can I find out if my executable is affected? Well, if you never experience the problem that CAESES complains about result files not being there, although they are when you look in the file browser, or that a computation finished although you know that the external program is still running, you don't need to worry about it. If, however, you do experience one of those two problems, or are planning to use a grid engine like the Oracle Grid Engine, you can do the following to ensure that the described problem does no affect you: Run the program from a terminal (cmd.exe on Windows). If the terminal returns to the command prompt right away although the program continues to run, you have identified the problem. As the Firefox example shows, even if it does not return right away, you may have to double check by starting a second instance of the program through another terminal. If you have identified, that your external program indeed shows this behavior and is the problem why your integration does not work, one way to solve the problem, is to wrap the call to the executable in a shell script that stores the process ID of the started exectuable in a variable and periodically looks whether the PID still exists. Once it does not exist anymore, the shell script should exit. An alternative to a shell script would be to write a simple program using your favorite programming or scripting language.

The attached project shows the export feature I presented at this year's User's Meeting. Its purpose is to show how to use Features to export and import custom file formats using Features. Additionally, it shows how to connect such an export to a SoftwareConnector. [update 4.0]: Please read the second post in this topic to see the improvements regarding connecting your export feature with the Software connector. The geometry is based on the "Basic Hull Model" sample. There are two FStructPanelMeshes (main and stem) that are grouped in a FPanelMeshGroup and transformed using the combination of FImagePanelMeshGroup and FScaling. Running the Feature "exporter" will export the scaled PanelMeshes to a file (see the console output for the full path). The Feature "importer" will import files of the correct format (e.g. one that was exported using the "exporter") to structured panel meshes. The "success" value that is calculated in the export Feature is added to the configuration of a SoftwareConnector, so that it is ran prior to running the external computation. [update]: In 4.0 this is no longer necessary (see next post). Notes: The export feature is written to be used within a SoftwareConnector. In order to make sure that the feature is triggered before running the computation, it takes the file path as an argument with the value set to "getInputDir()". That command yields the input directory of the currently running computation. By setting that command as an argument value, the feature becomes dependent (i.e. a "client") of the so-called PathManager. This ensures, that the export will be triggered before running a computation. If you execute the feature outside of the computation-run the exported file will not be in the correct location for your external program. [update]: In 4.0 this is no longer necessary (see next post). Using "getInputDir()" instead of "getResultsDir()" (which gets the actual working directory of the external process) will export the file to the "input" sub-directory of the computation's results directory. This is done in order to allow the feature to be used in conjunction with remote executions of computations using the SshResourceManager. [update]: In 4.0 this is no longer necessary, files referenced with a relative path are automatically written into the input directory (see next post). featureExport.fdb

The editor for the Create Function of a feature has some (so far) undocumented (which will change in 3.0) possibilities that can come in handy. So, here's a list of things that you may not be aware of: Copy/Cut whole lines If you press the keyboard shortcut for copy or cut (i.e. Ctrl+c and Ctrl+x) without having any text selected, the whole line will be copied/cut. If you then paste it, it will be inserted as a whole line Select words Select words works by double clicking on them. Nothing new? Do the same by single clicking an pressing CTRL. Find next occurence When using the text search (Ctrl+f) you can press F3 to find the next occurence of the search string Help By pressing F1 (or using the question mark button in the toolbar) the documentation browser will open the documentation (if available) of the type/command that is currently under the cursor. Debugging shortcuts The debugger can also be controlled by keyboard shortcuts: F5 : start debuggingF9 : toggle breakpoint in the current lineF10 : execute next lineShift + F5 : stop debuggingMore shortcuts Additional shortcuts: F7 : Evaluate the feature (same as the "Evaluate" button on the lower left)Ctrl+F7 : Apply the changes to the feature (same as the "Apply" button on the lower right)Collapse code You may have already noticed that parts of the code can be collapsed by pressing on the little arrow on the left of the code (in between the code and the line numbers). This applies to "block"-statements (e.g. if, while) and can help to get a clearer view of the code as parts that you are currently not interested in can be hidden. What you probably do not know is the ##region/##endregion functionality. This way you can define collapsable regions yourself, e.g.: ##region myRegion // this is some code point p() line c() ##endregionThe identifier behind the ##region directive allows you to give a name to the region, so you see what's inside when collapsing it. So the example can be collapsed to ##region myRegion { . . . }Of course, this only works if collapsing is enabled in the editor settings. Column select When using the mouse to select text it is not only possible to select lines, but you can also create a selection that only spans part of a line but expands to other lines. To do so, hold Ctrl+Alt while selecting with the mouse. This can come in handy when you want to change the type or name of multiple objects, for example. These screenshots should illustrate it a little better: feature defining 4 points: Select using Ctrl+Alt: Type fvector3: That's about it for now, however here's a little teaser about the news in the next release: Auto close blocks The editor automagically adds the corresponding keyword when starting a new block statement (e.g. if/while). New language features The feature language has been expanded a little further. The biggest news are: foreach -> iterate over objectlists and entitygroups in a very convenient way. Forget about all those castTo commands and counter variables! You probably know code like this: unsigned i(0) while(i < myObjectList.getcount()) f3dpoint p (myObjectList.at(i).castTo(F3dpoint)) if (!(!p)) p.setx(5) endif i += 1 endwhileNow, you can just write this: foreach(f3dpoint p in myObjectList) p.setX(5) endforswitch -> getting lost in all those if/elseif cascades? The new switch statement should solve that. unsigned i(2) switch (i) case 1 echo("i is 1") case 2 echo("i is 2") case 3,4,5 echo("i is 3, 4 or 5") default echo("i is larger than 5) endswitchfunction -> writing the same code inside your feature over and over again? Put it in a function and just call it! Example: function calculatePlusTwo(double d) : fdouble return(d + 2) endfunction echo(""+calculatePlusTwo(1)) echo(""+calculatePlusTwo(2)) echo(""+calculatePlusTwo(3))You can even reimplement existing commands inside your feature: function echo(string text) ::echo("My Feature" + text) endfunction echo(" wants to say hello")will output: "My Feature wants to say hello" on the console