Jump to content
Mr. Henrik Philipsen

Trying to average imported .csv data.

Recommended Posts

Hi again,

 

I have completed some designs and evaluated them externally in StarCCM+. I have the .csv files copied back into

this directory with full path for one design instance as an example:

 

home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/02_Sobol/Sobol_02_des0003/computation/DragTotal.csv

 

I use the attached averaging definition, as I need to time average the data. But I get the below error message:

 

*** INFO FGenIntReader : Could not open file /home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/02_Sobol/Sobol_02_des0003/Runner//DragTotal.csv for reading [parsing process aborted]

 

Seems it is trying to get the data from another directory than where it should look.

I have tried to change the code, but without luck. Maybe you can assist me?

Kind regards, Henrik

 

 

Here is the bulk of the code:

 

 

// --- This feature creator has been automatically modified to suit the new syntax of if-commands. ---

double evalDummy(dummy)

 

int iter(numIterations)

 

string path(getDesignDir()+"/"+computation+"/"+fileName)

 

if (pathSwitch)

path=absPath

endif

 

FFileIn in(path)

 

if (NOT(in.openRead()))

echo("*** INFO " + this.getName() + " : Error Opening File")

goto(end)

endif

 

loopHeader:

 

FObjectList vals(in.readline().simplified().splitByRegExp(","))

 

if (vals.at(0).castTo(FString).toDouble()==0)

goto(loopHeader)

endif

 

double firstTimeStepAfterHeader(vals.at(0).castTo(FString).toDouble())

 

//----------------------------------------------------------------------

 

unsigned i(2)

 

loopNot:

 

in.readLine()

 

i+=1

 

if (i <= (numIterations - itRead))

goto(loopNot)

endif

 

//----------------------------------------------------------------------

 

double temp()

double sum(0)

 

loopRead:

 

FObjectList vals(in.readline().simplified().splitByRegExp(","))

 

temp.setValue(-vals.at(1).castTo(FString).toDouble())

 

sum+=temp

 

i+=1

 

if (i <= numIterations)

goto(loopRead)

endif

 

//----------------------------------------------------------------------

 

double average(sum/itRead)

 

in.close()

 

end:

computeAverageStar.fdfc

Share this post


Link to post
Share on other sites

Hi Henrik,

 

I guess your path is wrong, not sure (runner vs. computation).

 

Note that there is a inbuilt command for this, see the attachment for more information.

 

In the upcoming version 4.0, there will be easier mechanisms for this common task.

 

Cheers

Joerg

post-8-0-45493500-1436341229_thumb.png

post-8-0-20783300-1436341234_thumb.png

Share this post


Link to post
Share on other sites

Thank you very much Jörg,

That seems a very useful function. I am having a bit of problems implementing it though. I have the complete series of 30 designs that has run, then specifies DragTotal.csv in the result files from the baseline directory,

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/manual_results/baseline/computation/DragTotal.csv

 

Shouldn't it then automatically update the available DragTotal.csv file for each design variant already simulated externally, and with the the result file available in eg:

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/02_Sobol/Sobol_02_des0001/computation/DragTotal.csv

 

When I try to press "read existing result" files in the runner object window, I get this error message again:

 

*** INFO FResultsGenInt : File read error [Failed to open file /home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/manual_results/baseline/Runner/DragTotal.csv]

Even I specified the DragTotal.csv to be in:

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/manual_results/baseline/computation/DragTotal.csv

 

Somehow it specifies .../Runner/... as default dir even I have loaded the .csv file from the computation dir?

 

What am I doing wrong?

Share this post


Link to post
Share on other sites

Hi Henrik,

 

could you explain a little further what do you want to do and what the problem is?

Do you want to use the feature to import the data and to calculate the average? If you want to get the average from the csv. data, i would recommend to use the command, that Joerg has already posted.

 

So I guess your problem is to import the csv. data from the external simulation?

Do you want the average of all designs or for each design?

 

Cheers

Patricia

Share this post


Link to post
Share on other sites

Hello Patricia,

 

I am using Joergs command, and am getting the error messages in my previos post. It seems like it is looking for the .csv file in .../baseline/Runner/.... as default. I could move the files over to .../Runner ??

 

I just want to average over a specified number of timesteps for each result file DragTotal.csv, to be able to compare the designs

Share this post


Link to post
Share on other sites

Hey Henrik,

 

The Command from Jörg is:

Runner.getResults().getAverageOfLastRows("test.csv.table",2,0)

Runner is the Computation of the Software connector, which is in your case: not Runner its "computation"

 

So simply change Runner to computation and it should work, if the file has been defined in the software Connector.

Also note, that "test.csv.table" you have to change "test" to your csv file name.

 

This is your new line:

computation.getResults().getAverageOfLastRows("test.csv.table",2,0)

In 3.1.4 there is already a build in function to average columns.

Nevertheless it is easier if you define the csv file in the software connector in the Result Value section:

 

1) add the csv file in the software connector in the buttom right section "Result Value"

post-41-0-75688800-1436375985_thumb.png

2) Double click the file

3) Click on the green "+" button to create a parameter

4) check the "Average" box

5) enter a number next to the checkbox, which will be the number of rows to average

6) choose the column

post-41-0-32450200-1436375991_thumb.png

7) click on the blue symbol this will add/create the parameter in you objecttree

This value will always be evaluated after a finished run, if the file exists in the "computation" folder

 

I hope this will solve your problem if not we will find a way to get it done.

 

A last hint because I think you try to change the baseline design after you have already run the design engine.

The objects you create in the baseline after a design engine has been finished, will not be added to the existing results.

If you want that changes you have to run the design engine again.

 

Cheers,

 

Karsten

Share this post


Link to post
Share on other sites

Thank you very much Jörg,

That seems a very useful function. I am having a bit of problems implementing it though. I have the complete series of 30 designs that has run, then specifies DragTotal.csv in the result files from the baseline directory,

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/manual_results/baseline/computation/DragTotal.csv

 

Shouldn't it then automatically update the available DragTotal.csv file for each design variant already simulated externally, and with the the result file available in eg:

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/02_Sobol/Sobol_02_des0001/computation/DragTotal.csv

 

When I try to press "read existing result" files in the runner object window, I get this error message again:

 

*** INFO FResultsGenInt : File read error [Failed to open file /home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/manual_results/baseline/Runner/DragTotal.csv]

Even I specified the DragTotal.csv to be in:

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/manual_results/baseline/computation/DragTotal.csv

 

Somehow it specifies .../Runner/... as default dir even I have loaded the .csv file from the computation dir?

 

What am I doing wrong?

Hey Henrik,

 

I have highlighted the important differences in the quote above.

 

That is related to my former post. In the parameter you reference RUNNER but your results are in the folder COMPUTATION.

That is the reason why the read file error occurs.

 

Some more details about designs:

 

And even if you change now the Baseline design these changes will not change the already existing Designs. Because they are all stand alone designs which are not any more connected to the baseline design.

 

If you open a design (double click) and try to change them you notice that you cannot change them because the arguments of the objects are grey.

If you want to change a design you need to "unlock" it than you can change this design add new parameters etc.

 

If you want more details about why we do it. I can elaborate a bit more.

 

Cheers,

 

Karsten

Share this post


Link to post
Share on other sites

Hi Karsten,

 

Yes, it is a Sobol series that I have already run the simulations on (3 days appr. wall clock runtime on a cluster), and the results are already available in (for one design instance):

/home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/02_Sobol/Sobol_02_des0001/computation/DragTotal.csv

 

So I need to read the DragTotal.csv and average the last 397 timesteps for each instance and then get it to update the Sobol Design list with the values.

I cannot get it to accept the new set result value in the "evaluations" of each design after the first run, even after unlocking all variants? Is that possible?

 

Is there a default directory structure for it to find the result values? Because it keeps looking through Runner.

And the command

 

 

computation.getResults().getAverageOfLastRows("DragTotal.csv.table",397,1)

 

When I try to assign the above command in a FParameter in one individual design variant (no 15 this case) inside the CAD edit menu of this design no, I get the following error:

 

*** INFO FGenIntReader : Could not open file /home/henrik/Documents/Dropbox/HullOptimDB/Friendship/FeSTL02/02_Sobol/Sobol_02_des0015/Runner//DragTotal.csv for reading [parsing process aborted]

 

I have loaded the DragTotal.csv file from the baseline design into the "Result files" and "Result values" field in the software connector, shouldn't that be sufficient for all following designs to do the same, when the direcotory structure is the same?

Share this post


Link to post
Share on other sites

Hey Henrik,

 

Is it possible that you share your project file and the results with me? Send me a download link

if it is to big just send me the project file and the baseline and one or two sobol design results.

 

Then I have the change to check it myself what the problem might be.

 

In addition:

 

Even if you unlock the design the changes of the baseline will not be in the already evaluated and finished design.

 

I have an idea how to realize it but i would like to test it with your project

and if i works I will explain how I have done it.

 

I can send you an upload link to our cloud service if you don't have enough space in your dropbox folder.

 

Cheers,

 

Karsten

Share this post


Link to post
Share on other sites

Here is another example where we needed to read a result file from FLOW-3D. The goal was to find the minimum value in this file, plus the corresponding row index.

 

For this task, a simple feature can be used. In this example, the identifier is the word "time" i.e. after the row with this expression, the data starts and we can iterate (and search).

 

The result file itself is an entry in the software connector (section "Result Files"). You get the full name incl. design directory as a string via

Runner.getResults().getResultFileName("myFileWithResults.txt")

 ... if the name of your computation is "Runner" (otherwise, use another one, of course). This string is input to your feature.

 

You can then reference the minimum value by putting it into a parameter (e.g. for setting up an objective function for an optimization).

 

Hope this helps...

Joerg

post-8-0-73558200-1457536400_thumb.png

post-8-0-93658700-1457536406_thumb.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...