Menu

Discussion Forum

New Discussion
Filter

Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.

How to properly export and import data?

Posted Apr 3, 2017, 3:49 a.m. EDT 3 Replies

Ton Nguyen
Send Private Message Flag post as spam

Please login with a confirmed email address before reporting spam

Hi, I have two models where the second one uses results of the first one.

I export the results of model one to a spreadsheet of 4000x4000 points (regular grid), corresponding to a resolution of 0.5 nm. This results in a huge csv-file of almost 1 GB. I then import and plot this file into the model two via an interpolation function, but this can take up to 15 minutes. This is all done on a dedicated Linux server with 128 GB memory, which is normally very fast.

Is there a better way for me to export/import data between models?
3 Replies Last Post Apr 13, 2017, 4:05 a.m. EDT
Gunnar Andersson COMSOL Employee

Please login with a confirmed email address before reporting spam

Posted: 7 years ago Apr 7, 2017, 8:41 a.m. EDT

I export the results of model one to a spreadsheet of 4000x4000 points (regular grid)
...

Is there a better way for me to export/import data between models?


What data format are you using - Spreadsheet or Grid? The Spreadsheet format gives a file that is easy to process in external software, but the Grid data format gives a smaller file which is processed much faster when you use it in an interpolation function in COMSOL Multiphysics.

[QUOTE] I export the results of model one to a spreadsheet of 4000x4000 points (regular grid) ... Is there a better way for me to export/import data between models? [/QUOTE] What data format are you using - Spreadsheet or Grid? The Spreadsheet format gives a file that is easy to process in external software, but the Grid data format gives a smaller file which is processed much faster when you use it in an interpolation function in COMSOL Multiphysics.
Nicholas Goldring Certified Consultant
Send Private Message Flag post as spam

Please login with a confirmed email address before reporting spam

Posted: 7 years ago Apr 7, 2017, 7:00 p.m. EDT

Hi, I have two models where the second one uses results of the first one.

I export the results of model one to a spreadsheet of 4000x4000 points (regular grid), corresponding to a resolution of 0.5 nm. This results in a huge csv-file of almost 1 GB. I then import and plot this file into the model two via an interpolation function, but this can take up to 15 minutes. This is all done on a dedicated Linux server with 128 GB memory, which is normally very fast.

Is there a better way for me to export/import data between models?

I think that if it's absolutely necessary for you to have that kind of resolution then there probably isn't a way around how long it's going to take for the import. You can always specify the step-size of your grid and cut down the number of data points. Also, when you say that you take data from one model and import it in to another are you talking about two separate model files or two components?

Edit: I wrote my response before reading Gunnar's comment - I have only ever imported data using the Grid format and although I don't have 4000X4000 data points, I can tell you that importing with something around 500x500 in grid format is almost instant on a machine with 32 GB memory.
[QUOTE] Hi, I have two models where the second one uses results of the first one. I export the results of model one to a spreadsheet of 4000x4000 points (regular grid), corresponding to a resolution of 0.5 nm. This results in a huge csv-file of almost 1 GB. I then import and plot this file into the model two via an interpolation function, but this can take up to 15 minutes. This is all done on a dedicated Linux server with 128 GB memory, which is normally very fast. Is there a better way for me to export/import data between models? [/QUOTE] I think that if it's absolutely necessary for you to have that kind of resolution then there probably isn't a way around how long it's going to take for the import. You can always specify the step-size of your grid and cut down the number of data points. Also, when you say that you take data from one model and import it in to another are you talking about two separate model files or two components? Edit: I wrote my response before reading Gunnar's comment - I have only ever imported data using the Grid format and although I don't have 4000X4000 data points, I can tell you that importing with something around 500x500 in grid format is almost instant on a machine with 32 GB memory.
Ton Nguyen
Send Private Message Flag post as spam

Please login with a confirmed email address before reporting spam

Posted: 7 years ago Apr 13, 2017, 4:05 a.m. EDT


I export the results of model one to a spreadsheet of 4000x4000 points (regular grid)
...

Is there a better way for me to export/import data between models?


What data format are you using - Spreadsheet or Grid? The Spreadsheet format gives a file that is easy to process in external software, but the Grid data format gives a smaller file which is processed much faster when you use it in an interpolation function in COMSOL Multiphysics.


Thanks, that did the trick! I was using Spreadsheet before, but Grid is significantly faster to the point that loading and plotting is almost instant.
[QUOTE] [QUOTE] I export the results of model one to a spreadsheet of 4000x4000 points (regular grid) ... Is there a better way for me to export/import data between models? [/QUOTE] What data format are you using - Spreadsheet or Grid? The Spreadsheet format gives a file that is easy to process in external software, but the Grid data format gives a smaller file which is processed much faster when you use it in an interpolation function in COMSOL Multiphysics. [/QUOTE] Thanks, that did the trick! I was using Spreadsheet before, but Grid is significantly faster to the point that loading and plotting is almost instant.

Note that while COMSOL employees may participate in the discussion forum, COMSOL® software users who are on-subscription should submit their questions via the Support Center for a more comprehensive response from the Technical Support team.

Suggested Content