FEFLOW model run (s) can be executed in the cloud using DHI MIKE Cloud infrastructure. The deployments of the model run (s) is a one-click solution directly from FEFLOW and/or FePEST graphical interfaces through the Cloud Simulations panel. DHI MIKE Cloud offers a variety of virtual machines, which are intended to cover a set of modelling applications.
Accessing cloud resources through FEFLOW and FePEST requires at least the installation of FEFLOW 7.5.1. All the necessary software components are installed together with the FEFLOW installation package. Beside the FEFLOW installation, the new Cloud Simulations panel requires the installation of .NET Desktop Runtime 6.x (x64) and ASP.NET Core Runtime 6.x (x64).
If the .NET installation is missing, FEFLOW will automatically deactivate the panel. The reactivation of the Cloud Simulations panel can only be done subsequently through the FEFLOW Global Settings (menu Tool).
A company subscription to the MIKE Cloud is required for accessing MIKE Data Admin (DHI's cloud storage portal) and MIKE Cloud Engine Execution services. If you are unsure about your access rights or whether your company has access to a subscription, you can contact the technical support at firstname.lastname@example.org for any question.
Cloud Interface option in the FEFLOW installation package.
Overview of functionalities
In FEFLOW, the Cloud Simulations panel is activate / deactivate as any other GUI panel through the menu View - Panels. Cloud simulations can be executed individually, i.e. one simulation instance per virtual machine (VM), or several FEFLOW models within one VM.
Cloud Simulations panel in FEFLOW.
Before getting started, you need a proper account to access the cloud services. You must log in directly through the panel (option at the uppermost right corner). The log-in operation can only be done through a company email address linked to the MIKE Cloud subscription.
MIKE Cloud project folder
When using the Cloud Simulations panel, the FEFLOW *.fem and *.dac files are uploaded to a specific project storage in DHI MIKE Data Admin. The project folder is user-defined and can contain one or more FEFLOW projects. For every cloud run, the result folder within the selected MIKE Cloud project folder is created. The name of the result folder consists on the <name of the FEFLOW model> + <_output>.
The details about the operations and workflows in MIKE Data Admin are presented here. After the FEFLOW cloud simulation is finished, it is recommended to download the result files locally and delete the *.dac files from MIKE Data Admin. The user has the full responsibility to maintain the cloud storage and not exceed the maximum capacity provided within the company MIKE Cloud Subscription.
MIKE Data Admin offers three project access levels: Confidential, Private and Shared. The access level is defined in the Data Admin portal and not in the FEFLOW / FePEST GUIs. In case several users are collaborating, the MIKE Cloud Project needs to be shared throughout all the users in order to make it accessible for the FEFLOW runs.
A FEFLOW model, which is to be used for the cloud execution, can be defined using any of the three options:
Current: The actual document opened in the FEFLOW GUI.
User-defined (local): A *.fem document located and reachable in a local directory.
User-defined (cloud): A *.fem document previously uploaded to the cloud and within the selected MIKE Cloud project folder.
In this release, we support cloud simulations using FEFLOW version 7.4, 7.5, and 8.0 engine.
The Configuration list indicates the virtual machine where the FEFLOW model will run. There are several configurations available and suitable for both FEFLOW and FePEST. Basic information such as memory, number of CPUs and number of GPUs is listed as reference together with the configuration name. A detailed hardware information is also available via a tool-tip displayed under each configuration name.
Note that a larger VM configuration would not necessary speed-up the model simulations. The final model runtime depends on the model Problem Settings such as Problem Class (e.g., linear or nonlinear problems), Solver settings, among others. Configurations with more than 16 CPUs are intended for cases, where more than one FEFLOW run is executed in parallel. For example, in a FePEST parallelization, the VM could host several FEFLOW runs in parallel, thus the CPUs could be entirely occupied. A configuration supporting GPU is only relevant for FEFLOW models with the Algebraic Multigrid method under the PETSc solver.
VM configurations with larger number of CPUs (and with GPU) will reflect a high consumption of MIKE credits. If you are not sure available your billability and costs, you can contact the general support hotline (email@example.com) to ask such details.
Add and Delete Simulations
A Simulation is defined by the combination of a FEFLOW *.fem / *.dac file, a FEFLOW Version and a Configuration. Simulations can be created by pressing the Add Simulation button. Simulations are not started immediately. The simulation begins only after pressing the Upload and Run button.
Simulations can be also eliminated after selecting their names from the model list and pressing the Delete Simulations button. Please be aware that, after pressing such button:
Simulations which are not started yet are deleted without additional query.
If there is at least one completed (succeeded, failed, cancelled) simulation selected, the user is asked if only the simulation entry (model list in GUI) or the simulation including the cloud results should be deleted.
After deleting a completed simulation with model FEFLOW Results (*.dac) in the cloud storage, this event also deletes the corresponding output folder in the MIKE Cloud project. The user could delete any output folder directly in MIKE Data Admin with the appropriate access level. With such as operation, the Cloud Simulations panel in the GUI will also remove the simulation entry from the model list.
Import/Export from/to file
The Export functionality will save all selected simulation records into a JSON file. For the simulation records created using the option User-defined (local), the model file name is stored with a relative path (base on the exported/imported JSON file).
On the Import event, simulation records can be imported to define the modelling jobs. The file could contain also information such as configuration and FEFLOW version. Invalid records, e.g., those not matching input files, etc., are automatically discarded.
Upload and Run
The Upload and Run button uploads the model input files (*.fem and *.dac) to the corresponding MIKE Cloud project. If the file has been previously uploaded, the process will not be repeated. After the upload operation successes, the simulation starts.
The user is fully responsible of the content of the model files to be uploaded and subsequently used for the cloud execution. There is a minimum level of comparison to assess whether a file has been already uploaded into the cloud storage portal (so-called MD5 checksum). The difference between two identical file names with almost the same model content (e.g. just a change in a FEFLOW parameter) would probably not be recognized. Especially in those cases where models files are several GBs, it is strongly recommended that the user double check the file content before uploading.
The option requires a single simulation record to be selected from the job list and that the result file (*.dac) has been previously downloaded. The Open result button will automatically load the FEFLOW *.dac file in the FEFLOW GUI.
Download result (s)
After pressing the button, FEFLOW starts downloading all selected (completed) records to a local temporary file.
Save result as
The button Save result as downloads the result to a user-defined path/location or copies/renames the temporally-created file to the user-defined path/location, if the result has been already downloaded.
Show all completed simulations
The option adds all completed simulations in the list (together with "current" simulations which are always shown).
Real-time result monitoring
The user can monitor the Status of the model run (e.g., Success or In Progress), the Simulation time (Time step number and Absolute Simulation time) and the Elapsed Time. All this information is continuously updated during the model run. Especially, the Elapsed Time can be used to partially understand the model runtime. However, the preparation of a VM after the Upload and Run button has ben used, could cost additional time (in minutes) before the FEFLOW run really starts. Therefore, the Elapsed Time may not necessary always reflect the total duration of the job.
FEFLOW model results can be monitored during the cloud run without the need of downloading the FEFLOW results *.dac file. As in any local FEFLOW simulation, the user can follow partial results through a set of charts. In the case of cloud model runs, the possibility to review model results is done though the Cloud Charts. These charts are also accessible through the menu View.
List of Cloud Charts available in the FEFLOW GUI.
Example of Cloud Charts (Head Distribution and Fluid Rate Budget) in the FEFLOW GUI.
In the case multiple simulations are in progress, the Cloud Charts content reflects the selected simulation from the model list. The content of the chart will be automatically updated as soon as another simulation record in selected in the Cloud Simulations panel.
The chart functionalities (import/export, colours, names, etc.) are identical to any other FEFLOW charts.
Model list and multiple job execution
By right-clicking on the model list in the Cloud Simulations panel, several advanced options are offered (see picture below). The user can create a Model list, which is a list of several FEFLOW models to be executed within the same virtual machine. The model to be executed in the cloud can be defined via any of the following options:
Any existing model already available in the cloud platform (Option Add cloud models to list),
Local FEFLOW files (Option Add local models to list) or
The current FEFLOW file opened in the GUI (Option Add current model to list).
Due to the fact that several model runs are executed within the same virtual machine, the user may want to define the Number of Threads per simulation. So, there is proper CPU balance in between all the jobs. The option is equivalent to the switch -threads via FEFLOW console run (see FEFLOW Command Line) and can be specified for each FEFLOW model independently by simply right-clicking on the model name.
The default number of threads is automatic, which means that the maximum available threads at the chosen configuration is taken. This approach is analogue to the default option in the FEFLOW Global Settings in the case of a local model run.
Context menu of the model simulation table.
In the case of a model list, the progress of the simulation shown in the Cloud Simulations panel is a function of all the FEFLOW models linked to the job. The panel also presents Simulation time and the Elapsed Time for each model independently.
Example of model list with two cloud runs within the same virtual machine.
Extended DAC run
In the case a FEFLOW cloud execution was completed and the FEFLOW Result *.dac file still exists in the cloud storage, the Cloud Simulations panel is capable to extend the model run. The described operation is possible if a *.dac file is selected with the option User-defined (cloud) . The user will be asked to define the new simulation time through the DAC Simulation - End time dialog.
Extended simulation time in DAC file.
Plug-ins and Python scripts
The current implementation of the cloud execution supports only the use FEFLOW DHI's plug-ins (piFreeze, piChem and piHMC) attached to the FEFLOW *.fem or *.dac documents. These plug-ins can be installed together with the standard FEFLOW installation packages. If any other user plug-in is attached to the FEFLOW model, the job will be terminated and the Status marked as Failure.
Python scripts are currently not supported with the cloud simulations. If the selected model contains an embedded Python script, the model run will be executed, but the Python will be disabled.