When is a parameter study appropriate? And how should one proceed? We present a tool chain from the simulation model in Modelica, FMU export and the use of Python scripts to visual data exploration.
unbekannt
The motivation for conducting a parameter study is generally new knowledge that one wants to gain about a system. For example, the system can be a real technical plant or a simulation model. The procedure is as follows: You change something in the system (a model parameter or factor) and see what happens. In extreme cases, you play through all the combinations and then know how the system will behave in all possible cases. This simple approach is called a full factorial design. The problem here is that the effort can become very large very quickly: With 8 parameters (factors) with five levels each, a full-factorial experimental design with a total of 58=390625 experiments results.
If the effort required to carry out the experiments is too high, there are various approaches in the statistical design of experiments to reduce the number of experiments. These include, for example, fractional factor plans that require fewer experiments but provide less information. One disadvantage, for example, is that the interactions between the factors can no longer be fully grasped.
In practice, the sequential design of experiments is therefore often carried out. The number of experiments is limited and the execution and evaluation of the experiments are carried out alternately until the desired accuracy or information gain is achieved. “Driving on sight” so to speak. We have chosen a similar approach for the analysis of the heat demand of an electric bus, which we will explain in more detail below.
To study these parameters, we used a vehicle model in Modelica. Modelica, as an object-oriented, equation-based language, is perfectly suited for modeling physical systems, such as a vehicle cabin. However, the Modelica development environments (e.g. OpenModelica, Dymola, SimulationX) do not currently offer particularly good possibilities for conducting large-scale simulation studies.
Therefore, we have exported the vehicle model as a Functional Mock-Up Unit (FMU). An FMU contains a compiled and thus executable version of the model and can be simulated by applications that support the Functional Mock-Up Interface (FMI). A list of compatible tools can be found at https://fmi-standard.org.
We decided to use Python, firstly because we have our own FMU tool for Python, MoBA Automation, and secondly because Python offers a large number of very good libraries for post-processing and visualization of the simulation results. For visualization, or rather for visual data exploration, we have used the freely available library bokeh.
Since we are using a model for our study, we can of course investigate considerably more variants than is possible in an experiment. However, the model must fulfill two conditions: It must be able to calculate quickly and reliably.
The duration of the simulation study scales linearly with the speed of the model. Let’s take our above example with 8 parameters and 5 levels each, a calculation time of only one second per simulation results in a total time of 4.5 days (390,625 sec) for the calculation of the full-factorial design of experiments.
This also shows why the model should run robustly. If there are delays or crashes of the model during the simulation study, this can greatly increase the computing time. For this reason, it must be ensured already during the model creation that the model is robust and fast in the planned parameter space. For this purpose, the min-max values of the parameters should be tested at least individually in a fractional factor plan (2 levels x 8 parameters = 16 simulations), or better still the combinations of all min-max values (28=256 simulations).
But now we come to the actual model. We have set it up in Modelica with our library TIL and the add-on “TIL Cabin”. The add-on already contains ready-made vehicle cabin models, which we only had to modify for our application. A more detailed description of the model and the underlying physics can be found in the blog post: “Heating load calculation of a vehicle cabin using simulation”.
Our simulation object is a 4.5m long electric bus for a maximum of 15 people. In order to investigate the weather conditions, vehicle characteristics and operation, we selected a total of 8 parameters in our study:
The set temperature in the vehicle is not constant but is dependent on the outside temperature. In this respect, we have followed the specifications of the Association of German Transport Companies (VDV-236 Air Conditioning of Buses). The heating and cooling capacity finally determined corresponds to the capacity required to maintain these temperature limits for the steady state.
After exporting the model as FMU, we were able to perform the simulation study detached from our model development environment. The rest of the toolchain is thus no longer dependent on Modelica, but only requires a simulation model in the form of an FMU. This can also come from other development environments, such as ANSYS or MATLAB Simulink.
We then carried out the simulation study with our in-house tool MoBA Automation. MoBA Automation is a Python-based software tool with predefined but customizable functions and a graphical user interface. Internally, MoBA Automation uses the TLK Simulator to simulate the FMU. Alternatively, you can of course use other Python FMU simulators (see https://fmi-standard.org) and write the corresponding scripts and functions for the parameter study yourself. But we don’t want to reinvent the wheel every time 😉. MoBA Automation is designed for this kind of task and thus saves us a lot of programming and time.
Now to the course of the study: First we set up the experimental plan in MoBA Automation by combining the parameters (combine input parameter). The experimental plan is then divided into smaller packages (evaluate number of simulations & subdivide simulations), which enables us to run the experiments in parallel. MoBA Automation supports both the distribution of the simulations to the local cores of the CPU and the network-wide use of CPUs. The core function for calculating and storing the results (simulate and store results) essentially contains nested for-loops. It is important here that the loops contain a timeout so that individual simulations can be skipped if they take too long to calculate or even crash (despite all the effort involved in creating the model). Furthermore, all errors in the loop should be logged for later debugging and model adjustments.
Of course, the results of the simulations that have been run should be saved regularly, as a complete repetition of the study might be necessary in case of a crash. We use an SQL database for storage, which is successively filled with the parameter sets and the calculated heating and cooling demand. When choosing the frequency for storing the data, you have to consider the time needed for the storage processes and the duration of the simulations.
In addition to the features presented here, MoBA Automation includes many more functionalities for simulation studies, measurement data processing, and data analysis. If you are curious and want to learn more, please contact us: info@tlk-energy.de
Following the described procedure, we finally carried out the simulation study. We also proceeded sequentially: We simulated, analyzed the results, extended or refined the parameter sets, and simulated again. In total, we carried out several hundred thousand simulations and saved the results in our database.
With such a large amount of data, you also have to think about the evaluation. A look at the series of numbers hardly helps. Meaningful graphics are the measure of all things here since humans can grasp this information much faster and better. However, with these large amounts of data, it is helpful to display only a part of the data, i.e. to filter the data.
For filtering and plotting in Python there are different libraries available, of which we can recommend the following:
With these libraries, a variety of meaningful graphics (e.g. scatter plots, bar charts, box plots, heatmaps) can be created. Bokeh has a special feature for this: With interactive plots, the library allows a faster and more intuitive data exploration than is possible via the command line. In addition, the plots generated are available as HTML files and can be shared online. This is exactly what we have done with a part of our simulation results:
The data shown contain the results of our simulation study, whereby 7 parameters can be filtered in three levels (parameter variations) each. The outdoor temperature is plotted in 6 levels on the x-axis. In total, the data set thus contains 37×6=13122 tests. The heat and cold demand of the bus are plotted on the y-axis. When hovering over a data point an informations about the parameter set appears. You can also switch to a box plot to obtain statistical information on the data set.
A note on the results: Since a factorial experimental design was used in the compilation of the parameter sets, combinations may occur which in reality occur very rarely or not at all (e.g. low temperatures of -15°C and high solar radiation of 500 W/m²). This must be taken into account when interpreting the results.
By filtering the data, the influence of the various parameters can now be experienced interactively. This quickly shows that solar radiation and air exchange rate have a great influence on the heating and cooling requirements of the bus. It can also be seen that the air exchange rate, especially at high and low outside temperatures, has the strongest influence and this decreases as the outside temperature approaches closer to the bus nominal temperature.
By exporting the Modelica model as FMU, it is possible to carry out large parameter studies in Python with manageable effort. Here, automation and a corresponding visual presentation are highly recommended. The described toolchain has proven itself in our research project LatHe.GO (funded by the European Regional Development Fund (ERDF)) and especially the discussion of the results using the interactive plots was very helpful.
Are you facing similar challenges and could you benefit from our know-how? You are welcome to contact us: info@tlk-energy.de
Of course, I am also happy to receive comments and suggestions regarding my blog post.
© 2024 TLK Energy. All rights reserved.
