Sensitivity analysis (SA) is one of the main themes of the Water Programming Blog. There are several decent blog posts that go over theoretical aspects of sensitivity analysis (for example, here , here, and here). Also, many blog posts explain how to efficiently and elegantly visualize sensitivity analysis results (for example, here, and here). In addition, there are many blog posts related to SALib, a widely used Python library developed at Cornell University by former members of Dr. Reed research group (for example, here, here, and here).
Recently, I have been trying to put together a comprehensive list of other SA tools, and I thought it might be useful to write a blog post on this topic. I organized the following list based on the platforms I have explored so far, including MATLAB, Python, and R. After that, I will introduce some other open-source and commercialized SA tools.
Many MATLAB packages have been developed to perform sensitivity analysis and uncertainty quantification. As the following table shows, they have been created by a variety of universities and research institutes. Also, several of them cover different sensitivity analysis methods, such as Regression-based SA, Variance-based SA (e.g., Sobol), and derivative-based SA. All of them support at least two sampling techniques, such as Latin Hyper Cube Sampling. Many of them are generic (discipline-free) and can be used to answer different types of questions; however, a few of them (e.g., PeTTSy and DyGloSA) have been tailored to specific applications, such as biological models. Also, almost all of them include some post-processing and visualization components.
There are two toolboxes that work in platforms other than MATLAB. The SAFE package developed by Pianosi et al. (2015) has R and Python versions, and the SaSAT package developed at the University of New South Wales works in Microsoft Excel.
|Abbreviation||Full Name||Example of Methods Supported||Institution|
|GSAT||Global Sensitivity Analysis Toolbox||Sobol and FAST||MATLAB|
|SAFE||Sensitivity Analysis For Everybody||EET, or Morris method,RSA, Sobol, FAST, and PAWN||University of Bristol|
|GSUA||Global Sensitivity and Uncertainty Analysis Toolbox||Sobol||MATLAB|
|GUI-HDMR||Global Sensitivity Analysis Toolbox||Global Sensitivity Analysis using HDMR||University of Leeds|
|DyGloSA||Dynamical Global Sensitivity Analysis Toolbox||Dynamical Global parameter Sensitivity Analysis (GPSA) of ODE models||University of Luxembourg|
|PeTTSy||Perturbation Theory Toolbox for Systems||Perturbation analysis of complex systems biology models||University of Cambridge|
|SaSAT||Sampling and Sensitivity Analysis Tools||Regression-based (Pearson, Spearman, and Partial Rank Correlation Coefficients)||The University of New South Wales|
|SensSB||Sensitivity Analysis in Systems Biology models||Local SA, derivative and variance based global sensitivity analysis||Process Engineering Group at IIM-CSIC (Vigo, Spain)|
|SobolGSA||Global Sensitivity Analysis and Metamodeling Software||Morris, Sobol FAST and derivative-based||Imperial College London|
|SUMO||SUrrogate Modeling Toolbox||Surrogate models, sensitivity analysis||Ghent University|
|UQLab||The Framework for Uncertainty Quantification||Morris, Kucherenko,ANCOVA, Borgonovo, Sobol||ETH Zurich|
EET: Elementary Effects Test
RSA: Regional Sensitivity Analysis
Interestingly, I was not able to find many Python libraries, and most of the ones that I did find were developed for specific applications. Please leave a comment if you are aware of any other packages that have not been listed here. Among these packages, SALib seems to be the one that covers more SA and sampling methods. There are two SA and QU packages that have C++ versions (OpenTURNS and UQTk). Also, uncertainpy have been originally developed for neuroscience applications.
|Abbreviation||Description||Example of Methods Supported||Institution|
|SALib||Python sensitivity analysis library||Sobol, Morris, FAST, RBD-FAST, Delta Moment-Independent Measure, Derivative-based, Factorial||Cornell University|
|uncertainpy||Uncertainty quantification and sensitivity analysis library||Sobol||University of Oslo|
|MATK||Model analysis toolKit||FAST, Sobol||Los Alamos National Laboratory|
|UQTk||Quantification of uncertainty in numerical models||Sobol||Sandia National Lab|
|OpenTURNS||Open source initiative for the Treatment of Uncertainties||Spearman Correlation Coefficients, Sobol, ANCOVA, UQ||Technical University of Denmark|
|varsens||Variance Based Sensitivity Analysis||Sobol||Vanderbilt University|
QU: Quantification of Uncertainty
I was able to find about fifty R packages that have sensitivity analysis features. The following table lists the ones that have the most comprehensive SA functionalities. It seems that the rest of them were developed for specific areas of science and have limited SA functionality. I list some of these here (RMut, pksensi, ivmodel, FME, episensr, pse).
|Name||Example of Methods Supported|
|sensitivity||Sobol, Morris, FAST, RBD-FAST, Delsa, Derivative-based , Factorial|
|multisensi||SA on models with multivariate outputs|
|konfound||Robustness and sensitivity of empirical models|
There are many other SA tools that have been developed in other platforms, and the following table lists only a few of them. There are also several commercial SA platforms such as SDI, VISYOND, and SMARTUQ that seem to have nice graphical user interfaces (GUIs), but, because they are not freeware and the source codes are not available, they might have limited applications in academic research.
|Abbreviation||Main applications||Programming Language||Institution|
|Dakota||Optimization, QU, SA (Sobol, FAST, Morris)||C++||Sandia National Laboratory|
|PSUADE||QU, Spearman, Pearson Correlation Coefficient, Sobol, Morris, FAST||C++||Lawrence Livermore National Laboratory|
|SIMLab||Sobol, FAST, Morris||GUI-based||The European Commission’s science and knowledge service|
FAST: Fourier Amplitude Sensitivity Testing
Please leave a comment and let me know if you are aware of any other useful tools that I did not list here.