Open main menu

Changes

Relax 3.3.0

120 bytes removed, 20:34, 16 October 2020
m
Switch to the {{gna task link}} template to remove dead Gna! links.
* Further improved the profiling of relax curve fit. This profiling shows, that Python code is about twice as slow as the C code implemented. But it also shows that optimising with scipy.optimize.leastsq is 20 X faster. It also gives reasonable error values. Combining a function for a linear fit to guess the initial values, together with scipy optimise, will be an extreme time win for estimating R<sub>2eff</sub> values fast. A further test would be to use relax Monte Carlo simulations for say 1000-2000 iterations, and compare to the errors extracted from estimated covariance.
* Added verification script, that shows that using scipy.optimize.leastsq reaches the exact same parameters as minfx for exponential curve fitting. The profiling shows that scipy.optimize.leastsq is 10X as fast as using minfx (with no linear constraints). scipy.optimize.leastsq is a wrapper around wrapper around MINPACK's lmdif and lmder algorithms. MINPACK is a FORTRAN90 library which solves systems of nonlinear equations, or carries out the least squares minimization of the residual of a set of linear or nonlinear equations. The verification script also shows, that a very heavy and time consuming Monte Carlo simulation of 2000 steps, reaches the same errors as the errors reported by scipy.optimize.leastsq. The return from scipy.optimize.leastsq, gives the estimated covariance. Taking the square root of the covariance corresponds with 2X error reported by minfx. This could be an extremely time saving step, when performing model fitting in R<sub>1&rho;</sub>, where the errors of the R<sub>2eff</sub> values, are estimated by Monte Carlo simulations. The following setup illustrates the problem. This was analysed on a MacBook Pro, 13-inch, Late 2011 with no multi-core setup. Script running is: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. This script analyses just the R<sub>2eff</sub> values for 15 residues. It estimates the errors of R<sub>2eff</sub> based on 2000 Monte Carlo simulations. For each residues, there is 14 exponential graphs. The script was broken after 35 simulations. This was measured to 20 minutes. So 500 simulations would take about 4.8 Hours. The R<sub>2eff</sub> values and errors can by scipy.optimize.leastsq can instead be calculated in: 15 residues * 0.02 seconds = 0.3 seconds.
* Moved the target function for minimisation of exponential fit into the target functions folder. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented initial system test Relax_disp.test_estimate_r2eff for setting up the new user function to estimate R<sub>2eff</sub> and errors by scipy. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added front end user function relax_disp.r2eff_estimate to estimate R<sub>2eff</sub> and errors by exponential curve fitting in scipy.optimize.leastsq. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified check for model, to accept model as input, for error printing. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented back end for estimating R<sub>2eff</sub> and errors by exponential curve fitting with scipy.optimize.leastsq. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Documentation fix for new exponential target function. Also added new function to estimate R<sub>2eff</sub> and I<sub>0</sub> parameters, before minimisation. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Small changes to verification scripts, to use &chi;<sup>2</sup> function and use the scaling matrix correct. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Split up of system test test_r1rho_kjaergaard_missing_r1, into a verification part. This is to test the new R<sub>2eff</sub> estimation, which should get the parameter values, as a this 2000 Monto Carlo simulation. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Modified system test Relax_disp.test_estimate_r2eff. This is to compare against errors simulated with 2000 MC. The parameters are comparable, but not equal. Mostly, it seems that the errors from scipy.optimize.leastsq, are twice as high than the Monte Carlo simulations. This affect model fitting, and the calculated &chi;<sup>2</sup> value.
* Added system test Relax_disp.test_estimate_r2eff_error(). This is to get insight in the error difference between 2000 Monto Carlo simulations and then scipy.optimize.leastsq.
* Add dependency check for scipy.optimize.leastsq. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Lowering precision in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. This is R<sub>1</sub> estimation with MODEL_NS_R1RHO_2SITE. The lowering of precision is due different system precision.
* Reused the dependency check "scipy_module", since leastsq() has been part of Scipy since 2003.
* Isolated all code related to [http://www.nmr-relax.com/manual/relax_disp_r2eff_estimate.html user function relax_disp.r2eff_estimate] into independent module file. All has been isolated to: specific_analyses.relax_disp.estimate_r2eff.
* Split function to minimise with scipy.optimize.leastsq out in estimate_r2eff module. This is to prepare for implementing with minfx.
* Implemented first try to minimise with minfx in estimate_r2eff() function. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Implementation of the target_functions.relax_fit.jacobian() function. This follows from the discussions at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807. The function will calculate the Jacobian matrix for the exponential curve-fitting module. The Jacobian can be used to directly calculate the covariance matrix, for example as described at https://www.gnu.org/software/gsl/manual/html_node/Computing-the-covariance-matrix-of-best-fit-parameters.html. The Jacobian is calculated using the help of the new exponential_dI() and exponential_dR() functions in the target_functions/exponential.c file. These calculate the partial derivatives of the exponential curve with respect to each model parameter separately. The implementation still needs testing and debugging.
* Fixes for the new target_functions.relax_fit.jacobian() function. The Python list of lists is now correctly created and returned.
* Turned off the optimisation constraints for the [[R2eff]] model in the dispersion auto-analysis. This follows from http://thread.gmane.org/gmane.science.nmr.relax.scm/22977/focus=6829. This model does not require constraints at all, and the constraints only cause the optimisation to take 10x longer to complete. Therefore the constraint flag has been set to False for the model.
* Initial try to form the Jacobian and Hessian matrix for exponential decay. This can be tried with system test: relax -s Relax_disp.test_estimate_r2eff_error. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Intermediate step in estimate R<sub>2eff</sub> module. It seems that minfx is minimising in a quadratic space because of the power of &chi;<sup>2</sup>, while the general input to scipy.optimize does not do this. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Cleaned up target function for leastsq, since arguments to function can be extracted from class. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Tried to implement with scipy.optimize.fmin_ncg and scipy.optimize.fmin_cg, but cannot get it to work. The matrices are not aligned well. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Implemented the chi-squared gradient as a C module for the target functions. This simply translates the Python code into C to allow any target function C modules to build its own gradient function.
* Implemented the target_functions.relax_fit.dfunc() gradient function. This is using the Python/C interface to provide a Python function for calculating and returned the chi-squared gradient for the exponential curves.
* The parameter index is now passed into exponential_dI0() and exponential_dR(). This is for the relaxation curve-fitting C module so that the indices are not hardcoded.
* The I<sub>0</sub> and R parameter indices are now defined in the target_function/relax_fit.h header file. This is to abstract the exponential curve parameter indices even more.
* Big cleanup of estimate R<sub>2eff</sub> module. This is to make the documentation more easy to read and understand. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Created 2 unit tests for the target_functions.relax_fit relax C module. These check the func() and dfunc() Python methods exposed by the module.
* The relax_fit C module unit tests now check if the parameter scaling is functional.
* Added several comments to the R<sub>2eff</sub> estimate module. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Added a script and log file for calculating the numerical gradient for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the gradient using the scipy.misc.derivative() function both at the minimum and at a point away from the minimum. The values will be used to construct a unit test to check the C module implementation.
* Created a unit test to check the dfunc() function of the relax_fit C module off the minimum.
* Fix for the test_dfunc_off_minimum() unit test. This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit. The wrong gradient was being scaled.
* Switched the optimisation algorithm in test_suite/system_tests/scripts/relax_fit.py. This script, used by the Relax_fit.test_curve_fitting_height and Relax_fit.test_curve_fitting_volume system tests, now uses the BFGS optimisation. This is to demonstrate that the exponential curve gradient function dfunc() is implemented correctly and that more advanced optimisation algorithms can be used (excluding those that require the full Hessian d2func() function).
* Got the method of 'Steepest descent' to work properly, by specifying the Jacobian correctly. The Jacobian was derived according to the &chi;<sup>2</sup> function. The key point was to evaluate the two derivative arrays for all times points, and then sum each of the two arrays together, before constructing the Jacobian. This clearly shows the difference between minfx and scipy.optimize.leastsq. scipy.optimize.leastsq takes as input a function F(x0), which should return the array of weighted differences between function value and measured values: "1. / self.errors * (self.calc_exp(self.times, *params) - self.values)". This will be an array with number of elements 'i' corresponding to number of elements. scipy.optimize.leastsq then internally evaluates the sum of squares -> sum[ (O - E)<sup>2</sup> ], and minimises this. This is the &chi;<sup>2</sup>. Minfx requires the function to minimise before hand. So, the "func" should be &chi;<sup>2</sup>. Then the dfunc, and d2func, should be derivative of &chi;<sup>2</sup>, but all elements in the array should still be summed together. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Got the Quasi-Newton BFGS to work. This uses only the gradient, this gets the same results as 2000 Monte Carlo with simplex and scipy.optimize.leastsq. Error estimation still not provided. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Removed all code regarding scipy.optimize fmin_cg and fmin_ncg. This problem should soon be able to be solved with minfx. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added initial documentation for multifit_covar. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified profiling script to use the new estimate R<sub>2eff</sub> module. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified verify error script, to use new estimate R<sub>2eff</sub> module. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Removed all unnecessary code from estimate R<sub>2eff</sub> module. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* More removal of code. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Changed the array declarations in the target_functions/exponential C file and header. Instead of using the pointer format of *xyz, the array format of xyz[] is now being used. These are equivalent and the later is more obvious that this is an array.
* Changed the array declarations in the target_functions/c_chi2 C file and header. Instead of using the pointer format of *xyz, the array format of xyz[] is now being used. These are equivalent and the later is more obvious that this is an array.
* Fixes for the Hessian.py script for numerical integrating the Hessian for an exponential curve.
* Implemented two unit tests to check the Hessian of the target_functions.relax_fit.d2func() function. This compares the calculated Hessian to the numerically integrated values from the test_suite/shared_data/curve_fitting/numeric_gradient/Hessian.py script, showing that the d2func() function is implemented correctly.
* Modified profiling script, but it seems that the dfunc from target_functions.relax_fit does not work. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Modified estimate R<sub>2eff</sub> module, to use C code. But system test Relax_disp.test_estimate_r2eff_error shows that the Jacobian is not correctly implemented to be called in minfx.
* Created an initial test suite data directory for a mixed R<sub>1&rho;</sub> + CPMG dispersion analysis. The generate.py script will be extended in the future to generate both synthetic R<sub>1&rho;</sub> and CPMG data for a common exchange process. Such a data combination should show some minor flaws in the current design of the dispersion analysis and will help to solve these.
* Improvements to the pipe_control.minimise.reset_min_stats() function. The minimise statistics resetting is now more elegantly implemented. And the sim_index keyword argument is accepted by the function and individual Monte Carlo simulation elements can now be reset.
* Modified wrapper function for curve_fit, to only change to list type, if the type is a ndarray. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* The model-free reset_min_stats() function has been replaced with the pipe_control.minimise version. The specific_analyses.model_free.optimisation.reset_min_stats() function has been deleted and instead the pipe_control.minimise version is being used.
* Implemented the first try to compute the variance of R<sub>2eff</sub> and I<sub>0</sub>, by the covariance. This uses the Jacobian matrix. The errors calculated, are though way to small compared 2000 Monte Carlo simulations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Tried to implement the Jacobian from C code. This though also report errors which are to small. Maybe some scaling is wrong. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Modified profiling script to calculate timings. The timings for C code are: Simplex, with constraints = 2.192; Simplex, without constraints = 0.216; BFGS, without constraints = 0.079; Newton, without constraints = 0.031; This is pretty pretty fast. To this profiling script, I would also now add some verification on calculations.
* Tried to verify solution to profiling script. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Set the constraints=False when doing Monte Carlo simulations for R<sub>2eff</sub>. This is to speed up the Monte Carlo simulations by a factor X10, when estimating the error for R<sub>2eff</sub>. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented the use of "Newton" as minimisation algorithm for R<sub>2eff</sub> curve fitting instead of simplex. Running the test script: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. For 50 Monte Carlo simulations, the time drop from: 3 minutes and 13 s, to 1 min an 5 seconds. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Changed the relax_fit.py sample script to use Newton rather than Simplex optimisation. This can lead to significantly faster optimisation times, as shown in the commit message http://article.gmane.org/gmane.science.nmr.relax.scm/23081.
* Changed the optimisation description in the relaxation curve-fitting chapter of the manual. The script example has been converted to match the sample script, replacing the Nelder-Mead simplex algorithm with Newton optimisation, and removing the argument turning diagonal scaling off. All the text about only the simplex algorithm being supported due to the missing gradients and Hessians in the C module have been deleted. The text that linear constraints are not supported has also been removed - but this was fixed when the logarithmic barrier constraint algorithm was added to minfx.
* By using minfx, and the reported Jacobian, it is now possible to get the exact same error estimation as scipy.optimize.leastsq. The fatal error was to set the weighting matrix with diagonal elements as the error. There weights are 1/errors<sup>2</sup>. There is though some unanswered questions left. The Jacobian used, is the direct derivative of the function. It is not the &chi;<sup>2</sup> derivative Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fixed naming of functions, to better represent what they do in module of estimating R<sub>2eff</sub>. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented the Jacobian of exponential function in Python code. This now also gets the same error as leastsq and C code. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Tried to implement a safety test for linearly-dependent columns in the covariance matrix. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Fixes for the [http://www.nmr-relax.com/manual/relax_disp_r2eff_estimate.html relax_disp.r2eff_estimate user function documentation]. This is to allow the relax manual to compile again as the original documentation was causing LaTeX failures.
* Clean up of the declarations in the target_functions.relax_fit C module. The Python list objects are now declared at the start of the functions, and then PyList_New() is called later on. This allows the code to compile on certain Windows systems.
* Removed the user function to estimate the R<sub>2eff</sub> values and errors with scipy.optimize.leastsq. With the newly implemented Jacobian and Hessian of the exponential decay function, the front-end to scipy.optimize.leastsq does not serve a purpose. This is because minfx is now as fast as scipy.optimize.leastsq, and can estimate the errors from the Jacobian to the exact same numbers as scipy.optimize.leastsq. In addition to that, the covariance can be calculated by QR decomposition. This adds additional feature for checking for a singular matrix. The back-end will still be kept in place for the coming tim, but could be removed later. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added front-end to the new [http://www.nmr-relax.com/manual/relax_disp_r2eff_err_estimate.html user function relax_disp.r2eff_err_estimate], which will estimate the R<sub>2eff</sub> errors from a pipe and spins with optimised values of R<sub>2eff</sub> and I<sub>0</sub>. The covariance matrix can be calculated from the optimised parameters, and the Jacobian. Big care should be taken not to directly trust these results, since the errors are quite different compared to the Monte Carlo simulations. This implementation, will reach the exact same error estimation as scipy.optimize.leastsq. But with much better control over the data, and insight into the calculations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added method to automatically perform error analysis on peak heights. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified system test Relax_disp.test_estimate_r2eff() to first do a grid search, then minimise and then estimate the errors for R<sub>2eff</sub> and I<sub>0</sub>. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added back-end to estimate R<sub>2eff</sub> errors. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix to system test test_estimate_r2eff_error(), to first delete the old error estimations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added several tests to: test_estimate_r2eff_error, to compare different output from algorithms. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Cleaned up code in R<sub>2eff</sub> error module. Also removed a non working Hessian matrix. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Moved code around, and made function multifit_covar() independent of class object. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Inserted checks for C module is available in module for estimating R<sub>2eff</sub> error. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Removed unnecessary call to experimental Exp class. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Renamed system test, that test the user function for estimating the R<sub>2eff</sub> error: test_estimate_r2eff_err, test the user function. test_estimate_r2eff_err_methods, test different methods for getting the error. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added system test, Relax_disp.test_estimate_r2eff_err_auto and extended functionality to the auto-analyses protocol. If "exp_mc_sim_num" is set to "-1" and sent to the auto-analyses, the errors of R<sub>2eff</sub> will be estimated from the covariance matrix. These errors is HIGHLY likely to be wrong, but can be used in an initial test fase, to rapidly produce data for plotting data. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added script, to be used in GUI test. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added GUI test Relax_disp.test_r2eff_err_estimate, to test the setting of MC sim to -1 for exponential R<sub>2eff</sub> error estimation. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added keyword "exp_mc_sim_num", to the auto-analyses in the GUI. This sets the number of Monte Carlo simulations for R<sub>2eff</sub> error estimation in exponential curve fitting. When setting to -1, the errors are estimated from the covariance matrix. These errors are highly likely to be wrong, but can be used in Rapid testing of data and plotting. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Tried to click the "fit_r1" button in the GUI test, but receives an error: relax --gui-tests Relax_disp.test_r2eff_err_estimate, "AttributeError: 'SpinContainer' object has no attribute 'r1'". [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Moved the mc_sim_num GUI element in the analysis tab ip, as it is executed first. Also modified the tooltip. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added a warning to the auto-analyses about error estimation from the covariance. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Removed yet another comma from GUI tooltip. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Formatting changes for the lib.periodic_table module. This is in preparation for extending the information content of this module.
* Modified system test 'test_estimate_r2eff_err_auto', to use the GUI script. It seems to work perfect. This is to test against GUI script: test_r2eff_err_estimate [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified test_estimate_r2eff_err_auto, to set r1_fit to False. This still make the system test pass, and fit R<sub>1</sub>. So this means R<sub>1</sub> fit button is not functioning properly. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix for warning message in the auto-analyses in the GUI. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Tried to improve docstring for API documentation. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Added all of the IUPAC 2011 atomic weights to the lib.periodic_table module. These will be useful for correctly calculating the centre of mass of a molecule.
* The lib.periodic_table method for adding elements is now private.
* Created the unit test infrastructure for the lib.periodic_table module. This includes one unit test of the lib.periodic_table.periodic_table.atomic_weight() function which has not been implemented yet.
* Implemented the lib.periodic_table.periodic_table.atomic_weight() method. This returns the standard atomic weight of the atom as a float.
* Yet another try to make the API documentation working. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented system test Relax_disp.verify_estimate_r2eff_err_compare_mc for testing R<sub>2eff</sub> error as function of Monte Carlo simulation. Note, since the name does not start with "test", but with "verify", this test will not be issued in the system test suite. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Converted the periodic table in lib.periodic_table into a dictionary type object. The new Element container has been added for storing the information about each element in the table. The Periodic_table object used the atomic symbol as a key for each Element instance.
* Modified system test test Relax_disp.test_estimate_r2eff_err_methods() to show the difference between using the direct function Jacobian, or the &chi;<sup>2</sup> function Jacobian. Added also the functionality to the estimate R<sub>2eff</sub> module, to switch between using the different Jacobians. The results show, that R<sub>2eff</sub> can be estimated better.
* Tiny fix for the Diffusion_tensor.test_create_diff_tensor_pdb_ellipsoid system test. The switch to using the lib.periodic_table module for atomic masses has caused the centre of mass of the ellipsoid to shift just enough that one ATOM coordinate in the PDB file has changed its last significant digit.
* Created the lib.periodic_table.process_symbol() function. This will take an atomic symbol and return a copy of it with an uppercase first letter and lowercase second letter. This is used by the Periodic_table methods atomic_mass() and atomic_weight() to allow for non-standard symbol input, for example if the element name comes directly from the all uppercase PDB file format without translation.
* Tried to scale the covariance matrix, as explained here: http://www.orbitals.com/self/least/least.htm. This does not work better. Also replaced "errors" to "weights" to the multifit_covar(), to better determine control calculations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Added all gyromagnetic ratio information from lib.physical_constants to lib.periodic_table. The Periodic_table.gyromagnetic_ratio() method has been added to allow this value to be easily returned.
* Added to back-end of R<sub>2eff</sub> estimate module, to be able to switch between the function Jacobian or the &chi;<sup>2</sup> Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified [http://www.nmr-relax.com/manual/relax_disp_r2eff_err_estimate.html user function relax_disp.r2eff_err_estimate], to be able switch between the Jacobians. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified system test Relax_disp.verify_estimate_r2eff_err_compare_mc, to try the difference between the Jacobian. The results are: Printing the estimated R<sub>2eff</sub> error as function of estimation from covariance and number of Monte Carlo simulations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Deleted the gyromagnetic ratio values and return_gyromagnetic_ratio() function from lib.physical_constants.
* Shifted all of relax to use the lib.periodic_table module for gyromagnetic ratios. The values and value returning function have been removed from lib.physical_constants and replaced by the Periodic_table.gyromagnetic_ratio() method in the lib.periodic_table module.
* Started making functions in R<sub>2eff</sub> estimate module, independent on the informations stored in the Class. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Cleaned up code in R<sub>2eff</sub> estimate module, by making each function independent of class. This is to give a better overview, how the different functions connect together. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Made the user function, which estimates the R<sub>2eff</sub> errors, use the Jacobian derived from &chi;<sup>2</sup> function. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified system test verify_estimate_r2eff_err_compare_mc() to first use the direct function Jacobian, and then the &chi;<sup>2</sup> derived Jacobian. This shows the result better. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added digit to printout in R<sub>2eff</sub> estimate module. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Locked values for system test test_estimate_r2eff_err, to estimate how the R<sub>2eff</sub> error estimation reflects on fitted parameters. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* More locking of values, when trying to use different methods for estimating R<sub>2eff</sub> err values. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* More locking of values. This actually shows, that errors should be estimated from the direct Jacobian. Not, the &chi;<sup>2</sup> Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Reverted the logic, that the &chi;<sup>2</sup> Jacobian should be used. Instead, the direct Jacobian exponential is used instead. When fitting with the estimated errors from the direct Jacobian, the results are MUCH better, and comparable to 2000 Monte Carlo simulations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Various precision fixes for different machine precision. This is in: verify_r1rho_kjaergaard_missing_r1 [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* First attempt at properly implementing the target_functions.relax_fit.jacobian() function. This is now the Jacobian of the chi-squared function. A new jacobian_matrix data structure has been created for holding the matrix data prior to converting it into a Python list of lists. The equation used was simply the chi-squared gradient whereby the sum over i has been dropped and the i elements are stored in the second dimension of matrix.
* Speed up of the target_functions.relax_fit C module. The variances are now pre-calculated in the setup() function from the errors, so that the use of the square() function is minimised. The chi-squared equation, gradient, and Hessian functions now accept the variance rather than standard deviation argument and hence the squaring of errors has been removed. This avoids a lot of duplicated maths operations.
* Added RelaxError, if less than 2 time points is used for exponential curve fitting in R<sub>2eff</sub>. This follows: http://thread.gmane.org/gmane.science.nmr.relax.user/1718 http://thread.gmane.org/gmane.science.nmr.relax.user/1735 Specifically, data was attached here: http://thread.gmane.org/gmane.science.nmr.relax.user/1735/focus=1736.
* Added system test Relax_disp.test_bug_atul_srivastava(), to catch a bug missing raising a RelaxError, since the setup points to a situation where the data shows it is exponential fitting, but only one time point is added per file. This follows: http://thread.gmane.org/gmane.science.nmr.relax.user/1718 http://thread.gmane.org/gmane.science.nmr.relax.user/1735 Specifically, data was attached here: http://thread.gmane.org/gmane.science.nmr.relax.user/1735/focus=1736.
* Parameter precision lowered for Relax_disp.test_estimate_r2eff_err_auto(). This is due to change to C code. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Created the [http://www.nmr-relax.com/manual/select_display.html select.display user function]. This simply displays the current spin selections of all spins. In the future it can be extended to display the interatomic data container selections, domain selections, etc.
* Fix for system test: test_estimate_r2eff_err_auto(). The Jacobian to estimate the errors has been changed from the direct function Jacobian, to the Jacobian of the &chi;<sup>2</sup> function. This changes the R<sub>2eff</sub> error predictions, and hence parameter fitting. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented the direct Jacobian in Python, to be independent of C code in development phase. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Activated all method try in: system test Relax_disp.test_estimate_r2eff_err_methods. This is to quickly estimate errors from all different methods. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix to system test: test_estimate_r2eff_err_auto, which now checks the values for the direct Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Increased the number of time points for exponential curve fitting to 3.
* Fix to weight properly according to if minimising with direct Jacobian or &chi;<sup>2</sup> Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix to system test test_estimate_r2eff_err_methods, after modification of weighting. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Switched in estimate_r2eff_err() to use the &chi;<sup>2</sup> Jacobian from C code, and Jacobian from Python code. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Removed all references to test values which was received by wrong weighting. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Better error checking in the relaxation dispersion overfit_deselect() API method. The model must be set for this procedure to work, and the method now checks that this is the case.
* Better error checking for the specific_analyses.relax_disp.average_intensity() function. This function would fail with a traceback if a peak intensity error analysis had not yet been performed. Now it fails instead with a clean RelaxError so that the user knows what is wrong.
* Tried implementing getting the &chi;<sup>2</sup> gradient, using target_function.chi2.dchi2(). The output seem equal. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Replaced the way to calculate the &chi;<sup>2</sup> Jacobian, for exponential fit in minfx. This is only for the test class, but reuses library code. This should make it much easier in the future to implement &chi;<sup>2</sup> gradient functions to minfx, since it is only necessary to implement the direct gradient of the function, and then pass the direct gradient to &chi;<sup>2</sup> library, which turn it into the &chi;<sup>2</sup> gradient function which minfx use. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Moved unnecessary function in R<sub>2eff</sub> error estimate module into experimental class. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Implemented system test: test_bug_negative_intensities_cpmg, to show lack of error message to user. Maybe these spins should be de-selected, or at least show a better warning. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* An attempt at documenting the Monte Carlo simulation verses covariance matrix error estimates. This is for the R<sub>2eff</sub> and I<sub>0</sub> parameters of the exponential curves. For the Monte Carlo errors, 10000 simulations were preformed. This means that these errors can perform as a gold standard by which to judge the covariance matrix technique. Currently it can be seen that the [http://www.nmr-relax.com/manual/relax_disp_r2eff_err_estimate.html relax_disp.r2eff_err_estimate user function] with the chi2_jacobian flag set to True performs extremely poorly.
* Reintroduced the original target_functions.relax_fit.jacobian() function. The new function for the Jacobian of the chi-squared function has been renamed to target_functions.relax_fit.jacobian_chi2() so that both Python functions are accessible within the C module.
* Epydoc fixes for the pipe_control.mol_res_spin.format_info_full() function.
* Epydoc docstring fixes for many methods in the relaxation dispersion auto-analysis module.
* If math domain errors are found when calculating the two point R<sub>2eff</sub> values, the point is being skipped. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Moved intensity negative value from reference to CPMG point.
* Modified system test test_bug_negative_intensities_cpmg, to prepare for testing number of R<sub>2eff</sub> points. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Comparison of 10,000 Monte Carlo simulations to a different covariance matrix error estimate. The covariance_matrix.py script has been duplicated and the chi2_jacobian argument of the [http://www.nmr-relax.com/manual/relax_disp_r2eff_err_estimate.html relax_disp.r2eff_err_estimate user function] has been changed from True to False. As can be seen in the 2D Grace plots, this error estimate is incredibly different. The R<sub>2eff</sub> errors are overestimated by a factor of 1.9555, which indicates that the Jacobian or covariance matrix formula are not yet correct.
* The target_functions.relax_fit C module Python function jacobian_chi2() is now exposed. This was previously not visible from within Python.
* Added a script and log for calculating the numerical covariance matrix for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the covariance matrix via the Jacobian calculated using the numdifftools.Jacobian object construct and obtain the matrix, both at the minimum and at a point away from the minimum. The covariance is calculated as inv(J^T.W.J).
* Added a script and log for calculating the exponential curve parameter errors via bootstrapping. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the parameter errors via bootstrapping. As the parameters at the minimum are the exact parameter values, bootstrapping and Monte Carlo simulation converge and hence this is a true error estimate. 200,000 simulations where used, so the parameter errors are extremely accurate.
* Modified module to estimate R<sub>2eff</sub> errors, to use the C code Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified system test test_estimate_r2eff_err_methods, to check all Jacobian methods are correctly implemented. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Added more print out information, when log(I / I_ref) is negative, and raising errors. This can help the user track back information to the error more easily.
* Improved system test test_bug_negative_intensities_cpmg, by counting number of R<sub>2eff</sub> points. Spin 4, which has one negative intensity, is expected to have one less R<sub>2eff</sub> point. This makes sure, that all CPMG data set can be loaded and analysed, even if some peaks are very weak are fluctuating with error level.
* Fix for system test test_estimate_r2eff_err and test_r1rho_kjaergaard_missing_r1, where r1_fit=True, needed to be send to Auto_analyses. [https://gna.org/bugs/?22541 Bug #22541: The R<sub>1</sub> fit flag does not work in the GUI].
* API documentation fixes.
* Moved multifit_covar into lib.statistics, since it is an independent module. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Moved "func_exp_grad" into experimental class for different minimisation methods. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Improved [http://www.nmr-relax.com/manual/relax_disp_r2eff_err_estimate.html documentation to user function relax_disp.r2eff_err_estimate], and removed the possibility to use the &chi;<sup>2</sup> Jacobian, as this is rubbish. But the back-end still have this possibility, should one desire to try this. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Moved the argument 'chi2_jacobian' as the last argument in estimate_r2eff_err. This argument is highly likely not to be used, but is kept for future testing purposes. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix to experimental class for fitting with different methods. After moving the function into class, 'self' should be added to the function. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix to system test test_estimate_r2eff_err, after removing the possibility to use the &chi;<sup>2</sup> Jacobian. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Fix for system test test_estimate_r2eff_err_methods. The function was called wrong in experimental class. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Initial try write comments how to generalize the scaling of the covariance according to the reduced &chi;<sup>2</sup> distribution. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* First try to make a test script for estimating efficiency on R<sub>2eff</sub> error calculations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added number of simulations to 10,000 in test script, and varied the random number of time point per simulation between 3 and 10. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* In module for estimating R<sub>2eff</sub> errors, removed "values, errors" to be send to function for gradient, since they are not used. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added Jacobian to test script, and now correctly do simulations, per R<sub>2eff</sub> points. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Improved analysing test script, with plotting. It seems that R<sub>2eff</sub> error estimation always get the same result. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added initial dataset for test analysis. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Deleted test data set. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified data script generator, to handle a situation with fixed 5 time points, and a situations with variable number of time points. Also modified analysis script. It seems, this has an influence how the error estimation is performing. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added simulations that show, there is perfect agreement between Monte Carlo simulations and covariance estimation. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Inserted extra tests in system test Relax_disp.test_estimate_r2eff_err_methods to test that all values of R and I<sub>0</sub> are positive, and the standard deviation from Monte Carlo simulations are equal. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Inserted system test Relax_disp.test_finite_value, to illustrate the return of inf from C code exponential, when R is negative. This can be an issue, if minfx takes a wrong step when no constraints are implemented. [https://gna.org/bugs/?22552 Bug #22552: &chi;<sup>2</sup> value returned from C code curve-fitting return 0.0 for wrong parameters -> Expected influence on Monte Carlo sim].
* Inserted possibility for bootstrapping in system test Relax_disp.test_estimate_r2eff_err_methods. This shows, that the bootstrapping method get the SAME estimation for R<sub>2eff</sub> errors, as the estimate_r2eff_err() function. This must either mean, that the OLD Monte Carlo simulation was corrupted, or the creation of data in Monte Carlo simulations is corrupted.
* Modified system test Relax_disp.verify_estimate_r2eff_err_compare_mc to include bootstrapping method. This shows there is excellent agreement between bootstrapping and estimation of errors from covariance, while relax Monte Carlo simulations are very much different. Boot strapping is the "-2": [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added functionality to create peak lists, for virtual data. This is to compare the distribution of R<sub>2eff</sub> values made by bootstrapping and Monte Carlo simulations. Rest of the analysis will be performed in relax. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added initial peak lists to be analysed in relax for test purposes. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added relax analysis script, to profile distribution of errors drawn in relax, and from Python module "random". It seems that relax draw a lot more narrow distribution of Intensity with errors, than Python module "random". This has an influence on estimated parameter error. This is a potential huge error in relax. A possible example of a catastrophic implementation of Monte Carlo simulations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Added PNG image that show that the distribution which relax makes are to narrow. This is a potential huge flaw in implementation of Monte Carlo simulations. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Modified analysis script, to also make histogram of intensities. This shows that the created intensities are totally off the true intensity. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.* Comment fix to system test Relax_disp.test_estimate_r2eff_err_methods, after the found of bug in relax. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}. [https://gna.org/bugs/?22554 Bug #22554: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected].* Cleaned up user function for estimating R<sub>2eff</sub> errors. Extensive tests have shown, there is a very good agreement between the covariance estimation, and Monte Carlo simulations. This is indeed a very positive implementation. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}. [https://gna.org/bugs/?22554 Bug #22554: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected].* Removed all junk comments from module for R<sub>2eff</sub> error estimation. The module runs perfect as it does now. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}. [https://gna.org/bugs/?22554 Bug #22554: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected].
* Fix for inf values being returned from C code exponential function. Values are now converted to high values. This fixes system test Relax_disp.test_finite_value. Example: x = np.array([np.inf, -np.inf, np.nan, -128, 128]), np.nan_to_num(x), array([ 1.79769313e+308, -1.79769313e+308, 0.00000000e+000, -1.28000000e+002, 1.28000000e+002]). [https://gna.org/bugs/?22552 Bug #2255: &chi;<sup>2</sup> value returned from C code curve-fitting return 0.0 for wrong parameters -> Expected influence on Monte Carlo sim]. Ref: http://docs.scipy.org/doc/numpy/reference/generated/numpy.nan_to_num.html.
* Initial try to reach constrained methods in minfx through relax. This is in system test Relax_disp.verify_estimate_r2eff_err_compare_mc() This though not seem to be supported.
* Fix for time not extracted for CPMG experiments in target_function. [https://gna.org/bugs/?22461 Bug #22461: NS R1rho 2-site_fit_r1 has extremely high &chi;<sup>2</sup> value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1].
* Fix for interpolating time points, when producing xmgrace files for CPMG experiments. [https://gna.org/bugs/?22461 Bug #22461: NS R1rho 2-site_fit_r1 has extremely high &chi;<sup>2</sup> value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1].
* Correction for catastrophic implementation of Monte Carlo simulations for exponential curve-fitting R<sub>2eff</sub> values in the dispersion analysis. A wrong implemented "else if" statement, would add the intensity for the simulated intensity together with the original intensity. This means that all intensity values send to minimisation would be twice as high than usually (if spectra were not replicated). This was discovered for Monte Carlo simulations of R<sub>2eff</sub> errors in exponential fit. This will affect all analyses using full relaxation exponential curves until now. By pure luck, it seems that the effect of this would be that R<sub>2eff</sub> errors are half the value they should be. A further investigation shows, that for the selected data set, this had a minimum on influence on the fitted parameters, because the &chi;<sup>2</sup> value would be scaled up by a factor 4. [https://gna.org/bugs/?22554 Bug #22554: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected]. [https://{{gna.org/task/?link|7822 |text=Task #7822: Implement user function to estimate R<sub>2eff</sub> and associated errors for exponential curve fitting]}}.
* Added a minfx minimum version check to the dep_check module. This is to avoid problems such as that reported at [http://gna.org/bugs/?22408 bug #22408].
<section end=bugfixes/>
Trusted, Bureaucrats
4,223

edits