Trusted, Bureaucrats
4,228
edits
Changes
Jump to navigation
Jump to search
m
* Split out the interpolating in specific_analyses.relax_disp.data.plot_disp_curves() into separate function. This is to prepare for a interpolating function for spin-lock offset rather than spin-lock field strength for R1rho models. [https://gna.org/support/?3124 Sr #3124]: Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. [https* Split out the interpolating in specific_analyses.relax_disp.data.plot_disp_curves() into separate function. This is to prepare for a interpolating function for spin-lock offset rather than spin-lock field strength for R<sub>1ρ</sub> models. [https://gna.org/support/?3124 Sr #3124]: Grace graphs production for R
* Added graphs and results for run with MC=2000, for system test Relax_disp.test_r1rho_kjaergaard_auto(). This is to be able to extend graph testing for interpolated R1rho graphs, and to add figures to the latex manual.* Added list of R1rho models, which use R1 in their equations. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models. [https://* Added graphs and results for run with MC=2000, for system test Relax_disp.test_r1rho_kjaergaard_auto(). This is to be able to extend graph testing for interpolated R<sub>1ρ</sub> graphs, and to add figures to the latex manual.* Added list of R<sub>1ρ</sub> models, which use R1 in their equations. [https://gna.org/support/?3135 Sr #3135]:
→Changes: Pretty R1rho formatting.
* Switched to use self.ei, self.si, self.mi, self.oi, self.di. This is for better reading of code. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Comment out the sys.exit(), which would make the code fail for wrong calculation of dw. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling script for CPMG model [[CR72]] to R1rho R<sub>1ρ</sub> [[DPL94]] model. The framework of the script will be the same, but the data a little different. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Started converting profiling script to [[DPL94]]. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced self.(ei,si,mi,oi,di) with self.(NE,NS,NM,NO,ND). These numbers represents the maximum number of dimensions, instead of index. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Used the new class function: get_back_calc(), to get the data in the right structures when interpolating for graphs. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed superfluous check, after the returned data is now in right structure. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Made changes to the dir argument of system test Relax_disp.test_r1rho_kjaergaard. This is to prepare for: [https://gna.org/support/?3124 sr #3124]: Grace graphs production for R1rho R<sub>1ρ</sub> analysis with R2eff as function of Omega_eff and [https://gna.org/support/?3138 sr #3138]: Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1]. This is also to test an expected bug, if R1 is not loaded. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* The relaxation dispersion target function can now be set up when the optional frqs_H argument is None. This allows the profiling scripts to run.
* More stability fixes for the relaxation dispersion target function initialisation. The target function can now be initialised when the r1 and chemical_shift arguments are None.
* The 'relax -v' command is now used for the dispersion model profiling script initial printout. This is to show the two different relax versions being compared.
* Modifications to the dispersion model profiling master script. The info.print_sys_info() function of the current relax version is being called at the start to show all information about the current system. This is useful to know the speed of the machine, the OS, the Python version and numpy version. The numpy version is important as future versions might optimise certain functions that are currently very slow, hence that could be a cause of model speed differences. In addition, the path variables path1 and path2 have been renamed to path_new and path_old to make it clearer which is which. And the individual profiling scripts are no longer copied to the base directory of the relax versions being compared, and are run in place.
* Added a relaxation dispersion model profiling log file for relax version 3.2.2 vs. 3.2.1. This is the output from the dispersion model profiling master script. It shows that the relax 3.2.2 release did not in fact feature "a large speed up of all analytic relaxation dispersion models" as described in the release notes at https://gna.org/forum/forum.php?forum_id=2465. For the CPMG models there is a 1 to 2 times increase in speed. But for the R1rho R<sub>1ρ</sub> models, there is a 1 to 2 times decrease in speed.
* Added a relaxation dispersion model profiling log file for relax version 3.2.0 vs. 3.1.7. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
* Added a relaxation dispersion model profiling log file for relax version 3.1.7 vs. 3.1.6. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
* Created a unit test for the pipe_control.mol_res_spin.format_info_full() function. This comprehensive test covers all input argument combinations.
* Changed the behaviour of the pipe_control.structure.mass.pipe_centre_of_mass() function. This function returns the CoM and optionally the mass of the structural data loaded into the current data pipe. However it was matching the structural data to the molecule-residue-spin data structure and skipping spins that were deselected. This illogical deselection part has been eliminated, as spins can be deselected for various analysis purposes and this should not change the CoM. The deletion also significantly speeds up the function.
* Added Andy Baldwin's 2013 R1rho R<sub>1ρ</sub> relaxation dispersion model (BK13) to the manual. The model has been added to the table of dispersion models and to the dispersion software comparison table of the dispersion chapter of the manual. The citation has also been added to the bibliography.
* The BK13 dispersion model is now properly added to the software comparison table.
* Added the 'BK13' and 'BK13 full' dispersion models to the to do section of the manual.
* Expansion of the 'to do' section of the dispersion chapter of the manual.
* Editing of the 'to do' section of the dispersion chapter of the manual.
* Created the Bruker.test_bug_22411_T1_read_fail system test. This is to catch [https://gna.org/bugs/?22411 bug #22411] as reported by Olena Dobrovolska.
* Fix for system test Relax_disp.test_kteilum_fmpoulsen_makke_check_graphs where minimise has been extended with execute.
* Fixes for numpy version under 1.8, when striding through data.
* Fix to unit tests, after changing the name of matrix_exponential function.
* Removed triggering an error in test_r1rho_kjaergaard_missing_r1. There is a bug fetching the standard value of parameter 'r1_fit'.
* Fix for system test Relax_disp.test_r1rho_kjaergaard_missing_r1, running on 64 bit system.
* Replaced variable name: MODEL_PARAM_INV_RELAX_TIMES with MODEL_LIST_INV_RELAX_TIMES, to match all of the other MODEL_LIST_* variables. Also added a newline to end of file.
* Replaced remaining variable names: MODEL_PARAM_* with MODEL_LIST_*, to match all of the other MODEL_LIST_* variables.
* Renamed R1rho R<sub>1ρ</sub> off resonance models where R1 is fitted. This removes the "underscore". This is a better representation presented to the user, for example in the GUI model selection list or the relax_disp.select_model user function in all UI.
* Renamed the parameter "r1_fit" to "r1". This naming fits better to all other parameters.
* Split the unit test of specific_analyses.relax_disp.checks.get_times() into its own unit test file.
* Removed the unit test regarding model.py in test_variables.py.
* Added unit tests regarding model.py and its functions.
* Removed the auto-sorting of models, when performing auto analysis of Relaxation dispersion. This was discussed in: http://thread.gmane.org/gmane.science.nmr.relax.scm/22733, http://thread.gmane.org/gmane.science.nmr.relax.scm/22734, http://thread.gmane.org/gmane.science.nmr.relax.scm/22737. Through this discussion, it has appeared that the order of how models are sorted for analysis, and hence the possibility for nesting, is a complicated case. The order of analysis should be possible to manually put into the auto analysis. This was not the scope of [https://gna.org/support/?3135 sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho R<sub>1ρ</sub> relaxation dispersion models to implement such a feature. Such a feature could be implemented for the next version of relax. It could be designed as function to "suggest" an order in the GUI. But this functionality would have to wait.
* Removing the 'tex'->'kex' conversion, and 'phi_ex' from dw and p<sub>A</sub>. This solution is not a proper implementation, but these parameters should rather be found by grid search.
* Re-inserted "MODEL_NS_CPMG_2SITE_EXPANDED" to be tested in system test test_hansen_cpmg_data_missing_auto_analysis.
* Fix for earlier bug fix destroyed functionality. Altering the data-keys to early, meant that data was not fetched correctly. This is related to: https://gna.org/support/?3121 sr #3121: Support request for replacing space in header files for the value.write functions.
* Moved the unit test of specific_analyses.relax_disp.checks.check_missing_r1() from a unit test to a system test. This is because the unit test involved several functions of relax.
* Inserted dictionary, that will convert a R1rho R<sub>1ρ</sub> off-resonance without R1, to the corresponding model which fit R1.
* Inserted to the check of missing R1, that MODEL_NOREX_R1RHO also depends on R1.
* Implemented function that determine if any model in the list of all models should be replaced or inserted as the correct [[No Rex]] model. It also translate the R1rho R<sub>1ρ</sub> off-resonance model to the corresponding 'R1 fit' models, if R1 is not loaded.
* Inserted system test Relax_disp.test_convert_no_rex_fit_r1, which test the return for the function that will determine if models self.models in the relax_disp should be translated/corrected.
* Fix for unit test, where the standard value of R1 was lowered from 5.0 to 2.0. Also fixed an import error in another unit test.
* Minimised the dependencies of the version module. This no longer relies on the dep_check module.
* Inserted return of True/False flags from function which convert models. The flag tells if: flag if [[No Rex]] model for R1rho R<sub>1ρ</sub> off-resonance was translated; flag if [[No Rex]] model for R1rho R<sub>1ρ</sub> off-resonance was inserted; flag if R1rho R<sub>1ρ</sub> off-resonance was translated to 'R1 fit' models if no R1 data was found.
* Changes to system test, after the number of returns from function has been altered.
* Inserted into relax_disp auto_analyses, to convert the input models. This will convert/insert the correct [[No Rex]] to the corresponding [[No Rex]] model for R1rho R<sub>1ρ</sub> off-resonance models. It will also translate to the corresponding 'R1 fit' model, if no 'R1 data' has been loaded with relax_data.read() function.
* Lowering of precision in system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). This is due to 64/32-bit issues, between analysing on Linux computer, and testing on Mac computer.
* Made the GUI selection of models for relaxation dispersion more simple. After the implementation of a function which will translate the models, the [[No Rex]] model will be converted to the [[No Rex]] model for R1rho R<sub>1ρ</sub> off-resonance. Also the corresponding 'R1 fit' model will be chosen instead, if R1 data has not been loaded. This makes the model selection easier in the GUI interface.
* Bugfix for Relax_disp.test_bug_21715_clustered_indexerror, where only 'R2eff', [[No Rex]] is analysed. This special case was not tested in the translating function.
* Shortening the text in the auto_analysis, and raises a warning if R1 data has not been loaded.
* Removed all of the '* R1 fit' models out of the relax_disp.select_model user function frontend. These models are now redundant as the question of R1 fitting is now determined internally in relax.
* Removed all of the MODEL_*_FIT_R1 dependencies from the specific_analyses.relax_disp package. These models are now redundant as the question of R1 fitting is now determined internally in relax.
* Fix for the specific_analyses.relax_disp.data.is_r1_optimised() function for on-resonance R1rho R<sub>1ρ</sub> data. This function needs to specifically catch these models.* Fix for the MODEL_LIST_R1RHO variable. Recent changes causes this to not include the on-resonance R1rho R<sub>1ρ</sub> dispersion models.
* Import fix for the Relax_disp.test_model_nesting_and_param system test. Somehow the import of the convert_no_rex() function was lost.
* Modified the MODEL_LIST_R1RHO_OFF_RES list to include MODEL_NOREX_R1RHO.
* Updated the dispersion auto-analysis for the universal [[No Rex]] model. The 'No Rex R1rho off res' references have all been deleted. The model conversion logic is also no longer needed and has been deleted.
* Converted the relaxation dispersion GUI interface to the unified [[No Rex]] model. All of the MODEL_NOREX_R1RHO references have simply been deleted.
* Converted the relaxation dispersion target function class to the unified [[No Rex]] model design. On top of removing all references to MODEL_NOREX_R1RHO, the aliasing of self.func now checks the experiment type list to determine which target function to use. This is not an ideal solution and will not handle mixed CPMG and R1rho R<sub>1ρ</sub> experiment, however neither will the target functions yet. The creation of the off-resonance data structures has also been modified so that they are now R1rho R<sub>1ρ</sub> independent. This allows the structures to be properly created while at the same time enabling this code to be compatible with off-resonance CPMG data in the future.
* Removed the 'No Rex R1rho off res' model from the relax_disp.select_model user function frontend.
* Removed all references to the 'No Rex R1rho off res' model in the system tests. In addition, the Relax_disp.test_convert_no_rex system test has been deleted as it no longer has a purpose. For the Relax_disp.test_model_nesting_and_param system test, to allow this to work the cdp.exp_type_list list is set to EXP_TYPE_LIST.
* Removed all references to the 'No Rex R1rho off res' model in the unit tests.
* Updated the [[No Rex]] dispersion model description in the relax manual. The universal nature of the model is now described, including the addition of the off-resonance CPMG and R1rho R<sub>1ρ</sub> equations for the absence of chemical exchange. The R1 parameter optimisation is also shortly covered.
* Added a subsection to the dispersion chapter of the manual about R1 parameter optimisation.
* Added the R1 parameter fitting GUI element to the dispersion GUI tab. This is a simple Boolean toggle element that allows the R1 optimisation to be turned on. The value is passed into the auto-analysis.
* Added the offset column to the spectrum list GUI element for the dispersion analysis. This is to complete [https://gna.org/task/?7820 task #7820]. The spectrum list GUI element add_offset() method has been added to insert the offset column when the relax_disp_flag is set. This is called by the update_data() method to fill and update the GUI element.
* Implemented the GUI test Relax_disp.test_bug_22501_close_all_analyse to catch [https://gna.org/bugs/?22501 bug #22501], 'Close all analyses' raises error.
* Inserted intermediate system test, to profile R2eff calculation for R1rhoR<sub>1ρ</sub>. System test is: Relax_disp.test_bug_9999_slow_r1rho_r2eff_error_with_mc. This system test actually fails, if one tries to do a grid search. This is related to the R2eff values stored as dictionary, and pipe_control.minimise.grid_setup() will fail. The function 'isNaN(values[i])' cannot handle dictionary.
* Modified intermediate system test Relax_disp.test_bug_9999_slow_r1rho_r2eff_error_with_mc to see if the initial grid search for 'i0' and 'R2eff' estimation can be skipped. This is done by converting the exponential curve, to a linear curve, and calculate the best parameters by a line of best fit by least squares. This seems like a promising method as an initial estimator of 'i0' and 'r2eff'. For 500 to 2000 Monte Carlo simulations, this could have an influence on the timings, as all grid searchs could be skipped.
* Modified system test test_bug_9999_slow_r1rho_r2eff_error_with_mc to save data arrays. This is to prepare a profiling script.
* Further extended the profiling script for curve fitting. Now profiling is in place for the implemented C code method in relax. A similar code should now be devised for numpy array for comparing. But this profiling shows that when contraints=True, is slowing down this procedure by a factor 10 X.
* Further improved the profiling of relax curve fit. This profiling shows, that Python code is about twice as slow as the C code implemented. But it also shows that optimising with scipy.optimize.leastsq is 20 X faster. It also gives reasonable error values. Combining a function for a linear fit to guess the initial values, together with scipy optimise, will be an extreme time win for estimating R2eff values fast. A further test would be to use relax Monte Carlo simulations for say 1000-2000 iterations, and compare to the errors extracted from estimated covariance.
* Added verification script, that shows that using scipy.optimize.leastsq reaches the exact same parameters as minfx for exponential curve fitting. The profiling shows that scipy.optimize.leastsq is 10X as fast as using minfx (with no linear constraints). scipy.optimize.leastsq is a wrapper around wrapper around MINPACK's lmdif and lmder algorithms. MINPACK is a FORTRAN90 library which solves systems of nonlinear equations, or carries out the least squares minimization of the residual of a set of linear or nonlinear equations. The verification script also shows, that a very heavy and time consuming Monte Carlo simulation of 2000 steps, reaches the same errors as the errors reported by scipy.optimize.leastsq. The return from scipy.optimize.leastsq, gives the estimated covariance. Taking the square root of the covariance corresponds with 2X error reported by minfx. This could be an extremely time saving step, when performing model fitting in R1rhoR<sub>1ρ</sub>, where the errors of the R2eff values, are estimated by Monte Carlo simulations. The following setup illustrates the problem. This was analysed on a MacBook Pro, 13-inch, Late 2011 with no multi-core setup. Script running is: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. This script analyses just the R2eff values for 15 residues. It estimates the errors of R2eff based on 2000 Monte Carlo simulations. For each residues, there is 14 exponential graphs. The script was broken after 35 simulations. This was measured to 20 minutes. So 500 simulations would take about 4.8 Hours. The R2eff values and errors can by scipy.optimize.leastsq can instead be calculated in: 15 residues * 0.02 seconds = 0.3 seconds.
* Moved the target function for minimisation of exponential fit into the target functions folder. [https://gna.org/task/?7822 Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Implemented initial system test Relax_disp.test_estimate_r2eff for setting up the new user function to estimate R2eff and errors by scipy. [https://gna.org/task/?7822 Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Modified profiling script, but it seems that the dfunc from target_functions.relax_fit does not work. [https://gna.org/task/?7822 Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Modified estimate R2eff module, to use C code. But system test Relax_disp.test_estimate_r2eff_error shows that the Jacobian is not correctly implemented to be called in minfx.
* Created an initial test suite data directory for a mixed R1rho R<sub>1ρ</sub> + CPMG dispersion analysis. The generate.py script will be extended in the future to generate both synthetic R1rho R<sub>1ρ</sub> and CPMG data for a common exchange process. Such a data combination should show some minor flaws in the current design of the dispersion analysis and will help to solve these.
* Improvements to the pipe_control.minimise.reset_min_stats() function. The minimise statistics resetting is now more elegantly implemented. And the sim_index keyword argument is accepted by the function and individual Monte Carlo simulation elements can now be reset.
* Modified wrapper function for curve_fit, to only change to list type, if the type is a ndarray. [https://gna.org/task/?7822 Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Yet another try to implement constrained method in verify_estimate_r2eff_err_compare_mc.
* Another attempt to reach constrained method in minfx through relax. I would need to specify: l, lower bound constraint vector (l <= x <= u); u, upper bound constraint vector (l <= x <= u); c: user supplied constraint function; dc: user supplied constraint gradient function.
* Added a derivation of the R2eff/R1rho R<sub>1ρ</sub> error estimate for the two-point measurement to the manual. This is from http://thread.gmane.org/gmane.science.nmr.relax.devel/6929/focus=6993 and is for the rate uncertainty of a 2-parameter exponential curve when only two data points have been collected. The derivation has been added to the dispersion chapter of the manual.
* Equation fixes for the two-point exponential error derivation in the dispersion chapter of the manual.
* Updated the minfx version numbers in the release checklist document. The version is now [https://gna.org/forum/forum.php?forum_id=2475 1.0.10], which has not been released yet but will contain the implementation of the log-barrier constraint algorithm gradient and Hessian.
