Revision as of 08:48, 5 September 2014

Description

This is a major feature release which includes a huge number of changes, as can be seen below. The most important change is an incredible speed up of all relaxation dispersion models. See the table below for a comparison to the previous relax 3.2.3 release. The maximum possible advantage of linear algebra operations are used to eliminate all of the slow Python looping and to obtain the ultimate algorithms for speed. As this is using NumPy, conversion to C or FORTRAN will not result in any significant speed advantage. With these huge speed ups, relax should now be one of the fastest software packages for analysing relaxation dispersion phenomena.

Other important features include the implementation of a zooming grid search algorithm for use in all analysis types, expanded plotting capabilities for R1rho values in the relaxation dispersion analysis, the ability to optimise the R1 parameter in all off-resonance dispersion models, proper minimisation statistics resetting by the minimisation user functions, and a large expansion of the periodic table information for all elements in the relax library for correctly estimating molecular masses. Additional features are that there is better tab completion support in the prompt UI for Mac OS X, the addition of the 'time' user function for printing the current date and time, the value.copy user function accepting a force argument for overwriting values, model nesting in the dispersion auto-analysis has been extended, the spin-lock offset is now shown in the dispersion analysis in the GUI, the relax_disp.r2eff_estimate user function has been added for fast R2eff and I0 parameter value and error estimation, and gradient and Hessian functions have been added to the exponential curve-fitting C module allowing for more advanced optimisation in the relaxation curve-fitting and dispersion analyses.

Note that this new 3.3 relax series breaks compatibility with old relax scripts. The important change, which is the main reason for starting the relax 3.3.x line, is the renaming of the calc, grid_search and minimise user functions to minimise.calculate, minimise.grid_search and minimise.execute respectively. Please update your scripts appropriately. A new relax feature is that old user function calls are detected in the prompt and script UIs and a RelaxError raised explaining what to rename the user function to.

Important bugfixes in this release include that relax can run on MS Windows systems again, numerous Python 3 fixes, the ability to load Bruker DC files when the file format has corrupted whitespace, the GUI "close all analyses" feature works and no longer raises an error, structure.create_diff_tensor_pdb user function now works when no structural data is present, the geometric prolate diffusion 3D PDB representation in a model-free analysis now aligns with the axis in the PDB as it was previously rotated by 90 degrees, and the Monte Carlo simulations in the relaxation dispersion analysis for exponential curve-fitting for R2eff/R1rho parameter errors is now correct and no longer underestimating the errors by half. For more details about the new features and the bug fixes, please see below. For fully formatted and easy to navigate release notes, please see http://wiki.nmr-relax.com/Relax_3.3.0.

To demonstrate the huge speeds ups in the relaxation dispersion analysis, the following table compares the speed of dispersion models in relax 3.2.3 compared to the new 3.3.0 version:

100 single spins analysis (times in seconds):
No Rex:                      0.839+/-0.018 ->   0.279+/-0.007,   3.004x faster.
LM63:                        1.633+/-0.040 ->   0.778+/-0.007,   2.098x faster.
LM63 3-site:                 3.188+/-0.066 ->   1.022+/-0.010,   3.120x faster.
CR72:                        2.615+/-0.018 ->   1.614+/-0.024,   1.621x faster.
CR72 full:                   2.678+/-0.020 ->   1.839+/-0.165,   1.456x faster.
IT99:                        1.837+/-0.030 ->   0.881+/-0.010,   2.086x faster.
TSMFK01:                     1.665+/-0.049 ->   0.742+/-0.007,   2.243x faster.
B14:                         5.851+/-0.133 ->   3.978+/-0.049,   1.471x faster.
B14 full:                    5.789+/-0.102 ->   4.065+/-0.059,   1.424x faster.
NS CPMG 2-site expanded:     8.225+/-0.196 ->   4.140+/-0.062,   1.987x faster.
NS CPMG 2-site 3D:         240.027+/-3.182 ->  45.056+/-0.584,   5.327x faster.
NS CPMG 2-site 3D full:    240.910+/-4.882 ->  45.230+/-0.540,   5.326x faster.
NS CPMG 2-site star:       186.480+/-2.299 ->  36.400+/-0.397,   5.123x faster.
NS CPMG 2-site star full:  187.111+/-2.791 ->  36.745+/-0.689,   5.092x faster.
MMQ CR72:                    5.821+/-0.088 ->   4.261+/-0.049,   1.366x faster.
NS MMQ 2-site:             368.639+/-3.248 ->  81.435+/-0.776,   4.527x faster.
NS MMQ 3-site linear:      393.104+/-4.055 ->  91.227+/-1.184,   4.309x faster.
NS MMQ 3-site:             398.257+/-5.613 ->  92.818+/-1.723,   4.291x faster.
M61:                         1.622+/-0.023 ->   0.889+/-0.012,   1.823x faster.
DPL94:                      23.525+/-0.409 ->   1.138+/-0.035,  20.676x faster.
TP02:                       20.191+/-0.375 ->   1.238+/-0.020,  16.308x faster.
TAP03:                      31.993+/-0.235 ->   1.956+/-0.025,  16.353x faster.
MP05:                       24.892+/-0.354 ->   1.431+/-0.014,  17.395x faster.
NS R1rho 2-site:           245.961+/-2.637 ->  36.308+/-0.458,   6.774x faster.
NS R1rho 3-site linear:    292.916+/-4.264 ->  70.334+/-0.829,   4.165x faster.
NS R1rho 3-site:           296.054+/-3.808 ->  72.046+/-0.740,   4.109x faster.

Cluster of 100 spins analysis (times in seconds):
No Rex:                      0.826+/-0.016 ->   0.008+/-0.001, 100.732x faster. 
LM63:                        1.625+/-0.031 ->   0.037+/-0.000,  43.694x faster. 
LM63 3-site:                 3.125+/-0.075 ->   0.068+/-0.001,  45.827x faster. 
CR72:                        2.610+/-0.035 ->   0.118+/-0.001,  22.138x faster. 
CR72 full:                   2.674+/-0.021 ->   0.122+/-0.001,  21.882x faster. 
IT99:                        0.018+/-0.000 ->   0.009+/-0.000,   2.044x faster.
TSMFK01:                     1.691+/-0.091 ->   0.039+/-0.000,  43.369x faster. 
B14:                         5.891+/-0.127 ->   0.523+/-0.004,  11.264x faster. 
B14 full:                    5.818+/-0.127 ->   0.515+/-0.021,  11.295x faster. 
NS CPMG 2-site expanded:     8.167+/-0.159 ->   0.702+/-0.008,  11.638x faster. 
NS CPMG 2-site 3D:         238.717+/-3.025 ->  41.380+/-0.950,   5.769x faster. 
NS CPMG 2-site 3D full:    507.411+/-803.09 -> 41.852+/-1.317,  12.124x faster. 
NS CPMG 2-site star:       187.004+/-1.935 ->  30.823+/-0.371,   6.067x faster. 
NS CPMG 2-site star full:  187.852+/-2.698 ->  30.882+/-0.671,   6.083x faster. 
MMQ CR72:                    5.961+/-0.086 ->   0.880+/-0.013,   6.774x faster. 
NS MMQ 2-site:             369.150+/-3.742 ->  75.403+/-1.095,   4.896x faster. 
NS MMQ 3-site linear:      390.939+/-4.415 ->  81.999+/-0.500,   4.768x faster. 
NS MMQ 3-site:             394.257+/-7.500 ->  83.291+/-1.552,   4.733x faster. 
M61:                         1.607+/-0.023 ->   0.035+/-0.000,  45.404x faster. 
DPL94:                      23.872+/-0.505 ->   0.145+/-0.002, 164.180x faster. 
TP02:                       20.445+/-0.411 ->   0.172+/-0.004, 118.662x faster. 
TAP03:                      32.212+/-0.234 ->   0.294+/-0.002, 109.714x faster. 
MP05:                       25.013+/-0.362 ->   0.188+/-0.003, 132.834x faster. 
NS R1rho 2-site:           246.024+/-3.724 ->  33.119+/-0.320,   7.428x faster. 
NS R1rho 3-site linear:    286.851+/-3.294 ->  64.882+/-0.687,   4.421x faster. 
NS R1rho 3-site:           287.560+/-4.145 ->  64.679+/-1.241,   4.446x faster.

For CPMG statistics: 3 fields, each with 20 CPMG points. Total number of dispersion points per spin is 60.

For R1rho experiments: 3 fields, each with 10 spin lock offsets, and each offset has been measured at 5 different spin lock fields. Per field there is 50 dispersion points. Total number of dispersion points per spin is 150.

Download

The new relax versions can be downloaded from http://www.nmr-relax.com/download.html. If binary distributions are not yet available for your platform and you manage to compile the binary modules, please consider contributing these to the relax project (described in section 3.6 of the relax manual, http://www.nmr-relax.com/manual/relax_distribution_archives.html).

CHANGES file

Version 3.3.0 (3 September 2014, from /trunk) http://svn.gna.org/svn/relax/tags/3.3.0

Features

Huge speed ups for all of the relaxation dispersion models ranging from 1.366x to 164.180x times faster. The speed ups for the clustered spin analysis are far greater than for the single spin analysis.
Implementation of a zooming grid search algorithm for optimisation in all analyses. This includes the addition of the minimise.grid_zoom user function to set the zoom level. The grid width will be divided by 2**zoom_level and centred at the current parameter values. If the new grid is outside of the bounds of the original grid, the entire grid will be translated so that it lies entirely within the original.
Increased the amount of user feedback for the minimise.grid_search user function. Now a comment for each parameter is included in the printed grid search setup table. This includes if the lower or upper bounds, or both, have been supplied and if a preset value has been used instead.
Improved support for R1rho 2D graph plotting in the relax_disp.plot_disp_curves user function as the X-axis can now be the nu1 value, the effective field omega_eff, or the rotating frame title angle. And the plots are interpolation over the spin-lock offset.
Ability to optimise the R1 relaxation rate parameter in the off-resonance relaxation dispersion models.
Creation of the relax_disp.r1_fit user function for activating and deactivating R1 fitting in the dispersion analysis.
Better tab completion support in the prompt UI for Mac OS X users. For some Python versions, the Mac supplied libedit library is used rather than GNU readline. But this library uses a completely different language and hence tab completion was non-functional on these systems. The library difference is now detected and the correct language sent into libedit to activate tab completion.
Created the time user function. This is just a shortcut for printing out the output of the time.asctime() function.
The value.copy user function now accepts the force flag to allow destination values to be overwritten.
Expanded model nesting capabilities in the relaxation dispersion auto-analysis to speed up the protocol.
The spin-lock offset is now included in the spectra list GUI element for the relaxation dispersion analysis.
Creation of the relax_disp.r2eff_estimate user function for the fast estimation of R2eff/R1rho values and errors when full exponential curves have been collected. This experiment feature uses linearisation to estimate the R2eff and I0 parameters and the covariance matrix to estimate parameter errors.
Gradients and Hessians are implemented for the exponential curve-fitting, hence all optimisation algorithms and constraint algorithms are now available for this analysis type. Using Newton optimisation instead of Nelder-Mead Simplex can save over an order of magnitude in computation time. This is also available in the relaxation dispersion analysis.
The minimisation statistics are now being reset for all analysis types. The minimise.calculate, minimise.grid_search, and minimise.execute user functions now all reset the minimisation statistics for either the model or the Monte Carlo simulations prior to performing any optimisation. This is required for both parallelised grid searches and repetitive optimisation schemes to allow the result to overwrite an old result in all situations, as sometimes the original chi-squared value is lower and the new result hence is rejected.
Large expansion of the periodic table information in the relax library to include all elements, the IUPAC 2011 standard atomic weights for all elements, mass numbers and atomic masses for all stable isotopes, and gyromagnetic ratios.
Significant improvements to the structure centre of mass calculations by using the new periodic table information - all elements are now supported and exact masses are now used.
Added a button to the spectra list GUI element for the spectrum.error_analysis user function. This is placed after the 'Add' and 'Delete' buttons and is used in the NOE, R1 and R2 curve-fitting and relaxation dispersion analyses.
RelaxErrors are now raised in the prompt or script UI if an old user function is called, printing out the names of the old and new user functions. This is for help in upgrading old scripts and is currently for the calc(), grid_search(), and minimise() user function calls.

Changes

Improved model handling for the internal structural object. The set_model() method has been added to allow either a model number to be set to the first unnumbered model (in preparation for adding new models) or to allow models to be renumbered. The logic of the add_model() has also been changed. Rather than looping over all atoms of the first model and copying them, which does not work due to the model validity checks, the entire MolList (molecule list) data structure is copied using copy.deepcopy() to make a perfect copy of the structural data. The ModelList.add_item() method has also been modified to return the newly added or numbered model. This is used by the add_model() structural object method to obtain the model object.
Updated the Mac OS X framework setting up instruction document. New sections have been added for the nose and matplotlib Python packages, as nose is needed for the numpy and scipy testing frameworks and matplotlib might be a useful optional dependency in the future. The mpy4py section has been updated to avoid the non-framework fink version of mpicc which cannot produce universal binaries. A few other parts also have small edits.
Removed the Freecode section from the release checklist as Freecode has been permanently shut down. The old relax links are still there (http://freecode.com/projects/nmr-relax), but Freecode is dead (http://freecode.com/about).
Fix for the internal structural object MolContainer.last_residue() method. This can now operate when no structural information is present, returning 0 instead of resulting in an IndexError.
Updated the script for finding unused imports in the relax source code. Now the file name is only printed for Python files which have unused imports.
Completely removed all mentions of Freecode from the release document. The old relax links are still there (http://freecode.com/projects/nmr-relax), but Freecode is dead (http://freecode.com/about).
Updated the minfx version in the release checklist document to 1.0.8. This version has not been released yet, but it will include important fixes and additions for constrained parallelised grid searches.
Fix for a broken link in the development chapter of the relax manual.
Fixes for dead hyperlinks in the relaxation dispersion chapter of the relax manual. The B14 model links to http://www.nmr-relax.com/api/3.2/lib.dispersion.b14-module.html were broken as the B in B14 was capitalised.
Sent in the verbosity argument value to the minfx.grid.grid_split() function. The minfx function in the next release (1.0.8) will now be more verbose, so this will help with user feedback when running the model-free analysis on a cluster or multi-core system using MPI.
The time user function now uses the chronometer Oxygen icon in the GUI.
Removed the line wrapping in the epydoc parameter section of the optimisation function docstrings. This is for the pipe_control.minimise module.
More docstring line wrapping removal from pipe_control.minimise.
Bug fix for the parameter units descriptions. This only affects a few rare parameters. The specific analysis API parameter object units() method was incorrectly checking if the units value is a function - it was checking the parameter conversion factor instead.
Modified the align_tensor.init user function so that the parameters are now optional. This allows alignment tensors to be initialised without specifying the parameter values for that tensor.
Modified profiling script to have different number of NCYC points per frequency. This is to complicate the data, so any erroneous reshaping of data is discovered. It is expected, that experiments can have different number of NCYC points per spectrometer frequency. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Initial try to alter the target function calc_CR72_chi2. This is the first test to restructure the arrays, to allow for higher dimensional computation. All numpy arrays have to have same shape to allow to multiply together. The dimensions should be [ei][si][mi][oi][di]. [Experiment][spins][spec. frq][offset][disp points]. This is complicated with number of disp point can change per spectrometer frequency. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis. This implementation brings a high overhead. The first test shows no winning of time. The creation of arrays takes all the time.
Temporary changed the lib/dispersion/cr72.py function to unsafe state. This change turns-off all the safety measures, since they have to be re-implemented for higher dimensional structures.
Altered profiling script to report cumulative timings and save to temporary files. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis. This indeed shows that the efficiency has gone down.
Added print out of chi2 to profile script. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the creation of special numpy structures outside target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script to calculate correct values when setting up R2eff values. This is to test, that the return of chi2 gets zero. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removing looping over exp and offset indices in calc_chi2. They are always 0 anyway. This brings a little speed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
In profiling script, moved up the calculation of values one level. This is to better see the output of the profiling iterations for CR72.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for calculation of the Larmor frequency per spin in profiling script. The frq loop should also be up-shifted. It was now extracted as 0.0. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Re-inserted safety checks in lin/dispersion/CR72.py file. This is re-inserted for the rank_1 cases. This makes the unit-tests pass again. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Important fix for extracting the correct shape to create new arrays. If using just one field, or having the same number of dispersion points, the shape would extend to the dispersion number. It would report [ei][si][mi][oi][di] when calling ndarray.shape. Shape always has to be reported as: [ei][si][mi][oi]. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made it easier to switch between single and cluster reporting in profiling script. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Important fix for the creation of the multi dimensional pA numpy array. It should be created as numpy.zeros([ei][si][mi][oi]) instead of numpy.ones([ei][si][mi][oi]). This allows for rapid testing of all dimensions with np.allclose(pA, numpy.ones(dw.shape)). pA can have missing filled out values, when the number of dispersion points are different per spectrometer frequency. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added unit tests demonstrating edge cases 'no Rex' failures of the model 'CR72 full', for a clustered multi dimensional calculation. This is implemented for one field. This is to implement catching of math domain errors, before they occur. These tests cover all parameter value combinations which result in no exchange: Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Re-implemented safety checks in lib/dispersion/cr72.py. This is now implemented for both rank-1 float array and of higher dimensions. This makes the unit tests pass for multi dimensional computing. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added unit tests demonstrating edge cases 'no Rex' failures of the model 'CR72 full', for a clustered multi dimensional calculation. This is implemented for three fields. This is to implement catching of math domain errors, before they occur. These tests cover all parameter value combinations which result in no exchange: Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed that special numpy structure is also created for "CR72". This makes most system tests pass. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Critical fix for the slicing of values in target function. This makes system test: Relax_disp.test_sod1wt_t25_to_cr72 pass. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added self.has_missing keyword in initialization of the Dispersion class. This is to test once, per spin or cluster. This saves a looping over the dispersion points, when collection the data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Created multi dimensional error and value numpy arrays. This is to calculate the chi2 sum much faster. Reordered the loop over missing data points, so it is only initiated if missing points is detected. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Switch the looping from spin->frq to frq->spin. Since the number of dispersion points are the same for all spins, this allows to move the calculation of pA and kex array one level up. This saves a lot of computation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed all the creation of special numpy arrays to be of float64 type. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the data filling of special numpy array errors and values, to initialization of Dispersion class. These values does not change, and can safely be stored outside. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Just a tiny little more speed, by removing temporary storage of chi2 calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made copies of numpy arrays instead of creating from new. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a self.frqs_a as a multidimensional numpy array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Small fix for the indices to the errors and values numpy array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Lowered the number of iterations to the profiling scripts. This is to use the profiling script as bug finder. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation of dw_frq out of spin and spectrometer loop. This is done by having a special 1/0 spin numpy array, which turns on or off the values in the numpy array multiplication. The multiplication needs to first axis expand dw, and then tile the arrays according to the numpy structure. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation of pA and kex out off all loops. This was done by having two special 1/0 spin structure arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed dw_frq_a numpy array, as it was not necessary. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed all looping over spin and spectrometer frequency. This is the last loop! Wuhu. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Reordered arrays for beauty of code. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the back_calc array be initiated as copy of the values array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Small edit to profiling script, to help bug finding. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fixed that arrays are correctly initiated with one or zero values. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Very important fix, for only replacing part of data array which have Nan values. Before, all values were replaced, which was wrong. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Needed to increase the relative tolerance when testing if pA array is 1. Now system test Relax_disp.test_hansen_cpmg_data_missing_auto_analysis passes. Also added some comments lines, to prepare for mask replace of values. For example if only some of etapos values should be replaced. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Restored profiling script to normal. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the logic and comments much clearer about how to reshape, expand axis, and tile numpy arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented a masked array search for where "missing" array is equal 1. This makes it possible to replace all values with this mask, from the value array. This eliminates the last loops over the missing values. It took over 4 hours to figure out, that the mask should be called with mask.mask, to return the same fulls structure, Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Yet another small improvement for the profiling script. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the multi dimensional structure of pA. pA is not multi-dimensional, and can just be multiplied with numpy arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for testing of pA in lib function, when pA is just float. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified unit tests, so pA is sent to target function as float instead of array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the multi dimensional structure of kex. kex is not multi-dimensional, and can just be multiplied with numpy arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for testing of kex in lib function, when kex is just float. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified unit tests, so kex is sent to target function as float instead of array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Important fix for replacing values if eta_pos > 700 is violated. This fixes system test: Relax_disp.test_sod1wt_t25_to_cr72, which failed after making kex to a numpy float. The trick is to make a numpy mask which stores the position where to replace the values. Then replace the values just before last return. This makes sure, that not all values are changed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Increased the kex speed to 1e7 in clustered unit tests cases. This is to demonstrate where there will be no excange. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a multi-dimensional numpy array chi2 value calculation function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Called the newly created chi2 function to calculate for multi dimensional numpy arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Renamed chi2_ND to chi2_rankN. This is a better name for representing multiple axis calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made special ei, si, mi, and oi numpy structure array. This is for rapid speed-up of numpy array creation in target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced self.spins_a with self.disp_struct. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made initialisation structures for dw. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Initial try to reshape dw faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Switched to use self.ei, self.si, self.mi, self.oi, self.di. This is for better reading of code. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Comment out the sys.exit(), which would make the code fail for wrong calculation of dw. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script for CPMG model CR72 to R1rho DPL94 model. The framework of the script will be the same, but the data a little different. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started converting profiling script to DPL94. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced self.(ei,si,mi,oi,di) with self.(NE,NS,NM,NO,ND). These numbers represents the maximum number of dimensions, instead of index. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the ei index, when creating the first dw_mask. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Reordered how the structures dw init structures are created. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Clearing the dw_struct before calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started using the new way of constructing dw. This is for running system tests. Note, somewhere in the dw array, the frequencies will be different between the two implementations. But apparently, this does not matter. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Inserted temporary method to switch for profiling. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
First try to speed-up the old dw structure calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Simplified calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Yet another try to implement a fast dw structure method. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the fastest way to calculate the dw structure. This uses the numpy ufunc multiply.outer function to create the outer array, and then multiply with the frqs_structure. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Renamed dw temporary structure to generic structure. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Restructured the calculation of R20A and R20B to the most efficient way. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the lib/CR72.py to a numpy multi dimensional numpy array calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the catching when dw is zero, to use masked array. Implemented backwards compatibility with unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Bugfix for testing if kex is zero. It was tested if kex was equal 1.0. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented masked replacement if fact is less that 1.0. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced isnan mask with function that catches all invalid values.
Removed the masked replacement if fact is less than 1.0. This is very strange, but otherwise system test: Relax_disp.test_hansen_cpmg_data_missing_auto_analysis would fail. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the slow allclose() function to test if R20A and R20B is equal. It is MUCH faster to just subtract and check sum is not 0.0. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced the temporary variable R2eff with back_calc, and used numpy subtract to speed up. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the lib function into a pure numpy array calculation. This requires, that r20a, r20b and dw has same dimension as the dispersion points. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changes too unit tests, so data is sent to target function in numpy array format. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the creation of an unnecessary structure by using numpy multiply. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the mask which finds where to replace values into the __init__ function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script for CR72 to B14 model. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script for the B14 model. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified model B14 lib file to faster numpy multidimensional mode. The implementations comes almost directly from the CR72 model file. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Reverted the use of the mask "mask_set_blank". It did not work, and many system tests started failing. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the target function to handle the B14 model for faster numpy computation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed unit test for B14 to match numpy input requirement. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added additional tests in b14, when math errors can occur. This is very easy with a conditional masked search in arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Comment fix for finding when E0 is above 700 in lib function of B14. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed use of "asarray", since the variables are already arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed target function for model CR72. To CR72 is now also the input of the parameters of R20A, R20B and dw. dw is tested for zero, to return flat lines. It is faster to search in the smaller numpy array, than the 5 dimensional dw array. This is for speed-up. R20A and R20B is also subtracted, to see if the full model should be used. In the same way, it is faster to subtract the smaller array. These small tricks are expected to give 5-10 pct. speeed-up. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the lib function of CR72 accept the R20A, R20B and dw of the original array. This is for speed-up. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed unit-tests, to send in the original R20A, R20B and dw_orig to the testing of the lib function CR72. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed profiling script to send R20A, R20B and dw, as original parameters to the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed target function for model B14. To B14 now also send the input of the original parameters dw. dw is tested for zero, to return flat lines. It is faster to search in the smaller numpy array, than the 5 dimensional dw array. This is for speed-up. These small tricks are expected to give 5-10 pct. speed-up. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the lib function of B14 accept dw of the original array. This is for speed-up. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed unit-tests, to send in the original dw_orig to the testing of the lib function B14. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed profiling script to send dw as original parameters to the lib function B14. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script for CR72 model to TSMFK01 model. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script to be used for model TSMFK01. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified target function for model TSMFK01, to send in dw as original parameter. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified lib function for model TSMFK01 to accept dw_orig as input and replaced functions to find math domain errors into maske replacements. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made unit tests for model TSMFK01 send in R20A and dw as a numpy array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Large increase in speed for model TSMFK01 by changing target functions to use multidimensional numpy arrays in calculation. This is done by restructuring data into multidimensional arrays of dimension [NE][NS][NM][NO][ND], which are number of spins, number of magnetic field strength, number of offsets, maximum number of dispersion point. The speed comes from using numpy ufunc operations. The new version is 2.4X as fast per spin calculation, and 54X as fast for clustered analysis.
Replacing math domain checking in model DPL94, with masked array replacement. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
First try to speed up model DPL94. This has not succeeded, since system test: Relax_disp.test_dpl94_data_to_dpl94 still fails. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Trying to move some of the structures into its own part. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for forgetting to multiply frqs to power 2. This was found by inspecting all print out before and after implementation. New implementation of DPL94 now passes all system and unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the expansion of the R1 structure out of the for loops. This is to speed-up the __init__ of the class of the target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the packing of errors and values out of for loop in the __init__ class of target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the multi dimensional expansion of inv_relax_times out of for loop. This can be done for all structures, which does not have missing points. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
For inv_relax_times, expanded one axis, and tiled up to NR spins, before reshaping and blowing up to full structure. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the expansion of frqs out of for loops. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix for description of input arrays to lib functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Converted TAP03 model to use multi dimensional numpy arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made dw in unit tests of TAP03 be of numpy array. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced the loop structure in target function of TAP03 with numpy arrays. This makes the model faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Reordered the initialization structure of the special numpy arrays. This was done in the init part of the target function of relaxation dispersion. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added model MODEL_TSMFK01 also get self.tau_cpmg calculated in init part. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model TP02, has been replaced with numpy masks. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in dw as numpy array in unit tests of model TP02. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model TP02, to use higher dimensional numpy array structures. That makes the model much faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for adding model TP02 to part of init class to initialize preparation of higher dimension numpy structures. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the NOREX model a faster numpy array calculation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed an unnecessary frq_struct in init of target function. frqs can just be expanded, and back_calc is cleaned afterwards with disp_struct. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model M61, has been replaced with numpy masks. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in r1rho_prime and phi_ex_scaled as numpy array in unit tests of model M61. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model M61, to use higher dimensional numpy array structures. That makes the model much faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model M61b, has been replaced with numpy masks. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in r1rho_prime and dw as numpy array in unit tests of model M61b. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model M61b, to use higher dimensional numpy array structures. That makes the model much faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points to be send to lib function of model TSMFK01. These are not used anymore. Also removed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points and pB to be send to lib function of model TP02. Number of points are not used anymore. pB is calculated in lib function instead. Also removed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points and pB to send to lib function of model TP02. pB is calculated in lib function instead. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points, pB, k_AB, k_BA to be send to lib function of model B14. Number of points are not used anymore. pB is calculated in lib function instead. k_AB, and k_BA are calculated in lib functions instead. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending number of points in target function of TSMFK01. This was removed in lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points, pB, to be send to lib function of model TAP03. Number of points are not used anymore. pB is calculated in lib function instead. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points, to be send to lib function of model CR72. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points, to be send to lib function of model DPL94. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points, to be send to lib function of model M61. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points, to be send to lib function of model M61b. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model MP05, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. Calculation of pB, has been moved to lib function for simplicity. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in dw as numpy array in unit tests of model MP05. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model MP05, to use higher dimensional numpy array structures. That makes the model much faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model LM63, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in number of points in unit tests of model LM63. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model LM63, to use higher dimensional numpy array structures. That makes the model much faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for replacement of values with mask, when phi_ex is zero. This can be spin specific. System test: Relax_disp.test_hansen_cpmg_data_to_lm63 starts to fail: Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in r20 and phi_ex as numpy array in unit tests of LM63. This is after using masks as replacement. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
1 digit decrease in parameter check in system test: Relax_disp.test_hansen_cpmg_data_to_lm63. It is unknown, why this has occurred. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model IT99, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. pB is now moved to be calculated inside. This makes the lib function simpler. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in r20 and dw as numpy array in unit tests of IT99. This is after using masks as replacement. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model IT99, to use higher dimensional numpy array structures. That makes the model much faster. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model ns_cpmg_2site_expanded, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. pB is now moved to be calculated inside. This makes the lib function simpler. k_AB and k_BA is also now calculated here. Documentation is also fixed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for sending in r20 and dw as numpy array in unit tests of ns_cpmg_2site_expanded. This is after using masks as replacement. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced target function for model ns_cpmg_2site_expanded, to use higher dimensional numpy array structures. That makes the model much faster. I cannot get system test: Relax_disp.test_cpmg_synthetic_dx_map_points to pass. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for system test Relax_disp.test_cpmg_synthetic_dx_map_points. By just copying self.back_calc_a to self.back_calc, problem was solved. In specific_analysis.relax_disp.optimisation in function back_calc_r2eff(), the function gets the last values stores in the class function. This is in "class Disp_result_command(Result_command)" with self.back_calc = back_calc. And back_calc_r2eff() have return model.back_calc. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Methods to replace math domain errors in model ns_cpmg_2site_3d, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. pB is now moved to be calculated inside. This makes the lib function simpler. k_AB and k_BA is also now calculated here. Magnetization vector is also now filled in lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for unit tests of model NS CPMG 2-site 3D to the reduced input to the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Change to the target function to the model NS CPMG 2-site 3D to use the reduced input to the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed linked matrix/vector inner products into chained dot expressions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Wrote the essential dot matrix up to be initiated earlier. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Lowered the number of dot iterations, by pre-prepare the dot matrix another round. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Turned Mint vector into a 7,1 matrix, so dimensions fit with evolution matrix. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Lowered the number of dot operations, by pre-preparing the evolution matrix another round. The power is in system tests always even. The trick to removing this for loop, would be to make a general multi dot function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the bulk operation of model NS CPMG 2-site 3D into the lib file. This is to keep the API clean. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the unit test of NS CPMG 2-site 3D, after the input to the function has changed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the target function for NS CPMG 2-site 3D. This reflects the new API layout. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the lib function of NS CPMG 2-site star, to get input of dw and r20a+r20b of higher dimensional type. This is to move the main operations from the target function to the lib function, and make the API code clean and consistent. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the target function of NS CPMG 2-site star, to reflect the input to the function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the dot evolution structure faster for NS CPMG 2-site 3D. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the BLAS method of dot product, which should be faster. I cannot get the "out" argument to work. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Small fix for the dot method. But the out argument does not work. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the dot method via blas. This needs a array with one more axis. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Last try to use the out argument. In the last dotting loop, the out argument wont work, no matter what I do. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Inner product fix in model NS CPMG 2-site 3D. Fix for system tests: Relax_disp.test_cpmg_synthetic_ns3d_to_b14, Relax_disp.test_cpmg_synthetic_ns3d_to_CR72, Relax_disp.test_cpmg_synthetic_ns3d_to_CR72_noise_cluster. The number of dotting with Mint, should correspond to the power. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced the temporary structure self.frqs_a to self.frqs, which works for all target functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced the temporary structure self.cpmg_frqs_a to self.cpmg_frqs, which works for all target functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Restructured all data structures into higher dimension in target function. Fix for the input to the different models. Restructured how to detect the number of offset and dispersion points. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Various index fixes, after the data structures have been reordered. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for unit test, where the dimension of points has to be one lower. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for plotting, since the back_calc now can hold more data points that cpmg frequencies. This is because the numpy array has been expanded to the maximum number of points. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented a frqs_squared calculation in the init of target function. This is to speed up the calculations. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Restructured frqs_H to higher dimension in target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation of dw and dwH out of for loops for model MMQ CR72. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed looping over spin and frequencies for model MMQ CR72. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Temporary removed check for dw = 0.0 in MMQ CR72. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed number of points to be parsed to model MMQ CR72. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed power to be parsed to MMQ CR72, since it is not used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed MMQ CR72 to use multi dimensional data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed unit test of MMQ CR72 to pass. dw needs to be of numpy structure. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation of dw out of for loops for model NS MMQ 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified lib function for NS MMQ 2-site, to have looping over spins and frequencies inside lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fixed the use of higher dimensional data in NS MMQ 2-site SQ DQ ZQ. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for documentation in NS MMQ 2-site/SQ/DQ/ZQ/MQ. Now explains which dimension data should be in. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the reshaping of dw and dwH, since it is not dependent on experiment. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changed the calculation of inner product in model NS CPMG 2-site 3D. The out argument of numpy.dot is buggy, and should not be used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added missing instances of cleaning the data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Bug fix for model LM63 3-site. The index si has to be used to extract data to lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Temporary added system test: test_korzhnev_2005_all_data_disp_speed_bug. This makes a minimisation with 1 iteration, and so will give the chi2 value at the preset parameter values. This is chi2 value should give 162.5, but gives 74.7104. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Updated documentation on dimensionality of numpy array num_points. They are in dimension [NE][NS][[NM][NO], where oi gives the number of dispersion points at that offset. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for system test: test_korzhnev_2005_all_data. The masking for replacing values was wrong. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the cleaning of data points and replacing of values of out loop for model NS MMQ 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for structure cleaning and value replacing for model MMQ CR72. System test: test_korzhnev_2005_all_data revealed how this should be done properly. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for system test test_korzhnev_2005_all_data_disp_speed_bug. The precision is lowered, and now matches the original system test. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced index to numpy array from example [0][si][mi][oi] to [0, si, mi, oi]. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
More replacing of numpy index. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix, where a double bracket "[[" has been copied into all lib functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
More fixes for numpy index in lib functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Restructured target function for NS MMQ 3-site to the new API structure of higher dimensional data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Reordered the lib function for NS MMQ 3-site to use higher dimensional data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix for which dimensionality number of points have. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix for the initial data structure of cpmg_frqs, spin_lock_nu1, r1. They were incorrect. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
First attempt to implement target function for NS R1rho 2-site. But it does not work yet. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
First attempt to implement lib function for NS R1rho 2-site. But it does not work yet. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fatal fix for calling inv_relax_time from relax_time variable. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removal of the temporary offset argument. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix for the dimensionality of the input arrays. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the target function for NS R1rho 3-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the lib function for NS R1rho 3-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented target function for LM63 3-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the lib function for LM63 3-site, for higher dimensional data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the number of disp points in target function for LM63 3-site, since it is no longer used, but have been replaced with mask replacements. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented class function in target class, to return back_calc values as list of lists. This is the back and forth conversion between the data structures implemented when gathering the data, and the data send to the library function of higher dimensionality. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Used the new class function: get_back_calc(), to get the data in the right structures when interpolating for graphs. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed superfluous check, after the returned data is now in right structure. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made changes to the dir argument of system test Relax_disp.test_r1rho_kjaergaard. This is to prepare for: sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff and sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1]. This is also to test an expected bug, if R1 is not loaded. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
The relaxation dispersion target function can now be set up when the optional frqs_H argument is None. This allows the profiling scripts to run.
More stability fixes for the relaxation dispersion target function initialisation. The target function can now be initialised when the r1 and chemical_shift arguments are None.
Split the system test test_r1rho_kjaergaard into a setup function, and a test function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Renamed system test test_r1rho_kjaergaard to test_r1rho_kjaergaard_auto. This corresponds to the use of the automatic analysis method. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Split system test test_r1rho_kjaergaard into test_r1rho_kjaergaard_auto and test_r1rho_kjaergaard_man. This is to test use of the manual way to analyse. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified all of Troels' dispersion profiling scripts to work with older relax versions. This is in preparation for obtaining some powerful timing statistics. The calls to the r2eff_*() functions are unnecessary and are the only failure point in the scripts between the current code in the disp_spin_speed branch and trunk or older versions of relax. So these function calls have been eliminated.
Implemented system test test_r1rho_kjaergaard_missing_r1, for safety check if R1 data is not loaded. The system test passes, so target function is safe. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Python 3 support for the dispersion profiling scripts. The xrange() builtin function does not exist in Python 3, so this is now aliased to range() which is the same thing.
Replaced double or triple hash-tags "##" with single hash-tags "#". Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copyright fixes for all the models, where Troels E. Linnet have made changes to make them functional for higher dimensional data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copyright fix for model TSMFK01. Sebastien Morin did not take part of implementing the TSMFK01 model. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Created a super script for profiling the relaxation dispersion models. This script will execute all of the current profiling scripts in the directory test_suite/shared_data/dispersion/profiling for both the current version of relax and any other specified version (current set to the 3.2.2 relax tag). It will run the scripts and relax versions interleaved N=10 times and extract the func_*() target function call profile timings. This interleaving makes the numbers much more consistent. Averages and standard deviations are then calculated, as well as the speed up between the two relax versions. The results are printed out in a format suitable for the relax release messages.
Increased the number of iterations to 1000 in all of the profiling scripts. This is for better statistics in the disp_profile_all.py script, and makes the number consistent between the different models.
Added a log file for comparing the speed of the disp_speed_branch to relax 3.2.2. This is from the disp_profile_all.py statistics generating script.
Made the processor.return_object get the back_calc structure in the expected order. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fixed the ordering of the relax versions in the dispersion super profiling script disp_profile_all.py. This has also been fixed in the disp_spin_speed branch to relax 3.2.2 comparison log.
Added a log file for comparing the speed of the disp_speed_branch to relax 3.2.1. This is from the disp_profile_all.py statistics generating script.
Added a profiling script for the 'NS CPMG 2-site expanded' dispersion model. This is the model at http://wiki.nmr-relax.com/NS_CPMG_2-site_expanded. The script was copied from that of the CR72 model, and it only needed to be changed in a few places. This is the first numeric model profiling script.
Updated the profiling super script and log for the 'NS CPMG 2-site expanded' model. This is the model at http://wiki.nmr-relax.com/NS_CPMG_2-site_expanded. This shows that the single spin calculation is 1.8 times faster, and the cluster of 100 11.7 times faster, when compared to relax 3.2.2.
Modified all of the dispersion model profiling scripts. The single() function for timing the single spin target function speed has been modified to include a second outer loop over 100 'spins'. This means that the timing numbers are equivalent to the cluster timings, as both are then over 100 spins. This now allows not only relax version differences and model differences to be compared, but also the non-clustered and clustered analysis speeds.
Added a script for profiling the 'NS CPMG 2-site 3D' relaxation dispersion model. This is the model at http://wiki.nmr-relax.com/NS_CPMG_2-site_3D. Again this only involved copying one of the other scripts and modifying a few variable and function names.
Added the 'NS CPMG 2-site 3D' model to the dispersion super profiling script. To handle the fact that this script has nr_iter set to 100 rather than 1000 (as otherwise it is too slow), a list of scaling factors has been created to scale all timing numbers to equivalent values.
Added DPL94 profiling script. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script for TSMFK01, to use correct parameters k_AB and r20a. Or else, the lib functions is just calculating with zero? Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changes to profiling script of NS CPMG 2-site expanded. The model does not have r20a and r20b, but only r2. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made changes to the profiling script of NS CPMG 2-site 3D. Need to use the full model, when r2a and r2b is specified. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changes to profiling script of NS CPMG 2-site expanded. The unpacking can be removed. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for the profiling script of NS CPMG 2-site 3D. The model should also be specified to full. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
The disp_profile_all.py super script now prints out the current relax version information. This is so that the log files contain information about the repository revision and path.
Copied profiling script of DPL94 to NS R1rho 2-site.
Improved the final printout from the disp_profile_all.py dispersion model super profiling script.
Added profiling script for NS R1rho 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
The disp_profile_all.py dispersion model super profiling script is now executable.
Decreased all nr_iter values by 10 and added more dispersion models to the super profiling script. This is for the dispersion model profiling scripts in test_suite/shared_data/dispersion/profiling/, all controlled by the disp_profile_all.py super profiling script for generating statistics using all of the other profiling scripts. The number of iterations needed to be decreased as otherwise it would now take almost 1 day to generate the statistics table.
Moved the parter conversion in LM63 3-site into the lib function. This cleans up the target api function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script for DPL94 to TAP03.
Copied profiling script for DPL94 to TP02.
Copied profiling script for DPL94 to MP05.
Copied profiling script for DPL94 to M61.
Modified profiling script for TAP03 to be used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script for TP02, to be used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script for MP05. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script for M61. This is the last one. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Expansion of the disp_profile_all.py dispersion model super profiling scripts. The newly added profiling scripts for models 'M61', 'TP02', 'TAP03', and 'MP05' are now included in the super script to generate statistics for all of these as well. The nr_iter variable has also been changed to match the other analytic models, so that the standard deviations are lowered and the statistics are better.
Moved the parameter conversion of MMQ CR72 into lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the parameter conversions of k_AB, k_BA and pB into lib function of NS MMQ 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the parameter conversion from target function to lib function for NS R1rho 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Updated the dispersion model speed statistics for the disp_spin_speed branch vs. relax-3.2.2. This now includes the 'NS CPMG 2-site 3D', 'DPL94', and 'NS R1rho 2-site' dispersion models. The timings for the single spin analyses are now comparable to the clustered analysis, as the equivalent of 100 single spins is being used. The final printout is also in a better format to present for the relax release messages. These new results show the insane 160x speed up of the 'DPL94' model.
Alignment improvements for the final printout from the dispersion model super profiling script. The log file has been updated with what the new formatting will look like.
Updated the model names in the dispersion model super profiling script. The 'CR72', 'B14' and 'NS CPMG 2-site 3D' models are the full, slower versions rather than the faster models with R20 = R20A = R20B. The log file has been updated to match.
Moved the parameter conversion for NS MMQ 3-site into lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Updated the dispersion model profiling comparison of the disp_spin_speed branch vs. relax-3.2.2. The 'M61', 'TP02', 'TAP03', and 'MP05' models are now included. The final printout has been manually updated to reflect the newest version of the disp_profile_all.py super profiling script.
Moved the parameter conversion for NS R1rho 3-site into lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script for CR72, so there is now a normal and a full version.
Copied profiling for B14 to normal and full model.
Created a text file suitable for use as part of the relax release notes. This contains the statistically averaged profiling information of the speed of the dispersion models in the disp_spin_speed branch vs. relax-3.2.2. This file has been created so that it can be used as part of the release notes for the version of relax that contains the insane speed ups of this branch. This file will be updated as new models are profiled and if any more speed ups magically appear.
Copied profiling script for NS CPMG 2-site 3D.
Copied profiling script for NS CPMG 2-site star.
Copied profiling script for No Rex.
Modified profiling script for B14, to R20A=R20B. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented profiling script for NS CPMG 2-site 3D. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented profiling script for NS CPMG 2-site star and star full. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script to be used for LM63.
Copied profiling script to model IT99.
Added profiling script for IT99. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented profiling script for LM63. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the "eta_scale = 2.0**(-3.0/2.0)" out of lib function for MMQ CR72, since this is only needs to be computed once. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for spaces aroung "=" outside functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Critical fix for wrong space inserted in NS MMQ 3-site MQ. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fixed the input for unit test of MMQ CR72. The number of input parameters has been lowered. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added additional math domain checking in B14. This is when v1c is less than 1.0. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Comment fixing, for explaining the masking and replacing when dw is zero. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script to be used for profiling the use of higher dimensional data for the numpy eig function.
Implemented the collection of the 3D exchange matrix, for rank [NE][NS][NM][NO][ND][7][7]. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented test, to see if 3D exchange matrices are the same. This can be tested while running system test test_hansen_cpmg_data_to_ns_cpmg_2site_3D. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Shifted the computation of Rexpo two loops up. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added intermediate step with for loops. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added another intermediate step. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added function to compute the matrix exponential for higher dimensional data of shape [NE][NS][NM][NO][ND][7][7]. This is done by using numpy.einsum, to make the dot product of the last two axis. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Inserted intermediate step, to check if the matrix propagator to evolve the magnetization is equal when done for lower dimensional data of shape [7][7] and higher dimensional data of shape [NE][NS][NM][NO][ND][7][7]. A short example is shown at the wiki: http://wiki.nmr-relax.com/Numpy_linalg#Ellipsis_broadcasting_in_numpy.einsum. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented double speed of model NS CPMG 2-site 3D. This is done by moving the costly calculation of the matrix exponential out of the for loops. The trick was to find a method to do dot product of higher dimensions. This was done with numpy.einsum, example at: http://wiki.nmr-relax.com/Numpy_linalg#Ellipsis_broadcasting_in_numpy.einsum. Example: dot_V_W = einsum('...ij,...jk', V, W_exp_diag) where V, and W_exp_diag has shape [NE][NS][NM][NO][ND][7][7]. The profiling script shows a 2X speed up.
Made notation consistent for variables, using "_i" to clarify extracted data from matrix. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation the evolution matrix out of for loops. The trick is that numpy.einsum allows for dot product of higher dimension: The essential evolution matrix; This is a dot product of the outer [7][7] matrix of the Rexpo_mat and r180x_mat matrices, which have the shape [NE][NS][NM][NO][ND][7][7]; This can be achieved by using numpy einsum, and where ellipsis notation will use the last axis.
Implemented system test: test_cpmg_synthetic_b14_to_ns3d_cluster. This is to catch failures of the model, when data is clustered. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed unused variables in NS CPMG 2-site 3D, to clean up the code. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the NS matrices, rr1rho_3d_rankN, to collect the multi dimensional 3D exchange matrix, of rank [NE][NS][NM][NO][ND][6][6]. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a check in lib/dispersion/ns_r1hro_2site.py, to see if the newly created multidimensional ns matrix of rank NE][NS][NM][NO][ND][6][6], is equal to the previous [6][6] matrix. It is. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the relax_time to collection of rr1rho_3d_rankN matrix collection. This is to pre-multiply all elements with the time. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a check, that the pre- relax_time multiplied multidimensional array, equal the previous. It does, to the sum of 1.0e-13. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the function use the new multidimensional R_mat matrix. System test: test_tp02_data_to_ns_r1rho_2site still passes. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix to the matrix_exponential_rankN, to return the exact exponential for any higher dimensional square matrix of shape [NE][NS][NM][NO][ND][X][X]. The fix was to the eye(X), to make the shape the same as the input shape. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the costly calculation of the matrix exponential out of for loops. It was the numpy.eig and numpy.inv which was draining power. This speeds up model NS R1rho 2-site, by a factor 4X.
Made the returned multidimensional rr1rho_3d_rankN, be of float64 type. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Cleaned up the code of NS R1rho 2-site, and removed the matrix argument to the function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the parsing of a matrix to the lib function of NS R1rho 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the function "rcpmg_star_rankN" for the collection of the multidimensional relaxation matrix for model NS CPMG 2-site star. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Insert check, that the newly created multidimensional matrix is the same. They are, but only to the fifth digit. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started using the newly created multidimensional matrix. test_hansen_cpmg_data_to_ns_cpmg_2site_star. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the system test: test_cpmg_synthetic_b14_to_ns_star_cluster, to check for the model is still working after change. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started using the newly cR2 variable, extracted from higher dimensional data. This should be okay, but system test test_hansen_cpmg_data_to_ns_cpmg_2site_star, now fails.
Changes of values to system test test_hansen_cpmg_data_to_ns_cpmg_2site_star. The values are changed, since chi2 is lower than before. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the costly finding of matrix exponential out of for loops for eR_tcp. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Rearranged the code, to properly show the nested matrix exponentials in dot functions. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the costly matrix_exponential of cR2 out of for loops. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Rearranged the dot code, for better view. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Cleaned up the code in model NS CPMG 2-site star. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Simplified model NS CPMG 2-site 3D. The expansion of matrices to higher dimensionality is not necessary. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Further cleaned up the code in NS CPMG 2-site star. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed input of matrix, Rr, Rex, RCS and R to model NS CPMG 2-site star. These matrices is now extracted from NS matrix function rcpmg_star_rankN. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the collection of the multidimensional matrix m1 and m2 in model NS MMQ 2-site. Inserted also a check, that the newly computed matrix is equal. They are, to the 6 digit. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started using the newly created multidimensional m1 and m2 matrices. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the costly calculation of matrix_exponential of M1 and M2 out of for loop, in model ns_mmq_2site_mq. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the function matrix_exponential_rankN also find the exponential if the experiments indices are missing. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for an extra axis inserted in eye function, when dimensionality is only [NS][NM][NO][ND]. This also fixes the index in the lib function of ns_mmq_2site_mq. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented same functionality in mmq_2site_sq_dq_zq. Problem, following system test fails: test_korzhnev_2005_15n_dq_data, test_korzhnev_2005_15n_mq_data, test_korzhnev_2005_15n_sq_data, test_korzhnev_2005_1h_mq_data, test_korzhnev_2005_1h_sq_data, test_korzhnev_2005_all_data, test_korzhnev_2005_all_data_disp_speed_bug. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed grid search, and lowered number of iterations for system test: test_cpmg_synthetic_b14_to_ns3d_cluster, test_cpmg_synthetic_b14_to_ns_star_cluster. This is to speed them up, since they before took 30 seconds. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for ns_mmq_2site_mq. Variable was wrong called. There seems to be a serious problem more with MQ.
Reinserted old code. This fixes: test_korzhnev_2005_15n_mq_data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Forcing the dtype to be complex64, instead of complex128. This solves a range of system tests. The one who fails now is: test_korzhnev_2005_15n_zq_data, test_korzhnev_2005_1h_mq_data, test_korzhnev_2005_1h_sq_data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Forces complex64 in ns_mmq_2site_sq_dq_zq instead complex128. This fixes system tests: test_korzhnev_2005_15n_zq_data,test_korzhnev_2005_1h_sq_data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Force complex64 in ns_mmq_2site_mq. This solves all system tests. Forcing to be complex64, does not seems like a long standing solution, since complex128 is possible. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for using the old matrix_exponential of m1. One: test_korzhnev_2005_15n_sq_data is still failing. That still uses the matrix_exponential_rankN. There seems to be a problem with matrix_exponential_rankN, when doing complex numbers. Maybe the dtype has to get fixed? Use it as a input argument? It must be the einsum. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the "dtype" argument to function matrix_exponential_rankN. This is to force the conversion of dtype, if they are of other type. This can be conversion from complex128 to complex64. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix the bug: "M2_i = M1_mat", which was causing the problems getting system tests to pass. Removed the specifications of which dtype, the initial matrices are created. They can be converted later, with the specification of dtype to matrix_exponential_rankN(). All system tests now pass. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the Bloch-McConnell matrix for 2-site exchange into lib/dispersion/ns_matrices.py. This is for consistency with the other code. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the matrices for Bloch-McConnell from lib ns_mmq_2site, since they are now defined in ns_matrices.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the Bloch-McConnell matrix for 3-site exchange, into the lib/dispersion/ns_matrices.py. This is to standardize the code. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed m1 and m2 to be sent to lib function of NS MMQ 2-site, since they are now populated inside the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented the Bloch-McConnell matrix for 3-site exchange, for multidimensional data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Inserted a check, that the new higher dimensional m1 and m2 matrices are equal to before. They are, to the 5 digit. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started using the newly created higher dimensional Bloch-McConnell matrix for 3-site exchange. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation of the matrix exponential out of for loops for NS MMQ 3-site MQ. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Converted NS MMQ 3-site/SQ/DQ/ZQ to calculate the matrix exponential out of the for loops. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the complex64 to be used as dtype in matrix exponential. Fix for missing "_i" in variable. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed m1 and m2 to be sent to target function of ns_mmq_3site_chi2. They are now populated inside the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation and input fix for NS MMQ 2-site. The m1 and m2 matrices are populated inside the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Renamed some numerical matrices, to get consistency in naming. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented multidimensional NS R1rho 3-site exchange matrix. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Inserted check, that newly multi dimensional matrix is equal the old. It is, to the 13 digit. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Started using the newly multidimensional 3D exchange matrix. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved the calculation of the matrix exponential out of the for loops for NS R1rho 3-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed the parameter "matrix" to be send to lib function of NS R1rho 3-site, since it is now populated inside the lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved parameter conversion for NS R1rho 3-site inside lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Cleaned up the Dispersion class target function, for creation of matrices, which is now populated inside the lib functions instead. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed pA and pB from the matrix population function rcpmg_star_rankN, since they are not used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed pA and pB from the matrix population function rr1rho_3d_2site_rankN, since they are not used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix for the dimensionality for model NS R1rho 2-site. The data is lined up to be of form [NE][NS][NM][NO][ND]. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed pA, pB and pC from the matrix population function rr1rho_3d_3site_rankN, since they are not used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Deleted the profiling of eig function profiling script. This was never implemented. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
For all profiling scripts, added conversion to numpy array for CPMG frqs and spin_lock, since some models complained in 3.2.2, that they were of list types. Also fixed IT99 to only have 1 spin, since clustering is broken in 3.2.2. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified super profiling script, to allow input to script, where alternative version of relax is positioned. Collected the variables in a list of lists, for better overview. Added a print out comment to IT99, to remember the bug. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added comment field to super profiling script. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Math domain fix for NS CPMG 2-site expanded. This is when t108 or t112 is zero, in the multidimensional array, a division error occurs. The elements are first set to 1.0, to allow for computation. Then elements are later replaced with 1e100. Lastly, if the elements are not part of the "True" dispersion point structure, they are cleaned out. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Precision lowering of system test, test_korzhnev_2005_15n_sq_data and test_korzhnev_2005_1h_sq_data. The system tests does not fail on Linux 64-bit system, but only on Mac 32-bit system. This is due to floating error deviations. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added log files for super profiling against tags 3.2.1 and 3.2.2. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied lib.linear_algebra.matrix_exponential to lib.dispersion.matrix_exponential. The matrix exponential of higher dimensional data is only used in the dispersion part of relax.
Added to __init__, the new lib.dispersion.matrix_exponential module. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added to unit_tests/_lib/_dispersion/__init__.py, the new unit test file: test_matrix_exponential.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added numpy array save files. They are the numpy array structures, which are send in from system test: Relax_disp.test_hansen_cpmg_data_to_ns_cpmg_2site_3D. These numpy array structures, are used in unit tests. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added unit test unit_tests/_lib/_dispersion/test_matrix_exponential.py to test the matrix exponential from higher dimensional data. lib.dispersion.matrix_exponential.matrix_exponential_rankN will match against lib.linear_algebra.matrix_exponential. Data which is used for comparison, comes from system test: Relax_disp.test_hansen_cpmg_data_to_ns_cpmg_2site_3D. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Renamed function to return data in unit test _lib/_dispersion/test_matrix_exponential.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix to lib/dispersion/matrix_exponential.py, since the svn copy command was used on non-updated version of the file. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added unit test for doing the matrix exponential for complex data. This test shows, that the dtype=complex64, should be removed from lib/dispersion/ns_mmq_2site.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added data for unit test for the testing of the matrix_exponential_rankN. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Expanded the dispersion profiling master script to handle any two relax versions. To compare two relax versions, for example 3.2.2 to 3.2.1, either the path1 and path2 variables or two command line arguments can be supplied. The first path should be for the newest version. This will allow for comparing the speed differences between multiple relax versions in the future.
Split matrix_exponential_rankN into matrix_exponential_rank_NE_NS_NM_NO_ND_x_x and matrix_exponential_rank_NS_NM_NO_ND_x_x. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Moved numerical solution matrices into the corresponding lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling scripts, to be used for 3-site models and MMQ models.
Implemented profiling script for LM63 3-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Improved the relax version printouts for the dispersion model master profiling script. This now reports both relax versions.
Removed a tonne of unused imports from the dispersion model profiling scripts. This is to allow most of the scripts to run on the relax 3.1.x versions, as well as to clean up the scripts. The unused imports were found using the command: pylint test_suite/shared_data/dispersion/profiling/*.py --disable=all --enable=unused-import.
Added a relaxation dispersion model profiling log file for relax version 3.2.1 vs. 3.2.0. This is the output from the dispersion model profiling master script. It shows a 2.2 times increase in speed for the B14 and B14 full models, with all other models remaining at the same speed. This matches the changes for relax 3.2.1 (https://gna.org/forum/forum.php?forum_id=2462), the main feature of which is a major bugfix for the B14 models.
The 'relax -v' command is now used for the dispersion model profiling script initial printout. This is to show the two different relax versions being compared.
Modifications to the dispersion model profiling master script. The info.print_sys_info() function of the current relax version is being called at the start to show all information about the current system. This is useful to know the speed of the machine, the OS, the Python version and numpy version. The numpy version is important as future versions might optimise certain functions that are currently very slow, hence that could be a cause of model speed differences. In addition, the path variables path1 and path2 have been renamed to path_new and path_old to make it clearer which is which. And the individual profiling scripts are no longer copied to the base directory of the relax versions being compared, and are run in place.
Added a relaxation dispersion model profiling log file for relax version 3.2.2 vs. 3.2.1. This is the output from the dispersion model profiling master script. It shows that the relax 3.2.2 release did not in fact feature "a large speed up of all analytic relaxation dispersion models" as described in the release notes at https://gna.org/forum/forum.php?forum_id=2465. For the CPMG models there is a 1 to 2 times increase in speed. But for the R1rho models, there is a 1 to 2 times decrease in speed.
Added a relaxation dispersion model profiling log file for relax version 3.2.0 vs. 3.1.7. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
Added a relaxation dispersion model profiling log file for relax version 3.1.7 vs. 3.1.6. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
Modified profiling script for NS R1rho 3-site, to be functional. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Modified profiling script for NS R1rho 3-site linear to be functional. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a relaxation dispersion model profiling log file for relax version 3.1.3 vs. 3.1.2 vs. 3.1.1. This is the output from the dispersion model profiling master script. It shows that there are no major speed differences between these relax versions.
Added the system information printout to the dispersion model profiling master script output. This is for the log files comparing one version of relax to the previous version.
Added profiling script for model MMQ CR72, Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Fix for the replacement value for invalid values in model MMQ CR72. The value was set to use r20, but should instead be 1e100. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Copied profiling script from MMQ CR72, to NS MMQ 2-site and 3-site.
Copied profiling script to NS MMQ 3-site linear.
Implemented profiling script for NS MMQ 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented profiling script for NS MMQ 3-site and 3-site linear. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Speeded up model NS CPMG 2-site star, by moving the forming of the propagator matrix out of the for loops, and preform it. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a relaxation dispersion model profiling log file for relax version 3.1.4 vs. 3.1.3. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
Speeded up NS MMQ 2-site, by moving the forming of evolution matrix out of the for loops, and preform it. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Speeded up NS MMQ 3-site, by moving the forming of evolution matrix out of the for loops, and preform it. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added a relaxation dispersion model profiling log file for relax version 3.1.5 vs. 3.1.4. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
Speeded up NS R1rho 2-site, by preforming the evolution matrices, and the M0 matrix in the init part of the target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Speeded up NS R1rho 3-site, by preforming the evolution matrices, and the M0 matrix in the init part of the target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Expanded the dispersion model profiling master script to cover all the new profiling scripts. This includes all 3-site and MMQ models. The list is now complete and covers all models. The only model not included in 'M61 skew' which has redundant parameters and is not optimisable anyway.
Added a relaxation dispersion model profiling log file for relax version 3.1.6 vs. 3.1.5. This is the output from the dispersion model profiling master script. It shows that there are almost no speed differences, except for a slight decrease in speed in the 'CR72 full' model for single spins.
Split system test test_tp02_data_to_ns_r1rho_2site into a setup and test part. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Implemented a clustered version of system test test_tp02_data_to_ns_r1rho_2site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Inserted an extremely interesting development in NS R1rho 2-site. If one do a transpose of M0, one can calculate all the matrix evolutions in the start via numpy einsum. Since M0 is in higher a dimensions, one should not do a numpy transpose, but swap/roll the outer M0 6x1 axis. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Shortened the code dramatically for NS R1rho 2-site. It is possible to calculate all in "one" go, after having the transposed/rolled-back M0 magnetization. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Speeded up the code of NS R1rho 2-site. This was essential done to numpy einsum, and doing the dot operations in multiple dimensions. It was though necessary to realize, that to do the proper dot product operations, the outer two axis if M0 should be swapped, by rolling the outer axis one back. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Speeded up the code of NS R1rho 3-site. This was essential done to numpy einsum, and doing the dot operations in multiple dimensions. It was though necessary to realize, that to do the proper dot product operations, the outer two axis if M0 should be swapped, by rolling the outer axis one back. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
For model NS CPMG 2-site 3D, the M0 matrix was preformed for higher dimensionality in init of target function. The transposes/rolled axis versions was also initiated. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Swapped the dot product position, when propagating the magnetisation in model NS CPMG 2-site 3D. This it to try to align to same method as in NS R1rho 2-site. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Lowered the looping in NS CPMG 2-site 3D, by preforming the initial dot product. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Speeded up NS CPMG 2-site 3D, by preforming the magnetisation. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Got rid of the inner evolution of the magnetization. If the looping over the number of CPMG elements is given by the index l, and the initial magnetization has been formed, then the number of times for propagation of magnetization is l = power_si_mi_di-1. If the magnetization matrix "Mint" has the index Mint_(i,k) and the evolution matrix has the index Evol_(k,j), i=1, k=7, j=7 then the dot product is given by: Sum_{k=1}^{k} Mint_(1,k) * Evol_(k,j) = D_(1, j). The numpy einsum formula for this would be: einsum('ik,kj -> ij', Mint, Evol). Following evolution will be: Sum_{k=1}^{k} D_(1, j) * Evol_(k,j) = Mint_(1,k) * Evol_(k,j) * Evol_(k,j). We can then realize, that the evolution matrix can be raised to the power l. Evol_P = Evol**l. It will then be: einsum('ik,kj -> ij', Mint, Evol_P). Get which power to raise the matrix to. l = power_si_mi_di-1. Raise the square evolution matrix to the power l. evolution_matrix_T_pwer_i = matrix_power(evolution_matrix_T_i, l), Mint_T_i = dot(Mint_T_i, evolution_matrix_T_pwer_i) or Mint_T_i = einsum('ik,kj -> ij', Mint_T_i, evolution_matrix_T_pwer_i). Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Tried to implement using lib.linear_algebra.matrix_power.square_matrix_power instead of matrix_power from numpy in NS CPMG 2-site 3D. Strangely, then system test: test_hansen_cpmg_data_to_ns_cpmg_2site_3D_full starts to fail. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Changes to unit test of NS CPMG 2-site 3D. This is after the new initiated M0 matrix in init of target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Double speed in NS CPMG 2-site star, after using numpy.linalg.matrix_power instead of the lib version in relax. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Triple speed in NS MMQ 2-site, after using numpy.linalg.matrix_power instead of the lib version in relax. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Small fix for making sure that power is a integer in NS MMQ 2-site. Following system tests was failing: Relax_disp.test_korzhnev_2005_15n_dq_data, Relax_disp.test_korzhnev_2005_15n_sq_data, Relax_disp.test_korzhnev_2005_15n_zq_data, Relax_disp.test_korzhnev_2005_1h_sq_data, Relax_disp.test_korzhnev_2005_all_data, Relax_disp.test_korzhnev_2005_all_data_disp_speed_bug. They should already be integers, but is now solved. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Comment and spell fixing in NS CPMG 2-site 3D. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Triple speed in NS MMQ 3-site, after using numpy.linalg.matrix_power instead of the lib version in relax. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Updated the dispersion model profiling comparison of the disp_spin_speed branch vs. relax-3.2.2. This now includes all dispersion models. This shows the large speed increases in the numeric and MMQ models recently obtained. Note that something went wrong with the 'NS CPMG 2-site 3D full' model for the clustered analysis, most times were around 24 seconds except for the first which was strangely 292 seconds.
Updated the relaxation dispersion model profiling log file for relax version 3.2.2 vs. 3.2.1. This adds the MMQ and 3-site models to the log file. The new information shows that there was a 4.2 times speed up for the 'MMQ CR72' model between these two relax versions, both for single spins and clustered spins, a much greater improvement than any other of the models.
Removed the now redundant disp_profile_all_3.2.2.table.txt dispersion model profiling table. As the dispersion model profiling master script now covers all dispersion models, the output from this script produces this table exactly. Therefore the end of the log files saved from running this script contains this table.
Initiated lengthy profiling script, that shows that doing square numpy matrix_power on strided data, can speed up the calculation by factor 1.5. The profiling script can quickly be turned into a unit test, and includes small helper functions to calculate how to stride through the data. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
First try to implement function that will calculate the matrix exponential by striding through data. Interestingly, it does not work. These system tests will fail: test_hansen_cpmg_data_to_ns_cpmg_2site_3D, test_hansen_cpmg_data_to_ns_cpmg_2site_3D_full. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added matrix_power to the init file in lib/dispersion. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Deleted the printout in dep_check. The printouts are only used for the essential packages before calling sys.exit(). Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Added the missing "self.num_exp" to target function. Testing on older system, this was failing the system test. It is a wonder how these lines in __init__ could be performed without this.
Fix for unit test passing on old numpy systems. The error was: ValueError: setting an array element with a sequence. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Expanded the dispersion target function class documentation. The NE, NS, NM, NO, and ND notation is now explained.
Added Ti and NT to the dispersion target function class documentation.
Slight speed up of the 'B14' and 'B14 full' dispersion models by minimising repetitive maths.
Initial try to write up a 2x2 matrix by closed form. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Made the validation check in profiling of matrix_power check all values. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced all self.spins with self.NS in target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced all self.num_exp with self.NE in target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Replaced all self.num_frq with self.NM in target function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
A very small speed up to the CR72 dispersion models by minimising repetitive maths operations. The k_BA and k_AB rates are used to simplify the Psi calculation, dropping from 3 to 2 multiplications and removing a squaring operation. The Dpos and Dneg value calculations have been simplified to drop one multiplication operation. And the calculation of eta_scale / cpmg_frqs now only occurs once rather than twice.
Removal of a tonne of unused imports in the lib.dispersion package. These were identified using the command "pylint * --disable=all --enable=unused-import".
A very small speed up to the MMQ CR72 dispersion model by minimising repetitive maths operations. This matches the recent change for the CR72 model, though the Psi calculation was already using the fast form.
Created a master profiling script for comparing the speed between different dispersion models. This is similar to the disp_profile_all.py script except it only operates on a single relax version. The output is then simply the timings, with statistics, of the calculation time for 100 function calls for 100 spins (either 10,000 function calls for single spins or 100 function calls for the cluster of 100 spins). The output of the script for the current disp_spin_speed branch code has also been added.
Critical fix for the recalculation of tau cpmg, when plotting for numerical models. The interpolated dispersion points with tau_cpmg was calculated with frq, instead of cpmg_frq. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
The new dispersion model profiling master script now includes links to the relax wiki. The models are no longer presented by name but rather by the relax wiki links for each model (see http://wiki.nmr-relax.com/Category:Relaxation_dispersion for all these links). This is to improve the Google rank of the relax wiki, as these links may appear in a number of locations.
Removal of many unused imports in the disp_spin_speed branch. These were detected using the devel_scripts/find_unused_imports.py script which uses pylint to find all unused imports. The false positives also present in the trunk were ignored.
Code validation of lib/dispersion/b14.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/cr72.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/dpl94.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/lm63_3site.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/lm63.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/m61b.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/m61.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/matrix_exponential. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/mp05.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/ns_cpmg_2site_expanded.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/ns_cpmg_2site_star.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/ns_mmq_2site.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/ns_mmq_3site.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/ns_r1rho_2site.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/ns_r1rho_3site.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/tap03.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/tp02.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of lib/dispersion/two_point.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Code validation of target_functions/relax_disp.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
For model NS MMQ 3-site, moved the parameter conversion of dw_AB from target function to lib function. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed chi sum initialisation in func_ns_mmq_2site() as this is not used. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Documentation fix for the get_back_calc() function in target_function/relax_disp.py. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Removed unnecessary repetitive calculation of kex**2 in model DPL94. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
API documentation fixes, where a "\" is the last character on the line. There should be a space " ", ending this character. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.
Updated the minfx version number to 1.0.9 in the release checklist document. This as of yet unreleased version contains an important fix for parallelised grid searches when the number of increments is set to one (i.e. a preset parameter).
Fix for the _prompt.test_align_tensor.Test_align_tensor.test_init_argfail_params unit test. As the alignment tensor can now be initialised as None, the None value can be accepted and a different RelaxError is raised when the params argument is incorrectly supplied.
Added a new set of icons for use with the minimisation user functions. These are of the Rosenbrock function and are much better suited for small icons than the current OpenDX 3D isosurface plots. The matplotlib figure originates from public domain code at http://commons.wikimedia.org/wiki/File:Rosenbrock_function.svg.
Redesign of the optimisation user functions calculate, grid_search, and minimise. In preparation for expanding the number of optimisation user functions, these three current user functions have been shifted into the new minimise user function class. The calculate user function is now accessed as minimise.calculate. The grid search as minimise.grid_search. And minimisation is via the minimise.execute user function. The icon used for the new user function class is the Rosenbrock function or the banana optimisation problem. As this is such a radical change, a huge number of changes in the relax source code, the sample scripts, the user manual, and the test suite were required.
Created the new minimise.grid_zoom user function. This allows the grid zoom level to be set. The value is stored in the current data pipe and will be used later by the minimise.grid_search user function.
The minimise.grid_zoom user function now uses the zoom-in Oxygen icon.
Created the Relax_fit.test_zooming_grid_search system test. This will be used to test the implementation of the zooming grid search. The relaxation curve-fitting analysis should be one of the fastest for testing this.
Added the print_model_title() method to the specific analysis base API class. This will be used to format and print out the information returned by the model_info() API method.
Implemented the print_model_title() specific analysis API method for the dispersion analysis.
Modified the specific analysis API _model_loop_spin() common method. This now additionally returns the spin ID string to allow the corresponding spin container to be identified.
Implemented the specific analysis API common method _print_model_title_spin(). This is for the corresponding _model_loop_spin() method. It can be aliased in the specific analyses to provide the print_model_title() API method.
Aliased the _print_model_title_spin() specific analysis API common method in a few analyses. This provides the print_model_title() API method for the J(w) mapping, consistency testing, and relaxation curve fitting analyses.
Updated all the specific analysis methods affected by the _model_info_spin() API method change. This is for the change whereby the common API method now returns the spin ID string as well.
Implemented get_param_names() and get_param_values() for the relaxation curve-fitting analysis. These are part of the specific analysis API.
Created the specific analysis API return_parameter_object() function. This is used by the non-specific analysis code to obtain the parameter object (a singleton object). It will allow for more direct access to the parameter information.
Created the parameter object infrastructure for adding the grid search lower and upper bounds. The _add() method now accepts the grid_lower and grid_upper keyword arguments, which can be either values or functions. These are then stored in the _grid_lower and _grid_upper class dictionaries. The public methods grid_lower() and grid_upper() have been added to return the value corresponding to the given parameter.
Modified the specific analysis parameter object grid_lower() and grid_upper() methods. These now accept the model information from the model_loop() API method and send that into an grid lower and upper functions. These functions will require the information to pull out the correct spin, spin cluster, or other information from the current data pipe to determine what the bounds should be.
Implemented infrastructure in the grid_search user function in preparation for the zooming grid. The grid search backend now calls the new grid_bounds() function. This takes the lower and upper bounds as arguments, uses the specific API to determine the per-model parameter grid search bounds, and then returns a per model list of lower and upper bounds. The specific API get_param_names() and get_param_values() are called to obtain the current model parameter names and values, and then the parameter names and model info are used in the new parameter object grid_lower() and grid_upper() methods to obtain the bounds. This shifts all of the grid search bounds logic out of the specific analyses and into the grid search backend, so it should allow the specific analysis code to be simplified.
More modifications of the minimise.grid_search user function backend. The grid_bounds() function has been renamed to grid_setup(), and it now accepts and processes the inc user function argument. The error checking code of the relaxation curve-fitting grid_search_setup() optimisation function has been shifted into this analysis independent grid_setup() function to shift the minimise.grid_search user function error checking out of the specific analyses. The function now scales the parameter bounds, using the yet-to-be implemented scaling() method of the parameter object. And the grid search increments are converted into a per-model list of lists.
Created the parameter object infrastructure for registering parameter scalings. The _add() method now accepts the scaling keyword argument, which can be either a value or function. This is then stored in the _scaling dictionary. The public method scaling() has been added to return the scaling factor corresponding to the given parameter.
Modified the analysis specific API optimisation method. The base calculate(), grid_search() and minimise() methods now all accept the scaling_matrix argument, and the minimise() scaling argument has been removed. This scaling_matrix argument should be a per-model list of scaling matrices. To handle the change, the pipe_control.minimise.assemble_scaling_matrix() function has been created. This uses the new parameter object scaling values to create the list of scaling matrices. This will in the end replace all of the analysis specific assemble_scaling_matrix() functions and simplify their optimisation code paths.
Changed the order of operations in the minimisation user function backends. The specific analysis API overfit_deselect() method needs to be called before any grid bounds, increments, or the scaling matrices are assembled. This is for the cases when the grid bounds or scaling factors are functions rather than values.
Converted the relaxation curve-fitting analysis to the new grid bounds and scaling factor design. The parameter object now registers the grid bounds and scaling factors for all of the curve-fitting parameters. This includes three functions i_scaling(), i0() and iinf() in the specific_analyses.relax_fit.parameter_object module for calculating some of these values. The specific_analyses.relax_fit.parameters.assemble_scaling_matrix() function has been deleted as this is now provided by the upstream code in pipe_control.minimise. And the API methods grid_search() and minimise() has been modified to accept the list of scaling matrices. As the grid bounds and increments are now handled by the upstream pipe_control.minimise.grid_setup() function, the specific_analyses.relax_fit.optimisation.grid_search_setup() function was redundant and was deleted.
Created the consistency testing specific API method get_param_names(). This is now required for the minimise.calculate user function, specifically for the analysis independent assemble_scaling_matrix() function. The get_param_names() method simply returns the fixed list of parameter names.
Standardisation of the specific analysis API with respect to the model_loop() and base_data_loop(). The model information arguments for the data returned by model_loop(), and the data arguments for the data arguments for the data returned by base_data_loop() have been standardised throughout the API.
Epydoc parameter order rearrangement in the specific analysis API base class.
Updated the specific analysis API common methods for the recent model_info argument changes.
Updated all of the specific API calculate() methods to accept the scaling_matrix argument. The list of per-model scaling matrices is now created independently of the analysis type by the pipe_control.minimise methods for the minimise.calculate, minimise.grid_search and minimise.execute user functions and sent into the specific analysis backend.
Updated all of the specific API grid_search() methods to accept the scaling_matrix argument. The list of per-model scaling matrices is now created independently of the analysis type by the pipe_control.minimise methods for the minimise.calculate, minimise.grid_search and minimise.execute user functions and sent into the specific analysis backend. The argument is also passed into the minimise() API method from the grid_search() method when that is used.
Updated all of the specific API minimise() methods to accept the scaling_matrix argument. The list of per-model scaling matrices is now created independently of the analysis type by the pipe_control.minimise methods for the minimise.calculate, minimise.grid_search and minimise.execute user functions and sent into the specific analysis backend.
Fix for the Monte Carlo simulations for the model_info argument changes in the specific API.
Fixes for the consistency testing and J(w) mapping calculate() methods. This is for the changes to the data_init() specific analysis API method.
More fixes for the Monte Carlo simulations for the model_info argument changes in the specific API.
Updated all of the data_init() specific API calls where the spin ID is expected.
Fixes for the _data_init_spin() specific API common method. The data returned from _base_data_loop_spin() is just the spin ID, the spin container is not included.
Updated the eliminate user function backend to work with the model_info argument changes in the specific API.
The new pipe_control.minimise module functions can now handle models with no parameters. The new assemble_scaling_matrix() and grid_setup() functions will now handle models with no parameters, as this is required for the relaxation dispersion analysis.
More fixes for the eliminate user function backend. This is for the model_info argument changes in the specific API.
Fixes for the grid search backend for a recent breakage and expansion of its capabilities. The user supplied lower and upper bounds for the grid search were no longer being scaled via the scaling matrix. In addition, the code has been refactored to be simpler and more flexible. The user can now supply just the lower or just the upper bounds and the grid search will work.
The grid search setup function now prints out the grid search bounds to be used. This is in the pipe_control.minimise.grid_setup() function, hence it is analysis independent. This is useful feedback for the user to know what the grid search is doing. And it will be even more useful for the zooming grid search to understand what is happening.
The grid search setup printout now also included the number of increments for each parameter.
Modified the new print_model_title() specific analysis API method. This now accepts the prefix argument for creating different titles independently of the specific analysis.
The grid search setup function now uses the prefix argument to the print_model_title() API function. This is simply set to 'Grid search setup:'.
The relaxation dispersion API now uses the MODEL_R2EFF variable for identifying the R2eff model.
Changes to the minimise.grid_search user function frontend. The Boolean constraints argument has been shifted to the end, and empty lines have been removed.
Epydoc docstring fixes for the keyword arguments of the pipe_control.minimise module.
Shifted the constraints Boolean argument to the end of the grid_search() function argument list.
Major change to the grid search user function. The minimise.grid_search user function now accepts the skip_preset flag. When True, the grid search will skip any parameters with a preset value. This allows the user to set parameters via the value.set user function and then have these parameters skipped in the grid search. The new skip_preset argument is passed into the pipe_control.minimise.grid_setup() function in the backend. This then sets both the grid lower and upper bounds to the preset parameter value and sets the number of increments to 1 for that parameter so that it is essentially skipped in the grid search.
Small change to the table printed out during the minimise.grid_search setup.
Fix for the skipping of preset parameters in the grid search. Dictionary and list type parameters are now handled correctly.
Converted the relaxation dispersion analysis to the new grid bounds and scaling factor design. The parameter object now registers the grid bounds and scaling factors for all of the dispersion parameters. This includes the functions dw_lower(), dwH_lower(), pA_lower() and i0_upper() in the specific_analyses.relax_disp.parameter_object module for calculating some of these values. The specific_analyses.relax_disp.parameters.assemble_scaling_matrix() function has been deleted as this is now provided by the upstream code in pipe_control.minimise. And the API methods grid_search() and minimise() has been modified to accept the list of scaling matrices. As the grid bounds and increments are now handled by the upstream pipe_control.minimise.grid_setup() function, the specific_analyses.relax_disp.optimisation.grid_search_setup() function was redundant and was deleted. The specific_analyses.relax_disp.parameters.get_param_names() function was also modified with the full argument added, to allow either the base parameter names or an augmented form with the dictionary key for presenting to the user to be returned. Importantly to allow the changes to be operational, the model_loop() API method was redesigned so that, for the R2eff base model, the individual spins rather than spin clusters will be looped over. This allows the specific_analyses.relax_disp.optimisation.minimise_r2eff() function to continue to operate correctly.
Implemented the J(w) mapping analysis get_param_names() API method. This simply returns the hardcoded list of 3 parameters of the model, and allows the minimise.calculate user function to operate.
Updated the _print_model_title_spin() specific API common method. This now accepts the prefix argument and adds this to the title.
The minimise.grid_search user function can now properly handle preset values of NaN. This occurs when the parameter vector contains values of None due to the parameter not being set and then the Python list being converted to a numpy array. The value of NaN is now caught and the parameter is no longer identified as being preset.
Fixes for the relaxation curve-fitting grid search. The parameters which are not set are no longer defaulting to 0.0. This means that the parameter vector will sometimes contain NaN values, but this is important for the correct operation of the new minimise.grid_search user function backend.
Updated the NOE analysis to handle the changes for the minimise.calculate user function. This now requires the model_loop() and get_param_names() API methods to be implemented. The first is provided by the API common _model_loop_spin() method and the second simply returns a list of the single 'noe' parameter.
Created the _print_model_title_global() specific analysis API method. This is to be paired with the _model_loop_single_global() API method and it simply prints out the prefix as the title.
Created the specific analysis parameter object _add_align_tensor() method. When called by a specific analysis, this will add the [Axx, Ayy, Axy, Axz, Ayz] parameters to the corresponding parameter object.
Deleted the lib.optimisation module. The checks in the single test_grid_ops() function are implemented in the pipe_control.minimise.grid_setup() function and are now redundant.
Removed the import of the now deleted lib.optimisation module from the model-free analysis.
Converted the N-state model analysis to the new grid bounds and scaling factor design. The parameter object now registers the grid bounds and scaling factors for all of the N-state model parameters. The specific_analyses.n_state_model.parameters.assemble_scaling_matrix() function has been deleted as this is now provided by the upstream code in pipe_control.minimise. And the API methods grid_search() and minimise() has been modified to accept the list of scaling matrices. In addition, all of the lower bounds defined in the grid_search() API method have been deleted as this is now in the parameter object. The new API function print_model_title() has been aliased from _print_model_title_global(). And the get_param_names() and get_param_values() API methods have been implemented.
The grid search upper and lower bound functions now must accept the incs argument. For a few analyses, the number of grid search increments are used to remove the end point of the grid, to remove duplicate points due to the circular nature of angles. Therefore the parameter object grid_lower() and grid_upper() methods now send the grid increment number for each parameter into all grid bound determining functions. The relaxation dispersion and curve-fitting analyses have been updated for the change.
Converted the frame order analysis to the new grid bounds and scaling factor design. The parameter object now registers the grid bounds and scaling factors for all of the dispersion parameters. This includes the functions angle_upper_excluding_bound(), axis_alpha_upper(), cone_angle_lower(), cone_angle_upper(), pivot_grid_bound(), pivot_x_lower(), pivot_x_upper(), pivot_y_lower(), pivot_y_upper(), pivot_z_lower(), and pivot_z_upper() in the specific_analyses.frame_order.parameter_object module for calculating some of these values. The specific_analyses.frame_order.parameters.assemble_scaling_matrix() function has been deleted as this is now provided by the upstream code in pipe_control.minimise. And the API methods grid_search() and minimise() has been modified to accept the list of scaling matrices. As the grid bounds and increments are now handled by the upstream pipe_control.minimise.grid_setup() function, the setup and error checking code in the grid_search() API method was redundant and was deleted.
Modified all calls to the parameter object _add() base class method. These are now all spread across multiple lines, with each argument on a separate line. This is for easier maintenance of the specific analysis parameters, as the code is now much cleaner and argument changes will only have diffs for that argument. It is also visually easier to see all the settings for each parameter.
Fix for the grid search setup printout when parameters are preset.
Changes to the diffusion tensor initialisation in the model-free auto-analysis. The values of the tensor are now initialised to None. This is to allow for the new grid search preset flag which defaults to True, setting the values to None indicates that a grid search should be performed.
The diffusion_tensor.init user function can now set initial tensor parameter values of None. This is to allow for the new grid search preset flag. Therefore allowing the values to be None allows for a grid search to be performed by default.
Created two new model-free system tests. These are Mf.test_m0_grid_with_grid_search and Mf.test_m0_grid_vs_m1_with_grid_search. Their aim is to better test the grid search in a model-free analysis when parameters are preset.
Converted the model-free analysis to the new grid bounds and scaling factor design. The parameter object now registers the grid bounds and scaling factors for all of the model-free parameters. This includes the functions rex_scaling() and rex_upper() in the specific_analyses.model_free.parameter_object module for calculating some of these values. The base parameter object has also been updated as that is where the diffusion parameters are defined. Here the da_lower() and da_upper() have been defined to handle the different Da value constraints. The specific_analyses.model_free.parameters.assemble_scaling_matrix() function has been deleted as this is now provided by the upstream code in pipe_control.minimise. And the API methods grid_search() and minimise() has been modified to accept the list of scaling matrices. As the grid bounds and increments are now handled by the upstream pipe_control.minimise.grid_setup() function, the grid_search_config(), grid_search_diff_bounds() and grid_search_spin_bounds() functions in the specific_analyses.model_free.optimisation module were redundant and were deleted. The new API function print_model_title() has been implemented to handle the grid search setup printouts.
Modified the pipe_control.minimise.grid_setup() function for when no parameters are present. For the current version of minfx to function correctly (1.0.8), the lower, upper and inc values should be set to [] rather than None.
Fix for the lib.arg_check.is_num_or_num_tuple(). When the can_be_none flag is set to True, the tuple of None values is now considered valid. This enable the diffusion_tensor.init user function to accept the spheroid tensor values of (None, None, None, None), and the ellipsoid tensor values as a tuple of 6 None.
Fix for the _prompt.test_diffusion_tensor.Test_diffusion_tensor.test_init_argfail_params unit test. As the diffusion tensor can now be initialised as None, the None value can be accepted and a different RelaxError is raised when the params argument is incorrectly supplied.
Modified the behaviour of the parameter object units() method. If the unit is set to the default of None, this method will now return an empty string instead of None.
The rx parameter of the relaxation curve-fitting analysis now has 'rad.s^-1' units defined.
Implemented the zooming grid search. If the zoom level is set to any value other than 0, then the grid width will be divided by 2**zoom_level and centred at the current parameter values. If the new grid is outside of the bounds of the original grid, the entire grid will be translated so that it lies entirely within the original.
Modified the zooming grid search algorithm. If the zoom level is negative, hence the grid will be larger than the original, the checks that the grid is within the original are no longer active.
Changed the minimise.grid_zoom user function. The zoom level can now be any floating point number or integer, including negative values. The user function docstring has been significantly expanded to explain the entire zooming grid search concept.
Alphabetical ordering of the minimisation user functions in the user_functions.minimisation module.
Large expansion of the minimise.grid_search user function documentation. The previous documentation was essentially non-existent.
Expanded the minimise.grid_zoom user function documentation. A few sentences about the limitations of the algorithm have been added.
Completed the Relax_fit.test_zooming_grid_search system test. Now only a single spin is optimised. The zooming levels increase in integer increments from 0 to 50 so that the final zoomed grid is insanely small (as the curve-fitting C modules are incredibly fast, this test is nevertheless relatively quick). The final zooming grid search parameter values are checked to see if they are the same as those optimised in the Relax_fit.test_curve_fitting_height system test to demonstrate the success of the algorithm.
Modified the grid search upper bounds functions for the relaxation curve-fitting. This is for both the exponential relaxation curve-fitting analysis and the same fitting in the dispersion analysis. The intensity values are doubled and then rounded to the next order. This ensures that I0 and Iinf will be within the grid bounds. Hence the zooming grid search can be used for these curves.
Expanded the documentation for the minimise.calculate user function. This now explains the dual operations of the user function.
Fixes for some relaxation dispersion system tests not converted to the new optimisation user functions. Minimisation is now via the minimise.execute user function, which used to be the minimise user function.
Added a 128x128 pixel version of the zoom-in Oxygen icon. This icon size is not available in the repository located at svn://anonsvn.kde.org/home/kde/trunk/kdesupport/oxygen-icons. Therefore the scalable/actions/small/48x48/zoom-in.svgz file was copied and then exported into a 128x128 PNG, and finally converted to a Gzipped EPS file for the relax manual.
The frame order grid search bound functions can now handle increment values of None or 1. These cases are now caught and the full lower or upper bound is now returned.
More even spacing for the frame order grid search. This is for the parameters which exclude end points in the grid search, as these excluded points do not decrease the number of increments searched over.
Even more even spacing for the frame order grid search. This is for the parameters which exclude end points in the grid search, as these excluded points do not decrease the number of increments searched over. This fixes the algorithm for all of the bounds.
Improved the logic for skipping parameters in the grid search. The logic is also fully explained in the user function documentation.
Removal of all unused imports. These were identified using the devel_scripts/find_unused_imports.py script.
Reverted the deletion of the Relax_disp.test_hansen_cpmg_data_to_lm63_3site system test which occurred in relax 3.2.3. See the thread at http://thread.gmane.org/gmane.science.nmr.relax.scm/21774/focus=6300 for the request that this deletion be reverted. This is the only system test for the 'LM63 3-site' dispersion model using real data. Having this test allows for better coverage of the code.
Updated the Relax_disp.test_hansen_cpmg_data_to_lm63_3site system test. This is for the changes to the optimisation user functions.
Updated the checks in the Relax_disp.test_hansen_cpmg_data_to_lm63_3site system test. The values were incorrect due to a bug in relax and a non-optimal minfx setting (https://gna.org/bugs/?22210 and https://gna.org/bugs/?22211).
Fix for a fatal bug for the prompt UI on MS Windows. The improvements in the tab completion support for the prompt UI on Mac OS X systems was fatal for certain Python readline modules on MS Windows, as readline.__doc__ can be None. This is now correctly handled.
Decreased the precision the Relax_disp.test_hansen_cpmg_data_to_lm63_3site system test. This is to allow the test to pass on Mac OS X systems.
Unit test fix for Mac OS X. This is for the test_ns_mmq_2site_korzhnev_2005_15n_dq_data_complex128 test of test_suite.unit_tests._lib._dispersion.test_matrix_exponential.Test_matrix_exponential. The tests no longer check for exact values, but use the assertAlmostEqual() calls instead.
Deleted the ancient optimisation_testing.py development script, as this no longer works and is of no use.
Implemented the pipe_control.mol_res_spin.format_info_full() function. This follows from http://thread.gmane.org/gmane.science.nmr.relax.scm/22522/focus=6534. This is a verbose representation of the spin information which can be used for presenting to the user. Functions for shorter string versions will also be of great use, for example as described by Troels at http://thread.gmane.org/gmane.science.nmr.relax.scm/22522/focus=6535.
Created a unit test for the pipe_control.mol_res_spin.format_info_full() function. This comprehensive test covers all input argument combinations.
Changed the behaviour of the pipe_control.structure.mass.pipe_centre_of_mass() function. This function returns the CoM and optionally the mass of the structural data loaded into the current data pipe. However it was matching the structural data to the molecule-residue-spin data structure and skipping spins that were deselected. This illogical deselection part has been eliminated, as spins can be deselected for various analysis purposes and this should not change the CoM. The deletion also significantly speeds up the function.
Added Andy Baldwin's 2013 R1rho relaxation dispersion model (BK13) to the manual. The model has been added to the table of dispersion models and to the dispersion software comparison table of the dispersion chapter of the manual. The citation has also been added to the bibliography.
The BK13 dispersion model is now properly added to the software comparison table.
Added the 'BK13' and 'BK13 full' dispersion models to the to do section of the manual.
Standardisation of the author names in the bibliography of the relax manual.
Added links for the BK13 model to https://gna.org/support/?3155 in the manual.
Expansion of the 'to do' section of the dispersion chapter of the manual.
Editing of the 'to do' section of the dispersion chapter of the manual.
Split out the interpolating in specific_analyses.relax_disp.data.plot_disp_curves() into separate function. This is to prepare for a interpolating function for spin-lock offset rather than spin-lock field strength for R1rho models. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Split out the looping over frequency and offset into its own function, in function of specific_analysis.relax_disp.data.plot_disp_curves(). Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Split out the writing of dispersion graph files in specific_analyses.relax_disp.data.plot_disp_curves(). This is to prepare for a stand-alone function to plot R1rho graphs, interpolating theta through spin-lock offset rather than spin-lock field strength. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for function calling and default values of None in sub-plotting functions. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Large extension of specific_analyses.relax_disp.data(), by adding several helper plotting functions. This is to prepare for plotting R1rho/R2 as function of effective field in rotating frame: w_eff. R_2 = R1rho / sin^2(theta) - R_1 / tan^2(theta) = (R1rho - R_1 * cos^2(theta) ) / sin^2(theta). Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Extended specific_analyses.relax_disp.optimisation.back_calc_r2eff() to handle interpolated spin-lock offset values. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Removed a wrong adding of empty offset dimension in the get_back_calc() function of target_functions.relax_disp(). Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added the back calculated R2 as function of effective field in rotating frame: w_eff. R1rho R_2 is defined as: R_2 = R1rho / sin^2(theta) - R_1 / tan^2(theta) = (R1rho - R_1 * cos^2(theta) ) / sin^2(theta). This is described more at: http://wiki.nmr-relax.com/DPL94#Equation_-_re-writed_forms. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added an intermediate attempt to show the back calculated data in the graph for R1rho/R2 as function of the effective field in rotating frame: w_eff. The graph is aiming for the representation of Figure 2 in Kjaergaard et al 2013. (http://dx.doi.org/10.1021/bi4001062). The figure can be seen at https://gna.org/support/download.php?file_id=20208. It becomes clear, that it is not necessary interpolate through the spin-lock offset, but it is sufficient to interpolate through the spin-lock field strengths. The necessary step was the extraction of the effective field in rotating frame, w_eff. In earlier attempt is shown at: http://wiki.nmr-relax.com/File:Matplotlib_52_N_R1_rho_R2eff_w_eff.png This though show lines for 6 offset values. The question is how to show the single line of interpolation. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added NMR library function to convert the given frequency from rad/s to ppm units. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Hard-coded restriction of R1rho extra plotting to model DPL94, TP02, TAP03, MP05, NS_R1RHO_2SITE. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for sending the correct data structures to target function, and fix for the spin index which is always zero in graph production. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Moved the file_name creation out of the interpolate function, to make it a general function for interpolating. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Made both interpolation functions of the spin-lock field strength and spin-lock offset return the offset data. This is make it possible to switch between the interpolating functions, when plotting R1rho graphs. This is necessary to produce the R2 as function of effective field in rotating frame w_eff, and to produce R1rho as function of theta, when ramping the spin-lock offset. These graphs can be seen at: http://wiki.nmr-relax.com/Matplotlib_DPL94_R1rho_R2eff. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Changed interpolation function from offset to spin lock field strength, to plot R1rho/R2 as function of effective field. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Improved axis label for plotting R1rho/R2 as function of effective field w_eff. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added sub-title to the plot of R1rho/R2 as function of effective field. This is to add information about how the effective field has been interpolated. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added functionality to plot R1rho/R2 as function of effective field w_eff, for the R2eff model. Also renamed a function, to better reflect is functionality. The hard-coding of which models to plot, has been removed. If the exp-type is R1rho, then the plotting will commence. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added the spin specific residue name and spin_id to the title of the dispersion plots. This is handy, since it is often of interest to have this information at hand, when looking through many graphs. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed and improved epydoc information for interpolating function for dispersion values. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed the interpolating function for offset, and improved the epydoc information. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Corrected the epydoc information for the return_offset_data() function in specific_analyses.relax_disp.data. The function has been extended to return more data. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed function to represent the functionality of returning data in correct xmgrace form. Also improved the epydoc information, for the return of values. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed the other function to represent the functionality of returning data in correct xmgrace form. Also improved the epydoc information, for the return of values. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added function to calculate rotating frame parameters for lib/nmr.py. This function is called several times in specific_analyses/relax_disp/data.py by plotting functions. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Replaced repeated calculation of rotating frame parameters to use function in lib/nmr.py. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Skip production of plotting R1rho/R2 as function of effective field w_eff, when spin.isotope is not present. This can happen when it is 'exp_fit' model curve fitting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added functionality to function to accept how the first part of the filename is formed. This is to prepare to reuse the same plotting function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed plotting sub function, to accept different file name arguments, and X-axis arguments. This is to reuse the sub-plot function to plot against different X-axis. Added plot of R1rho as function of theta, where interpolated against spin-lock field strength. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Prepared flag, to tell which data type to interpolate through. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Preparation to interpolate through the offset to plot R1rho as function of theta, interpolated through spin-lock offset. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added graph functionality, to plot R1rho as function of theta, when spin-lock offset is interpolated. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Ensured production of plotting R1rho/R2 as function of effective field w_eff, when spin.isotope is not present. The offset in radians would be set to 0.0 instead. This can happen when it is 'exp_fit' model curve fitting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added function to return spin info, and a function to return a spin string for graphs. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Replaced the spin info string in the title of graphs, with the new function method. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added keywords to be used to backend function of plot_disp_curves. The keyword 'y_axis', determine which y data to be plotted on this axis. The keyword 'x_axis', determine which x data to be plotted on this axis. The keyword 'extend_hz' to determine how far to extend interpolated CPMG frequency or spin-lock field strenght. The keyword 'extend_ppm' to determine how far to extend interpolated spin-lock offsets. The keyword 'interpolate' to determine to interpolate dispersion points, or offset. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Made the plotting function uniform into accepting both CPMG and R1rho data. Also made a function to return data, depending if it is data, back calculated, interpolated or residual. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed return grace data function, to a better shorter name. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed the other return grace data function, to a better shorter name. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Hardcoded xmgrace colour_order, and made function to return data label, and data plot settings, depending on data type. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed plotting function to represent the function of writing to file. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Removed unused plotting function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Streamlined plotting functions, to have similar input. Reordered the output from return_offset_data(), interpolate_disp() and interpolate_offset(), to reflect the order of data type. Made the input to return_grace_data_vs_disp(), return_grace_data_vs_offset the same(). Added the interpolate flag to return_grace_data_vs_disp() and return_grace_data_vs_offset(), to help return correct X-value. Added the interpolate flag to return_x_y_point(), to help determine if "disp point" or "offset point" should be returned. Added the "offset point" to return_x_y_point() function, to make it possible to plot against offset. Cleaned up the return_grace_data_vs_offset() function, to use newly created return_x_y_labels() function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for output catching after reorder or return_offset_data() function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Removed unused return_grace_data() function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Made uniform function for returning x_axis and y_axis labels for xmgrace plotting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Renamed return functions, to reflect they are specific for xmgrace plotting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Extended relax_disp auto_analyses to plot special R1rho graphs. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Modified front-end user function relax_disp.plot_disp_curves to send new arguments to back-end function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added new Unicode symbols to be used by the GUI drop-down menu. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Modified back-end of plot_disp_curves() to reflect changes to the front-end function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Extended system test Relax_disp.test_r1rho_kjaergaard_auto(), to check that the expected graphs exist. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added model 'No Rex' to system test Relax_disp.test_r1rho_kjaergaard_auto(), to check all graphs are produced. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added function to return the initial part of the file name for grace plotting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Modified system test to use the new function to return initial part of file name for grace plotting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Started testing all possible combinations of graphs for R1rho analysis. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added more printout, to detect which graphs are not working. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added forgotten "interpolate" type to function which return X,Y point to xmgrace graphs. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for interpolation graph when plotting R1rho/R2 as function of offset (ppm). Missed to extract the offset value from list. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Forcing overwrite of special R1rho graphs in auto analyses in relax_disp. The other graphs are also auto forced. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added system test Relax_disp.test_r1rho_kjaergaard_auto_check_graphs, to check that the contents of all combinations of graphs are consistent. The system test actual show that the error is changing per run-through. This is a bug, which should be corrected. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added graphs to check against in system test test_r1rho_kjaergaard_auto_check_graphs. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Code validation of system test file for Relax_disp. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix to system test Relax_disp.test_r1rho_kjaergaard_auto_check_graphs by only comparing X,Y values, and skipping the error. This is a hack until the error difference bug gets corrected. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added system test Relax_disp.test_kteilum_fmpoulsen_makke_check_graphs() to check all possible combinations of dispersion plotting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for GUI text in MS Windows, since subscript 1 and Greek theta symbol is not working in this Unicode system. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for desc_short, in user function relax_disp.plot_disp_curves. The text "The " is preformed in the formatting. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for system test, after moving graphs to check against. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Moved graph files up one level in system test. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for forgotten removal of counter. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added check function for relax_disp. This function check if interpolating against offset for non-R1rho exp types, and will raise an Error. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added check function to plot_disp_curves, to check that CPMG exp types are not interpolated against offset, which is not implemented. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for forgotten "1" in lib text GUI. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Improved description in GUI text for user function relax_disp.plot_disp_curves(). The improved description now explains the new features. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Extended graph labelling, file naming and return of data for multiple CPMG graphs types. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added system test Relax_disp.test_kteilum_fmpoulsen_makke_check_graphs to check all CPMG graph combinations of: y_axis_types = [Y_AXIS_R2_EFF, Y_AXIS_R2_R1RHO]; x_axis_types = [X_AXIS_DISP, X_AXIS_THETA, X_AXIS_W_EFF]; interpolate_types = [INTERPOLATE_DISP]. This is a total of 6 graphs. The graphs will in most cases be totally equal, since the theta angle is calculated to 90 degrees, and R1 is returned as 0.0, then R2=(R1rho - R1 cos^2(theta)) / sin^2(theta) = R1rho = R2eff for CPMG models. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Added graphs to check against for system test: Relax_disp.test_kteilum_fmpoulsen_makke_check_graphs. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Expanded ex. to example in help text for function. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Fix for unit test, where return_offset() function return has been expanded and reordered. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 (https://gna.org/support/?3138): Interpolating theta through spin-lock offset [Omega], rather than spin-lock field strength [w1].
Created the Bruker.test_bug_22411_T1_read_fail system test. This is to catch bug #22411 (https://gna.org/bugs/?22411) as reported by Olena Dobrovolska.
Fix for system test Relax_disp.test_kteilum_fmpoulsen_makke_check_graphs where minimise has been extended with execute.
Changed graphs after new minimisation algorithm has been implemented. The values are now slightly different.
Implemented first try to stride through data, when computing the eig() of higher dimensional data. System test test_cpmg_synthetic_b14_to_ns3d_cluster survived this transformation. The system test will go from about 11 seconds to 22 seconds.
Implemented second try to stride through data, when computing the eig() of higher dimensional data. This of data of form: NS, NM, NO, ND, Row, Col. System test test_sprangers_data_to_ns_mmq_2site survived this transformation. The system test will go from about 2 seconds to 4 seconds.
Created function to create the helper index numpy array, to help figuring out the indices to store in the exchange data matrix. This is for striding through data and store the data correct in the data matrix. This is for a special situation where numpy version is < 1.8, and where the numpy.linalg.eig() function can only be performed on square matrices, and not on higher dimensional data. For this situation, it is necessary to stride through the data.
Created the numpy array self.index target function which contains index to store the data. This is for situations where numpy version is under 1.8.
Added function to get the data view via striding through a higher dimensional column numpy array.
Extracted the data view of the index indices numpy array in the target_function.
A profiling showed, that it was not faster to preform the index view.
Made new general stride helper function and matrix_exponential function.
Changed to the matrix_exponential function for NS R1rho 2site.
Removed all unused helper functions, and matrix exponential functions. They are now condensed to the fewest possible functions.
Fix for eye matrix being formed incorrectly.
Replaced all matrix_exponential functions in numerical models to use the new general matrix_exponential function.
Added warning message to auto analysis in relax disp, if numpy is below 1.8 and using numerical model. This will make the analysis 5-6 times slower.
Fixes for numpy version under 1.8, when striding through data.
Fix to unit tests, after changing the name of matrix_exponential function.
Added graphs and results for run with MC=2000, for system test Relax_disp.test_r1rho_kjaergaard_auto(). This is to be able to extend graph testing for interpolated R1rho graphs, and to add figures to the latex manual.
Added list of R1rho models, which use R1 in their equations. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models. Bug #21788 (https://gna.org/bugs/?21788): Only Warning is raised for missing loading R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Modified the warning and error messages being raised when calling return_r1_data(). Now warnings is raised if no R1 data is available. An error is raised if the R1rho model is expected to have R1 data, and it is not available. That makes system test Relax_disp.test_r1rho_kjaergaard_missing_r1() fail, which is the expected behaviour. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models. Bug #21788 (https://gna.org/bugs/?21788): Only Warning is raised for missing loading R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added the model "MODEL_DPL94_FIT_R1", to the full list of models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
In system test Relax_disp.test_r1rho_kjaergaard_missing_r1(), started using the new model MODEL_DPL94_FIT_R1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added "r1_fit" as a parameter object. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added functionality to return r1_fit parameter in loop_parameters() function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added model variable MODEL_DPL94_FIT_R1, to relax_disp target function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Split the target function of model DPL94 into a func_DPL94 and calc_DPL94. This is to prepare for a target function func_DPL94_fit_r1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added initial target function for model DPL94_fit_r1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added list of R1rho models, which can fit R1 in their equations. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Modified return_r1_data() function, to return numpy array of "None", if model is in list of "MODEL_LIST_R1RHO_FIT_R1". Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Modified target function func_DPL94_fit_r1(), to unpack fitted parameters correct. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added linear linear_constraints for parameter "r1_fit". Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added warning message when using function return_r1_data(), and model is in list MODEL_LIST_R1RHO_FIT_R1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added model DPL94_FIT_R1 to the list of MODEL_LIST_R1RHO and MODEL_LIST_R1RHO_FULL. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Modified return_r1_data(), to be dependent on fitting model. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Made function return_r1_err_data() be dependent on model type. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Replaced instances of "['r2', 'r2a', 'r2b']" with variable PARAMS_R20. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented model list, which uses parameter of inverted relax delay times. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented model list, which uses parameter of R20B. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix in target function for relax_disp, where model IT99 does not belong to model list with several chemical shift correlated parameters. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added new variable, for models which has parameters with mixed dw, and has two variables. For example with both dw and dwH or dw_AB and dw_BC or phi_ex_B and phi_ex_C. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added model MODEL_NS_R1RHO_3SITE, MODEL_NS_R1RHO_3SITE_LINEAR, to list of models who has who dw parameters. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added new variable, for models which has parameters with mixed dw, and has four variables Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added to relax_disp auto_analyses, that R1_fit should be plotted and written out. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added 2 new models, MODEL_NOREX_R1RHO and MODEL_NOREX_R1RHO_FIT_R1. The "NOREX" model is not covering R1rho models. The target function for "NOREX" is calculated as: back_calc = R20 R20 is for R1hro models equivalent to R1rho prime (R1rho'), which for example in the model of DPL94 would mean: R1rho = R1rho' But for the "NOREX" case, the return should be R1rho = R1 * cos^2(theta) + (R1rho' + 0) * sin^2(theta). This affects all off-resonance model calculations. These two target functions will be implemented. Bug #22440 (https://gna.org/bugs/?22440): The "NOREX" model is not covering R1rho models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Made the model MODEL_NOREX_R1RHO_FIT_R1, be tested in system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). Bug #22440 (https://gna.org/bugs/?22440): The "NOREX" model is not covering R1rho models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added models MODEL_NOREX_R1RHO and MODEL_NOREX_R1RHO_FIT_R1 to MODEL_LIST_FULL. Bug #22440 (https://gna.org/bugs/?22440): The "NOREX" model is not covering R1rho models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented target and calculation function for MODEL_NOREX_R1RHO, MODEL_NOREX_R1RHO_FIT_R1. Bug #22440 (https://gna.org/bugs/?22440): The "NOREX" model is not covering R1rho models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Made the model "MODEL_NOREX_R1RHO", be testes in system test Relax_disp.test_r1rho_kjaergaard_auto. This is for system test where R1 has been loaded from earlier results, which was not analysed in relax. Bug #22440 (https://gna.org/bugs/?22440): The "NOREX" model is not covering R1rho models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Made list of models which fit pA or pA and pB. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added function to auto_analysis, to test if it give meaning to write and plot out the parameter. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Made the writing out of parameter pC, be tested with the new function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented models list with phi_ex, phi_ex_B, and phi_ex_C, and added to test in auto_analyses of relax_disp. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Made use of the dictionary MODEL_PARAMS, to determine if parameter is present. This makes the list of models belonging to parameter lists superfluous. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed unnecessary list of models which support a parameter. This functionality already exists with the dictionary MODEL_PARAMS. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Rearranged the writing out of parameters in auto_analysis of relax_disp. This is to prevent writing out all possible parameters in the final round, if any of those parameters have not been tested. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Extended the writing and checking of parameters, to use different file name, than the parameter name. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Moved the auto-analyses writing out of w_eff and theta into check for has_r1rho_exp_type(). Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Re-arranged all model variable lists, to be able to re-use earlier lists. This is to prevent user errors, when setting up the lists, and re-use the lists through all code. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for MODEL_NOREX_R1RHO_FIT_R1 not being part of list: MODEL_LIST_DISP. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Moved the auto_analyses plot of special R1rho graphs into the check of has_r1rho_exp_type(). Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Moved the auto-analyses plotting and writing of r2, r2a and r2b for CPMG models into test of has_cpmg_exp_type(). Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Replaced in auto analysis all instances of 'No Rex' and 'R2eff' with its equivalent defined variables. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Re-arranged plotting and writing in auto-analyses of relax disp, when model is R2eff. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed unused variables in auto-analyses of relax_disp. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented the list "MODEL_LIST_NEST", which define which model are used for nesting. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for the nested copying of r2, if using a nested list. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added the equivalent R1 fit models for: TP02, TAP03, MP05 and NS_R1RHO_2SITE. The R1 fit models will no be implemented for 3-site models, because there will be to many variables. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added the new R1 fit models to system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Split target function of model TP02, into a calc and two func_TP02* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for error checking covering R1rho off resonance models in target function. This is for checking presence of chemical shifts and r1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Split target function of model TAP03, into a calc and two func_TAP03* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Split target function of model MP05, into a calc and two func_MP05* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for system test Relax_disp.test_r1rho_kjaergaard_auto_check_graphs(), where the special R1rho graphs is no longer produced for the R2eff models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Split target function of model ns_r1rho_2site, into a calc and two func_ns_r1rho_2site* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed num_points to be used in target and lib function of model ns_r1rho_2site. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added a variable describing the model year, for all relaxation dispersion models. This could be used, when trying to write up an intelligent detect+select nesting function. This function need some meta-data describing the models, in order to sort the self.models before calculations, and to select a proper nested model pipe. Other meta data could be: Accept of Exp_type, full or normal model for CPMG, fitted or loaded R1 for R1rho, is analytic, silico or numeric type. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added initial unit test class for testing specific_analysis.relax_disp.variables. There will be more tests added, when a nesting selection function has been written here. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added dictionary for returning year, when using model as key. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added unit test for the dictionary of model years. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Reused the EXP_TYPE_LIST_CPMG and EXP_TYPE_LIST_R1RHO, to combine for the list EXP_TYPE_LIST, the list of all dispersion experiment types. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added a common EXP_TYPE_CPMG MMQ description for models which handle MMQ. This is part of adding meta data for each model, making it possible to device a sensible nesting selection function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added meta data about EXP_TYPE per model, and made a dictionary for it. Added unit test for the new dictionary. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added three new EXP_TYPE variables: EXP_TYPE_R2EFF = 'R2eff/R1rho', EXP_TYPE_NOREX = 'No Rex', EXP_TYPE_NOREX_R1RHO = 'No Rex: R1rho off res'. These are used to add meta-data information to each model, making it possible to make a nesting function, determining which model to nest from. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added model meta information about number of chemical exchange sites. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added meta information about equation type. The models are divided into: analytic, silico or numeric. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented nesting function, which will determine which model to nest from. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
In auto analysis of relax_disp, started implementing the new nesting function. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Modified nesting function to return all model info for the current model, and the comparable model. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for nesting kex, when model is CR72, and analysed models is LM63. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for nesting kex, when model is CR72, and analysed models is IT99. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added a return from the nesting model, if all fails. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Replaced the test, if acquiring the model info to a numerical model from a analytical model. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Improved the printing when nesting parameters from equivalent models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed unused import of models in auto-analyses of relax_disp. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Improved the printing of system test Relax_disp.test_r1rho_kjaergaard_missing_r1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Improved the printing of system test Relax_disp.test_r1rho_kjaergaard_missing_r1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added function to sort models before auto-analyses. They are sorted in order: exp_type: EXP_TYPE_R2EFF, EXP_TYPE_NOREX, EXP_TYPE_NOREX_R1RHO, EXP_TYPE_CPMG_SQ, EXP_TYPE_CPMG_MMQ, EXP_TYPE_R1RHO; equation: EQ_SILICO, EQ_ANALYTIC, EQ_NUMERIC; Nr of chemical sites: 2 or 3; Year: Newest models first; Nr of parameters. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added unit test, to test the expected sorting of models for auto-analyses. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added more models to be tested in system test Relax_disp.test_r1rho_kjaergaard_auto. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented the sorting of models, for auto-analyses. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Implemented partial reading of results file. Before reading a results file, it is determined if the file exists. This makes is possible to read a directory with partial results from a previous analysis. This can be handsome, if reading R2eff values in R1rho experiments, and the error estimation has been prepared with a high number of Monte Carlo simulations. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added keyword to relax_disp auto analysis, if R2eff values should be optimised. Here optimisation means minimisation and Monte Carlo simulations of the error. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Modified system test Relax_disp.test_r1rho_kjaergaard_missing_r1 to load previous R2eff values, and not optimise them. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix in back end for relax_disp.parameter_copy, where r2a and r2b should be skipped, since it has already been copied. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Locked values in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. This is possible after locking the R2eff values and errors from a previous run. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed MODEL_NS_CPMG_2SITE_EXPANDED to be analysed in system test test_hansen_cpmg_data_missing_auto_analysis. The new ordering of models, will make MODEL_NS_CPMG_2SITE_EXPANDED be analysed first, and results copied to model CR72. This will interfere with the old results. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Lowered the precision of dw for model NS CPMG 2-site expanded, in system test est_hansen_cpmg_data_auto_analysis_numeric. Model NS CPMG 2-site expanded is now analysed before MODEL_CR72, which alter the values a bit. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added GUI text for parameter r1_fit. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added front-end description of the 6 new R1 fit R1rho models for relax_disp.select_model. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added a paragraph for the no chemical exchange model in help text description for selecting models. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added the new R1rho models where R1 is fitted, to the GUI model selection. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Deleted system test test_r1rho_kjaergaard_man, since it was no necessary. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Deleted unused script files in data folder for Kjaergaard_et_al_2013. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for the linear constraints where parameter r1_fit was written as R1_fit. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Fix for the numbers of parameters not getting counted correct. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Triggering an error in test_r1rho_kjaergaard_missing_r1. There is a bug fetching the standard value of parameter 'r1_fit'. AttributeError: 'float' object has no attribute 'keys'. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed unused scripts in folder of Kjaergaard et al., 2013. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Added new analysis scripts in folder of Kjaergaard et al., 2013. Sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
Removed triggering an error in test_r1rho_kjaergaard_missing_r1. There is a bug fetching the standard value of parameter 'r1_fit'.
Fix for system test Relax_disp.test_r1rho_kjaergaard_missing_r1, running on 64 bit system.
Inserted LaTeX bibliography for reference to linear constraints of the exchange rate.
Equation fix for $kex$ in manual.
Added system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis and data. This is for bug #22477 (https://gna.org/bugs/?22477): Not possible to perform grace.write() on k_AB parameter for mixed CPMG analysis.
Added more printout to system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis and data.
Fix for system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis. Bug #22477 (https://gna.org/bugs/?22477): Not possible to perform grace.write() on k_AB parameter for mixed CPMG analysis.
Set the default value of r1_fit to 5.0.
Added the relax_disp pipe type to be setup for unit tests of value function.
Setup a unit test for the value.set functionality for param r1_fit. Bug #22470 (https://gna.org/bugs/?22470): value.set does not work for parameter r1_fit.
Fix in relax_disp API, how to handle the r1_fit parameter type. Bug #22470 (https://gna.org/bugs/?22470): value.set does not work for parameter r1_fit.
Modified system test Relax_disp.test_r1rho_kjaergaard_missing_r1 to use GRID_INC=None, and thereby speeding up the analysis. Bug #22470 (https://gna.org/bugs/?22470): value.set does not work for parameter r1_fit.
Added to system test a count of number of headers and values, when issuing a value.write(). Sr #3121 (https://gna.org/support/?3121): Support request for replacing space in header files for the value.write functions.
Fix for replacing spaces " " with "_" in header files. Sr #3121 (https://gna.org/support/?3121): Support request for replacing space in header files for the value.write functions.
Fix for comment, which mentions R2 parameter, when it relates to R1 fit.
Replaced variable name: MODEL_PARAM_INV_RELAX_TIMES with MODEL_LIST_INV_RELAX_TIMES, to match all of the other MODEL_LIST_* variables. Also added a newline to end of file.
Replaced remaining variable names: MODEL_PARAM_* with MODEL_LIST_*, to match all of the other MODEL_LIST_* variables.
Renamed R1rho off resonance models where R1 is fitted. This removes the "underscore". This is a better representation presented to the user, for example in the GUI model selection list or the relax_disp.select_model user function in all UI.
Renamed the parameter "r1_fit" to "r1". This naming fits better to all other parameters.
Split the unit test of specific_analyses.relax_disp.checks.get_times() into its own unit test file.
Added a "check" function, what will determine if R1 data is missing for a model to analyse. Also added corresponding unit tests, to test the functionality.
Modified in documentation, that the "No Rex" model have one chemical exchange site, namely itself.
Copied variables.py to model.py. There should not exist any functions in variables.py. It should only consist of hardcoded variables, and the functions related to model sorting and nesting is split into its own file.
Parted the file of variables.py into model.py.
Removed the unit test regarding model.py in test_variables.py.
Added unit tests regarding model.py and its functions.
Removed the auto-sorting of models, when performing auto analysis of Relaxation dispersion. This was discussed in: http://thread.gmane.org/gmane.science.nmr.relax.scm/22733, http://thread.gmane.org/gmane.science.nmr.relax.scm/22734, http://thread.gmane.org/gmane.science.nmr.relax.scm/22737. Through this discussion, it has appeared that the order of how models are sorted for analysis, and hence the possibility for nesting, is a complicated case. The order of analysis should be possible to manually put into the auto analysis. This was not the scope of sr #3135 (https://gna.org/support/?3135): Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models to implement such a feature. Such a feature could be implemented for the next version of relax. It could be designed as function to "suggest" an order in the GUI. But this functionality would have to wait.
Removing the 'tex'->'kex' conversion, and 'phi_ex' from dw and pA. This solution is not a proper implementation, but these parameters should rather be found by grid search.
Re-inserted "MODEL_NS_CPMG_2SITE_EXPANDED" to be tested in system test test_hansen_cpmg_data_missing_auto_analysis.
Removed special cases for nesting. Now following order is determined. First sort completed models into: EQ_NUMERIC, EQ_SILICO, EQ_ANALYTIC. Then into year, with newest first. Then number of chemical sites, which reflects number of parameters. Go through the completed models. If the experiment types are the same, then look for: If a completed model has same parameters, then nest from this. If a completed model has all other parameters than R20 parameters, then nest from this. Special cases are taken care of by: MODEL_LM63_3SITE from MODEL_LM63, MODEL_NS_MMQ_3SITE, MODEL_NS_MMQ_3SITE_LINEAR from MODEL_NS_MMQ_2SITE, MODEL_NS_R1RHO_3SITE, MODEL_NS_R1RHO_3SITE_LINEAR from MODEL_NS_R1RHO_2SITE, MODEL_MMQ_CR72 from MODEL_CR72. This functionality represents the hard-coding from previous implementation.
Moved the nesting lists down in variables file.
Small verb tense fix for the descriptions of the '* R1 fit' relaxation dispersion models.
Added definition and dictionary, for each model, to determine which model they nest from. This is better to hardcode, since it makes it possible to produce a table with an overview, and accurately determine which model is nested from. This is discussed in thread http://thread.gmane.org/gmane.science.nmr.relax.devel/6684.
Moved the lookup in dictionaries for model information, into the class of model info.
Division of unit tests of specific_analyses/relax_disp/model.py into different functions. Also added more print information to each tests.
Added the ordered list of nest models to the class of model information.
Modified the defined list of nesting, according to thread: http://thread.gmane.org/gmane.science.nmr.relax.devel/6684. More specific, thread: http://thread.gmane.org/gmane.science.nmr.relax.devel/6694.
Modified unit test regarding finding correct nested model. This was discussed in: http://thread.gmane.org/gmane.science.nmr.relax.devel/6684. More specific, thread: http://thread.gmane.org/gmane.science.nmr.relax.devel/6694.
Modified nesting function, to pull list of possible models from dictionary, and check if these models are available in the completed models.
Added initial Python script, to help print each model and its corresponding nested models. It can be executed by: relax test_suite/shared_data/dispersion/print_model_info/print_model_info.py.
Added a relaxation dispersion example to show how certain literature statements are just utter crap. This follows from http://thread.gmane.org/gmane.science.nmr.relax.scm/22774/focus=6693, and the change http://thread.gmane.org/gmane.science.nmr.relax.scm/22774 which implement such dangerous literature conjectures. To see how a real minimum is excluded from the optimisation space, here for residue :2, execute the R1rho_analysis.py script in relax. This is synthetic data generated with kex = 1e5 assuming the model TP02. For the case of residue :2, this still produces an optimisable minimum in the space and dispersion curves. However the change blocks optimisation to the minimum.
Added a function to determine how to nest/copy the parameters, when nesting from another model. It takes the list of parameters from the current model, and the list of parameters available in the nested model, and return a dictionary of parameter conversion for the current model params.
Added unit test for the new function which determine how parameters are copied for a nested model parameters.
Added a table for dispersion model nesting in the auto-analysis to the manual. This adds the ideas discussed in the thread http://thread.gmane.org/gmane.science.nmr.relax.devel/6684.
Added the string 'me' for Methods in Enzymology to the bibtex file for the manual.
Set the average value of R1 to 2.0 instead of 5.0. This is "normally" a better guess for R1.
Implemented the function, which translates how parameters are copied from a nested model in the auto_analyses for relax_disp. This makes it possible to test the translating code, and makes logic clearer in the auto_analyses.
Modified the r1rho_off_res_tp02_high_kex dispersion test data. The kex value is now set to 2e5.
Removed the 'DPL94 R1 fit' model nesting from the table in the dispersion chapter of the manual. This was identified using the dispersion test suite data script print_model_info/print_model_info.py (http://thread.gmane.org/gmane.science.nmr.relax.scm/22823).
Added to system test Relax_disp.test_r1rho_kjaergaard_missing_r1 to check, that values are not None when writing .out files. This is related to: https://gna.org/support/?3121 sr #3121: Support request for replacing space in header files for the value.write functions. The fix for this bug, destroyed getting the values.
Fix for earlier bug fix destroyed functionality. Altering the data-keys to early, meant that data was not fetched correctly. This is related to: https://gna.org/support/?3121 sr #3121: Support request for replacing space in header files for the value.write functions.
Moved the unit test of specific_analyses.relax_disp.checks.check_missing_r1() from a unit test to a system test. This is because the unit test involved several functions of relax.
Inserted dictionary, that will convert a R1rho off-resonance without R1, to the corresponding model which fit R1.
Inserted to the check of missing R1, that MODEL_NOREX_R1RHO also depends on R1.
Implemented function that determine if any model in the list of all models should be replaced or inserted as the correct 'No Rex' model. It also translate the R1rho off-resonance model to the corresponding 'R1 fit' models, if R1 is not loaded.
Inserted system test Relax_disp.test_convert_no_rex_fit_r1, which test the return for the function that will determine if models self.models in the relax_disp should be translated/corrected.
Fix for unit test, where the standard value of R1 was lowered from 5.0 to 2.0. Also fixed an import error in another unit test.
Minimised the dependencies of the version module. This no longer relies on the dep_check module.
Inserted return of True/False flags from function which convert models. The flag tells if: flag if 'No Rex' model for R1rho off-resonance was translated; flag if 'No Rex' model for R1rho off-resonance was inserted; flag if R1rho off-resonance was translated to 'R1 fit' models if no R1 data was found.
Changes to system test, after the number of returns from function has been altered.
Inserted into relax_disp auto_analyses, to convert the input models. This will convert/insert the correct 'No Rex' to the corresponding 'No Rex' model for R1rho off-resonance models. It will also translate to the corresponding 'R1 fit' model, if no 'R1 data' has been loaded with relax_data.read() function.
Lowering of precision in system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). This is due to 64/32-bit issues, between analysing on Linux computer, and testing on Mac computer.
Made the GUI selection of models for relaxation dispersion more simple. After the implementation of a function which will translate the models, the 'No Rex' model will be converted to the 'No Rex' model for R1rho off-resonance. Also the corresponding 'R1 fit' model will be chosen instead, if R1 data has not been loaded. This makes the model selection easier in the GUI interface.
Bugfix for Relax_disp.test_bug_21715_clustered_indexerror, where only 'R2eff', 'No Rex' is analysed. This special case was not tested in the translating function.
Shortening the text in the auto_analysis, and raises a warning if R1 data has not been loaded.
Rewrote the logic of the key-word 'optimise_r2eff' in the auto-analyses of relax disp. If R2eff result file exist in the 'pre_run_dir', this is loaded. If the results contain both values, and errors, then no optimisation is performed on the R2eff model. Unless the 'optimise_r2eff' flag is raised, which is not standard.
Fixes for the Relax_disp.test_hansen_cpmg_data_missing_auto_analysis system test on MS Windows. This is for 32-bit MS Windows systems where the parameter checks need to be loosened.
Fixes for the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test on MS Windows. This is for 32-bit MS Windows systems where the parameter checks need to be loosened.
Created the specific_analyses.relax_disp.data.is_r1_optimsed() function. This follows from an idea for handling R1 optimisation in the tread at http://thread.gmane.org/gmane.science.nmr.relax.scm/22850/focus=6736. This relaxation dispersion function can be used anywhere that requires the knowledge that R1 values should be fitted during optimisation or if loaded values should be used.
Implemented the very basic relax_disp.r1_fit user function. This is as discussed at http://thread.gmane.org/gmane.science.nmr.relax.scm/22850/focus=6737.
Implemented the specific_analyses.relax_disp.parameters.r1_setup() function. This matches the description at http://thread.gmane.org/gmane.science.nmr.relax.scm/22850/focus=6739 and http://thread.gmane.org/gmane.science.nmr.relax.scm/22850/focus=6736.
Spelling fix for the is_r1_optimised() function name.
Implemented to the back_end of the value.copy, a "force" flag to overwrite destination value.
Implemented to the front_end of the value.copy, a "force" flag to overwrite destination value.
Copying the R2eff value from the reading of R2eff results in the auto-analyses of relax_disp.
Fix for the misspelled is_r1_optimised() function.
The r1_fit flag is stored in the Disp_minimise_command class and passed into the target function. This matches the details at http://thread.gmane.org/gmane.science.nmr.relax.scm/22850/focus=6736. The specific_analyses.relax_disp.optimisation.Disp_minimise_command class calls is_r1_optimised() and stores the result. This is then passed into the relaxation dispersion target function class.
Modified that 'NS CPMG 2-site expanded' will be preferred before 'NS CPMG 2-site 3D' and 'NS CPMG 2-site star'.
Modified the nesting for 'NS MMQ 3-site linear'. 'NS MMQ 3-site linear' should be able to nest from 'NS MMQ 3-site'.
Fix to unit test, after MODEL_NS_CPMG_2SITE_EXPANDED has been preferred over other numerical CPMG models.
Modified to unit test, that when model: MODEL_PARAMS_NS_R1RHO_3SITE nest from: MODEL_PARAMS_NS_R1RHO_2SITE the conversion should be: 'r2', 'r2' 'pA', 'pA' 'dw_AB', 'dw' 'kex_AB', 'kex' 'pB', '1 - pA' 'dw_BC', 'dw' 'kex_BC', 'kex' 'kex_AC', 'kex'. Here '1 - pA' is a special conversion flag.
Implemented the special flag '1 - pA', when nesting parameters from models with fewer chemical sites.
Implemented the nesting of parameters from a model with fewer chemical sites when nesting for NS R1rho 3-site.
Inserted system test Relax_disp.test_model_nesting_and_param() which will go through all models, and then through all it nested models, testing that all parameters have a conversion.
Modified system test Relax_disp.test_model_nesting_and_param() to only print, when the converted parameter is different from the original parameter.
Fix for parameter conversion when model is: MODEL_PARAMS_NS_R1RHO_3SITE or MODEL_PARAMS_NS_R1RHO_3SITE_LINEAR.
Fix for parameter conversion for: MODEL_PARAMS_NS_MMQ_3SITE and MODEL_PARAMS_NS_MMQ_3SITE_LINEAR
Fixes to unit tests, after parameter conversion have been corrected.
Replaced that folder names for writing out results should be with replaced with underscores "_". This is for the dispersion auto-analysis.
Fixes for the relaxation dispersion loop_parameters() function. The R1, R20, R20A, R20B (and R1rho_prime, R1rho_primeA, R1rho_primeB) parameters are now checked for in each spin container rather than just the first of the cluster. This should make no difference as all spins should have the same model and parameters, but it might be a source of bugs in the future.
The r1_fit flag is now used to switch between dispersion target functions. This is as described in http://thread.gmane.org/gmane.science.nmr.relax.scm/22850/focus=6736. The change makes the '* R1 fit' models now redundant.
Removed all of the '* R1 fit' models out of the relax_disp.select_model user function frontend. These models are now redundant as the question of R1 fitting is now determined internally in relax.
Removed all of the MODEL_*_FIT_R1 dependencies from the specific_analyses.relax_disp package. These models are now redundant as the question of R1 fitting is now determined internally in relax.
Fix for the specific_analyses.relax_disp.data.is_r1_optimised() function for on-resonance R1rho data. This function needs to specifically catch these models.
Fix for the MODEL_LIST_R1RHO variable. Recent changes causes this to not include the on-resonance R1rho dispersion models.
Import fix for the Relax_disp.test_model_nesting_and_param system test. Somehow the import of the convert_no_rex() function was lost.
Modified the MODEL_LIST_R1RHO_OFF_RES list to include MODEL_NOREX_R1RHO.
The specific_analyses.relax_disp.parameters.r1_setup() function is now being called. This happens before the R1 data is returned in the Disp_minimise_command class.
The dispersion auto-analysis now handles the optional R1 parameter correctly. The value.set user function was no longer setting the R1 parameter to the default value when the grid search was deactivated, as it is no longer in MODEL_PARAMS. So instead the new is_r1_optimised() function is being used to decide if the value.set user function should set the 'r1' parameter value.
The dispersion loop_parameter() function now calls r1_setup() to handle R1 parameters correctly. This allows the R1 parameter to be removed or added to the parameter list prior to looping over the parameters of the model. The change is required to allow for the dynamic handling of R1 parameters.
The dispersion back_calc_r2eff() function can now handle the dynamic R1 parameter. This required a call to r1_setup() to add or remove the parameter, and is_r1_optimised() to obtain the r1_fit flag to be sent into the target function class.
Updated the specific_analyses.relax_disp.model.Model_class class to handle the dynamic R1 parameter. The class variable self.params now has the 'r1' parameter prepended to the list if is_r1_optimised() returns True.
More changes for specific_analyses.relax_disp.model.Model_class for the dynamic R1 parameter. The 'r1' parameter is only prepended to self.params if it is not already in the list.
Created the MODEL_LIST_FIT_R1 variable to keep track of dispersion models with R1 fitting support.
The is_r1_optimised() function now checks MODEL_LIST_FIT_R1. If the model is not in MODEL_LIST_FIT_R1, i.e. R1 optimisation is not supported, then the function will return False.
Fix for the test_nesting_param_5 unit test. The 'r1' parameter is now dynamic and hence will not be present in the initial list.
One final fix for the Model_class.params list with 'r1'. The is_r1_optimised() function is now called with the model name argument, as required.
Updated the relax_disp.r1_fit user function docstring. This now includes information about which models support R1 parameter optimisation.
Removed results files to allow the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test to pass. These are the test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/check_graphs/mc_2000/ results files for the 'No Rex' and 'DPL94' models, as well as the final run. This commit is to allow the test to temporarily pass. It can be reverted once a better solution is discussed and decided upon.
Altered the number of Monte Carlo simulations in test script to 2000.
Merger of the 'No Rex' and 'No Rex R1rho off res' models in the specific_analyses.relax_disp package. In the 'variables' module, all *_NOREX_R1RHO variables have simply been deleted and the MODEL_LIST_* structures updated. For the 'data' module, the is_r1_optimised() function was modified to catch the 'No Rex' model and to then use the cdp.exp_type_list structure to determine if the experiment type is EXP_TYPE_R1RHO. This will be modified in the future by using a function for determining if the current experiment is on or off-resonance. The return_r1_data() and return_r1_err_data() functions have also been modified to check if R1 values are fit rather than if the model is in MODEL_LIST_R1RHO_OFF_RES. In the 'model' module, in addition to deleting all *_NOREX_R1RHO variables, the convert_no_rex() function has also been deleted as it no longer serves a purpose. In the 'checks' module, all 'No Rex R1rho off res' model references have been replaced with 'No Rex'.
Updated the dispersion auto-analysis for the universal 'No Rex' model. The 'No Rex R1rho off res' references have all been deleted. The model conversion logic is also no longer needed and has been deleted.
Converted the relaxation dispersion GUI interface to the unified 'No Rex' model. All of the MODEL_NOREX_R1RHO references have simply been deleted.
Converted the relaxation dispersion target function class to the unified 'No Rex' model design. On top of removing all references to MODEL_NOREX_R1RHO, the aliasing of self.func now checks the experiment type list to determine which target function to use. This is not an ideal solution and will not handle mixed CPMG and R1rho experiment, however neither will the target functions yet. The creation of the off-resonance data structures has also been modified so that they are now R1rho independent. This allows the structures to be properly created while at the same time enabling this code to be compatible with off-resonance CPMG data in the future.
Removed the 'No Rex R1rho off res' model from the relax_disp.select_model user function frontend.
Removed all references to the 'No Rex R1rho off res' model in the system tests. In addition, the Relax_disp.test_convert_no_rex system test has been deleted as it no longer has a purpose. For the Relax_disp.test_model_nesting_and_param system test, to allow this to work the cdp.exp_type_list list is set to EXP_TYPE_LIST.
Removed all references to the 'No Rex R1rho off res' model in the unit tests.
Updated the 'No Rex' dispersion model description in the relax manual. The universal nature of the model is now described, including the addition of the off-resonance CPMG and R1rho equations for the absence of chemical exchange. The R1 parameter optimisation is also shortly covered.
Added a subsection to the dispersion chapter of the manual about R1 parameter optimisation.
Added the R1 parameter fitting GUI element to the dispersion GUI tab. This is a simple Boolean toggle element that allows the R1 optimisation to be turned on. The value is passed into the auto-analysis.
Added the r1_fit argument to the relaxation dispersion auto-analysis. When this is True, the relax_disp.r1_fit user function will be called to turn R1 parameter fitting on.
Added the relax_disp.spin_lock_offset user function to the dispersion GUI. This has been added to the pop up menu in the spectrum list GUI element, when the relax_disp_flag has been set. It simply mimics the relax_disp.spin_lock_field functionality already present. This follows from task #7820 (https://gna.org/task/?7820).
Fix for the relax_disp.spin_lock_offset user function in the dispersion GUI tab. This is in the spectrum list element popup menu.
Added the offset column to the spectrum list GUI element for the dispersion analysis. This is to complete task #7820 (https://gna.org/task/?7820). The spectrum list GUI element add_offset() method has been added to insert the offset column when the relax_disp_flag is set. This is called by the update_data() method to fill and update the GUI element.
Implemented the GUI test Relax_disp.test_bug_22501_close_all_analyse to catch bug #22501 (https://gna.org/bugs/?22501), 'Close all analyses' raises error.
Inserted intermediate system test, to profile R2eff calculation for R1rho. System test is: Relax_disp.test_bug_9999_slow_r1rho_r2eff_error_with_mc. This system test actually fails, if one tries to do a grid search. This is related to the R2eff values stored as dictionary, and pipe_control.minimise.grid_setup() will fail. The function 'isNaN(values[i])' cannot handle dictionary.
Modified intermediate system test Relax_disp.test_bug_9999_slow_r1rho_r2eff_error_with_mc to see if the initial grid search for 'i0' and 'R2eff' estimation can be skipped. This is done by converting the exponential curve, to a linear curve, and calculate the best parameters by a line of best fit by least squares. This seems like a promising method as an initial estimator of 'i0' and 'r2eff'. For 500 to 2000 Monte Carlo simulations, this could have an influence on the timings, as all grid searchs could be skipped.
Modified system test test_bug_9999_slow_r1rho_r2eff_error_with_mc to save data arrays. This is to prepare a profiling script.
Added start script with basic data for profiling the relax curve fit.
Created the Structure.test_create_diff_tensor_pdb system test. This is to show the failure of the structure.create_diff_tensor_pdb user function when no structural data is present.
Created the Structure.test_create_diff_tensor_pdb2 system test. This is to catch another situation leading to bug #22505 (https://gna.org/bugs/?22505), the failure of the structure.create_diff_tensor_pdb user function when no structural data is present.
Added an optimisation script for the test_suite/shared_data/diffusion_tensor/ellipsoid relaxation data. This is to help catch bug #22502 (https://gna.org/bugs/?22502), the geometric prolate diffusion representation does not align with axis in PDB, as reported by Martin Ballaschk (https://gna.org/users/mab). The PDB files of the optimised tensor demonstrate exactly the same problem as seen in the files attached to the bug report. The oblate and spherical diffusion tensor representations match that of the ellipsoid. But the prolate axis and tensor orientation are both different from the ellipsoid as well as themselves.
Updated the diffusion tensor PDB representation files. This replaces the broken prolate representation with the corrected representation.
Deleted the duplicated Structure.test_create_diff_tensor_pdb system test.
Created a number of system tests to check the diffusion tensor PDB representation. This is to prevent bugs such as #22502 (https://gna.org/bugs/?22502) from ever reappearing. The PDB file contents are hardcoded into the tests and checked. The tests include Structure.test_create_diff_tensor_pdb_ellipsoid, Structure.test_create_diff_tensor_pdb_oblate, Structure.test_create_diff_tensor_pdb_prolate, and Structure.test_create_diff_tensor_pdb_sphere.
Improved data checking for all of the Structure system tests. Before looping over the structural data, the number of lines in the real file and the newly generated file are compared. This avoids the situation whereby an empty file is produced, accidentally allowing the test to pass.
Modified following functions (time points are now saved at the [ei][mi][oi][di] index level, at this index level all time points are saved for the R2eff point): interpolate_disp(), to interpolate time points, all time points through the original dispersion points di, are collected and then made unique - this time list can potentially be the largest of all time lists; interpolate_offset(), to interpolate time points, all time points through the original offset points, and then dispersion points di, are collected and then made unique - this time list can potentially be the largest of all time lists; plot_disp_curves_to_file(), to acquire the original relax_times points; return_r2eff_arrays(), to save all time points on the level of [ei][mi][oi][di]. At this index level, it will be a numpy array list with all time values used for fitting. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Modified back_calc_r2eff() to accept interpolated time points. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Modified target function of relax dispersion, to use the new list of time points, which are of higher dimension. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Fix to system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). After the relaxation times have been fixed, this model now return reasonable chi2 values. The reported parameters are though quite different from all other models, and it seems something may still be wrong. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Fix for system test Relax_disp.test_exp_fit(), where the spin.isotope was not set. The new call to return_r2eff_arrays(), when producing graphs, raise RelaxSpinTypeError() if no isotope is set. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Modified the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test to pass on 64-bit Linux systems. The accuracy of the checks of the optimised values have been decreased.
Moved the storing of relax time up before check of missing data in return_r2eff_arrays(). Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Fix for system test not adding spin.isotope to setup information. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Fix for looping over data indices, where tilt_angles has the si index. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Added Nikolai's original Matlab code to the lib.dispersion.ns_r1rho_2site module docstring. This is the code taken directly form the original funNumrho.m file, which was the origin of the code in this module.
Further extended the profiling script for curve fitting. Now profiling is in place for the implemented C code method in relax. A similar code should now be devised for numpy array for comparing. But this profiling shows that when contraints=True, is slowing down this procedure by a factor 10 X.
Further improved the profiling of relax curve fit. This profiling shows, that Python code is about twice as slow as the C code implemented. But it also shows that optimising with scipy.optimize.leastsq is 20 X faster. It also gives reasonable error values. Combining a function for a linear fit to guess the initial values, together with scipy optimise, will be an extreme time win for estimating R2eff values fast. A further test would be to use relax Monte Carlo simulations for say 1000-2000 iterations, and compare to the errors extracted from estimated covariance.
Added verification script, that shows that using scipy.optimize.leastsq reaches the exact same parameters as minfx for exponential curve fitting. The profiling shows that scipy.optimize.leastsq is 10X as fast as using minfx (with no linear constraints). scipy.optimize.leastsq is a wrapper around wrapper around MINPACK's lmdif and lmder algorithms. MINPACK is a FORTRAN90 library which solves systems of nonlinear equations, or carries out the least squares minimization of the residual of a set of linear or nonlinear equations. The verification script also shows, that a very heavy and time consuming Monte Carlo simulation of 2000 steps, reaches the same errors as the errors reported by scipy.optimize.leastsq. The return from scipy.optimize.leastsq, gives the estimated covariance. Taking the square root of the covariance corresponds with 2X error reported by minfx. This could be an extremely time saving step, when performing model fitting in R1rho, where the errors of the R2eff values, are estimated by Monte Carlo simulations. The following setup illustrates the problem. This was analysed on a MacBook Pro, 13-inch, Late 2011 with no multi-core setup. Script running is: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. This script analyses just the R2eff values for 15 residues. It estimates the errors of R2eff based on 2000 Monte Carlo simulations. For each residues, there is 14 exponential graphs. The script was broken after 35 simulations. This was measured to 20 minutes. So 500 simulations would take about 4.8 Hours. The R2eff values and errors can by scipy.optimize.leastsq can instead be calculated in: 15 residues * 0.02 seconds = 0.3 seconds.
Moved the target function for minimisation of exponential fit into the target functions folder. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented initial system test Relax_disp.test_estimate_r2eff for setting up the new user function to estimate R2eff and errors by scipy. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added front end user function relax_disp.r2eff_estimate to estimate R2eff and errors by exponential curve fitting in scipy.optimize.leastsq. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified check for model, to accept model as input, for error printing. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented back end for estimating R2eff and errors by exponential curve fitting with scipy.optimize.leastsq. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Documentation fix for new exponential target function. Also added new function to estimate R2eff and i0 parameters, before minimisation. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Small changes to verification scripts, to use chi2 function and use the scaling matrix correct. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Split up of system test test_r1rho_kjaergaard_missing_r1, into a verification part. This is to test the new r2eff estimation, which should get the parameter values, as a this 2000 Monto Carlo simulation. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified system test Relax_disp.test_estimate_r2eff. This is to compare against errors simulated with 2000 MC. The parameters are comparable, but not equal. Mostly, it seems that the errors from scipy.optimize.leastsq, are twice as high than the Monte Carlo simulations. This affect model fitting, and the calculated chi2 value.
Added system test Relax_disp.test_estimate_r2eff_error(). This is to get insight in the error difference between 2000 Monto Carlo simulations and then scipy.optimize.leastsq.
Add dependency check for scipy.optimize.leastsq. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Lowering precision in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. This is R1 estimation with MODEL_NS_R1RHO_2SITE. The lowering of precision is due different system precision.
Reused the dependency check "scipy_module", since leastsq() has been part of Scipy since 2003.
Moved target function for curve fitting with scipy into specific_analyses.relax_disp.estimate_r2eff. This will later include the backend specific_analyses.relax_disp.optimisation.estimate_r2eff() function and the code in the target_functions package. The code in target_functions.relax_disp_curve_fit is a lot more than just a target function, so it doesn't really belong in this package. This is also to isolate this experimental feature.
Isolated all code related to user function relax_disp.r2eff_estimate into independent module file. All has been isolated to: specific_analyses.relax_disp.estimate_r2eff.
Split function to minimise with scipy.optimize.leastsq out in estimate_r2eff module. This is to prepare for implementing with minfx.
Implemented first try to minimise with minfx in estimate_r2eff() function. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implementation of the target_functions.relax_fit.jacobian() function. This follows from the discussions at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807. The function will calculate the Jacobian matrix for the exponential curve-fitting module. The Jacobian can be used to directly calculate the covariance matrix, for example as described at https://www.gnu.org/software/gsl/manual/html_node/Computing-the-covariance-matrix-of-best-fit-parameters.html. The Jacobian is calculated using the help of the new exponential_dI() and exponential_dR() functions in the target_functions/exponential.c file. These calculate the partial derivatives of the exponential curve with respect to each model parameter separately. The implementation still needs testing and debugging.
Fixes for the new target_functions.relax_fit.jacobian() function. The Python list of lists is now correctly created and returned.
Turned off the optimisation constraints for the 'R2eff' model in the dispersion auto-analysis. This follows from http://thread.gmane.org/gmane.science.nmr.relax.scm/22977/focus=6829. This model does not require constraints at all, and the constraints only cause the optimisation to take 10x longer to complete. Therefore the constraint flag has been set to False for the model.
Initial try to form the Jacobian and Hessian matrix for exponential decay. This can be tried with system test: relax -s Relax_disp.test_estimate_r2eff_error. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Intermediate step in estimate R2eff module. It seems that minfx is minimising in a quadratic space because of the power of chi2, while the general input to scipy.optimize does not do this. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Cleaned up target function for leastsq, since arguments to function can be extracted from class. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Tried to implement with scipy.optimize.fmin_ncg and scipy.optimize.fmin_cg, but cannot get it to work. The matrices are not aligned well. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented the chi-squared gradient as a C module for the target functions. This simply translates the Python code into C to allow any target function C modules to build its own gradient function.
Implemented the target_functions.relax_fit.dfunc() gradient function. This is using the Python/C interface to provide a Python function for calculating and returned the chi-squared gradient for the exponential curves.
Implementation of the specific_analyses.relax_fit.optimisation.dfunc_wrapper() function. This interfaces with the target_functions.relax_fit C module and converts the gradient from a Python list to a numpy array.
The exponential curve-fitting gradient is now scaled by the scaling matrix.
Clean up of the end of the target_functions.relax_fit.dfunc() function.
Fixes for the target function chi-squared gradient C function.
Fixes for all of the exponential functions in target_functions/exponential.c. The condition whereby Rx is zero is now setting the value correctly - the exponential will be 1, not zero, hence the intensity and gradient values should not be zero.
Clean up of the target function C files (spacing fixes and removal of unused code).
Changed the argument and variable names in the C code chi-squared gradient function.
Modified all of the exponential curve functions and gradients in the C target function module. Instead of passing in the parameter vector, now the I0 and R values are passed in separately. This is for greater code flexibility as the parameter order does not need to be known.
The parameter index is now passed into exponential_dI0() and exponential_dR(). This is for the relaxation curve-fitting C module so that the indices are not hardcoded.
The I0 and R parameter indices are now defined in the target_function/relax_fit.h header file. This is to abstract the exponential curve parameter indices even more.
Big cleanup of estimate R2eff module. This is to make the documentation more easy to read and understand. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Created 2 unit tests for the target_functions.relax_fit relax C module. These check the func() and dfunc() Python methods exposed by the module.
The relax_fit C module unit tests now check if the parameter scaling is functional.
Added several comments to the R2eff estimate module. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added a script and log file for calculating the numerical gradient for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the gradient using the scipy.misc.derivative() function both at the minimum and at a point away from the minimum. The values will be used to construct a unit test to check the C module implementation.
Created a unit test to check the dfunc() function of the relax_fit C module off the minimum.
Fixes for the target_functions.relax_fit C module unit tests. All values are now set to floats to avoid integer division issues.
Activated parameter scaling of the gradient in the test_dfunc_off_minimum() unit test. This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit.
The exponential curve numeric gradient script now uses only floating point numbers. This is to avoid integer truncation problems.
Fix for the script for calculating the numerical gradient for an exponential curve. The off-minimum derivative was not correctly calculated.
Increased the printouts for the script for calculating the numerical gradient for an exponential curve.
Bug fix for the chi-squared gradient calculation in the C module. The definition of the square() function needed extra brackets so that the 1/error**2 calculation would be 1/(error*error) rather than the incorrect 1/error*error form.
Fix for the test_dfunc_off_minimum() unit test. This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit. The wrong gradient was being scaled.
Switched the optimisation algorithm in test_suite/system_tests/scripts/relax_fit.py. This script, used by the Relax_fit.test_curve_fitting_height and Relax_fit.test_curve_fitting_volume system tests, now uses the BFGS optimisation. This is to demonstrate that the exponential curve gradient function dfunc() is implemented correctly and that more advanced optimisation algorithms can be used (excluding those that require the full Hessian d2func() function).
Got the method of 'Steepest descent' to work properly, by specifying the Jacobian correctly. The Jacobian was derived according to the chi2 function. The key point was to evaluate the two derivative arrays for all times points, and then sum each of the two arrays together, before constructing the Jacobian. This clearly shows the difference between minfx and scipy.optimize.leastsq. scipy.optimize.leastsq takes as input a function F(x0), which should return the array of weighted differences between function value and measured values: "1. / self.errors * (self.calc_exp(self.times, *params) - self.values)". This will be an array with number of elements 'i' corresponding to number of elements. scipy.optimize.leastsq then internally evaluates the sum of squares -> sum[ (O - E)**2 ], and minimises this. This is the chi2. Minfx requires the function to minimise before hand. So, the "func" should be chi2. Then the dfunc, and d2func, should be derivative of chi2, but all elements in the array should still be summed together. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Got the Quasi-Newton BFGS to work. This uses only the gradient, this gets the same results as 2000 Monte Carlo with simplex and scipy.optimize.leastsq. Error estimation still not provided. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Removed all code regarding scipy.optimize fmin_cg and fmin_ncg. This problem should soon be able to be solved with minfx. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added initial documentation for multifit_covar. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified profiling script to use the new estimate R2eff module. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified verify error script, to use new estimate R2eff module. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Removed all unnecessary code from estimate R2eff module. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
More removal of code. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Changed the array declarations in the target_functions/exponential C file and header. Instead of using the pointer format of *xyz, the array format of xyz[] is now being used. These are equivalent and the later is more obvious that this is an array.
Changed the array declarations in the target_functions/c_chi2 C file and header. Instead of using the pointer format of *xyz, the array format of xyz[] is now being used. These are equivalent and the later is more obvious that this is an array.
Shifted all of the parameter Python lists to C arrays into the new param_to_c() function. This is for the target_functions.relax_fit C module to avoid much duplicated code.
Removed the comment and docstring saying that the exponential curve-fitting gradient is unimplemented.
Updated the copyright notices in the C files of the target_functions directory.
Implemented the C version of the chi-squared Hessian. This is a direct translation of the Python code.
Changed the internal variables of the chi-squared gradient C code to match the Python code.
Standardisation of the array dimensionality in the target function C code. The new target_functions/dimensions.h header file defines MAX_PARAMS and MAX_DATA which is then included in the header files of all the other C files. All array declarations now explicitly specify the length of each dimension. The values of MAX_PARAMS and MAX_DATA have increased from 3 and 50 to 20 and 5000. This is to allow for models with more parameters and to allow a much larger number of input data points to be supported before memory corruptions happen. The data structures now take up more memory, but as the functions do not loop up to maximum but only the number of parameters and points specified, this will not make the code slower.
All of the C code chi-squared functions now have the array argument dimensions explicitly declared.
The target_functions/exponential.c file no longer includes exponential.h. This is not needed as exponential.h only contains the function definitions of the exponential.c file.
Clean up of the header and includes of the target_functions/c_chi2.c file. The square() function macro has been shifted to the header file and the stdio.h and math.h standard library headers are no longer included as they are not used.
Partly implemented the front end target_functions.relax_fit.d2func() C module Python function. This is not fully implemented as the exponential curve double partial derivatives are not implemented.
Implemented the exponential curve second partial derivative C functions. These are declared in the exponential.h header file and are now used by the Python function target_functions.relax_fit.d2func().
The square() function macro is now defined for the target_functions/exponential.c file. This is defined in the header, and now the exponential.h is included in the C file to access it.
Implemented the specific_analyses.relax_fit.optimisation.d2func_wrapper() function. This converts the numpy parameter array into a Python list, calls the target_functions.relax_fit.d2func() function with this list, converts the Hessian output list of lists into a numpy rank-2 array, and returns it. This will allow Newton optimisation to be used for the relaxation curve-fitting analysis.
Switched the optimisation algorithm in test_suite/system_tests/scripts/relax_fit.py. This script, used by the Relax_fit.test_curve_fitting_height and Relax_fit.test_curve_fitting_volume system tests, now uses Newton optimisation. This is to demonstrate that the exponential curve gradient function dfunc() and Hessian function d2func() are implemented correctly.
Added a script and log file for calculating the numerical gradient for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the Hessian using the numdifftools.Hessian object construct and obtain the matrix, both at the minimum and at a point away from the minimum. The values will be used to construct a unit test to check the C module implementation.
Fixes for the Hessian.py script for numerical integrating the Hessian for an exponential curve.
Implemented two unit tests to check the Hessian of the target_functions.relax_fit.d2func() function. This compares the calculated Hessian to the numerically integrated values from the test_suite/shared_data/curve_fitting/numeric_gradient/Hessian.py script, showing that the d2func() function is implemented correctly.
Modified profiling script, but it seems that the dfunc from target_functions.relax_fit does not work. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified estimate R2eff module, to use C code. But system test Relax_disp.test_estimate_r2eff_error shows that the Jacobian is not correctly implemented to be called in minfx.
Created an initial test suite data directory for a mixed R1rho + CPMG dispersion analysis. The generate.py script will be extended in the future to generate both synthetic R1rho and CPMG data for a common exchange process. Such a data combination should show some minor flaws in the current design of the dispersion analysis and will help to solve these.
Improvements to the pipe_control.minimise.reset_min_stats() function. The minimise statistics resetting is now more elegantly implemented. And the sim_index keyword argument is accepted by the function and individual Monte Carlo simulation elements can now be reset.
Modified wrapper function for curve_fit, to only change to list type, if the type is a ndarray. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
The model-free reset_min_stats() function has been replaced with the pipe_control.minimise version. The specific_analyses.model_free.optimisation.reset_min_stats() function has been deleted and instead the pipe_control.minimise version is being used.
Implemented the first try to compute the variance of R2eff and i0, by the covariance. This uses the Jacobian matrix. The errors calculated, are though way to small compared 2000 Monte Carlo simulations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Tried to implement the Jacobian from C code. This though also report errors which are to small. Maybe some scaling is wrong. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified profiling script to calculate timings. The timings for C code are: Simplex, with constraints = 2.192; Simplex, without constraints = 0.216; BFGS, without constraints = 0.079; Newton, without constraints = 0.031; This is pretty pretty fast. To this profiling script, I would also now add some verification on calculations.
Tried to verify solution to profiling script. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Set the constraints=False when doing Monte Carlo simulations for R2eff. This is to speed up the Monte Carlo simulations by a factor X10, when estimating the error for R2eff. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented the use of "Newton" as minimisation algorithm for R2eff curve fitting instead of simplex. Running the test script: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. For 50 Monte Carlo simulations, the time drop from: 3 minutes and 13 s, to 1 min an 5 seconds. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Changed the relax_fit.py sample script to use Newton rather than Simplex optimisation. This can lead to significantly faster optimisation times, as shown in the commit message http://article.gmane.org/gmane.science.nmr.relax.scm/23081.
Changed the optimisation description in the relaxation curve-fitting chapter of the manual. The script example has been converted to match the sample script, replacing the Nelder-Mead simplex algorithm with Newton optimisation, and removing the argument turning diagonal scaling off. All the text about only the simplex algorithm being supported due to the missing gradients and Hessians in the C module have been deleted. The text that linear constraints are not supported has also been removed - but this was fixed when the logarithmic barrier constraint algorithm was added to minfx.
By using minfx, and the reported Jacobian, it is now possible to get the exact same error estimation as scipy.optimize.leastsq. The fatal error was to set the weighting matrix with diagonal elements as the error. There weights are 1/errors**2. There is though some unanswered questions left. The Jacobian used, is the direct derivative of the function. It is not the chi2 derivative Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fixed naming of functions, to better represent what they do in module of estimating R2eff. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented the Jacobian of exponential function in Python code. This now also gets the same error as leastsq and C code. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Tried to implement a safety test for linearly-dependent columns in the covariance matrix. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fixes for the relax_disp.r2eff_estimate user function documentation. This is to allow the relax manual to compile again as the original documentation was causing LaTeX failures.
Clean up of the declarations in the target_functions.relax_fit C module. The Python list objects are now declared at the start of the functions, and then PyList_New() is called later on. This allows the code to compile on certain Windows systems.
Removed the user function to estimate the R2eff values and errors with scipy.optimize.leastsq. With the newly implemented Jacobian and Hessian of the exponential decay function, the front-end to scipy.optimize.leastsq does not serve a purpose. This is because minfx is now as fast as scipy.optimize.leastsq, and can estimate the errors from the Jacobian to the exact same numbers as scipy.optimize.leastsq. In addition to that, the covariance can be calculated by QR decomposition. This adds additional feature for checking for a singular matrix. The back-end will still be kept in place for the coming tim, but could be removed later. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added front-end to the new user function relax_disp.r2eff_err_estimate(), which will estimate the R2eff errors from a pipe and spins with optimised values of R2eff and i0. The covariance matrix can be calculated from the optimised parameters, and the Jacobian. Big care should be taken not to directly trust these results, since the errors are quite different compared to the Monte Carlo simulations. This implementation, will reach the exact same error estimation as scipy.optimize.leastsq. But with much better control over the data, and insight into the calculations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added method to automatically perform error analysis on peak heights. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified system test Relax_disp.test_estimate_r2eff() to first do a grid search, then minimise and then estimate the errors for R2eff and i0. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added back-end to estimate R2eff errors. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix to system test test_estimate_r2eff_error(), to first delete the old error estimations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added several tests to: test_estimate_r2eff_error, to compare different output from algorithms. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Cleaned up code in R2eff error module. Also removed a non working Hessian matrix. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Moved code around, and made function multifit_covar() independent of class object. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Inserted checks for C module is available in module for estimating R2eff error. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Removed unnecessary call to experimental Exp class. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Renamed system test, that test the user function for estimating the R2eff error: test_estimate_r2eff_err, test the user function. test_estimate_r2eff_err_methods, test different methods for getting the error. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added system test, Relax_disp.test_estimate_r2eff_err_auto and extended functionality to the auto-analyses protocol. If "exp_mc_sim_num" is set to "-1" and sent to the auto-analyses, the errors of R2eff will be estimated from the covariance matrix. These errors is HIGHLY likely to be wrong, but can be used in an initial test fase, to rapidly produce data for plotting data. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added script, to be used in GUI test. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added GUI test Relax_disp.test_r2eff_err_estimate, to test the setting of MC sim to -1 for exponential R2eff error estimation. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added keyword "exp_mc_sim_num", to the auto-analyses in the GUI. This sets the number of Monte Carlo simulations for R2eff error estimation in exponential curve fitting. When setting to -1, the errors are estimated from the covariance matrix. These errors are highly likely to be wrong, but can be used in Rapid testing of data and plotting. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Tried to click the "fit_r1" button in the GUI test, but receives an error: relax --gui-tests Relax_disp.test_r2eff_err_estimate, "AttributeError: 'SpinContainer' object has no attribute 'r1'". Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Moved the mc_sim_num GUI element in the analysis tab ip, as it is executed first. Also modified the tooltip. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added a warning to the auto-analyses about error estimation from the covariance. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Removed yet another comma from GUI tooltip. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Formatting changes for the lib.periodic_table module. This is in preparation for extending the information content of this module.
Modified system test 'test_estimate_r2eff_err_auto', to use the GUI script. It seems to work perfect. This is to test against GUI script: test_r2eff_err_estimate Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified test_estimate_r2eff_err_auto, to set r1_fit to False. This still make the system test pass, and fit R1. So this means R1 fit button is not functioning properly. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix for warning message in the auto-analyses in the GUI. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Tried to improve docstring for API documentation. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added all of the IUPAC 2011 atomic weights to the lib.periodic_table module. These will be useful for correctly calculating the centre of mass of a molecule.
The lib.periodic_table method for adding elements is now private.
Created the unit test infrastructure for the lib.periodic_table module. This includes one unit test of the lib.periodic_table.periodic_table.atomic_weight() function which has not been implemented yet.
Implemented the lib.periodic_table.periodic_table.atomic_weight() method. This returns the standard atomic weight of the atom as a float.
Yet another try to make the API documentation working. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented system test Relax_disp.verify_estimate_r2eff_err_compare_mc for testing R2eff error as function of Monte Carlo simulation. Note, since the name does not start with "test", but with "verify", this test will not be issued in the system test suite. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Converted the periodic table in lib.periodic_table into a dictionary type object. The new Element container has been added for storing the information about each element in the table. The Periodic_table object used the atomic symbol as a key for each Element instance.
Modified system test test Relax_disp.test_estimate_r2eff_err_methods() to show the difference between using the direct function Jacobian, or the chi2 function Jacobian. Added also the functionality to the estimate R2eff module, to switch between using the different Jacobians. The results show, that R2eff can be estimated better.
Added isotope information to the lib.periodic_table module including mass number and atomic mass. A new Isotope data container has been added to store this information. The Periodic_table._add() method now returns the initialised Element container. This container has the _add_isotope() method which is used to initialise Isotope data containers with the mass number and atomic mass and append it to the list.
Created a unit test for the Periodic_table.atomic_mass() method. This method is not implemented yet.
Changed the method call in the new Test_periodic_table.test_get_atomic_mass unit test.
Fix for the Test_periodic_table.test_get_atomic_mass unit test - the method calls were incorrect.
Implemented the lib.periodic_table module Periodic_table.atomic_mass() method. This method will return either the atomic mass of an isotope or the standard atomic weight.
Changed the operation of the lib.structure.mass.centre_of_mass() function. Instead of using the lib.physical_constants.return_atomic_mass() function, the centre_of_mass() function instead uses the lib.periodic_table.periodic_table.atomic_mass() method. This is a huge improvement in that the exact mass of absolutely all elements are taken into account.
Deletion of the lib.physical_constants.return_atomic_mass() function and all relative atomic masses. These were inaccurate and only included a tiny subset of all standard atomic weight and isotope masses. The functionality has been replaced by the complete and 100% accurate complete Periodic_table object in the lib.periodic_table module.
Fix for the MolContainer.fill_object_from_gaussian() method. This is in the lib.structure.internal.molecules module. The Periodic_table.lookup_z_to_symbol() method in the lib.periodic_table module has been renamed to lookup_symbol().
Fix for the Periodic_table.lookup_symbol() method. The __init__() method of the Periodic_table has been reintroduced to initialise a fast atomic symbol lookup table. The _add() method then updates this table. And the lookup_symbol() method now uses this lookup table to correctly return the symbol.
Tiny fix for the Diffusion_tensor.test_create_diff_tensor_pdb_ellipsoid system test. The switch to using the lib.periodic_table module for atomic masses has caused the centre of mass of the ellipsoid to shift just enough that one ATOM coordinate in the PDB file has changed its last significant digit.
Created the lib.periodic_table.process_symbol() function. This will take an atomic symbol and return a copy of it with an uppercase first letter and lowercase second letter. This is used by the Periodic_table methods atomic_mass() and atomic_weight() to allow for non-standard symbol input, for example if the element name comes directly from the all uppercase PDB file format without translation.
Tried to scale the covariance matrix, as explained here: http://www.orbitals.com/self/least/least.htm. This does not work better. Also replaced "errors" to "weights" to the multifit_covar(), to better determine control calculations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added all gyromagnetic ratio information from lib.physical_constants to lib.periodic_table. The Periodic_table.gyromagnetic_ratio() method has been added to allow this value to be easily returned.
Added to back-end of R2eff estimate module, to be able to switch between the function Jacobian or the chi2 Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified user function 'relax_disp.r2eff_err_estimate', to be able switch between the Jacobians. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified system test Relax_disp.verify_estimate_r2eff_err_compare_mc, to try the difference between the Jacobian. The results are: Printing the estimated R2eff error as function of estimation from covariance and number of Monte Carlo simulations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Deleted the gyromagnetic ratio values and return_gyromagnetic_ratio() function from lib.physical_constants.
Shifted all of relax to use the lib.periodic_table module for gyromagnetic ratios. The values and value returning function have been removed from lib.physical_constants and replaced by the Periodic_table.gyromagnetic_ratio() method in the lib.periodic_table module.
Started making functions in R2eff estimate module, independent on the informations stored in the Class. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Cleaned up code in R2eff estimate module, by making each function independent of class. This is to give a better overview, how the different functions connect together. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Made the user function, which estimates the R2eff errors, use the Jacobian derived from chi2 function. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified system test verify_estimate_r2eff_err_compare_mc() to first use the direct function Jacobian, and then the chi2 derived Jacobian. This shows the result better. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added digit to printout in R2eff estimate module. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Locked values for system test test_estimate_r2eff_err, to estimate how the R2eff error estimation reflects on fitted parameters. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
More locking of values, when trying to use different methods for estimating R2eff err values. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
More locking of values. This actually shows, that errors should be estimated from the direct Jacobian. Not, the chi2 Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Reverted the logic, that the chi2 Jacobian should be used. Instead, the direct Jacobian exponential is used instead. When fitting with the estimated errors from the direct Jacobian, the results are MUCH better, and comparable to 2000 Monte Carlo simulations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Various precision fixes for different machine precision. This is in: verify_r1rho_kjaergaard_missing_r1 Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
First attempt at properly implementing the target_functions.relax_fit.jacobian() function. This is now the Jacobian of the chi-squared function. A new jacobian_matrix data structure has been created for holding the matrix data prior to converting it into a Python list of lists. The equation used was simply the chi-squared gradient whereby the sum over i has been dropped and the i elements are stored in the second dimension of matrix.
Speed up of the target_functions.relax_fit C module. The variances are now pre-calculated in the setup() function from the errors, so that the use of the square() function is minimised. The chi-squared equation, gradient, and Hessian functions now accept the variance rather than standard deviation argument and hence the squaring of errors has been removed. This avoids a lot of duplicated maths operations.
Alphabetical ordering of global variable declarations in the target_functions.relax_fit header file.
Added RelaxError, if less than 2 time points is used for exponential curve fitting in R2eff. This follows: http://thread.gmane.org/gmane.science.nmr.relax.user/1718 http://thread.gmane.org/gmane.science.nmr.relax.user/1735 Specifically, data was attached here: http://thread.gmane.org/gmane.science.nmr.relax.user/1735/focus=1736.
Added system test Relax_disp.test_bug_atul_srivastava(), to catch a bug missing raising a RelaxError, since the setup points to a situation where the data shows it is exponential fitting, but only one time point is added per file. This follows: http://thread.gmane.org/gmane.science.nmr.relax.user/1718 http://thread.gmane.org/gmane.science.nmr.relax.user/1735 Specifically, data was attached here: http://thread.gmane.org/gmane.science.nmr.relax.user/1735/focus=1736.
Parameter precision lowered for Relax_disp.test_estimate_r2eff_err_auto(). This is due to change to C code. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Created the select.display user function. This simply displays the current spin selections of all spins. In the future it can be extended to display the interatomic data container selections, domain selections, etc.
Fix for system test: test_estimate_r2eff_err_auto(). The Jacobian to estimate the errors has been changed from the direct function Jacobian, to the Jacobian of the chi2 function. This changes the R2eff error predictions, and hence parameter fitting. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented the direct Jacobian in Python, to be independent of C code in development phase. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Activated all method try in: system test Relax_disp.test_estimate_r2eff_err_methods. This is to quickly estimate errors from all different methods. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix to system test: test_estimate_r2eff_err_auto, which now checks the values for the direct Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Increased the number of time points for exponential curve fitting to 3.
Fix to weight properly according to if minimising with direct Jacobian or chi2 Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix to system test test_estimate_r2eff_err_methods, after modification of weighting. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Switched in estimate_r2eff_err() to use the chi2 Jacobian from C code, and Jacobian from Python code. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Removed all references to test values which was received by wrong weighting. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Better error checking in the relaxation dispersion overfit_deselect() API method. The model must be set for this procedure to work, and the method now checks that this is the case.
Better error checking for the specific_analyses.relax_disp.average_intensity() function. This function would fail with a traceback if a peak intensity error analysis had not yet been performed. Now it fails instead with a clean RelaxError so that the user knows what is wrong.
Tried implementing getting the chi2 gradient, using target_function.chi2.dchi2(). The output seem equal. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Replaced the way to calculate the chi2 Jacobian, for exponential fit in minfx. This is only for the test class, but reuses library code. This should make it much easier in the future to implement chi2 gradient functions to minfx, since it is only necessary to implement the direct gradient of the function, and then pass the direct gradient to chi2 library, which turn it into the chi2 gradient function which minfx use. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Moved unnecessary function in R2eff error estimate module into experimental class. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Implemented system test: test_bug_negative_intensities_cpmg, to show lack of error message to user. Maybe these spins should be de-selected, or at least show a better warning. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
An attempt at documenting the Monte Carlo simulation verses covariance matrix error estimates. This is for the R2eff and I0 parameters of the exponential curves. For the Monte Carlo errors, 10000 simulations were preformed. This means that these errors can perform as a gold standard by which to judge the covariance matrix technique. Currently it can be seen that the relax_disp.r2eff_err_estimate user function with the chi2_jacobian flag set to True performs extremely poorly.
Reintroduced the original target_functions.relax_fit.jacobian() function. The new function for the Jacobian of the chi-squared function has been renamed to target_functions.relax_fit.jacobian_chi2() so that both Python functions are accessible within the C module.
Epydoc fixes for the pipe_control.mol_res_spin.format_info_full() function.
Epydoc docstring fixes for many methods in the relaxation dispersion auto-analysis module.
If math domain errors are found when calculating the two point R2eff values, the point is being skipped. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Moved intensity negative value from reference to CPMG point.
Modified system test test_bug_negative_intensities_cpmg, to prepare for testing number of R2eff points. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Comparison of 10,000 Monte Carlo simulations to a different covariance matrix error estimate. The covariance_matrix.py script has been duplicated and the chi2_jacobian argument of the relax_disp.r2eff_err_estimate user function has been changed from True to False. As can be seen in the 2D Grace plots, this error estimate is incredibly different. The R2eff errors are overestimated by a factor of 1.9555, which indicates that the Jacobian or covariance matrix formula are not yet correct.
The target_functions.relax_fit C module Python function jacobian_chi2() is now exposed. This was previously not visible from within Python.
Added a script and log file for calculating the numerical Jacobian for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the Jacobian using the numdifftools.Jacobian object construct and obtain the matrix, both at the minimum and at a point away from the minimum. The values will be used to construct a unit test to check the C module implementation.
Created two unit tests showing the target_functions.relax_fit.jacobian() function is correct. This compares the calculated Jacobian to the numerically integrated values from the test_suite/shared_data/curve_fitting/numeric_gradient/jacobian.py script.
Renamed the test_data/shared_data/curve_fitting/numeric_gradient/ directory to numeric_topology. This is to better reflect that it contains numeric approximations to the gradient, Hessian, and Jacobian.
Added a script and log for calculating the numerical chi-squared Jacobian for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the chi-squared Jacobian using the numdifftools.Jacobian object construct and obtain the matrix, both at the minimum and at a point away from the minimum. The values will be used to construct a unit test to check the C module implementation.
Fix for the chi-squared Jacobian numerical approximation script. The function was modified to that a list of chi-squared elements are returned, i.e. the sum part of the chi-squared equation has been removed.
Created two unit tests showing the target_functions.relax_fit.jacobian_chi2() function is correct. This compares the calculated chi-squared Jacobian to the numerically integrated values from the test_suite/shared_data/curve_fitting/numeric_topology/jacobian_chi2.py script.
Added a script and log for calculating the numerical covariance matrix for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the covariance matrix via the Jacobian calculated using the numdifftools.Jacobian object construct and obtain the matrix, both at the minimum and at a point away from the minimum. The covariance is calculated as inv(J^T.W.J).
Added a script and log for calculating the exponential curve parameter errors via bootstrapping. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the parameter errors via bootstrapping. As the parameters at the minimum are the exact parameter values, bootstrapping and Monte Carlo simulation converge and hence this is a true error estimate. 200,000 simulations where used, so the parameter errors are extremely accurate.
Modified module to estimate R2eff errors, to use the C code Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified system test test_estimate_r2eff_err_methods, to check all Jacobian methods are correctly implemented. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added more print out information, when log(I / I_ref) is negative, and raising errors. This can help the user track back information to the error more easily.
Improved system test test_bug_negative_intensities_cpmg, by counting number of R2eff points. Spin 4, which has one negative intensity, is expected to have one less R2eff point. This makes sure, that all CPMG data set can be loaded and analysed, even if some peaks are very weak are fluctuating with error level.
Fix for also storing 'r1_fit' to cdp even though it is set to False. Bug #22541 (https://gna.org/bugs/?22541): The R1 fit flag does not work in the GUI.
Cleanup in GUI test Relax_disp.test_r2eff_err_estimate. This now passes after previous commit. Bug #22541 (https://gna.org/bugs/?22541): The R1 fit flag does not work in the GUI.
Added model DPL94, to be tested in GUI test Relax_disp.test_r2eff_err_estimate. This shows that the bug is still there. Bug #22541 (https://gna.org/bugs/?22541): The R1 fit flag does not work in the GUI.
Fix for system test test_estimate_r2eff_err and test_r1rho_kjaergaard_missing_r1, where r1_fit=True, needed to be send to Auto_analyses. Bug #22541 (https://gna.org/bugs/?22541): The R1 fit flag does not work in the GUI.
API documentation fixes.
Moved multifit_covar into lib.statistics, since it is an independent module. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Moved "func_exp_grad" into experimental class for different minimisation methods. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Improved documentation to user function relax_disp.r2eff_err_estimate, and removed the possibility to use the chi2 Jacobian, as this is rubbish. But the back-end still have this possibility, should one desire to try this. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Moved the argument 'chi2_jacobian' as the last argument in estimate_r2eff_err. This argument is highly likely not to be used, but is kept for future testing purposes. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix to experimental class for fitting with different methods. After moving the function into class, 'self' should be added to the function. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix to system test test_estimate_r2eff_err, after removing the possibility to use the chi2 Jacobian. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Fix for system test test_estimate_r2eff_err_methods. The function was called wrong in experimental class. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Initial try write comments how to generalize the scaling of the covariance according to the reduced chi2 distribution. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
First try to make a test script for estimating efficiency on R2eff error calculations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added number of simulations to 10,000 in test script, and varied the random number of time point per simulation between 3 and 10. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
In module for estimating R2eff errors, removed "values, errors" to be send to function for gradient, since they are not used. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added Jacobian to test script, and now correctly do simulations, per R2eff points. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Improved analysing test script, with plotting. It seems that R2eff error estimation always get the same result. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added initial dataset for test analysis. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Deleted test data set. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified data script generator, to handle a situation with fixed 5 time points, and a situations with variable number of time points. Also modified analysis script. It seems, this has an influence how the error estimation is performing. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added simulations that show, there is perfect agreement between Monte Carlo simulations and covariance estimation. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Inserted extra tests in system test Relax_disp.test_estimate_r2eff_err_methods to test that all values of R and i0 are positive, and the standard deviation from Monte Carlo simulations are equal. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Inserted system test Relax_disp.test_finite_value, to illustrate the return of inf from C code exponential, when R is negative. This can be an issue, if minfx takes a wrong step when no constraints are implemented. Bug #22552 (https://gna.org/bugs/?22552): Chi2 value returned from C code curve-fitting return 0.0 for wrong parameters -> Expected influence on Monte Carlo sim.
Inserted possibility for bootstrapping in system test Relax_disp.test_estimate_r2eff_err_methods. This shows, that the bootstrapping method get the SAME estimation for R2eff errors, as the estimate_r2eff_err() function. This must either mean, that the OLD Monte Carlo simulation was corrupted, or the creation of data in Monte Carlo simulations is corrupted.
Modified system test Relax_disp.verify_estimate_r2eff_err_compare_mc to include bootstrapping method. This shows there is excellent agreement between bootstrapping and estimation of errors from covariance, while relax Monte Carlo simulations are very much different. Boot strapping is the "-2": Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added functionality to create peak lists, for virtual data. This is to compare the distribution of R2eff values made by bootstrapping and Monte Carlo simulations. Rest of the analysis will be performed in relax. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added initial peak lists to be analysed in relax for test purposes. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added relax analysis script, to profile distribution of errors drawn in relax, and from Python module "random". It seems that relax draw a lot more narrow distribution of Intensity with errors, than Python module "random". This has an influence on estimated parameter error. This is a potential huge error in relax. A possible example of a catastrophic implementation of Monte Carlo simulations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added PNG image that show that the distribution which relax makes are to narrow. This is a potential huge flaw in implementation of Monte Carlo simulations. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Modified analysis script, to also make histogram of intensities. This shows that the created intensities are totally off the true intensity. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Comment fix to system test Relax_disp.test_estimate_r2eff_err_methods, after the found of bug in relax. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting. Bug #22554 (https://gna.org/bugs/?22554): The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected.
Cleaned up user function for estimating R2eff errors. Extensive tests have shown, there is a very good agreement between the covariance estimation, and Monte Carlo simulations. This is indeed a very positive implementation. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting. Bug #22554 (https://gna.org/bugs/?22554): The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected.
Removed all junk comments from module for R2eff error estimation. The module runs perfect as it does now. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting. Bug #22554 (https://gna.org/bugs/?22554): The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected.
Fix for inf values being returned from C code exponential function. Values are now converted to high values. This fixes system test Relax_disp.test_finite_value. Example: x = np.array([np.inf, -np.inf, np.nan, -128, 128]), np.nan_to_num(x), array([ 1.79769313e+308, -1.79769313e+308, 0.00000000e+000, -1.28000000e+002, 1.28000000e+002]). Bug #22552(https://gna.org/bugs/?22552): Chi2 value returned from C code curve-fitting return 0.0 for wrong parameters -> Expected influence on Monte Carlo sim. Ref: http://docs.scipy.org/doc/numpy/reference/generated/numpy.nan_to_num.html.
Initial try to reach constrained methods in minfx through relax. This is in system test Relax_disp.verify_estimate_r2eff_err_compare_mc() This though not seem to be supported.
Allow R2eff model to reach constrained methods in minfx through relax. This is in system test Relax_disp.verify_estimate_r2eff_err_compare_mc() This though not seem to be supported.
Modified specific_analyses.relax_disp.parameters.r1_setup() to initialise the 'r1' variable. This relates to bug #22541 (https://gna.org/bugs/?22541), the R1 fit flag does not work in the GUI. This is a hack, as all of the dispersion analysis code assumes that all parameters are initialised. This is a dangerous assumption that will have to be eliminated in the future.
The dispersion get_param_values() API method now calls the r1_setup() function. This relates to bug #22541 (https://gna.org/bugs/?22541), the R1 fit flag does not work in the GUI. This is to make sure that the parameters are correctly set up prior to obtaining all parameter values. The R1 parameter is dynamic hence r1_setup() needs to be called at any point model parameters are accessed, as the R1 parameter can be turned on or off at any time with the relax_disp.r1_fit user function.
Yet another try to implement constrained method in verify_estimate_r2eff_err_compare_mc.
Another attempt to reach constrained method in minfx through relax. I would need to specify: l, lower bound constraint vector (l <= x <= u); u, upper bound constraint vector (l <= x <= u); c: user supplied constraint function; dc: user supplied constraint gradient function.
Added a derivation of the R2eff/R1rho error estimate for the two-point measurement to the manual. This is from http://thread.gmane.org/gmane.science.nmr.relax.devel/6929/focus=6993 and is for the rate uncertainty of a 2-parameter exponential curve when only two data points have been collected. The derivation has been added to the dispersion chapter of the manual.
Equation fixes for the two-point exponential error derivation in the dispersion chapter of the manual.
Updated the minfx version numbers in the release checklist document. The version is now 1.0.10, which has not been released yet but will contain the implementation of the log-barrier constraint algorithm gradient and Hessian.
Fix for the minfx version checking logic in the dep_check module. Now newer versions of minfx will be handled.
Fixes for the Relax_disp.test_estimate_r2eff_err system test. The kex parameter value checks have all been scaled by 1e-5 to allow for a meaningful floating point number comparison. The number of significant figures have also been scaled. This allows the test to now pass on one 64-bit GNU/Linux system.
Another fix for the minfx version checking in the dep_check module. The version_comparison() function has been created to perform a proper version number comparison by stripping trailing zeros, converting the two version numbers to lists of int and comparing the lists using the Python cmp() function. This will return -1 when the version number is too low, 0 when the versions are equal, and 1 when the version is higher than the minimum.
Added a button for the spectrum.error_analysis user function to the spectra list GUI element. This is placed after the 'Add' and 'Delete' buttons. The functionality could be improved by presetting the spectrum ID argument to anything the user has selected in the spectrum list.
Modified the behaviour of the spectrum.error_analysis button in the spectrum list GUI element. Now the subset argument of this user function will be pre-set to any spectra selected in the list.
Improvements for the spectrum.error_analysis button in the spectrum list GUI element. The user function is now launched as being modal so that the rest of the GUI freezes, and after the user function is executed the relax controller window is show and scrolled to the bottom.
Added Relax_disp system tests to black-list, if they depend on C code module.
Improvements for the spectrum.error_analysis button in the spectrum list GUI element. The subset argument is set to None if no spectra are selected.
Loosened a value check to allow the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test to pass. This test fails on MS Windows systems.
Fix for the Relax_disp.test_estimate_r2eff_err_auto system test on MS Windows systems. One of the value checks has been loosened.
Python 2 vs. 3 compatibility fix for the pickle module. This is for the estimate_errors*.py scripts in the directory test_suite/shared_data/curve_fitting/numeric_topology/. The lib.compat.pickle module is now used to allow both Python versions to run relax.
Python 3 fix, the cmp(v1, v2) notation in the dep_check.version_comparison() function has been replaced with (v1 > v2) - (v1 < v2). This allows relax to run on Python 3.
Python 3 fix for the lib.periodic_table module, the Python string module does not exist in Python 3.
Created the user_functions.uf_translation_table list. The elements of this list are the names of user functions before and after a renaming. The list is provided for backwards compatibility for relax scripts, though it is not used yet.
Converted the user_functions.uf_translation_table object to a dictionary. This is for faster access which does not require looping.
The prompt UI now uses the user_functions.uf_translation_table dictionary. The modified runcode() function will now check if the command typed by the user is a function or method call and then will raise a RelaxError if the command name is in the user_functions.uf_translation_table dictionary, telling the user that the user function has been renamed to the new name in the translation table. This appears to have no effect in the script UI however.
Hack in the script UI for handling missing user functions due to it being renamed. This script UI requires a different solution as the prompt UI. The script is executed via the runpy Python module and there appears to be no clean way of catching each command before it is executed. So instead, prior to executing the script, the contents of the script are read and old user functions are searched for using re.search(). The old user function name has "(" appended to it in the search so that it is sure that it is a user function call. And the old function must have a space or newline character preceding it.

Bugfixes

MS Windows fixes to allow relax to run again. The code for eliminating the GNU readline ^[[?1034h escape code emission on Linux systems fails on Windows as the 'TERM' environmental variable does not exist in os.environ.
Fix for the relaxation dispersion analysis Monte Carlo simulation printouts on clusters. The multi-processor code was calling the print() function from the Slave_command.run() method, however this runs on the slave processor. This has been shifted to the Results_command.run() method which runs on the master once the results have been returned via the Results_command. Now the printout of the simulation number and cluster ID will be visible when running via OpenMPI on a cluster.
Bug fix for the lib.arg_check.is_num_tuple() function. There was a typo in two of the RelaxError objects so that non-existent errors were being raised.
Grace string fixes for the alignment tensor parameters defined in the base parameter_object module. This is essentially for allowing relax to run using Python 3. All Grace '\' characters need to be escaped as '\\' in Python strings.
Another Python 3 fix - the string.split function no longer exists, it is now only a string method.
Fix for replacing reduce function. This is a Python 3 fix, where this function has been removed. This was reported as a necessity in thread: http://thread.gmane.org/gmane.science.nmr.relax.devel/6544
Fix for bug #22411 (https://gna.org/bugs/?22411), the failure in loading a Bruker DC T1 data file. The problem was that there was an empty line with spaces. The logic for skipping empty lines could not handle lines with just whitespace. This has now been fixed.
Fix for bug #22501 (https://gna.org/bugs/?22501), "Close all analyses" raises error in the GUI. The problem was general for all analysis types. This used to work, but as it was not tested in the test suite, a regression occurred.
Fix for the return_r2eff_arrays() dispersion function for exponential curves. This is a partial solution for bug #22461 (https://gna.org/bugs/?22461). For the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test, there are multiple relaxation times for each data set. For example, printing out the exp_type, frq, offset, point, ei, mi, oi, di, and relax_times data gives: R1rho 799777399.1 118.078 431.0 0 0 0 0 [0.0, 0.04, 0.1, 0.2]; R1rho 799777399.1 118.078 651.2 0 0 0 1 [0.0, 0.04, 0.1, 0.2, 0.4]. Instead of taking the first relaxation time of 0.0, now the maximum time is taken.
Fix for bug #22505 (https://gna.org/bugs/?22505). This is the failure of the structure.create_diff_tensor_pdb user function when no structural data is present. The solution was simple - the CoM of the representation is set to the origin when no structural data is present, and the check for the presence of data removed.
Another fix for bug #22505 (https://gna.org/bugs/?22505). This is the failure of the structure.create_diff_tensor_pdb user function when no structural data is present. Now the cdp.structure data structure is checked, when present, if it contains any data using its own empty() method.
Fix for bug #22502 (https://gna.org/bugs/?22502). This is the problem whereby the geometric prolate diffusion representation does not align with axis in PDB, as reported by Martin Ballaschk (https://gna.org/users/mab). This problem was not the main prolate tensor axis, but that the geometric object needed to be rotated 90 degrees about the z-axis to bring the object and axis into the same frame.
Fix for time not extracted for CPMG experiments in target_function. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Fix for interpolating time points, when producing xmgrace files for CPMG experiments. Bug #22461 (https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
Correction for catastrophic implementation of Monte Carlo simulations for exponential curve-fitting R2eff values in the dispersion analysis. A wrong implemented "else if" statement, would add the intensity for the simulated intensity together with the original intensity. This means that all intensity values send to minimisation would be twice as high than usually (if spectra were not replicated). This was discovered for Monte Carlo simulations of R2eff errors in exponential fit. This will affect all analyses using full relaxation exponential curves until now. By pure luck, it seems that the effect of this would be that R2eff errors are half the value they should be. A further investigation shows, that for the selected data set, this had a minimum on influence on the fitted parameters, because the chi2 value would be scaled up by a factor 4. Bug #22554 (https://gna.org/bugs/?22554): The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected. Task #7822 (https://gna.org/task/?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
Added a minfx minimum version check to the dep_check module. This is to avoid problems such as that reported at bug #22408 (http://gna.org/bugs/?22408).

@@ Line 9: / Line 9: @@
 Note that this new 3.3 relax series breaks compatibility with old relax scripts.  The important change, which is the main reason for starting the relax 3.3.x line, is the renaming of the calc, grid_search and minimise user functions to minimise.calculate, minimise.grid_search and minimise.execute respectively.  Please update your scripts appropriately.  A new relax feature is that old user function calls are detected in the prompt and script UIs and a RelaxError raised explaining what to rename the user function to.
-Important bugfixes in this release include that relax can run on MS Windows systems again, numerous Python 3 fixes, the ability to load Bruker DC files when the file format has corrupted whitespace, the GUI "close all analyses" feature works and no longer raises an error, structure.create_diff_tensor_pdb user function now works when no structural data is present, the geometric prolate diffusion 3D PDB representation in a model-free analysis now aligns with the axis in the PDB as it was previously rotated by 90 degrees, and the Monte Carlo simulations in the relaxation dispersion analysis for exponential curve-fitting for R2eff/R1rho parameter errors is now correct and no longer underestimating the errors by half.  For more details about the [[relax 3.3.0#Features|new features]] and the [[relax 3.3.0#Bugfixes|bug fixes]], please see below.  For fully formatted and easy to navigate release notes, please see http://wiki.nmr-relax.com/Relax_3.3.0.
+Important bugfixes in this release include that relax can run on MS Windows systems again, numerous Python 3 fixes, the ability to load Bruker DC files when the file format has corrupted whitespace, the GUI "close all analyses" feature works and no longer raises an error, structure.create_diff_tensor_pdb user function now works when no structural data is present, the geometric prolate diffusion 3D PDB representation in a model-free analysis now aligns with the axis in the PDB as it was previously rotated by 90 degrees, and the [[Monte Carlo simulation]]s in the relaxation dispersion analysis for exponential curve-fitting for R2eff/R1rho parameter errors is now correct and no longer underestimating the errors by half.  For more details about the [[relax 3.3.0#Features|new features]] and the [[relax 3.3.0#Bugfixes|bug fixes]], please see below.  For fully formatted and easy to navigate release notes, please see http://wiki.nmr-relax.com/Relax_3.3.0.
 To demonstrate the huge speeds ups in the relaxation dispersion analysis, the following table compares the speed of dispersion models in [[relax 3.2.3]] compared to the new 3.3.0 version:

Difference between revisions of "Relax 3.3.0"

Revision as of 08:48, 5 September 2014

Contents

Description

Download

CHANGES file

Features

Changes

Bugfixes

See also

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools