Changes

Jump to navigation Jump to search

Relax 3.3.0

No change in size, 19:36, 7 September 2014
→‎Changes: More links.
* Bug fix for the parameter units descriptions. This only affects a few rare parameters. The specific analysis API parameter object units() method was incorrectly checking if the units value is a function - it was checking the parameter conversion factor instead.
* Modified the [http://www.nmr-relax.com/manual/align_tensor_init.html align_tensor.init user function] so that the parameters are now optional. This allows alignment tensors to be initialised without specifying the parameter values for that tensor.
* Modified profiling script to have different number of NCYC points per frequency. This is to complicate the data, so any erroneous reshaping of data is discovered. It is expected, that experiments can have different number of NCYC points per spectrometer frequency. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Initial try to alter the target function calc_CR72_chi2. This is the first test to restructure the arrays, to allow for higher dimensional computation. All numpy arrays have to have same shape to allow to multiply together. The dimensions should be [ei][si][mi][oi][di]. [Experiment][spins][spec. frq][offset][disp points]. This is complicated with number of disp point can change per spectrometer frequency. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis. This implementation brings a high overhead. The first test shows no winning of time. The creation of arrays takes all the time.
* Temporary changed the lib/dispersion/cr72.py function to unsafe state. This change turns-off all the safety measures, since they have to be re-implemented for higher dimensional structures.
* Altered profiling script to report cumulative timings and save to temporary files. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis. This indeed shows that the efficiency has gone down.* Added print out of chi2 to profile script. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the creation of special numpy structures outside target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script to calculate correct values when setting up R2eff values. This is to test, that the return of chi2 gets zero. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removing looping over exp and offset indices in calc_chi2. They are always 0 anyway. This brings a little speed. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* In profiling script, moved up the calculation of values one level. This is to better see the output of the profiling iterations for CR72.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for calculation of the Larmor frequency per spin in profiling script. The frq loop should also be up-shifted. It was now extracted as 0.0. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Re-inserted safety checks in lin/dispersion/CR72.py file. This is re-inserted for the rank_1 cases. This makes the unit-tests pass again. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Important fix for extracting the correct shape to create new arrays. If using just one field, or having the same number of dispersion points, the shape would extend to the dispersion number. It would report [ei][si][mi][oi][di] when calling ndarray.shape. Shape always has to be reported as: [ei][si][mi][oi]. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made it easier to switch between single and cluster reporting in profiling script. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Important fix for the creation of the multi dimensional pA numpy array. It should be created as numpy.zeros([ei][si][mi][oi]) instead of numpy.ones([ei][si][mi][oi]). This allows for rapid testing of all dimensions with np.allclose(pA, numpy.ones(dw.shape)). pA can have missing filled out values, when the number of dispersion points are different per spectrometer frequency. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added unit tests demonstrating edge cases 'no Rex' failures of the model [[CR72 full]], for a clustered multi dimensional calculation. This is implemented for one field. This is to implement catching of math domain errors, before they occur. These tests cover all parameter value combinations which result in no exchange: Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Re-implemented safety checks in lib/dispersion/cr72.py. This is now implemented for both rank-1 float array and of higher dimensions. This makes the unit tests pass for multi dimensional computing. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added unit tests demonstrating edge cases 'no Rex' failures of the model [[CR72 full]], for a clustered multi dimensional calculation. This is implemented for three fields. This is to implement catching of math domain errors, before they occur. These tests cover all parameter value combinations which result in no exchange: Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed that special numpy structure is also created for [[CR72]]. This makes most system tests pass. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Critical fix for the slicing of values in target function. This makes system test: Relax_disp.test_sod1wt_t25_to_cr72 pass. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added self.has_missing keyword in initialization of the Dispersion class. This is to test once, per spin or cluster. This saves a looping over the dispersion points, when collection the data. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Created multi dimensional error and value numpy arrays. This is to calculate the chi2 sum much faster. Reordered the loop over missing data points, so it is only initiated if missing points is detected. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Switch the looping from spin->frq to frq->spin. Since the number of dispersion points are the same for all spins, this allows to move the calculation of pA and kex array one level up. This saves a lot of computation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed all the creation of special numpy arrays to be of float64 type. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the data filling of special numpy array errors and values, to initialization of Dispersion class. These values does not change, and can safely be stored outside. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Just a tiny little more speed, by removing temporary storage of chi2 calculation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made copies of numpy arrays instead of creating from new. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added a self.frqs_a as a multidimensional numpy array. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Small fix for the indices to the errors and values numpy array. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Lowered the number of iterations to the profiling scripts. This is to use the profiling script as bug finder. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the calculation of dw_frq out of spin and spectrometer loop. This is done by having a special 1/0 spin numpy array, which turns on or off the values in the numpy array multiplication. The multiplication needs to first axis expand dw, and then tile the arrays according to the numpy structure. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the calculation of pA and kex out off all loops. This was done by having two special 1/0 spin structure arrays. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed dw_frq_a numpy array, as it was not necessary. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed all looping over spin and spectrometer frequency. This is the last loop! Wuhu. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Reordered arrays for beauty of code. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the back_calc array be initiated as copy of the values array. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Small edit to profiling script, to help bug finding. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fixed that arrays are correctly initiated with one or zero values. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Very important fix, for only replacing part of data array which have Nan values. Before, all values were replaced, which was wrong. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Needed to increase the relative tolerance when testing if pA array is 1. Now system test Relax_disp.test_hansen_cpmg_data_missing_auto_analysis passes. Also added some comments lines, to prepare for mask replace of values. For example if only some of etapos values should be replaced. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Restored profiling script to normal. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the logic and comments much clearer about how to reshape, expand axis, and tile numpy arrays. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented a masked array search for where "missing" array is equal 1. This makes it possible to replace all values with this mask, from the value array. This eliminates the last loops over the missing values. It took over 4 hours to figure out, that the mask should be called with mask.mask, to return the same fulls structure, Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Yet another small improvement for the profiling script. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the multi dimensional structure of pA. pA is not multi-dimensional, and can just be multiplied with numpy arrays. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for testing of pA in lib function, when pA is just float. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified unit tests, so pA is sent to target function as float instead of array. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the multi dimensional structure of kex. kex is not multi-dimensional, and can just be multiplied with numpy arrays. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for testing of kex in lib function, when kex is just float. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified unit tests, so kex is sent to target function as float instead of array. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Important fix for replacing values if eta_pos > 700 is violated. This fixes system test: Relax_disp.test_sod1wt_t25_to_cr72, which failed after making kex to a numpy float. The trick is to make a numpy mask which stores the position where to replace the values. Then replace the values just before last return. This makes sure, that not all values are changed. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Increased the kex speed to 1e7 in clustered unit tests cases. This is to demonstrate where there will be no excange. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added a multi-dimensional numpy array chi2 value calculation function. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Called the newly created chi2 function to calculate for multi dimensional numpy arrays. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Renamed chi2_ND to chi2_rankN. This is a better name for representing multiple axis calculation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made special ei, si, mi, and oi numpy structure array. This is for rapid speed-up of numpy array creation in target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Replaced self.spins_a with self.disp_struct. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made initialisation structures for dw. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Initial try to reshape dw faster. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Switched to use self.ei, self.si, self.mi, self.oi, self.di. This is for better reading of code. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Comment out the sys.exit(), which would make the code fail for wrong calculation of dw. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Copied profiling script for CPMG model [[CR72]] to R1rho [[DPL94]] model. The framework of the script will be the same, but the data a little different. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Started converting profiling script to [[DPL94]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Replaced self.(ei,si,mi,oi,di) with self.(NE,NS,NM,NO,ND). These numbers represents the maximum number of dimensions, instead of index. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added the ei index, when creating the first dw_mask. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Reordered how the structures dw init structures are created. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Clearing the dw_struct before calculation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Started using the new way of constructing dw. This is for running system tests. Note, somewhere in the dw array, the frequencies will be different between the two implementations. But apparently, this does not matter. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Inserted temporary method to switch for profiling. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* First try to speed-up the old dw structure calculation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Simplified calculation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Yet another try to implement a fast dw structure method. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented the fastest way to calculate the dw structure. This uses the numpy ufunc multiply.outer function to create the outer array, and then multiply with the frqs_structure. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Renamed dw temporary structure to generic structure. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Restructured the calculation of R20A and R20B to the most efficient way. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the lib/CR72.py to a numpy multi dimensional numpy array calculation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed the catching when dw is zero, to use masked array. Implemented backwards compatibility with unit tests. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Bugfix for testing if kex is zero. It was tested if kex was equal 1.0. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented masked replacement if fact is less that 1.0. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced isnan mask with function that catches all invalid values.
* Removed the masked replacement if fact is less than 1.0. This is very strange, but otherwise system test: Relax_disp.test_hansen_cpmg_data_missing_auto_analysis would fail. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the slow allclose() function to test if R20A and R20B is equal. It is MUCH faster to just subtract and check sum is not 0.0. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Replaced the temporary variable R2eff with back_calc, and used numpy subtract to speed up. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the lib function into a pure numpy array calculation. This requires, that r20a, r20b and dw has same dimension as the dispersion points. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changes too unit tests, so data is sent to target function in numpy array format. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the creation of an unnecessary structure by using numpy multiply. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the mask which finds where to replace values into the __init__ function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Copied profiling script for [[CR72]] to [[B14]] model. Task #7807 ([https://gna.org/task/?7807): Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script for the [[B14]] model. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified model [[B14]] lib file to faster numpy multidimensional mode. The implementations comes almost directly from the [[CR72]] model file. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Reverted the use of the mask "mask_set_blank". It did not work, and many system tests started failing. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed the target function to handle the [[B14]] model for faster numpy computation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed unit test for [[B14]] to match numpy input requirement. Task #7807 ([https://gna.org/task/?7807): Task #7807]: Speed-up of dispersion models for clustered analysis.* Added additional tests in b14, when math errors can occur. This is very easy with a conditional masked search in arrays. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Comment fix for finding when E0 is above 700 in lib function of [[B14]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed use of "asarray", since the variables are already arrays. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed target function for model [[CR72]]. To [[CR72]] is now also the input of the parameters of R20A, R20B and dw. dw is tested for zero, to return flat lines. It is faster to search in the smaller numpy array, than the 5 dimensional dw array. This is for speed-up. R20A and R20B is also subtracted, to see if the full model should be used. In the same way, it is faster to subtract the smaller array. These small tricks are expected to give 5-10 pct. speeed-up. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the lib function of [[CR72]] accept the R20A, R20B and dw of the original array. This is for speed-up. Task #7807 ([https://gna.org/task/?7807): Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed unit-tests, to send in the original R20A, R20B and dw_orig to the testing of the lib function [[CR72]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed profiling script to send R20A, R20B and dw, as original parameters to the lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed target function for model [[B14]]. To [[B14]] now also send the input of the original parameters dw. dw is tested for zero, to return flat lines. It is faster to search in the smaller numpy array, than the 5 dimensional dw array. This is for speed-up. These small tricks are expected to give 5-10 pct. speed-up. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the lib function of [[B14]] accept dw of the original array. This is for speed-up. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed unit-tests, to send in the original dw_orig to the testing of the lib function [[B14]]. Task #7807 ([https://gna.org/task/?7807): Task #7807]: Speed-up of dispersion models for clustered analysis.* Changed profiling script to send dw as original parameters to the lib function [[B14]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Copied profiling script for [[CR72]] model to [[TSMFK01]] model. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script to be used for model [[TSMFK01]]. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified target function for model [[TSMFK01]], to send in dw as original parameter. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified lib function for model [[TSMFK01]] to accept dw_orig as input and replaced functions to find math domain errors into maske replacements. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made unit tests for model [[TSMFK01]] send in R20A and dw as a numpy array. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Large increase in speed for model [[TSMFK01]] by changing target functions to use multidimensional numpy arrays in calculation. This is done by restructuring data into multidimensional arrays of dimension [NE][NS][NM][NO][ND], which are number of spins, number of magnetic field strength, number of offsets, maximum number of dispersion point. The speed comes from using numpy ufunc operations. The new version is 2.4X as fast per spin calculation, and 54X as fast for clustered analysis.
* Replacing math domain checking in model [[DPL94]], with masked array replacement. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.* First try to speed up model [[DPL94]]. This has not succeeded, since system test: Relax_disp.test_dpl94_data_to_dpl94 still fails. Task #7807 (https://gna.org/task/?7807): Speed-up of dispersion models for clustered analysis.* Trying to move some of the structures into its own part. Task #* Replacing math domain checking in model [[DPL94]], with masked array replacement. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* First try to speed up model [[DPL94]]. This has not succeeded, since system test: Relax_disp.test_dpl94_data_to_dpl94 still fails. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* Trying to move some of the structures into its own part. [https
* The relaxation dispersion target function can now be set up when the optional frqs_H argument is None. This allows the profiling scripts to run.
* More stability fixes for the relaxation dispersion target function initialisation. The target function can now be initialised when the r1 and chemical_shift arguments are None.
* Split the system test test_r1rho_kjaergaard into a setup function, and a test function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Renamed system test test_r1rho_kjaergaard to test_r1rho_kjaergaard_auto. This corresponds to the use of the automatic analysis method. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Split system test test_r1rho_kjaergaard into test_r1rho_kjaergaard_auto and test_r1rho_kjaergaard_man. This is to test use of the manual way to analyse. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Modified all of Troels' dispersion profiling scripts to work with older relax versions. This is in preparation for obtaining some powerful timing statistics. The calls to the r2eff_*() functions are unnecessary and are the only failure point in the scripts between the current code in the disp_spin_speed branch and trunk or older versions of relax. So these function calls have been eliminated.
* Implemented system test test_r1rho_kjaergaard_missing_r1, for safety check if R1 data is not loaded. The system test passes, so target function is safe. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Python 3 support for the dispersion profiling scripts. The xrange() builtin function does not exist in Python 3, so this is now aliased to range() which is the same thing.
* Replaced double or triple hash-tags "##" with single hash-tags "#". Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Copyright fixes for all the models, where Troels E. Linnet have made changes to make them functional for higher dimensional data. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Copyright fix for model [[TSMFK01]]. Sebastien Morin did not take part of implementing the [[TSMFK01]] model. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Created a super script for profiling the relaxation dispersion models. This script will execute all of the current profiling scripts in the directory test_suite/shared_data/dispersion/profiling for both the current version of relax and any other specified version (current set to the 3.2.2 relax tag). It will run the scripts and relax versions interleaved N=10 times and extract the func_*() target function call profile timings. This interleaving makes the numbers much more consistent. Averages and standard deviations are then calculated, as well as the speed up between the two relax versions. The results are printed out in a format suitable for the relax release messages.
* Increased the number of iterations to 1000 in all of the profiling scripts. This is for better statistics in the disp_profile_all.py script, and makes the number consistent between the different models.
* Added a log file for comparing the speed of the disp_speed_branch to relax 3.2.2. This is from the disp_profile_all.py statistics generating script.
* Made the processor.return_object get the back_calc structure in the expected order. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fixed the ordering of the relax versions in the dispersion super profiling script disp_profile_all.py. This has also been fixed in the disp_spin_speed branch to relax 3.2.2 comparison log.
* Added a log file for comparing the speed of the disp_speed_branch to relax 3.2.1. This is from the disp_profile_all.py statistics generating script.
* Added a script for profiling the [[NS CPMG 2-site 3D]] relaxation dispersion model. This is the model at http://wiki.nmr-relax.com/NS_CPMG_2-site_3D. Again this only involved copying one of the other scripts and modifying a few variable and function names.
* Added the [[NS CPMG 2-site 3D]] model to the dispersion super profiling script. To handle the fact that this script has nr_iter set to 100 rather than 1000 (as otherwise it is too slow), a list of scaling factors has been created to scale all timing numbers to equivalent values.
* Added [[DPL94]] profiling script. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script for [[TSMFK01]], to use correct parameters k_AB and r20a. Or else, the lib functions is just calculating with zero? Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changes to profiling script of [[NS CPMG 2-site expanded]]. The model does not have r20a and r20b, but only r2. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made changes to the profiling script of [[NS CPMG 2-site 3D]]. Need to use the full model, when r2a and r2b is specified. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changes to profiling script of [[NS CPMG 2-site expanded]]. The unpacking can be removed. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for the profiling script of [[NS CPMG 2-site 3D]]. The model should also be specified to full. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* The disp_profile_all.py super script now prints out the current relax version information. This is so that the log files contain information about the repository revision and path.
* Copied profiling script of [[DPL94]] to [[NS R1rho 2-site]].
* Improved the final printout from the disp_profile_all.py dispersion model super profiling script.
* Added profiling script for [[NS R1rho 2-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* The disp_profile_all.py dispersion model super profiling script is now executable.
* Decreased all nr_iter values by 10 and added more dispersion models to the super profiling script. This is for the dispersion model profiling scripts in test_suite/shared_data/dispersion/profiling/, all controlled by the disp_profile_all.py super profiling script for generating statistics using all of the other profiling scripts. The number of iterations needed to be decreased as otherwise it would now take almost 1 day to generate the statistics table.
* Moved the parter conversion in [[LM63 3-site]] into the lib function. This cleans up the target api function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling script for [[DPL94]] to [[TAP03]].
* Copied profiling script for [[DPL94]] to [[TP02]].
* Copied profiling script for [[DPL94]] to [[MP05]].
* Copied profiling script for [[DPL94]] to [[M61]].
* Modified profiling script for [[TAP03]] to be used. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script for [[TP02]], to be used. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script for [[MP05]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script for [[M61]]. This is the last one. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Expansion of the disp_profile_all.py dispersion model super profiling scripts. The newly added profiling scripts for models [[M61]], [[TP02]], [[TAP03]], and [[MP05]] are now included in the super script to generate statistics for all of these as well. The nr_iter variable has also been changed to match the other analytic models, so that the standard deviations are lowered and the statistics are better.
* Moved the parameter conversion of [[MMQ CR72]] into lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the parameter conversions of k_AB, k_BA and pB into lib function of [[NS MMQ 2-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the parameter conversion from target function to lib function for [[NS R1rho 2-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Updated the dispersion model speed statistics for the disp_spin_speed branch vs. relax-3.2.2. This now includes the [[NS CPMG 2-site 3D]], [[DPL94]], and [[NS R1rho 2-site]] dispersion models. The timings for the single spin analyses are now comparable to the clustered analysis, as the equivalent of 100 single spins is being used. The final printout is also in a better format to present for the relax release messages. These new results show the insane 160x speed up of the [[DPL94]] model.
* Alignment improvements for the final printout from the dispersion model super profiling script. The log file has been updated with what the new formatting will look like.
* Updated the model names in the dispersion model super profiling script. The [[CR72]], [[B14]] and [[NS CPMG 2-site 3D]] models are the full, slower versions rather than the faster models with R20 = R20A = R20B. The log file has been updated to match.
* Moved the parameter conversion for [[NS MMQ 3-site]] into lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Updated the dispersion model profiling comparison of the disp_spin_speed branch vs. relax-3.2.2. The [[M61]], [[TP02]], [[TAP03]], and [[MP05]] models are now included. The final printout has been manually updated to reflect the newest version of the disp_profile_all.py super profiling script.
* Moved the parameter conversion for [[NS R1rho 3-site]] into lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling script for [[CR72]], so there is now a normal and a full version.
* Copied profiling for [[B14]] to normal and full model.
* Copied profiling script for [[NS CPMG 2-site star]].
* Copied profiling script for [[No Rex]].
* Modified profiling script for [[B14]], to R20A=R20B. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented profiling script for [[NS CPMG 2-site 3D]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented profiling script for [[NS CPMG 2-site star]] and [[NS CPMG 2-site star full|star full]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling script to be used for [[LM63]].
* Copied profiling script to model IT99.
* Added profiling script for IT99. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented profiling script for [[LM63]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the "eta_scale = 2.0**(-3.0/2.0)" out of lib function for [[MMQ CR72]], since this is only needs to be computed once. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for spaces aroung "=" outside functions. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Critical fix for wrong space inserted in [[NS MMQ 3-site]] MQ. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fixed the input for unit test of [[MMQ CR72]]. The number of input parameters has been lowered. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added additional math domain checking in [[B14]]. This is when v1c is less than 1.0. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Comment fixing, for explaining the masking and replacing when dw is zero. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling script to be used for profiling the use of higher dimensional data for the numpy eig function.
* Implemented the collection of the 3D exchange matrix, for rank [NE][NS][NM][NO][ND][7][7]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented test, to see if 3D exchange matrices are the same. This can be tested while running system test test_hansen_cpmg_data_to_ns_cpmg_2site_3D. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Shifted the computation of Rexpo two loops up. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added intermediate step with for loops. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added another intermediate step. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added function to compute the matrix exponential for higher dimensional data of shape [NE][NS][NM][NO][ND][7][7]. This is done by using numpy.einsum, to make the dot product of the last two axis. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Inserted intermediate step, to check if the matrix propagator to evolve the magnetization is equal when done for lower dimensional data of shape [7][7] and higher dimensional data of shape [NE][NS][NM][NO][ND][7][7]. A short example is shown at the wiki: http://wiki.nmr-relax.com/Numpy_linalg#Ellipsis_broadcasting_in_numpy.einsum. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Implemented double speed of model [[NS CPMG 2-site 3D]]. This is done by moving the costly calculation of the matrix exponential out of the for loops. The trick was to find a method to do dot product of higher dimensions. This was done with numpy.einsum, example at: http://wiki.nmr-relax.com/Numpy_linalg#Ellipsis_broadcasting_in_numpy.einsum. Example: dot_V_W = einsum('...ij,...jk', V, W_exp_diag) where V, and W_exp_diag has shape [NE][NS][NM][NO][ND][7][7]. The profiling script shows a 2X speed up.
* Made notation consistent for variables, using "_i" to clarify extracted data from matrix. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the calculation the evolution matrix out of for loops. The trick is that numpy.einsum allows for dot product of higher dimension: The essential evolution matrix; This is a dot product of the outer [7][7] matrix of the Rexpo_mat and r180x_mat matrices, which have the shape [NE][NS][NM][NO][ND][7][7]; This can be achieved by using numpy einsum, and where ellipsis notation will use the last axis.
* Implemented system test: test_cpmg_synthetic_b14_to_ns3d_cluster. This is to catch failures of the model, when data is clustered. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed unused variables in [[NS CPMG 2-site 3D]], to clean up the code. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added the NS matrices, rr1rho_3d_rankN, to collect the multi dimensional 3D exchange matrix, of rank [NE][NS][NM][NO][ND][6][6]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added a check in lib/dispersion/ns_r1hro_2site.py, to see if the newly created multidimensional ns matrix of rank NE][NS][NM][NO][ND][6][6], is equal to the previous [6][6] matrix. It is. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added the relax_time to collection of rr1rho_3d_rankN matrix collection. This is to pre-multiply all elements with the time. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added a check, that the pre- relax_time multiplied multidimensional array, equal the previous. It does, to the sum of 1.0e-13. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the function use the new multidimensional R_mat matrix. System test: test_tp02_data_to_ns_r1rho_2site still passes. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix to the matrix_exponential_rankN, to return the exact exponential for any higher dimensional square matrix of shape [NE][NS][NM][NO][ND][X][X]. The fix was to the eye(X), to make the shape the same as the input shape. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the costly calculation of the matrix exponential out of for loops. It was the numpy.eig and numpy.inv which was draining power. This speeds up model [[NS R1rho 2-site]], by a factor 4X.
* Made the returned multidimensional rr1rho_3d_rankN, be of float64 type. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Cleaned up the code of [[NS R1rho 2-site]], and removed the matrix argument to the function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the parsing of a matrix to the lib function of [[NS R1rho 2-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added the function "rcpmg_star_rankN" for the collection of the multidimensional relaxation matrix for model [[NS CPMG 2-site star]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Insert check, that the newly created multidimensional matrix is the same. They are, but only to the fifth digit. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Started using the newly created multidimensional matrix. test_hansen_cpmg_data_to_ns_cpmg_2site_star. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added the system test: test_cpmg_synthetic_b14_to_ns_star_cluster, to check for the model is still working after change. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Started using the newly cR2 variable, extracted from higher dimensional data. This should be okay, but system test test_hansen_cpmg_data_to_ns_cpmg_2site_star, now fails.
* Changes of values to system test test_hansen_cpmg_data_to_ns_cpmg_2site_star. The values are changed, since chi2 is lower than before. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the costly finding of matrix exponential out of for loops for eR_tcp. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Rearranged the code, to properly show the nested matrix exponentials in dot functions. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the costly matrix_exponential of cR2 out of for loops. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Rearranged the dot code, for better view. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Cleaned up the code in model [[NS CPMG 2-site star]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Simplified model [[NS CPMG 2-site 3D]]. The expansion of matrices to higher dimensionality is not necessary. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Further cleaned up the code in [[NS CPMG 2-site star]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed input of matrix, Rr, Rex, RCS and R to model [[NS CPMG 2-site star]]. These matrices is now extracted from NS matrix function rcpmg_star_rankN. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented the collection of the multidimensional matrix m1 and m2 in model [[NS MMQ 2-site]]. Inserted also a check, that the newly computed matrix is equal. They are, to the 6 digit. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Started using the newly created multidimensional m1 and m2 matrices. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the costly calculation of matrix_exponential of M1 and M2 out of for loop, in model ns_mmq_2site_mq. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the function matrix_exponential_rankN also find the exponential if the experiments indices are missing. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for an extra axis inserted in eye function, when dimensionality is only [NS][NM][NO][ND]. This also fixes the index in the lib function of ns_mmq_2site_mq. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented same functionality in mmq_2site_sq_dq_zq. Problem, following system test fails: test_korzhnev_2005_15n_dq_data, test_korzhnev_2005_15n_mq_data, test_korzhnev_2005_15n_sq_data, test_korzhnev_2005_1h_mq_data, test_korzhnev_2005_1h_sq_data, test_korzhnev_2005_all_data, test_korzhnev_2005_all_data_disp_speed_bug. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed grid search, and lowered number of iterations for system test: test_cpmg_synthetic_b14_to_ns3d_cluster, test_cpmg_synthetic_b14_to_ns_star_cluster. This is to speed them up, since they before took 30 seconds. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for ns_mmq_2site_mq. Variable was wrong called. There seems to be a serious problem more with MQ.
* Reinserted old code. This fixes: test_korzhnev_2005_15n_mq_data. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Forcing the dtype to be complex64, instead of complex128. This solves a range of system tests. The one who fails now is: test_korzhnev_2005_15n_zq_data, test_korzhnev_2005_1h_mq_data, test_korzhnev_2005_1h_sq_data. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Forces complex64 in ns_mmq_2site_sq_dq_zq instead complex128. This fixes system tests: test_korzhnev_2005_15n_zq_data,test_korzhnev_2005_1h_sq_data. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Force complex64 in ns_mmq_2site_mq. This solves all system tests. Forcing to be complex64, does not seems like a long standing solution, since complex128 is possible. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for using the old matrix_exponential of m1. One: test_korzhnev_2005_15n_sq_data is still failing. That still uses the matrix_exponential_rankN. There seems to be a problem with matrix_exponential_rankN, when doing complex numbers. Maybe the dtype has to get fixed? Use it as a input argument? It must be the einsum. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added the "dtype" argument to function matrix_exponential_rankN. This is to force the conversion of dtype, if they are of other type. This can be conversion from complex128 to complex64. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix the bug: "M2_i = M1_mat", which was causing the problems getting system tests to pass. Removed the specifications of which dtype, the initial matrices are created. They can be converted later, with the specification of dtype to matrix_exponential_rankN(). All system tests now pass. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the Bloch-McConnell matrix for 2-site exchange into lib/dispersion/ns_matrices.py. This is for consistency with the other code. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the matrices for Bloch-McConnell from lib ns_mmq_2site, since they are now defined in ns_matrices.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the Bloch-McConnell matrix for 3-site exchange, into the lib/dispersion/ns_matrices.py. This is to standardize the code. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed m1 and m2 to be sent to lib function of [[NS MMQ 2-site]], since they are now populated inside the lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented the Bloch-McConnell matrix for 3-site exchange, for multidimensional data. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Inserted a check, that the new higher dimensional m1 and m2 matrices are equal to before. They are, to the 5 digit. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Started using the newly created higher dimensional Bloch-McConnell matrix for 3-site exchange. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the calculation of the matrix exponential out of for loops for [[NS MMQ 3-site]] MQ. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Converted [[NS MMQ 3-site]]/SQ/DQ/ZQ to calculate the matrix exponential out of the for loops. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the complex64 to be used as dtype in matrix exponential. Fix for missing "_i" in variable. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed m1 and m2 to be sent to target function of ns_mmq_3site_chi2. They are now populated inside the lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Documentation and input fix for [[NS MMQ 2-site]]. The m1 and m2 matrices are populated inside the lib function. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Renamed some numerical matrices, to get consistency in naming. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented multidimensional [[NS R1rho 3-site]] exchange matrix. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Inserted check, that newly multi dimensional matrix is equal the old. It is, to the 13 digit. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Started using the newly multidimensional 3D exchange matrix. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved the calculation of the matrix exponential out of the for loops for [[NS R1rho 3-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed the parameter "matrix" to be send to lib function of [[NS R1rho 3-site]], since it is now populated inside the lib function. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved parameter conversion for [[NS R1rho 3-site]] inside lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Cleaned up the Dispersion class target function, for creation of matrices, which is now populated inside the lib functions instead. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed pA and pB from the matrix population function rcpmg_star_rankN, since they are not used. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed pA and pB from the matrix population function rr1rho_3d_2site_rankN, since they are not used. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Documentation fix for the dimensionality for model [[NS R1rho 2-site]]. The data is lined up to be of form [NE][NS][NM][NO][ND]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed pA, pB and pC from the matrix population function rr1rho_3d_3site_rankN, since they are not used. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Deleted the profiling of eig function profiling script. This was never implemented. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* For all profiling scripts, added conversion to numpy array for CPMG frqs and spin_lock, since some models complained in 3.2.2, that they were of list types. Also fixed IT99 to only have 1 spin, since clustering is broken in 3.2.2. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified super profiling script, to allow input to script, where alternative version of relax is positioned. Collected the variables in a list of lists, for better overview. Added a print out comment to IT99, to remember the bug. Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added comment field to super profiling script. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Math domain fix for [[NS CPMG 2-site expanded]]. This is when t108 or t112 is zero, in the multidimensional array, a division error occurs. The elements are first set to 1.0, to allow for computation. Then elements are later replaced with 1e100. Lastly, if the elements are not part of the "True" dispersion point structure, they are cleaned out. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Precision lowering of system test, test_korzhnev_2005_15n_sq_data and test_korzhnev_2005_1h_sq_data. The system tests does not fail on Linux 64-bit system, but only on Mac 32-bit system. This is due to floating error deviations. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added log files for super profiling against tags 3.2.1 and 3.2.2. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied lib.linear_algebra.matrix_exponential to lib.dispersion.matrix_exponential. The matrix exponential of higher dimensional data is only used in the dispersion part of relax.
* Added to __init__, the new lib.dispersion.matrix_exponential module. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added to unit_tests/_lib/_dispersion/__init__.py, the new unit test file: test_matrix_exponential.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added numpy array save files. They are the numpy array structures, which are send in from system test: Relax_disp.test_hansen_cpmg_data_to_ns_cpmg_2site_3D. These numpy array structures, are used in unit tests. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added unit test unit_tests/_lib/_dispersion/test_matrix_exponential.py to test the matrix exponential from higher dimensional data. lib.dispersion.matrix_exponential.matrix_exponential_rankN will match against lib.linear_algebra.matrix_exponential. Data which is used for comparison, comes from system test: Relax_disp.test_hansen_cpmg_data_to_ns_cpmg_2site_3D. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Renamed function to return data in unit test _lib/_dispersion/test_matrix_exponential.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix to lib/dispersion/matrix_exponential.py, since the svn copy command was used on non-updated version of the file. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added unit test for doing the matrix exponential for complex data. This test shows, that the dtype=complex64, should be removed from lib/dispersion/ns_mmq_2site.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added data for unit test for the testing of the matrix_exponential_rankN. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Expanded the dispersion profiling master script to handle any two relax versions. To compare two relax versions, for example 3.2.2 to 3.2.1, either the path1 and path2 variables or two command line arguments can be supplied. The first path should be for the newest version. This will allow for comparing the speed differences between multiple relax versions in the future.
* Split matrix_exponential_rankN into matrix_exponential_rank_NE_NS_NM_NO_ND_x_x and matrix_exponential_rank_NS_NM_NO_ND_x_x. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Moved numerical solution matrices into the corresponding lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling scripts, to be used for 3-site models and MMQ models.
* Implemented profiling script for [[LM63 3-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Improved the relax version printouts for the dispersion model master profiling script. This now reports both relax versions.
* Removed a tonne of unused imports from the dispersion model profiling scripts. This is to allow most of the scripts to run on the relax 3.1.x versions, as well as to clean up the scripts. The unused imports were found using the command: pylint test_suite/shared_data/dispersion/profiling/*.py --disable=all --enable=unused-import.
* Added a relaxation dispersion model profiling log file for relax version 3.2.0 vs. 3.1.7. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
* Added a relaxation dispersion model profiling log file for relax version 3.1.7 vs. 3.1.6. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
* Modified profiling script for [[NS R1rho 3-site]], to be functional. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified profiling script for [[NS R1rho 3-site linear]] to be functional. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added a relaxation dispersion model profiling log file for relax version 3.1.3 vs. 3.1.2 vs. 3.1.1. This is the output from the dispersion model profiling master script. It shows that there are no major speed differences between these relax versions.
* Added the system information printout to the dispersion model profiling master script output. This is for the log files comparing one version of relax to the previous version.
* Added profiling script for model [[MMQ CR72]], Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for the replacement value for invalid values in model [[MMQ CR72]]. The value was set to use r20, but should instead be 1e100. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Copied profiling script from [[MMQ CR72]], to [[NS MMQ 2-site]] and [[NS MMQ 3-site|3-site]].
* Copied profiling script to [[NS MMQ 3-site linear]].
* Implemented profiling script for [[NS MMQ 2-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented profiling script for [[NS MMQ 3-site]] and [[NS MMQ 3-site linear|3-site linear]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Speeded up model [[NS CPMG 2-site star]], by moving the forming of the propagator matrix out of the for loops, and preform it. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added a relaxation dispersion model profiling log file for relax version 3.1.4 vs. 3.1.3. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
* Speeded up [[NS MMQ 2-site]], by moving the forming of evolution matrix out of the for loops, and preform it. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Speeded up [[NS MMQ 3-site]], by moving the forming of evolution matrix out of the for loops, and preform it. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added a relaxation dispersion model profiling log file for relax version 3.1.5 vs. 3.1.4. This is the output from the dispersion model profiling master script. It shows that there are no speed differences.
* Speeded up [[NS R1rho 2-site]], by preforming the evolution matrices, and the M0 matrix in the init part of the target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Speeded up [[NS R1rho 3-site]], by preforming the evolution matrices, and the M0 matrix in the init part of the target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Expanded the dispersion model profiling master script to cover all the new profiling scripts. This includes all 3-site and MMQ models. The list is now complete and covers all models. The only model not included in [[M61 skew]] which has redundant parameters and is not optimisable anyway.
* Added a relaxation dispersion model profiling log file for relax version 3.1.6 vs. 3.1.5. This is the output from the dispersion model profiling master script. It shows that there are almost no speed differences, except for a slight decrease in speed in the [[CR72 full]] model for single spins.
* Split system test test_tp02_data_to_ns_r1rho_2site into a setup and test part. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Implemented a clustered version of system test test_tp02_data_to_ns_r1rho_2site. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Inserted an extremely interesting development in [[NS R1rho 2-site]]. If one do a transpose of M0, one can calculate all the matrix evolutions in the start via numpy einsum. Since M0 is in higher a dimensions, one should not do a numpy transpose, but swap/roll the outer M0 6x1 axis. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Shortened the code dramatically for [[NS R1rho 2-site]]. It is possible to calculate all in "one" go, after having the transposed/rolled-back M0 magnetization. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Speeded up the code of [[NS R1rho 2-site]]. This was essential done to numpy einsum, and doing the dot operations in multiple dimensions. It was though necessary to realize, that to do the proper dot product operations, the outer two axis if M0 should be swapped, by rolling the outer axis one back. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Speeded up the code of [[NS R1rho 3-site]]. This was essential done to numpy einsum, and doing the dot operations in multiple dimensions. It was though necessary to realize, that to do the proper dot product operations, the outer two axis if M0 should be swapped, by rolling the outer axis one back. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* For model [[NS CPMG 2-site 3D]], the M0 matrix was preformed for higher dimensionality in init of target function. The transposes/rolled axis versions was also initiated. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Swapped the dot product position, when propagating the magnetisation in model [[NS CPMG 2-site 3D]]. This it to try to align to same method as in [[NS R1rho 2-site]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Lowered the looping in [[NS CPMG 2-site 3D]], by preforming the initial dot product. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Speeded up [[NS CPMG 2-site 3D]], by preforming the magnetisation. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Got rid of the inner evolution of the magnetization. If the looping over the number of CPMG elements is given by the index l, and the initial magnetization has been formed, then the number of times for propagation of magnetization is l = power_si_mi_di-1. If the magnetization matrix "Mint" has the index Mint_(i,k) and the evolution matrix has the index Evol_(k,j), i=1, k=7, j=7 then the dot product is given by: Sum_{k=1}^{k} Mint_(1,k) * Evol_(k,j) = D_(1, j). The numpy einsum formula for this would be: einsum('ik,kj -> ij', Mint, Evol). Following evolution will be: Sum_{k=1}^{k} D_(1, j) * Evol_(k,j) = Mint_(1,k) * Evol_(k,j) * Evol_(k,j). We can then realize, that the evolution matrix can be raised to the power l. Evol_P = Evol**l. It will then be: einsum('ik,kj -> ij', Mint, Evol_P). Get which power to raise the matrix to. l = power_si_mi_di-1. Raise the square evolution matrix to the power l. evolution_matrix_T_pwer_i = matrix_power(evolution_matrix_T_i, l), Mint_T_i = dot(Mint_T_i, evolution_matrix_T_pwer_i) or Mint_T_i = einsum('ik,kj -> ij', Mint_T_i, evolution_matrix_T_pwer_i). Task #7807 (https[https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Tried to implement using lib.linear_algebra.matrix_power.square_matrix_power instead of matrix_power from numpy in [[NS CPMG 2-site 3D]]. Strangely, then system test: test_hansen_cpmg_data_to_ns_cpmg_2site_3D_full starts to fail. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Changes to unit test of [[NS CPMG 2-site 3D]]. This is after the new initiated M0 matrix in init of target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Double speed in [[NS CPMG 2-site star]], after using numpy.linalg.matrix_power instead of the lib version in relax. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Triple speed in [[NS MMQ 2-site]], after using numpy.linalg.matrix_power instead of the lib version in relax. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Small fix for making sure that power is a integer in [[NS MMQ 2-site]]. Following system tests was failing: Relax_disp.test_korzhnev_2005_15n_dq_data, Relax_disp.test_korzhnev_2005_15n_sq_data, Relax_disp.test_korzhnev_2005_15n_zq_data, Relax_disp.test_korzhnev_2005_1h_sq_data, Relax_disp.test_korzhnev_2005_all_data, Relax_disp.test_korzhnev_2005_all_data_disp_speed_bug. They should already be integers, but is now solved. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Comment and spell fixing in [[NS CPMG 2-site 3D]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Triple speed in [[NS MMQ 3-site]], after using numpy.linalg.matrix_power instead of the lib version in relax. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Updated the dispersion model profiling comparison of the disp_spin_speed branch vs. relax-3.2.2. This now includes all dispersion models. This shows the large speed increases in the numeric and MMQ models recently obtained. Note that something went wrong with the [[NS CPMG 2-site 3D full]] model for the clustered analysis, most times were around 24 seconds except for the first which was strangely 292 seconds.
* Updated the relaxation dispersion model profiling log file for relax version 3.2.2 vs. 3.2.1. This adds the MMQ and 3-site models to the log file. The new information shows that there was a 4.2 times speed up for the [[MMQ CR72]] model between these two relax versions, both for single spins and clustered spins, a much greater improvement than any other of the models.
* Removed the now redundant disp_profile_all_3.2.2.table.txt dispersion model profiling table. As the dispersion model profiling master script now covers all dispersion models, the output from this script produces this table exactly. Therefore the end of the log files saved from running this script contains this table.
* Initiated lengthy profiling script, that shows that doing square numpy matrix_power on strided data, can speed up the calculation by factor 1.5. The profiling script can quickly be turned into a unit test, and includes small helper functions to calculate how to stride through the data. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* First try to implement function that will calculate the matrix exponential by striding through data. Interestingly, it does not work. These system tests will fail: test_hansen_cpmg_data_to_ns_cpmg_2site_3D, test_hansen_cpmg_data_to_ns_cpmg_2site_3D_full. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Added matrix_power to the init file in lib/dispersion. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Deleted the printout in dep_check. The printouts are only used for the essential packages before calling sys.exit(). Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added the missing "self.num_exp" to target function. Testing on older system, this was failing the system test. It is a wonder how these lines in __init__ could be performed without this.
* Fix for unit test passing on old numpy systems. The error was: ValueError: setting an array element with a sequence. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Expanded the dispersion target function class documentation. The NE, NS, NM, NO, and ND notation is now explained.
* Added Ti and NT to the dispersion target function class documentation.
* Slight speed up of the [[B14]] and [[B14 full]] dispersion models by minimising repetitive maths.
* Initial try to write up a 2x2 matrix by closed form. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Made the validation check in profiling of matrix_power check all values. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Replaced all self.spins with self.NS in target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Replaced all self.num_exp with self.NE in target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Replaced all self.num_frq with self.NM in target function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* A very small speed up to the [[CR72]] dispersion models by minimising repetitive maths operations. The k_BA and k_AB rates are used to simplify the Psi calculation, dropping from 3 to 2 multiplications and removing a squaring operation. The Dpos and Dneg value calculations have been simplified to drop one multiplication operation. And the calculation of eta_scale / cpmg_frqs now only occurs once rather than twice.
* Removal of a tonne of unused imports in the lib.dispersion package. These were identified using the command "pylint * --disable=all --enable=unused-import".
* A very small speed up to the [[MMQ CR72]] dispersion model by minimising repetitive maths operations. This matches the recent change for the [[CR72]] model, though the Psi calculation was already using the fast form.
* Created a master profiling script for comparing the speed between different dispersion models. This is similar to the disp_profile_all.py script except it only operates on a single relax version. The output is then simply the timings, with statistics, of the calculation time for 100 function calls for 100 spins (either 10,000 function calls for single spins or 100 function calls for the cluster of 100 spins). The output of the script for the current disp_spin_speed branch code has also been added.
* Critical fix for the recalculation of tau cpmg, when plotting for numerical models. The interpolated dispersion points with tau_cpmg was calculated with frq, instead of cpmg_frq. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* The new dispersion model profiling master script now includes links to the relax wiki. The models are no longer presented by name but rather by the relax wiki links for each model (see http://wiki.nmr-relax.com/Category:Relaxation_dispersion for all these links). This is to improve the Google rank of the relax wiki, as these links may appear in a number of locations.
* Removal of many unused imports in the disp_spin_speed branch. These were detected using the devel_scripts/find_unused_imports.py script which uses pylint to find all unused imports. The false positives also present in the trunk were ignored.
* Code validation of lib/dispersion/b14.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/cr72.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/dpl94.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/lm63_3site.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/lm63.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/m61b.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/m61.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/matrix_exponential. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/mp05.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/ns_cpmg_2site_expanded.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/ns_cpmg_2site_star.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/ns_mmq_2site.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/ns_mmq_3site.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/ns_r1rho_2site.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/ns_r1rho_3site.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/tap03.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/tp02.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of lib/dispersion/two_point.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Code validation of target_functions/relax_disp.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* For model [[NS MMQ 3-site]], moved the parameter conversion of dw_AB from target function to lib function. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed chi sum initialisation in func_ns_mmq_2site() as this is not used. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Documentation fix for the get_back_calc() function in target_function/relax_disp.py. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed unnecessary repetitive calculation of kex**2 in model [[DPL94]]. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.* API documentation fixes, where a "\" is the last character on the line. There should be a space " ", ending this character. Task #7807 ([https://gna.org/task/?7807)Task #7807]: Speed-up of dispersion models for clustered analysis.
* Updated the [https://gna.org/forum/forum.php?forum_id=2472 minfx version number to 1.0.9] in the release checklist document. This as of yet unreleased version contains an important fix for parallelised grid searches when the number of increments is set to one (i.e. a preset parameter).
* Fix for the _prompt.test_align_tensor.Test_align_tensor.test_init_argfail_params unit test. As the alignment tensor can now be initialised as None, the None value can be accepted and a different RelaxError is raised when the params argument is incorrectly supplied.
* Expansion of the 'to do' section of the dispersion chapter of the manual.
* Editing of the 'to do' section of the dispersion chapter of the manual.
* Split out the interpolating in specific_analyses.relax_disp.data.plot_disp_curves() into separate function. This is to prepare for a interpolating function for spin-lock offset rather than spin-lock field strength for R1rho models. Sr #3124 (https://gna.org/support/?3124): Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. Sr #3138 * Split out the interpolating in specific_analyses.relax_disp.data.plot_disp_curves() into separate function. This is to prepare for a interpolating function for spin-lock offset rather than spin-lock field strength for R1rho models. [https://gna.org/support/?3124 Sr #3124]: Grace graphs production for R1rho analysis with R2eff as function of Omega_eff. [https:
* Fix for system test Relax_disp.test_kteilum_fmpoulsen_makke_check_graphs where minimise has been extended with execute.
* Changed graphs after new minimisation algorithm has been implemented. The values are now slightly different.
* Fix to unit tests, after changing the name of matrix_exponential function.
* Added graphs and results for run with MC=2000, for system test Relax_disp.test_r1rho_kjaergaard_auto(). This is to be able to extend graph testing for interpolated R1rho graphs, and to add figures to the latex manual.
* Added list of R1rho models, which use R1 in their equations. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models. Bug #21788 ([https://gna.org/bugs/?21788)Bug #21788]: Only Warning is raised for missing loading R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Modified the warning and error messages being raised when calling return_r1_data(). Now warnings is raised if no R1 data is available. An error is raised if the R1rho model is expected to have R1 data, and it is not available. That makes system test Relax_disp.test_r1rho_kjaergaard_missing_r1() fail, which is the expected behaviour. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models. Bug #21788 ([https://gna.org/bugs/?21788)Bug #21788]: Only Warning is raised for missing loading R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added the model "MODEL_DPL94_FIT_R1", to the full list of models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* In system test Relax_disp.test_r1rho_kjaergaard_missing_r1(), started using the new model MODEL_DPL94_FIT_R1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added "r1_fit" as a parameter object. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added functionality to return r1_fit parameter in loop_parameters() function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added model variable MODEL_DPL94_FIT_R1, to relax_disp target function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Split the target function of model [[DPL94]] into a func_DPL94 and calc_DPL94. This is to prepare for a target function func_DPL94_fit_r1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added initial target function for model DPL94_fit_r1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added list of R1rho models, which can fit R1 in their equations. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Modified return_r1_data() function, to return numpy array of "None", if model is in list of "MODEL_LIST_R1RHO_FIT_R1". Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Modified target function func_DPL94_fit_r1(), to unpack fitted parameters correct. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added linear linear_constraints for parameter "r1_fit". Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added warning message when using function return_r1_data(), and model is in list MODEL_LIST_R1RHO_FIT_R1. Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added model DPL94_FIT_R1 to the list of MODEL_LIST_R1RHO and MODEL_LIST_R1RHO_FULL. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Modified return_r1_data(), to be dependent on fitting model. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Made function return_r1_err_data() be dependent on model type. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Replaced instances of "['r2', 'r2a', 'r2b']" with variable PARAMS_R20. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented model list, which uses parameter of inverted relax delay times. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented model list, which uses parameter of R20B. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix in target function for relax_disp, where model IT99 does not belong to model list with several chemical shift correlated parameters. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added new variable, for models which has parameters with mixed dw, and has two variables. For example with both dw and dwH or dw_AB and dw_BC or phi_ex_B and phi_ex_C. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added model MODEL_NS_R1RHO_3SITE, MODEL_NS_R1RHO_3SITE_LINEAR, to list of models who has who dw parameters. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added new variable, for models which has parameters with mixed dw, and has four variables Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added to relax_disp auto_analyses, that R1_fit should be plotted and written out. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added 2 new models, MODEL_NOREX_R1RHO and MODEL_NOREX_R1RHO_FIT_R1. The "NOREX" model is not covering R1rho models. The target function for "NOREX" is calculated as: back_calc = R20 R20 is for R1hro models equivalent to R1rho prime (R1rho'), which for example in the model of [[DPL94]] would mean: R1rho = R1rho' But for the "NOREX" case, the return should be R1rho = R1 * cos^2(theta) + (R1rho' + 0) * sin^2(theta). This affects all off-resonance model calculations. These two target functions will be implemented. Bug #22440 ([https://gna.org/bugs/?22440)Bug #22440]: The "NOREX" model is not covering R1rho models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Made the model MODEL_NOREX_R1RHO_FIT_R1, be tested in system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). Bug #22440 ([https://gna.org/bugs/?22440)Bug #22440]: The "NOREX" model is not covering R1rho models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added models MODEL_NOREX_R1RHO and MODEL_NOREX_R1RHO_FIT_R1 to MODEL_LIST_FULL. Bug #22440 ([https://gna.org/bugs/?22440)Bug #22440]: The "NOREX" model is not covering R1rho models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented target and calculation function for MODEL_NOREX_R1RHO, MODEL_NOREX_R1RHO_FIT_R1. Bug #22440 ([https://gna.org/bugs/?22440)Bug #22440]: The "NOREX" model is not covering R1rho models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Made the model "MODEL_NOREX_R1RHO", be testes in system test Relax_disp.test_r1rho_kjaergaard_auto. This is for system test where R1 has been loaded from earlier results, which was not analysed in relax. Bug #22440 (https[https://gna.org/bugs/?22440)Bug #22440]: The "NOREX" model is not covering R1rho models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Made list of models which fit pA or pA and pB. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added function to auto_analysis, to test if it give meaning to write and plot out the parameter. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Made the writing out of parameter pC, be tested with the new function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented models list with phi_ex, phi_ex_B, and phi_ex_C, and added to test in auto_analyses of relax_disp. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Made use of the dictionary MODEL_PARAMS, to determine if parameter is present. This makes the list of models belonging to parameter lists superfluous. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Removed unnecessary list of models which support a parameter. This functionality already exists with the dictionary MODEL_PARAMS. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Rearranged the writing out of parameters in auto_analysis of relax_disp. This is to prevent writing out all possible parameters in the final round, if any of those parameters have not been tested. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Extended the writing and checking of parameters, to use different file name, than the parameter name. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Moved the auto-analyses writing out of w_eff and theta into check for has_r1rho_exp_type(). Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Re-arranged all model variable lists, to be able to re-use earlier lists. This is to prevent user errors, when setting up the lists, and re-use the lists through all code. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for MODEL_NOREX_R1RHO_FIT_R1 not being part of list: MODEL_LIST_DISP. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Moved the auto_analyses plot of special R1rho graphs into the check of has_r1rho_exp_type(). Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Moved the auto-analyses plotting and writing of r2, r2a and r2b for CPMG models into test of has_cpmg_exp_type(). Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Replaced in auto analysis all instances of [[No Rex]] and 'R2eff' with its equivalent defined variables. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Re-arranged plotting and writing in auto-analyses of relax disp, when model is R2eff. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Removed unused variables in auto-analyses of relax_disp. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented the list "MODEL_LIST_NEST", which define which model are used for nesting. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for the nested copying of r2, if using a nested list. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added the equivalent R1 fit models for: [[TP02]], [[TAP03]], [[MP05]] and [[NS R1rho 2-site]]. The R1 fit models will no be implemented for 3-site models, because there will be to many variables. Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added the new R1 fit models to system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Split target function of model [[TP02]], into a calc and two func_TP02* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for error checking covering R1rho off resonance models in target function. This is for checking presence of chemical shifts and r1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Split target function of model [[TAP03]], into a calc and two func_TAP03* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Split target function of model [[MP05]], into a calc and two func_MP05* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for system test Relax_disp.test_r1rho_kjaergaard_auto_check_graphs(), where the special R1rho graphs is no longer produced for the R2eff models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Split target function of model ns_r1rho_2site, into a calc and two func_ns_r1rho_2site* variants. One target function will use measured R1 values, while one target function will use the fitted R1 values. They will use the same calculation function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Removed num_points to be used in target and lib function of model ns_r1rho_2site. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added a variable describing the model year, for all relaxation dispersion models. This could be used, when trying to write up an intelligent detect+select nesting function. This function need some meta-data describing the models, in order to sort the self.models before calculations, and to select a proper nested model pipe. Other meta data could be: Accept of Exp_type, full or normal model for CPMG, fitted or loaded R1 for R1rho, is analytic, silico or numeric type. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added initial unit test class for testing specific_analysis.relax_disp.variables. There will be more tests added, when a nesting selection function has been written here. Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added dictionary for returning year, when using model as key. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added unit test for the dictionary of model years. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Reused the EXP_TYPE_LIST_CPMG and EXP_TYPE_LIST_R1RHO, to combine for the list EXP_TYPE_LIST, the list of all dispersion experiment types. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added a common EXP_TYPE_CPMG MMQ description for models which handle MMQ. This is part of adding meta data for each model, making it possible to device a sensible nesting selection function. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added meta data about EXP_TYPE per model, and made a dictionary for it. Added unit test for the new dictionary. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added three new EXP_TYPE variables: EXP_TYPE_R2EFF = 'R2eff/R1rho', EXP_TYPE_NOREX = 'No Rex', EXP_TYPE_NOREX_R1RHO = 'No Rex: R1rho off res'. These are used to add meta-data information to each model, making it possible to make a nesting function, determining which model to nest from. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added model meta information about number of chemical exchange sites. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added meta information about equation type. The models are divided into: analytic, silico or numeric. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented nesting function, which will determine which model to nest from. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* In auto analysis of relax_disp, started implementing the new nesting function. Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Modified nesting function to return all model info for the current model, and the comparable model. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for nesting kex, when model is [[CR72]], and analysed models is [[LM63]]. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for nesting kex, when model is [[CR72]], and analysed models is IT99. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added a return from the nesting model, if all fails. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Replaced the test, if acquiring the model info to a numerical model from a analytical model. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Improved the printing when nesting parameters from equivalent models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Removed unused import of models in auto-analyses of relax_disp. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Improved the printing of system test Relax_disp.test_r1rho_kjaergaard_missing_r1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Improved the printing of system test Relax_disp.test_r1rho_kjaergaard_missing_r1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added function to sort models before auto-analyses. They are sorted in order: exp_type: EXP_TYPE_R2EFF, EXP_TYPE_NOREX, EXP_TYPE_NOREX_R1RHO, EXP_TYPE_CPMG_SQ, EXP_TYPE_CPMG_MMQ, EXP_TYPE_R1RHO; equation: EQ_SILICO, EQ_ANALYTIC, EQ_NUMERIC; Nr of chemical sites: 2 or 3; Year: Newest models first; Nr of parameters. Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added unit test, to test the expected sorting of models for auto-analyses. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added more models to be tested in system test Relax_disp.test_r1rho_kjaergaard_auto. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented the sorting of models, for auto-analyses. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Implemented partial reading of results file. Before reading a results file, it is determined if the file exists. This makes is possible to read a directory with partial results from a previous analysis. This can be handsome, if reading R2eff values in R1rho experiments, and the error estimation has been prepared with a high number of Monte Carlo simulations. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added keyword to relax_disp auto analysis, if R2eff values should be optimised. Here optimisation means minimisation and Monte Carlo simulations of the error. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Modified system test Relax_disp.test_r1rho_kjaergaard_missing_r1 to load previous R2eff values, and not optimise them. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix in back end for relax_disp.parameter_copy, where r2a and r2b should be skipped, since it has already been copied. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Locked values in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. This is possible after locking the R2eff values and errors from a previous run. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Removed MODEL_NS_CPMG_2SITE_EXPANDED to be analysed in system test test_hansen_cpmg_data_missing_auto_analysis. The new ordering of models, will make MODEL_NS_CPMG_2SITE_EXPANDED be analysed first, and results copied to model [[CR72]]. This will interfere with the old results. Sr #3135 (https[https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Lowered the precision of dw for model [[NS CPMG 2-site expanded]], in system test est_hansen_cpmg_data_auto_analysis_numeric. Model [[NS CPMG 2-site expanded]] is now analysed before MODEL_CR72, which alter the values a bit. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added GUI text for parameter r1_fit. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added front-end description of the 6 new R1 fit R1rho models for relax_disp.select_model. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added a paragraph for the no chemical exchange model in help text description for selecting models. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added the new R1rho models where R1 is fitted, to the GUI model selection. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Deleted system test test_r1rho_kjaergaard_man, since it was no necessary. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Deleted unused script files in data folder for Kjaergaard_et_al_2013. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for the linear constraints where parameter r1_fit was written as R1_fit. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Fix for the numbers of parameters not getting counted correct. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Triggering an error in test_r1rho_kjaergaard_missing_r1. There is a bug fetching the standard value of parameter 'r1_fit'. AttributeError: 'float' object has no attribute 'keys'. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Removed unused scripts in folder of Kjaergaard et al., 2013. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.* Added new analysis scripts in folder of Kjaergaard et al., 2013. Sr #3135 ([https://gna.org/support/?3135)Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Removed triggering an error in test_r1rho_kjaergaard_missing_r1. There is a bug fetching the standard value of parameter 'r1_fit'.
* Fix for system test Relax_disp.test_r1rho_kjaergaard_missing_r1, running on 64 bit system.
* Inserted LaTeX bibliography for reference to linear constraints of the exchange rate.
* Equation fix for $kex$ in manual.
* Added system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis and data. This is for bug #22477 ([https://gna.org/bugs/?22477)bug #22477]: Not possible to perform grace.write() on k_AB parameter for mixed CPMG analysis.
* Added more printout to system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis and data.
* Fix for system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis. Bug #22477 ([https://gna.org/bugs/?22477)Bug #22477]: Not possible to perform grace.write() on k_AB parameter for mixed CPMG analysis.
* Set the default value of r1_fit to 5.0.
* Added the relax_disp pipe type to be setup for unit tests of value function.
* Setup a unit test for the value.set functionality for param r1_fit. Bug #22470 ([https://gna.org/bugs/?22470)Bug #22470]: value.set does not work for parameter r1_fit.* Fix in relax_disp API, how to handle the r1_fit parameter type. Bug #22470 ([https://gna.org/bugs/?22470)Bug #22470]: value.set does not work for parameter r1_fit.* Modified system test Relax_disp.test_r1rho_kjaergaard_missing_r1 to use GRID_INC=None, and thereby speeding up the analysis. Bug #22470 ([https://gna.org/bugs/?22470)Bug #22470]: value.set does not work for parameter r1_fit.* Added to system test a count of number of headers and values, when issuing a value.write(). Sr #3121 ([https://gna.org/support/?3121)Sr #3121]: Support request for replacing space in header files for the value.write functions.* Fix for replacing spaces " " with "_" in header files. Sr #3121 ([https://gna.org/support/?3121)Sr #3121]: Support request for replacing space in header files for the value.write functions.
* Fix for comment, which mentions R2 parameter, when it relates to R1 fit.
* Replaced variable name: MODEL_PARAM_INV_RELAX_TIMES with MODEL_LIST_INV_RELAX_TIMES, to match all of the other MODEL_LIST_* variables. Also added a newline to end of file.
* Removed the unit test regarding model.py in test_variables.py.
* Added unit tests regarding model.py and its functions.
* Removed the auto-sorting of models, when performing auto analysis of Relaxation dispersion. This was discussed in: http://thread.gmane.org/gmane.science.nmr.relax.scm/22733, http://thread.gmane.org/gmane.science.nmr.relax.scm/22734, http://thread.gmane.org/gmane.science.nmr.relax.scm/22737. Through this discussion, it has appeared that the order of how models are sorted for analysis, and hence the possibility for nesting, is a complicated case. The order of analysis should be possible to manually put into the auto analysis. This was not the scope of sr #3135 ([https://gna.org/support/?3135)sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models to implement such a feature. Such a feature could be implemented for the next version of relax. It could be designed as function to "suggest" an order in the GUI. But this functionality would have to wait.
* Removing the 'tex'->'kex' conversion, and 'phi_ex' from dw and pA. This solution is not a proper implementation, but these parameters should rather be found by grid search.
* Re-inserted "MODEL_NS_CPMG_2SITE_EXPANDED" to be tested in system test test_hansen_cpmg_data_missing_auto_analysis.
* Added the R1 parameter fitting GUI element to the dispersion GUI tab. This is a simple Boolean toggle element that allows the R1 optimisation to be turned on. The value is passed into the auto-analysis.
* Added the r1_fit argument to the relaxation dispersion auto-analysis. When this is True, the relax_disp.r1_fit user function will be called to turn R1 parameter fitting on.
* Added the relax_disp.spin_lock_offset user function to the dispersion GUI. This has been added to the pop up menu in the spectrum list GUI element, when the relax_disp_flag has been set. It simply mimics the relax_disp.spin_lock_field functionality already present. This follows from task #7820 ([https://gna.org/task/?7820)task #7820].
* Fix for the relax_disp.spin_lock_offset user function in the dispersion GUI tab. This is in the spectrum list element popup menu.
* Added the offset column to the spectrum list GUI element for the dispersion analysis. This is to complete task #7820 ([https://gna.org/task/?7820)task #7820]. The spectrum list GUI element add_offset() method has been added to insert the offset column when the relax_disp_flag is set. This is called by the update_data() method to fill and update the GUI element.* Implemented the GUI test Relax_disp.test_bug_22501_close_all_analyse to catch bug #22501 ([https://gna.org/bugs/?22501)bug #22501], 'Close all analyses' raises error.
* Inserted intermediate system test, to profile R2eff calculation for R1rho. System test is: Relax_disp.test_bug_9999_slow_r1rho_r2eff_error_with_mc. This system test actually fails, if one tries to do a grid search. This is related to the R2eff values stored as dictionary, and pipe_control.minimise.grid_setup() will fail. The function 'isNaN(values[i])' cannot handle dictionary.
* Modified intermediate system test Relax_disp.test_bug_9999_slow_r1rho_r2eff_error_with_mc to see if the initial grid search for 'i0' and 'R2eff' estimation can be skipped. This is done by converting the exponential curve, to a linear curve, and calculate the best parameters by a line of best fit by least squares. This seems like a promising method as an initial estimator of 'i0' and 'r2eff'. For 500 to 2000 Monte Carlo simulations, this could have an influence on the timings, as all grid searchs could be skipped.
* Added start script with basic data for profiling the relax curve fit.
* Created the Structure.test_create_diff_tensor_pdb system test. This is to show the failure of the structure.create_diff_tensor_pdb user function when no structural data is present.
* Created the Structure.test_create_diff_tensor_pdb2 system test. This is to catch another situation leading to bug #22505 ([https://gna.org/bugs/?22505)bug #22505], the failure of the structure.create_diff_tensor_pdb user function when no structural data is present.* Added an optimisation script for the test_suite/shared_data/diffusion_tensor/ellipsoid relaxation data. This is to help catch bug #22502 ([https://gna.org/bugs/?22502)bug #22502], the geometric prolate diffusion representation does not align with axis in PDB, as reported by Martin Ballaschk (https://gna.org/users/mab). The PDB files of the optimised tensor demonstrate exactly the same problem as seen in the files attached to the bug report. The oblate and spherical diffusion tensor representations match that of the ellipsoid. But the prolate axis and tensor orientation are both different from the ellipsoid as well as themselves.
* Updated the diffusion tensor PDB representation files. This replaces the broken prolate representation with the corrected representation.
* Deleted the duplicated Structure.test_create_diff_tensor_pdb system test.
* Created a number of system tests to check the diffusion tensor PDB representation. This is to prevent bugs such as #22502 ([https://gna.org/bugs/?22502) bugs such as #22502] from ever reappearing. The PDB file contents are hardcoded into the tests and checked. The tests include Structure.test_create_diff_tensor_pdb_ellipsoid, Structure.test_create_diff_tensor_pdb_oblate, Structure.test_create_diff_tensor_pdb_prolate, and Structure.test_create_diff_tensor_pdb_sphere.
* Improved data checking for all of the Structure system tests. Before looping over the structural data, the number of lines in the real file and the newly generated file are compared. This avoids the situation whereby an empty file is produced, accidentally allowing the test to pass.
* Modified following functions (time points are now saved at the [ei][mi][oi][di] index level, at this index level all time points are saved for the R2eff point): interpolate_disp(), to interpolate time points, all time points through the original dispersion points di, are collected and then made unique - this time list can potentially be the largest of all time lists; interpolate_offset(), to interpolate time points, all time points through the original offset points, and then dispersion points di, are collected and then made unique - this time list can potentially be the largest of all time lists; plot_disp_curves_to_file(), to acquire the original relax_times points; return_r2eff_arrays(), to save all time points on the level of [ei][mi][oi][di]. At this index level, it will be a numpy array list with all time values used for fitting. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.* Modified back_calc_r2eff() to accept interpolated time points. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.* Modified target function of relax dispersion, to use the new list of time points, which are of higher dimension. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.* Fix to system test Relax_disp.test_r1rho_kjaergaard_missing_r1(). After the relaxation times have been fixed, this model now return reasonable chi2 values. The reported parameters are though quite different from all other models, and it seems something may still be wrong. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.* Fix for system test Relax_disp.test_exp_fit(), where the spin.isotope was not set. The new call to return_r2eff_arrays(), when producing graphs, raise RelaxSpinTypeError() if no isotope is set. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
* Modified the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test to pass on 64-bit Linux systems. The accuracy of the checks of the optimised values have been decreased.
* Moved the storing of relax time up before check of missing data in return_r2eff_arrays(). Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.* Fix for system test not adding spin.isotope to setup information. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.* Fix for looping over data indices, where tilt_angles has the si index. Bug #22461 ([https://gna.org/bugs/?22461)Bug #22461]: NS R1rho 2-site_fit_r1 has extremely high chi2 value in system test Relax_disp.test_r1rho_kjaergaard_missing_r1.
* Added Nikolai's original Matlab code to the lib.dispersion.ns_r1rho_2site module docstring. This is the code taken directly form the original funNumrho.m file, which was the origin of the code in this module.
* Further extended the profiling script for curve fitting. Now profiling is in place for the implemented C code method in relax. A similar code should now be devised for numpy array for comparing. But this profiling shows that when contraints=True, is slowing down this procedure by a factor 10 X.
* Further improved the profiling of relax curve fit. This profiling shows, that Python code is about twice as slow as the C code implemented. But it also shows that optimising with scipy.optimize.leastsq is 20 X faster. It also gives reasonable error values. Combining a function for a linear fit to guess the initial values, together with scipy optimise, will be an extreme time win for estimating R2eff values fast. A further test would be to use relax Monte Carlo simulations for say 1000-2000 iterations, and compare to the errors extracted from estimated covariance.
* Added verification script, that shows that using scipy.optimize.leastsq reaches the exact same parameters as minfx for exponential curve fitting. The profiling shows that scipy.optimize.leastsq is 10X as fast as using minfx (with no linear constraints). scipy.optimize.leastsq is a wrapper around wrapper around MINPACK's lmdif and lmder algorithms. MINPACK is a FORTRAN90 library which solves systems of nonlinear equations, or carries out the least squares minimization of the residual of a set of linear or nonlinear equations. The verification script also shows, that a very heavy and time consuming Monte Carlo simulation of 2000 steps, reaches the same errors as the errors reported by scipy.optimize.leastsq. The return from scipy.optimize.leastsq, gives the estimated covariance. Taking the square root of the covariance corresponds with 2X error reported by minfx. This could be an extremely time saving step, when performing model fitting in R1rho, where the errors of the R2eff values, are estimated by Monte Carlo simulations. The following setup illustrates the problem. This was analysed on a MacBook Pro, 13-inch, Late 2011 with no multi-core setup. Script running is: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. This script analyses just the R2eff values for 15 residues. It estimates the errors of R2eff based on 2000 Monte Carlo simulations. For each residues, there is 14 exponential graphs. The script was broken after 35 simulations. This was measured to 20 minutes. So 500 simulations would take about 4.8 Hours. The R2eff values and errors can by scipy.optimize.leastsq can instead be calculated in: 15 residues * 0.02 seconds = 0.3 seconds.
* Moved the target function for minimisation of exponential fit into the target functions folder. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented initial system test Relax_disp.test_estimate_r2eff for setting up the new user function to estimate R2eff and errors by scipy. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added front end user function relax_disp.r2eff_estimate to estimate R2eff and errors by exponential curve fitting in scipy.optimize.leastsq. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified check for model, to accept model as input, for error printing. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented back end for estimating R2eff and errors by exponential curve fitting with scipy.optimize.leastsq. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Documentation fix for new exponential target function. Also added new function to estimate R2eff and i0 parameters, before minimisation. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Small changes to verification scripts, to use chi2 function and use the scaling matrix correct. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Split up of system test test_r1rho_kjaergaard_missing_r1, into a verification part. This is to test the new r2eff estimation, which should get the parameter values, as a this 2000 Monto Carlo simulation. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Modified system test Relax_disp.test_estimate_r2eff. This is to compare against errors simulated with 2000 MC. The parameters are comparable, but not equal. Mostly, it seems that the errors from scipy.optimize.leastsq, are twice as high than the Monte Carlo simulations. This affect model fitting, and the calculated chi2 value.
* Added system test Relax_disp.test_estimate_r2eff_error(). This is to get insight in the error difference between 2000 Monto Carlo simulations and then scipy.optimize.leastsq.
* Add dependency check for scipy.optimize.leastsq. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Lowering precision in system test Relax_disp.test_r1rho_kjaergaard_missing_r1. This is R1 estimation with MODEL_NS_R1RHO_2SITE. The lowering of precision is due different system precision.
* Reused the dependency check "scipy_module", since leastsq() has been part of Scipy since 2003.
* Isolated all code related to user function relax_disp.r2eff_estimate into independent module file. All has been isolated to: specific_analyses.relax_disp.estimate_r2eff.
* Split function to minimise with scipy.optimize.leastsq out in estimate_r2eff module. This is to prepare for implementing with minfx.
* Implemented first try to minimise with minfx in estimate_r2eff() function. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Implementation of the target_functions.relax_fit.jacobian() function. This follows from the discussions at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807. The function will calculate the Jacobian matrix for the exponential curve-fitting module. The Jacobian can be used to directly calculate the covariance matrix, for example as described at https://www.gnu.org/software/gsl/manual/html_node/Computing-the-covariance-matrix-of-best-fit-parameters.html. The Jacobian is calculated using the help of the new exponential_dI() and exponential_dR() functions in the target_functions/exponential.c file. These calculate the partial derivatives of the exponential curve with respect to each model parameter separately. The implementation still needs testing and debugging.
* Fixes for the new target_functions.relax_fit.jacobian() function. The Python list of lists is now correctly created and returned.
* Turned off the optimisation constraints for the 'R2eff' model in the dispersion auto-analysis. This follows from http://thread.gmane.org/gmane.science.nmr.relax.scm/22977/focus=6829. This model does not require constraints at all, and the constraints only cause the optimisation to take 10x longer to complete. Therefore the constraint flag has been set to False for the model.
* Initial try to form the Jacobian and Hessian matrix for exponential decay. This can be tried with system test: relax -s Relax_disp.test_estimate_r2eff_error. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Intermediate step in estimate R2eff module. It seems that minfx is minimising in a quadratic space because of the power of chi2, while the general input to scipy.optimize does not do this. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Cleaned up target function for leastsq, since arguments to function can be extracted from class. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Tried to implement with scipy.optimize.fmin_ncg and scipy.optimize.fmin_cg, but cannot get it to work. The matrices are not aligned well. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Implemented the chi-squared gradient as a C module for the target functions. This simply translates the Python code into C to allow any target function C modules to build its own gradient function.
* Implemented the target_functions.relax_fit.dfunc() gradient function. This is using the Python/C interface to provide a Python function for calculating and returned the chi-squared gradient for the exponential curves.
* The parameter index is now passed into exponential_dI0() and exponential_dR(). This is for the relaxation curve-fitting C module so that the indices are not hardcoded.
* The I0 and R parameter indices are now defined in the target_function/relax_fit.h header file. This is to abstract the exponential curve parameter indices even more.
* Big cleanup of estimate R2eff module. This is to make the documentation more easy to read and understand. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Created 2 unit tests for the target_functions.relax_fit relax C module. These check the func() and dfunc() Python methods exposed by the module.
* The relax_fit C module unit tests now check if the parameter scaling is functional.
* Added several comments to the R2eff estimate module. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Added a script and log file for calculating the numerical gradient for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the gradient using the scipy.misc.derivative() function both at the minimum and at a point away from the minimum. The values will be used to construct a unit test to check the C module implementation.
* Created a unit test to check the dfunc() function of the relax_fit C module off the minimum.
* Fix for the test_dfunc_off_minimum() unit test. This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit. The wrong gradient was being scaled.
* Switched the optimisation algorithm in test_suite/system_tests/scripts/relax_fit.py. This script, used by the Relax_fit.test_curve_fitting_height and Relax_fit.test_curve_fitting_volume system tests, now uses the BFGS optimisation. This is to demonstrate that the exponential curve gradient function dfunc() is implemented correctly and that more advanced optimisation algorithms can be used (excluding those that require the full Hessian d2func() function).
* Got the method of 'Steepest descent' to work properly, by specifying the Jacobian correctly. The Jacobian was derived according to the chi2 function. The key point was to evaluate the two derivative arrays for all times points, and then sum each of the two arrays together, before constructing the Jacobian. This clearly shows the difference between minfx and scipy.optimize.leastsq. scipy.optimize.leastsq takes as input a function F(x0), which should return the array of weighted differences between function value and measured values: "1. / self.errors * (self.calc_exp(self.times, *params) - self.values)". This will be an array with number of elements 'i' corresponding to number of elements. scipy.optimize.leastsq then internally evaluates the sum of squares -> sum[ (O - E)**2 ], and minimises this. This is the chi2. Minfx requires the function to minimise before hand. So, the "func" should be chi2. Then the dfunc, and d2func, should be derivative of chi2, but all elements in the array should still be summed together. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Got the Quasi-Newton BFGS to work. This uses only the gradient, this gets the same results as 2000 Monte Carlo with simplex and scipy.optimize.leastsq. Error estimation still not provided. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Removed all code regarding scipy.optimize fmin_cg and fmin_ncg. This problem should soon be able to be solved with minfx. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added initial documentation for multifit_covar. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified profiling script to use the new estimate R2eff module. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified verify error script, to use new estimate R2eff module. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Removed all unnecessary code from estimate R2eff module. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* More removal of code. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Changed the array declarations in the target_functions/exponential C file and header. Instead of using the pointer format of *xyz, the array format of xyz[] is now being used. These are equivalent and the later is more obvious that this is an array.
* Changed the array declarations in the target_functions/c_chi2 C file and header. Instead of using the pointer format of *xyz, the array format of xyz[] is now being used. These are equivalent and the later is more obvious that this is an array.
* Fixes for the Hessian.py script for numerical integrating the Hessian for an exponential curve.
* Implemented two unit tests to check the Hessian of the target_functions.relax_fit.d2func() function. This compares the calculated Hessian to the numerically integrated values from the test_suite/shared_data/curve_fitting/numeric_gradient/Hessian.py script, showing that the d2func() function is implemented correctly.
* Modified profiling script, but it seems that the dfunc from target_functions.relax_fit does not work. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Modified estimate R2eff module, to use C code. But system test Relax_disp.test_estimate_r2eff_error shows that the Jacobian is not correctly implemented to be called in minfx.
* Created an initial test suite data directory for a mixed R1rho + CPMG dispersion analysis. The generate.py script will be extended in the future to generate both synthetic R1rho and CPMG data for a common exchange process. Such a data combination should show some minor flaws in the current design of the dispersion analysis and will help to solve these.
* Improvements to the pipe_control.minimise.reset_min_stats() function. The minimise statistics resetting is now more elegantly implemented. And the sim_index keyword argument is accepted by the function and individual Monte Carlo simulation elements can now be reset.
* Modified wrapper function for curve_fit, to only change to list type, if the type is a ndarray. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* The model-free reset_min_stats() function has been replaced with the pipe_control.minimise version. The specific_analyses.model_free.optimisation.reset_min_stats() function has been deleted and instead the pipe_control.minimise version is being used.
* Implemented the first try to compute the variance of R2eff and i0, by the covariance. This uses the Jacobian matrix. The errors calculated, are though way to small compared 2000 Monte Carlo simulations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Tried to implement the Jacobian from C code. This though also report errors which are to small. Maybe some scaling is wrong. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Modified profiling script to calculate timings. The timings for C code are: Simplex, with constraints = 2.192; Simplex, without constraints = 0.216; BFGS, without constraints = 0.079; Newton, without constraints = 0.031; This is pretty pretty fast. To this profiling script, I would also now add some verification on calculations.
* Tried to verify solution to profiling script. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Set the constraints=False when doing Monte Carlo simulations for R2eff. This is to speed up the Monte Carlo simulations by a factor X10, when estimating the error for R2eff. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented the use of "Newton" as minimisation algorithm for R2eff curve fitting instead of simplex. Running the test script: test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py. For 50 Monte Carlo simulations, the time drop from: 3 minutes and 13 s, to 1 min an 5 seconds. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Changed the relax_fit.py sample script to use Newton rather than Simplex optimisation. This can lead to significantly faster optimisation times, as shown in the commit message http://article.gmane.org/gmane.science.nmr.relax.scm/23081.
* Changed the optimisation description in the relaxation curve-fitting chapter of the manual. The script example has been converted to match the sample script, replacing the Nelder-Mead simplex algorithm with Newton optimisation, and removing the argument turning diagonal scaling off. All the text about only the simplex algorithm being supported due to the missing gradients and Hessians in the C module have been deleted. The text that linear constraints are not supported has also been removed - but this was fixed when the logarithmic barrier constraint algorithm was added to minfx.
* By using minfx, and the reported Jacobian, it is now possible to get the exact same error estimation as scipy.optimize.leastsq. The fatal error was to set the weighting matrix with diagonal elements as the error. There weights are 1/errors**2. There is though some unanswered questions left. The Jacobian used, is the direct derivative of the function. It is not the chi2 derivative Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fixed naming of functions, to better represent what they do in module of estimating R2eff. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented the Jacobian of exponential function in Python code. This now also gets the same error as leastsq and C code. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Tried to implement a safety test for linearly-dependent columns in the covariance matrix. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Fixes for the relax_disp.r2eff_estimate user function documentation. This is to allow the relax manual to compile again as the original documentation was causing LaTeX failures.
* Clean up of the declarations in the target_functions.relax_fit C module. The Python list objects are now declared at the start of the functions, and then PyList_New() is called later on. This allows the code to compile on certain Windows systems.
* Removed the user function to estimate the R2eff values and errors with scipy.optimize.leastsq. With the newly implemented Jacobian and Hessian of the exponential decay function, the front-end to scipy.optimize.leastsq does not serve a purpose. This is because minfx is now as fast as scipy.optimize.leastsq, and can estimate the errors from the Jacobian to the exact same numbers as scipy.optimize.leastsq. In addition to that, the covariance can be calculated by QR decomposition. This adds additional feature for checking for a singular matrix. The back-end will still be kept in place for the coming tim, but could be removed later. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added front-end to the new user function relax_disp.r2eff_err_estimate(), which will estimate the R2eff errors from a pipe and spins with optimised values of R2eff and i0. The covariance matrix can be calculated from the optimised parameters, and the Jacobian. Big care should be taken not to directly trust these results, since the errors are quite different compared to the Monte Carlo simulations. This implementation, will reach the exact same error estimation as scipy.optimize.leastsq. But with much better control over the data, and insight into the calculations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added method to automatically perform error analysis on peak heights. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified system test Relax_disp.test_estimate_r2eff() to first do a grid search, then minimise and then estimate the errors for R2eff and i0. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added back-end to estimate R2eff errors. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix to system test test_estimate_r2eff_error(), to first delete the old error estimations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added several tests to: test_estimate_r2eff_error, to compare different output from algorithms. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Cleaned up code in R2eff error module. Also removed a non working Hessian matrix. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Moved code around, and made function multifit_covar() independent of class object. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Inserted checks for C module is available in module for estimating R2eff error. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Removed unnecessary call to experimental Exp class. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Renamed system test, that test the user function for estimating the R2eff error: test_estimate_r2eff_err, test the user function. test_estimate_r2eff_err_methods, test different methods for getting the error. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added system test, Relax_disp.test_estimate_r2eff_err_auto and extended functionality to the auto-analyses protocol. If "exp_mc_sim_num" is set to "-1" and sent to the auto-analyses, the errors of R2eff will be estimated from the covariance matrix. These errors is HIGHLY likely to be wrong, but can be used in an initial test fase, to rapidly produce data for plotting data. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added script, to be used in GUI test. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added GUI test Relax_disp.test_r2eff_err_estimate, to test the setting of MC sim to -1 for exponential R2eff error estimation. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added keyword "exp_mc_sim_num", to the auto-analyses in the GUI. This sets the number of Monte Carlo simulations for R2eff error estimation in exponential curve fitting. When setting to -1, the errors are estimated from the covariance matrix. These errors are highly likely to be wrong, but can be used in Rapid testing of data and plotting. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Tried to click the "fit_r1" button in the GUI test, but receives an error: relax --gui-tests Relax_disp.test_r2eff_err_estimate, "AttributeError: 'SpinContainer' object has no attribute 'r1'". Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Moved the mc_sim_num GUI element in the analysis tab ip, as it is executed first. Also modified the tooltip. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added a warning to the auto-analyses about error estimation from the covariance. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Removed yet another comma from GUI tooltip. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Formatting changes for the lib.periodic_table module. This is in preparation for extending the information content of this module.
* Modified system test 'test_estimate_r2eff_err_auto', to use the GUI script. It seems to work perfect. This is to test against GUI script: test_r2eff_err_estimate Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified test_estimate_r2eff_err_auto, to set r1_fit to False. This still make the system test pass, and fit R1. So this means R1 fit button is not functioning properly. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix for warning message in the auto-analyses in the GUI. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Tried to improve docstring for API documentation. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Added all of the IUPAC 2011 atomic weights to the lib.periodic_table module. These will be useful for correctly calculating the centre of mass of a molecule.
* The lib.periodic_table method for adding elements is now private.
* Created the unit test infrastructure for the lib.periodic_table module. This includes one unit test of the lib.periodic_table.periodic_table.atomic_weight() function which has not been implemented yet.
* Implemented the lib.periodic_table.periodic_table.atomic_weight() method. This returns the standard atomic weight of the atom as a float.
* Yet another try to make the API documentation working. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented system test Relax_disp.verify_estimate_r2eff_err_compare_mc for testing R2eff error as function of Monte Carlo simulation. Note, since the name does not start with "test", but with "verify", this test will not be issued in the system test suite. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Converted the periodic table in lib.periodic_table into a dictionary type object. The new Element container has been added for storing the information about each element in the table. The Periodic_table object used the atomic symbol as a key for each Element instance.
* Modified system test test Relax_disp.test_estimate_r2eff_err_methods() to show the difference between using the direct function Jacobian, or the chi2 function Jacobian. Added also the functionality to the estimate R2eff module, to switch between using the different Jacobians. The results show, that R2eff can be estimated better.
* Tiny fix for the Diffusion_tensor.test_create_diff_tensor_pdb_ellipsoid system test. The switch to using the lib.periodic_table module for atomic masses has caused the centre of mass of the ellipsoid to shift just enough that one ATOM coordinate in the PDB file has changed its last significant digit.
* Created the lib.periodic_table.process_symbol() function. This will take an atomic symbol and return a copy of it with an uppercase first letter and lowercase second letter. This is used by the Periodic_table methods atomic_mass() and atomic_weight() to allow for non-standard symbol input, for example if the element name comes directly from the all uppercase PDB file format without translation.
* Tried to scale the covariance matrix, as explained here: http://www.orbitals.com/self/least/least.htm. This does not work better. Also replaced "errors" to "weights" to the multifit_covar(), to better determine control calculations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Added all gyromagnetic ratio information from lib.physical_constants to lib.periodic_table. The Periodic_table.gyromagnetic_ratio() method has been added to allow this value to be easily returned.
* Added to back-end of R2eff estimate module, to be able to switch between the function Jacobian or the chi2 Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified user function 'relax_disp.r2eff_err_estimate', to be able switch between the Jacobians. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified system test Relax_disp.verify_estimate_r2eff_err_compare_mc, to try the difference between the Jacobian. The results are: Printing the estimated R2eff error as function of estimation from covariance and number of Monte Carlo simulations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Deleted the gyromagnetic ratio values and return_gyromagnetic_ratio() function from lib.physical_constants.
* Shifted all of relax to use the lib.periodic_table module for gyromagnetic ratios. The values and value returning function have been removed from lib.physical_constants and replaced by the Periodic_table.gyromagnetic_ratio() method in the lib.periodic_table module.
* Started making functions in R2eff estimate module, independent on the informations stored in the Class. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Cleaned up code in R2eff estimate module, by making each function independent of class. This is to give a better overview, how the different functions connect together. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Made the user function, which estimates the R2eff errors, use the Jacobian derived from chi2 function. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified system test verify_estimate_r2eff_err_compare_mc() to first use the direct function Jacobian, and then the chi2 derived Jacobian. This shows the result better. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added digit to printout in R2eff estimate module. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Locked values for system test test_estimate_r2eff_err, to estimate how the R2eff error estimation reflects on fitted parameters. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* More locking of values, when trying to use different methods for estimating R2eff err values. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* More locking of values. This actually shows, that errors should be estimated from the direct Jacobian. Not, the chi2 Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Reverted the logic, that the chi2 Jacobian should be used. Instead, the direct Jacobian exponential is used instead. When fitting with the estimated errors from the direct Jacobian, the results are MUCH better, and comparable to 2000 Monte Carlo simulations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Various precision fixes for different machine precision. This is in: verify_r1rho_kjaergaard_missing_r1 Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* First attempt at properly implementing the target_functions.relax_fit.jacobian() function. This is now the Jacobian of the chi-squared function. A new jacobian_matrix data structure has been created for holding the matrix data prior to converting it into a Python list of lists. The equation used was simply the chi-squared gradient whereby the sum over i has been dropped and the i elements are stored in the second dimension of matrix.
* Speed up of the target_functions.relax_fit C module. The variances are now pre-calculated in the setup() function from the errors, so that the use of the square() function is minimised. The chi-squared equation, gradient, and Hessian functions now accept the variance rather than standard deviation argument and hence the squaring of errors has been removed. This avoids a lot of duplicated maths operations.
* Added RelaxError, if less than 2 time points is used for exponential curve fitting in R2eff. This follows: http://thread.gmane.org/gmane.science.nmr.relax.user/1718 http://thread.gmane.org/gmane.science.nmr.relax.user/1735 Specifically, data was attached here: http://thread.gmane.org/gmane.science.nmr.relax.user/1735/focus=1736.
* Added system test Relax_disp.test_bug_atul_srivastava(), to catch a bug missing raising a RelaxError, since the setup points to a situation where the data shows it is exponential fitting, but only one time point is added per file. This follows: http://thread.gmane.org/gmane.science.nmr.relax.user/1718 http://thread.gmane.org/gmane.science.nmr.relax.user/1735 Specifically, data was attached here: http://thread.gmane.org/gmane.science.nmr.relax.user/1735/focus=1736.
* Parameter precision lowered for Relax_disp.test_estimate_r2eff_err_auto(). This is due to change to C code. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Created the select.display user function. This simply displays the current spin selections of all spins. In the future it can be extended to display the interatomic data container selections, domain selections, etc.
* Fix for system test: test_estimate_r2eff_err_auto(). The Jacobian to estimate the errors has been changed from the direct function Jacobian, to the Jacobian of the chi2 function. This changes the R2eff error predictions, and hence parameter fitting. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented the direct Jacobian in Python, to be independent of C code in development phase. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Activated all method try in: system test Relax_disp.test_estimate_r2eff_err_methods. This is to quickly estimate errors from all different methods. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix to system test: test_estimate_r2eff_err_auto, which now checks the values for the direct Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Increased the number of time points for exponential curve fitting to 3.
* Fix to weight properly according to if minimising with direct Jacobian or chi2 Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix to system test test_estimate_r2eff_err_methods, after modification of weighting. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Switched in estimate_r2eff_err() to use the chi2 Jacobian from C code, and Jacobian from Python code. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Removed all references to test values which was received by wrong weighting. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Better error checking in the relaxation dispersion overfit_deselect() API method. The model must be set for this procedure to work, and the method now checks that this is the case.
* Better error checking for the specific_analyses.relax_disp.average_intensity() function. This function would fail with a traceback if a peak intensity error analysis had not yet been performed. Now it fails instead with a clean RelaxError so that the user knows what is wrong.
* Tried implementing getting the chi2 gradient, using target_function.chi2.dchi2(). The output seem equal. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Replaced the way to calculate the chi2 Jacobian, for exponential fit in minfx. This is only for the test class, but reuses library code. This should make it much easier in the future to implement chi2 gradient functions to minfx, since it is only necessary to implement the direct gradient of the function, and then pass the direct gradient to chi2 library, which turn it into the chi2 gradient function which minfx use. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Moved unnecessary function in R2eff error estimate module into experimental class. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Implemented system test: test_bug_negative_intensities_cpmg, to show lack of error message to user. Maybe these spins should be de-selected, or at least show a better warning. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* An attempt at documenting the Monte Carlo simulation verses covariance matrix error estimates. This is for the R2eff and I0 parameters of the exponential curves. For the Monte Carlo errors, 10000 simulations were preformed. This means that these errors can perform as a gold standard by which to judge the covariance matrix technique. Currently it can be seen that the relax_disp.r2eff_err_estimate user function with the chi2_jacobian flag set to True performs extremely poorly.
* Reintroduced the original target_functions.relax_fit.jacobian() function. The new function for the Jacobian of the chi-squared function has been renamed to target_functions.relax_fit.jacobian_chi2() so that both Python functions are accessible within the C module.
* Epydoc fixes for the pipe_control.mol_res_spin.format_info_full() function.
* Epydoc docstring fixes for many methods in the relaxation dispersion auto-analysis module.
* If math domain errors are found when calculating the two point R2eff values, the point is being skipped. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Moved intensity negative value from reference to CPMG point.
* Modified system test test_bug_negative_intensities_cpmg, to prepare for testing number of R2eff points. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Comparison of 10,000 Monte Carlo simulations to a different covariance matrix error estimate. The covariance_matrix.py script has been duplicated and the chi2_jacobian argument of the relax_disp.r2eff_err_estimate user function has been changed from True to False. As can be seen in the 2D Grace plots, this error estimate is incredibly different. The R2eff errors are overestimated by a factor of 1.9555, which indicates that the Jacobian or covariance matrix formula are not yet correct.
* The target_functions.relax_fit C module Python function jacobian_chi2() is now exposed. This was previously not visible from within Python.
* Added a script and log for calculating the numerical covariance matrix for an exponential curve. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the covariance matrix via the Jacobian calculated using the numdifftools.Jacobian object construct and obtain the matrix, both at the minimum and at a point away from the minimum. The covariance is calculated as inv(J^T.W.J).
* Added a script and log for calculating the exponential curve parameter errors via bootstrapping. This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and calculates the parameter errors via bootstrapping. As the parameters at the minimum are the exact parameter values, bootstrapping and Monte Carlo simulation converge and hence this is a true error estimate. 200,000 simulations where used, so the parameter errors are extremely accurate.
* Modified module to estimate R2eff errors, to use the C code Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified system test test_estimate_r2eff_err_methods, to check all Jacobian methods are correctly implemented. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.
* Added more print out information, when log(I / I_ref) is negative, and raising errors. This can help the user track back information to the error more easily.
* Improved system test test_bug_negative_intensities_cpmg, by counting number of R2eff points. Spin 4, which has one negative intensity, is expected to have one less R2eff point. This makes sure, that all CPMG data set can be loaded and analysed, even if some peaks are very weak are fluctuating with error level.
* Fix for also storing 'r1_fit' to cdp even though it is set to False. Bug #22541 ([https://gna.org/bugs/?22541)Bug #22541]: The R1 fit flag does not work in the GUI.* Cleanup in GUI test Relax_disp.test_r2eff_err_estimate. This now passes after previous commit. Bug #22541 ([https://gna.org/bugs/?22541)Bug #22541]: The R1 fit flag does not work in the GUI.* Added model [[DPL94]], to be tested in GUI test Relax_disp.test_r2eff_err_estimate. This shows that the bug is still there. Bug #22541 ([https://gna.org/bugs/?22541)Bug #22541]: The R1 fit flag does not work in the GUI.* Fix for system test test_estimate_r2eff_err and test_r1rho_kjaergaard_missing_r1, where r1_fit=True, needed to be send to Auto_analyses. Bug #22541 ([https://gna.org/bugs/?22541)Bug #22541]: The R1 fit flag does not work in the GUI.
* API documentation fixes.
* Moved multifit_covar into lib.statistics, since it is an independent module. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Moved "func_exp_grad" into experimental class for different minimisation methods. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Improved documentation to user function relax_disp.r2eff_err_estimate, and removed the possibility to use the chi2 Jacobian, as this is rubbish. But the back-end still have this possibility, should one desire to try this. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Moved the argument 'chi2_jacobian' as the last argument in estimate_r2eff_err. This argument is highly likely not to be used, but is kept for future testing purposes. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix to experimental class for fitting with different methods. After moving the function into class, 'self' should be added to the function. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix to system test test_estimate_r2eff_err, after removing the possibility to use the chi2 Jacobian. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Fix for system test test_estimate_r2eff_err_methods. The function was called wrong in experimental class. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Initial try write comments how to generalize the scaling of the covariance according to the reduced chi2 distribution. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* First try to make a test script for estimating efficiency on R2eff error calculations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added number of simulations to 10,000 in test script, and varied the random number of time point per simulation between 3 and 10. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* In module for estimating R2eff errors, removed "values, errors" to be send to function for gradient, since they are not used. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added Jacobian to test script, and now correctly do simulations, per R2eff points. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Improved analysing test script, with plotting. It seems that R2eff error estimation always get the same result. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added initial dataset for test analysis. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Deleted test data set. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified data script generator, to handle a situation with fixed 5 time points, and a situations with variable number of time points. Also modified analysis script. It seems, this has an influence how the error estimation is performing. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added simulations that show, there is perfect agreement between Monte Carlo simulations and covariance estimation. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Inserted extra tests in system test Relax_disp.test_estimate_r2eff_err_methods to test that all values of R and i0 are positive, and the standard deviation from Monte Carlo simulations are equal. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Inserted system test Relax_disp.test_finite_value, to illustrate the return of inf from C code exponential, when R is negative. This can be an issue, if minfx takes a wrong step when no constraints are implemented. Bug #22552 ([https://gna.org/bugs/?22552)Bug #22552]: Chi2 value returned from C code curve-fitting return 0.0 for wrong parameters -> Expected influence on Monte Carlo sim.
* Inserted possibility for bootstrapping in system test Relax_disp.test_estimate_r2eff_err_methods. This shows, that the bootstrapping method get the SAME estimation for R2eff errors, as the estimate_r2eff_err() function. This must either mean, that the OLD Monte Carlo simulation was corrupted, or the creation of data in Monte Carlo simulations is corrupted.
* Modified system test Relax_disp.verify_estimate_r2eff_err_compare_mc to include bootstrapping method. This shows there is excellent agreement between bootstrapping and estimation of errors from covariance, while relax Monte Carlo simulations are very much different. Boot strapping is the "-2": Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added functionality to create peak lists, for virtual data. This is to compare the distribution of R2eff values made by bootstrapping and Monte Carlo simulations. Rest of the analysis will be performed in relax. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added initial peak lists to be analysed in relax for test purposes. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added relax analysis script, to profile distribution of errors drawn in relax, and from Python module "random". It seems that relax draw a lot more narrow distribution of Intensity with errors, than Python module "random". This has an influence on estimated parameter error. This is a potential huge error in relax. A possible example of a catastrophic implementation of Monte Carlo simulations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Added PNG image that show that the distribution which relax makes are to narrow. This is a potential huge flaw in implementation of Monte Carlo simulations. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Modified analysis script, to also make histogram of intensities. This shows that the created intensities are totally off the true intensity. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting.* Comment fix to system test Relax_disp.test_estimate_r2eff_err_methods, after the found of bug in relax. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting. Bug #22554 ([https://gna.org/bugs/?22554)Bug #22554]: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected.* Cleaned up user function for estimating R2eff errors. Extensive tests have shown, there is a very good agreement between the covariance estimation, and Monte Carlo simulations. This is indeed a very positive implementation. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting. Bug #22554 ([https://gna.org/bugs/?22554)Bug #22554]: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected.* Removed all junk comments from module for R2eff error estimation. The module runs perfect as it does now. Task #7822 ([https://gna.org/task/?7822)Task #7822]: Implement user function to estimate R2eff and associated errors for exponential curve fitting. Bug #22554 ([https://gna.org/bugs/?22554)Bug #22554]: The distribution of intensity with errors in Monte Carlo simulations are markedly more narrow than expected.* Fix for inf values being returned from C code exponential function. Values are now converted to high values. This fixes system test Relax_disp.test_finite_value. Example: x = np.array([np.inf, -np.inf, np.nan, -128, 128]), np.nan_to_num(x), array([ 1.79769313e+308, -1.79769313e+308, 0.00000000e+000, -1.28000000e+002, 1.28000000e+002]). Bug #22552([https://gna.org/bugs/?22552)Bug #2255]: Chi2 value returned from C code curve-fitting return 0.0 for wrong parameters -> Expected influence on Monte Carlo sim. Ref: http://docs.scipy.org/doc/numpy/reference/generated/numpy.nan_to_num.html.
* Initial try to reach constrained methods in minfx through relax. This is in system test Relax_disp.verify_estimate_r2eff_err_compare_mc() This though not seem to be supported.
* Allow R2eff model to reach constrained methods in minfx through relax. This is in system test Relax_disp.verify_estimate_r2eff_err_compare_mc() This though not seem to be supported.
* Modified specific_analyses.relax_disp.parameters.r1_setup() to initialise the 'r1' variable. This relates to bug #22541 ([https://gna.org/bugs/?22541)bug #22541], the R1 fit flag does not work in the GUI. This is a hack, as all of the dispersion analysis code assumes that all parameters are initialised. This is a dangerous assumption that will have to be eliminated in the future.* The dispersion get_param_values() API method now calls the r1_setup() function. This relates to bug #22541 ([https://gna.org/bugs/?22541)bug #22541], the R1 fit flag does not work in the GUI. This is to make sure that the parameters are correctly set up prior to obtaining all parameter values. The R1 parameter is dynamic hence r1_setup() needs to be called at any point model parameters are accessed, as the R1 parameter can be turned on or off at any time with the relax_disp.r1_fit user function.
* Yet another try to implement constrained method in verify_estimate_r2eff_err_compare_mc.
* Another attempt to reach constrained method in minfx through relax. I would need to specify: l, lower bound constraint vector (l <= x <= u); u, upper bound constraint vector (l <= x <= u); c: user supplied constraint function; dc: user supplied constraint gradient function.
Trusted, Bureaucrats
4,228

edits

Navigation menu