Trusted, Bureaucrats
4,228
edits
Changes
Jump to navigation
Jump to search
m
* Important fix for extracting the correct shape to create new arrays. If using just one field, or having the same number of dispersion points, the shape would extend to the dispersion number. It would report [ei][si][mi][oi][di] when calling ndarray.shape. Shape always has to be reported as: [ei][si][mi][oi]. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Made it easier to switch between single and cluster reporting in profiling script. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Important fix for the creation of the multi dimensional pA p<sub>A</sub> numpy array. It should be created as numpy.zeros([ei][si][mi][oi]) instead of numpy.ones([ei][si][mi][oi]). This allows for rapid testing of all dimensions with np.allclose(pA, numpy.ones(dw.shape)). pA p<sub>A</sub> can have missing filled out values, when the number of dispersion points are different per spectrometer frequency. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added unit tests demonstrating edge cases 'no Rex' failures of the model [[CR72 full]], for a clustered multi dimensional calculation. This is implemented for one field. This is to implement catching of math domain errors, before they occur. These tests cover all parameter value combinations which result in no exchange: [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Re-implemented safety checks in lib/dispersion/cr72.py. This is now implemented for both rank-1 float array and of higher dimensions. This makes the unit tests pass for multi dimensional computing. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added self.has_missing keyword in initialization of the Dispersion class. This is to test once, per spin or cluster. This saves a looping over the dispersion points, when collection the data. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Created multi dimensional error and value numpy arrays. This is to calculate the chi2 sum much faster. Reordered the loop over missing data points, so it is only initiated if missing points is detected. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Switch the looping from spin->frq to frq->spin. Since the number of dispersion points are the same for all spins, this allows to move the calculation of pA p<sub>A</sub> and kex array one level up. This saves a lot of computation. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Changed all the creation of special numpy arrays to be of float64 type. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the data filling of special numpy array errors and values, to initialization of Dispersion class. These values does not change, and can safely be stored outside. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Lowered the number of iterations to the profiling scripts. This is to use the profiling script as bug finder. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the calculation of dw_frq out of spin and spectrometer loop. This is done by having a special 1/0 spin numpy array, which turns on or off the values in the numpy array multiplication. The multiplication needs to first axis expand dw, and then tile the arrays according to the numpy structure. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the calculation of pA p<sub>A</sub> and kex out off all loops. This was done by having two special 1/0 spin structure arrays. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed dw_frq_a numpy array, as it was not necessary. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed all looping over spin and spectrometer frequency. This is the last loop! Wuhu. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fixed that arrays are correctly initiated with one or zero values. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Very important fix, for only replacing part of data array which have Nan values. Before, all values were replaced, which was wrong. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Needed to increase the relative tolerance when testing if pA p<sub>A</sub> array is 1. Now system test Relax_disp.test_hansen_cpmg_data_missing_auto_analysis passes. Also added some comments lines, to prepare for mask replace of values. For example if only some of etapos values should be replaced. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Restored profiling script to normal. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Made the logic and comments much clearer about how to reshape, expand axis, and tile numpy arrays. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Implemented a masked array search for where "missing" array is equal 1. This makes it possible to replace all values with this mask, from the value array. This eliminates the last loops over the missing values. It took over 4 hours to figure out, that the mask should be called with mask.mask, to return the same fulls structure, [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Yet another small improvement for the profiling script. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed the multi dimensional structure of pAp<sub>A</sub>. pA p<sub>A</sub> is not multi-dimensional, and can just be multiplied with numpy arrays. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* Fix for testing of pA p<sub>A</sub> in lib function, when pA p<sub>A</sub> is just float. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* Modified unit tests, so pA p<sub>A</sub> is sent to target function as float instead of array. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed the multi dimensional structure of kex. kex is not multi-dimensional, and can just be multiplied with numpy arrays. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for testing of kex in lib function, when kex is just float. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced target function for model [[M61b]], to use higher dimensional numpy array structures. That makes the model much faster. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points to be send to lib function of model [[TSMFK01]]. These are not used anymore. Also removed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points and pB p<sub>B</sub> to be send to lib function of model [[TP02]]. Number of points are not used anymore. pB p<sub>B</sub> is calculated in lib function instead. Also removed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed number of points and pB p<sub>B</sub> to send to lib function of model [[TP02]]. pB p<sub>B</sub> is calculated in lib function instead. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed number of points, pBp<sub>B</sub>, k_ABk<sub>AB</sub>, k_BA k<sub>BA</sub> to be send to lib function of model [[B14]]. Number of points are not used anymore. pB p<sub>B</sub> is calculated in lib function instead. k_ABk<sub>AB</sub>, and k_BA k<sub>BA</sub> are calculated in lib functions instead. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for sending number of points in target function of [[TSMFK01]]. This was removed in lib function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points, pBp<sub>B</sub>, to be send to lib function of model [[TAP03]]. Number of points are not used anymore. pB p<sub>B</sub> is calculated in lib function instead. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points, to be send to lib function of model [[CR72]]. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points, to be send to lib function of model [[DPL94]]. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points, to be send to lib function of model [[M61]]. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed number of points, to be send to lib function of model [[M61b]]. Number of points are not used anymore. Fixed in target function. Fixed in lib function. Fixed in corresponding unit tests. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Methods to replace math domain errors in model [[MP05]], has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. Calculation of pBp<sub>B</sub>, has been moved to lib function for simplicity. Documentation is also fixed. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for sending in dw as numpy array in unit tests of model [[MP05]]. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced target function for model [[MP05]], to use higher dimensional numpy array structures. That makes the model much faster. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for sending in r20 and phi_ex as numpy array in unit tests of [[LM63]]. This is after using masks as replacement. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* 1 digit decrease in parameter check in system test: Relax_disp.test_hansen_cpmg_data_to_lm63. It is unknown, why this has occurred. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Methods to replace math domain errors in model IT99, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. pB p<sub>B</sub> is now moved to be calculated inside. This makes the lib function simpler. Documentation is also fixed. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for sending in r20 and dw as numpy array in unit tests of IT99. This is after using masks as replacement. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced target function for model IT99, to use higher dimensional numpy array structures. That makes the model much faster. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Methods to replace math domain errors in model ns_cpmg_2site_expanded, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. pB p<sub>B</sub> is now moved to be calculated inside. This makes the lib function simpler. k_AB k<sub>AB</sub> and k_BA k<sub>BA</sub> is also now calculated here. Documentation is also fixed. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for sending in r20 and dw as numpy array in unit tests of ns_cpmg_2site_expanded. This is after using masks as replacement. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced target function for model ns_cpmg_2site_expanded, to use higher dimensional numpy array structures. That makes the model much faster. I cannot get system test: Relax_disp.test_cpmg_synthetic_dx_map_points to pass. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for system test Relax_disp.test_cpmg_synthetic_dx_map_points. By just copying self.back_calc_a to self.back_calc, problem was solved. In specific_analysis.relax_disp.optimisation in function back_calc_r2eff(), the function gets the last values stores in the class function. This is in "class Disp_result_command(Result_command)" with self.back_calc = back_calc. And back_calc_r2eff() have return model.back_calc. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Methods to replace math domain errors in model ns_cpmg_2site_3d, has been replaced with numpy masks. Number of points has been removed, as the masks utility replaces this. pB p<sub>B</sub> is now moved to be calculated inside. This makes the lib function simpler. k_AB k<sub>AB</sub> and k_BA k<sub>BA</sub> is also now calculated here. Magnetization vector is also now filled in lib function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Fix for unit tests of model [[NS CPMG 2-site 3D]] to the reduced input to the lib function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Change to the target function to the model [[NS CPMG 2-site 3D]] to use the reduced input to the lib function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Added the [[NS CPMG 2-site 3D]] model to the dispersion super profiling script. To handle the fact that this script has nr_iter set to 100 rather than 1000 (as otherwise it is too slow), a list of scaling factors has been created to scale all timing numbers to equivalent values.
* Added [[DPL94]] profiling script. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Modified profiling script for [[TSMFK01]], to use correct parameters k_AB k<sub>AB</sub> and r20a. Or else, the lib functions is just calculating with zero? [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Changes to profiling script of [[NS CPMG 2-site expanded]]. The model does not have r20a and r20b, but only r2. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Made changes to the profiling script of [[NS CPMG 2-site 3D]]. Need to use the full model, when r2a and r2b is specified. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Expansion of the disp_profile_all.py dispersion model super profiling scripts. The newly added profiling scripts for models [[M61]], [[TP02]], [[TAP03]], and [[MP05]] are now included in the super script to generate statistics for all of these as well. The nr_iter variable has also been changed to match the other analytic models, so that the standard deviations are lowered and the statistics are better.
* Moved the parameter conversion of [[MMQ CR72]] into lib function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the parameter conversions of k_ABk<sub>AB</sub>, k_BA k<sub>BA</sub> and pB p<sub>B</sub> into lib function of [[NS MMQ 2-site]]. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Moved the parameter conversion from target function to lib function for [[NS R1rho 2-site]]. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Updated the dispersion model speed statistics for the disp_spin_speed branch vs. relax-3.2.2. This now includes the [[NS CPMG 2-site 3D]], [[DPL94]], and [[NS R1rho 2-site]] dispersion models. The timings for the single spin analyses are now comparable to the clustered analysis, as the equivalent of 100 single spins is being used. The final printout is also in a better format to present for the relax release messages. These new results show the insane 160x speed up of the [[DPL94]] model.
* Moved parameter conversion for [[NS R1rho 3-site]] inside lib function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Cleaned up the Dispersion class target function, for creation of matrices, which is now populated inside the lib functions instead. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed pA p<sub>A</sub> and pB p<sub>B</sub> from the matrix population function rcpmg_star_rankN, since they are not used. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.* Removed pA p<sub>A</sub> and pB p<sub>B</sub> from the matrix population function rr1rho_3d_2site_rankN, since they are not used. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Documentation fix for the dimensionality for model [[NS R1rho 2-site]]. The data is lined up to be of form [NE][NS][NM][NO][ND]. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Removed pAp<sub>A</sub>, pB p<sub>B</sub> and pC p<sub>C</sub> from the matrix population function rr1rho_3d_3site_rankN, since they are not used. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Deleted the profiling of eig function profiling script. This was never implemented. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* For all profiling scripts, added conversion to numpy array for CPMG frqs and spin_lock, since some models complained in 3.2.2, that they were of list types. Also fixed IT99 to only have 1 spin, since clustering is broken in 3.2.2. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced all self.num_exp with self.NE in target function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* Replaced all self.num_frq with self.NM in target function. [https://gna.org/task/?7807 Task #7807]: Speed-up of dispersion models for clustered analysis.
* A very small speed up to the [[CR72]] dispersion models by minimising repetitive maths operations. The k_BA k<sub>BA</sub> and k_AB k<sub>AB</sub> rates are used to simplify the Psi calculation, dropping from 3 to 2 multiplications and removing a squaring operation. The Dpos and Dneg value calculations have been simplified to drop one multiplication operation. And the calculation of eta_scale / cpmg_frqs now only occurs once rather than twice.
* Removal of a tonne of unused imports in the lib.dispersion package. These were identified using the command "pylint * --disable=all --enable=unused-import".
* A very small speed up to the [[MMQ CR72]] dispersion model by minimising repetitive maths operations. This matches the recent change for the [[CR72]] model, though the Psi calculation was already using the fast form.
* Implemented target and calculation function for MODEL_NOREX_R1RHO, MODEL_NOREX_R1RHO_FIT_R1. [https://gna.org/bugs/?22440 Bug #22440]: The "NOREX" model is not covering R1rho models. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Made the model "MODEL_NOREX_R1RHO", be testes in system test Relax_disp.test_r1rho_kjaergaard_auto. This is for system test where R1 has been loaded from earlier results, which was not analysed in relax. [https://gna.org/bugs/?22440 Bug #22440]: The "NOREX" model is not covering R1rho models. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Made list of models which fit pA p<sub>A</sub> or pA p<sub>A</sub> and pBp<sub>B</sub>. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Added function to auto_analysis, to test if it give meaning to write and plot out the parameter. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Made the writing out of parameter pCp<sub>C</sub>, be tested with the new function. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Implemented models list with phi_ex, phi_ex_B, and phi_ex_C, and added to test in auto_analyses of relax_disp. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Made use of the dictionary MODEL_PARAMS, to determine if parameter is present. This makes the list of models belonging to parameter lists superfluous. [https://gna.org/support/?3135 Sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models.
* Inserted LaTeX bibliography for reference to linear constraints of the exchange rate.
* Equation fix for $kex$ in manual.
* Added system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis and data. This is for [https://gna.org/bugs/?22477 bug #22477]: Not possible to perform grace.write() on k_AB k<sub>AB</sub> parameter for mixed CPMG analysis.
* Added more printout to system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis and data.
* Fix for system test Relax_disp.test_bug_22477_grace_write_k_AB_mixed_analysis. [https://gna.org/bugs/?22477 Bug #22477]: Not possible to perform grace.write() on k_AB k<sub>AB</sub> parameter for mixed CPMG analysis.
* Set the default value of r1_fit to 5.0.
* Added the relax_disp pipe type to be setup for unit tests of value function.
* Added unit tests regarding model.py and its functions.
* Removed the auto-sorting of models, when performing auto analysis of Relaxation dispersion. This was discussed in: http://thread.gmane.org/gmane.science.nmr.relax.scm/22733, http://thread.gmane.org/gmane.science.nmr.relax.scm/22734, http://thread.gmane.org/gmane.science.nmr.relax.scm/22737. Through this discussion, it has appeared that the order of how models are sorted for analysis, and hence the possibility for nesting, is a complicated case. The order of analysis should be possible to manually put into the auto analysis. This was not the scope of [https://gna.org/support/?3135 sr #3135]: Optimisation of the R1 relaxation rate for the off-resonance R1rho relaxation dispersion models to implement such a feature. Such a feature could be implemented for the next version of relax. It could be designed as function to "suggest" an order in the GUI. But this functionality would have to wait.
* Removing the 'tex'->'kex' conversion, and 'phi_ex' from dw and pAp<sub>A</sub>. This solution is not a proper implementation, but these parameters should rather be found by grid search.
* Re-inserted "MODEL_NS_CPMG_2SITE_EXPANDED" to be tested in system test test_hansen_cpmg_data_missing_auto_analysis.
* Removed special cases for nesting. Now following order is determined. First sort completed models into: EQ_NUMERIC, EQ_SILICO, EQ_ANALYTIC. Then into year, with newest first. Then number of chemical sites, which reflects number of parameters. Go through the completed models. If the experiment types are the same, then look for: If a completed model has same parameters, then nest from this. If a completed model has all other parameters than R20 parameters, then nest from this. Special cases are taken care of by: MODEL_LM63_3SITE from MODEL_LM63, MODEL_NS_MMQ_3SITE, MODEL_NS_MMQ_3SITE_LINEAR from MODEL_NS_MMQ_2SITE, MODEL_NS_R1RHO_3SITE, MODEL_NS_R1RHO_3SITE_LINEAR from MODEL_NS_R1RHO_2SITE, MODEL_MMQ_CR72 from MODEL_CR72. This functionality represents the hard-coding from previous implementation.
