Changes

Relax 4.0.0

6,267 bytes added, 13:04, 13 October 2015

Links to the API documentation (part 1).

* Deletion of the frame_order.average_position user function and all of the associated backend code. This user function allowed the user to specify five different types of displacement to the average moving domain position: a pure rotation, with no translation, about the pivot of the motion in the system; a rotation about the pivot of the motion of the system together with a translation; a pure translation with no rotation; a rotation about the centre of mass of the moving domain with no translation; a rotation about the centre of mass of the moving domain together with a translation. Now the last option will be the default and only option. This option is equivalent to the standard superimposition algorithm (the Kabsch algorithm) to a hypothetical structure at the real average position. The other four are due to the history of the development of the theory. These limit the usefulness of the theory and will only cause confusion.

* Clean up of the frame order target function code. This matches the previous change of the deletion of the frame_order.average_position user function. The changes include the removal of the translation optimisation flag as this is now always performed, and the removal of the flag which causes the average domain rotation pivot point to match the motional pivot point as these are now permanently decoupled.

* Alphabetical ordering of functions in the [http://www.nmr-relax.com/api/4.0/lib.frame_order.pseudo_ellipse-module.html lib.frame_order.pseudo_ellipse module].

* Eliminated all of the 'line' frame order models, as they are not implemented yet. This is just frontend code - the backend does not exist.

* Updated the isotropic cone CaM frame order test model optimisation script. Due to all of the changes in the frame order analysis, the old script was no longer functional.

* Updated the CaM frame order test model superimposition script. The [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose user function] is now correctly called. The output log file has been added to the repository as it contains the correct translation and Euler rotation information needed for the test models.

* Parameter update for the isotropic cone CaM frame order test model optimisation script. The Euler angles for the rotation about the motional pivot have been replaced by the translation vector and Euler angle CoM rotation parameters.

* Fix for a number of the frame order models which do not have parameter constraints. The [http://www.nmr-relax.com/api/4.0/specific_analyses.frame_order.parameters-module.html#linear_constraints linear_constraint() function ] was returning A, b = [], [] for these models, but these empty numpy arrays were causing the ~~minfx library (~~[https://gna.org/projects/minfx/) minfx library] to fail. These values are now caught and the constraint algorithm turned off in the [http://www.nmr-relax.com/api/4.0/specific_analyses.frame_order.api.Frame_order-class.html#minimise minimise() specific API method].

* Increased the precision of all the data in the CaM frame order test data generation base script. These have all been converted from float16 to float64 numpy types.

* Fix for the RDC error setting in the CaM frame order test data generation base script. The rdc_err data structure is located in the interatomic data containers, no the spin containers.

* Huge speedup of the CaM frame order test data generation base script. By using multidimensional numpy arrays to store the atomic positions and XH unit vectors of all spins, and performing the rotations on these structures using numpy.tensordot(), the calculations are now a factor of 10 times faster. The progress meter had to be changed to show every 1000 rather than 100 iterations. The rotations of the positions and vectors are now performed sequentially, accidentally fixing a bug with the double motion models (i.e. the 'double rotor' model).

* Modified the CaM frame order test data generation base script to conserve computer RAM. The XH vector and atomic position data structures for all N rotations are now of the numpy.float32 rather than numpy.float64 type. The main change is to calculate the averaged RDCs and averaged PCSs separately, deleting the N-sized data structures once the data files are written.

* Complete redesign of the CaM frame order data generation base script for speed and memory savings. Although the rotated XH bond vector and atomic position code was very fast, the amount of memory needed to store these in the spin containers and interatomic data containers was huge when N > ~~1e6~~1e6. The subsequent [http://www.nmr-relax.com/manual/rdc_back_calc.html rdc.back_calc] and [http://www.nmr-relax.com/manual/pcs_back_calc.html pcs.back_calc] user function calls would also take far too long. Therefore the base script has been redesigned. The _create_distribution() method has been split into four: _calculate_pcs(), _calculate_rdc(), _create_distribution(), and _pipe_setup(). The _pipe_setup() method is called first to set up the data pipe with all required data. Then the _calculate_rdc() and _calculate_pcs() methods, and finally _create_distribution() if the DIST_PDB flag is set. The calls to the [http://www.nmr-relax.com/manual/rdc_back_calc.html rdc.back_calc] and [http://www.nmr-relax.com/manual/pcs_back_calc.html pcs.back_calc] user functions have been eliminated. Instead the _calculate_rdc() and _calculate_pcs() methods calculate the averaged RDC and PCS themselves as numpy array structures. Rather than storing the huge rotated vectors and atomic positions data structures, the RDCs and PCSs are summed. These are then divided by self.N at the end to average the values. Compared to the old code, when N is set to 20 million the RAM usage drops from ~20 GB to ~65 MB. The total run time is also decreased on one system from a few days to a few hours (an order or two of magnitude).

* Changed the progress meter updating for the CaM frame order test data generation base script. The spinner was far too fast, updating every 5 increments, and is now updated every 250. And the total number is now only printed every 10,000 increments.

* Improvements to the progress meter for the CaM frame order test data generation base script. Commas are now printed between the thousands and the numbers are now right justified.

* Large increase in accuracy of the RDC and PCS averaging. This is for the CaM frame order test data generation base script. By summing the RDCs and PCSs into 1D numpy.float128 arrays (for this, a 64-bit system is required), and then dividing by N at the end, the average value can be calculated with a much higher accuracy. As N becomes larger, the numerical averaging introduces greater and greater amounts of truncation artifacts. So this change alleviates this.

* Fix for the RDC and PCS averaging in the CaM frame order test data generation base script. For the double rotor model, or any multiple motional mode model, the averaging was incorrect. Instead of dividing by N, the values should be divided by NM, where M is the number of motional modes.

* Huge increase in precision for the CaM frame order free rotor model test data. The higher precision is because the number structures in the distribution is now twenty million rather than one million, and the much higher precision numpy.float128 averaging of the updated data generation base script has been used. This data should allow for a much better estimate of the β and γ average domain position parameter values for the free rotor models which are affected by the collapse of the α parameter to zero.

* Large increase in precision for the 2nd CaM frame order free rotor model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 999,999 and the numpy.float128 data averaging has been used.

* Updated the CaM frame order test model superimposition script. The Ca2+ atoms are now deleted from the structures before superimposition so that the centroid matches that used in the frame order analysis.

* The average domain rotation centroid is printed out when [http://www.nmr-relax.com/api/4.0/target_functions.frame_order.Frame_order-class.html#__init__ setting up the frame order target functions]. This is to help the user understand what is happening in the analysis.* Faster clearing of numpy arrays in the [http://www.nmr-relax.com/api/4.0/lib.frame_order-module.html lib.frame_order modules]. The x[:] = 0.0 notation is now used to set all elements to zero, rather than nested looping over all dimensions. This however has a negligible effect on the test suite timings.

* Large increase in precision for the CaM frame order pseudo-ellipse model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.

* Improved the value setting in the optimisation() method of the CaM frame order system tests. This is in the base script used by all scripts in test_suite/system_tests/scripts/frame_order/cam/.

* Changed the average domain position parameter values in the CaM frame order system tests. This is in the base script used by all scripts in test_suite/system_tests/scripts/frame_order/cam/. The translation vector coordinates are now set, as well as the CoM Euler angle rotations. These come from the log file of the test_suite/shared_data/frame_order/cam/superimpose.py script, and are needed due to the simplification of the average domain position mechanics now mimicking the Kabsch superimposition algorithm.

* The [http://www.nmr-relax.com/api/4.0/test_suite.system_tests.frame_order.Frame_order-class.html#mesg_opt_debug CaM frame order system test mesg_opt_debug() method ] now prints out the translation vector. This is printed out at the end of all CaM frame order system tests to help with debugging when the test fails.

* Change for how the CaM frame order system test scripts handle the average domain position rotation. The trick of pre-rotating the 3D coordinates was used to solve the {α, β, γ} -> {0, β', γ'} angle conversion problem in the rotor models no longer works now that the average domain position mechanics has been simplified. Instead, high precision optimised β' and γ' values are now set, and the ave_pos_alpha value set to None. The high precision parameters were obtained with the frame_order.py script located in the directory test_suite/shared_data/frame_order/cam/free_rotor. The free rotor target function was modified so that the translation vector is hard-coded to [-20.859750185691549, -2.450606987447843, -2.191854570352916] and the axis θ and φ angles to 0.96007997859534299767 and 4.0322755062196229403. These parameters were then commented out for the model in the module specific_analyses.frame_order.parameters so only β' and γ' were optimised. Iterative optimisation was used with increasing precision, ending up with high precision using 10,000 Sobol' points.

* Updated a number of the CaM frame order system tests for the higher precision data. The new data results in chi-squared values at the real solution to be much closer to zero.

* Fixes for the 2nd frame order free-rotor system tests. The average domain position parameters are now set to the correct values, matching those in the relax log file frame_order_ave_pos_opt.log in test_suite/shared_data/frame_order/cam/free_rotor2.

* Updated the 2nd CaM free-rotor frame order system tests for the correct average domain position. The chi-squared values are now significantly lower.

* Increased the precision of the chi-squared value testing in the CaM frame order system tests. The [http://www.nmr-relax.com/api/4.0/test_suite.system_tests.frame_order.Frame_order-class.html#check_chi2 check_chi2 method ] has been modified so that the chi-squared value is no longer scaled, and the precision has been increased from 1 significant figure to 4. All of the tests have been updated to match.

* The minimisation verbosity flag now effects the frame order RelaxWarning about turning constraints off.

* Preformed a frame order analysis on the 2nd CaM free-rotor model test data. This is to check that everything is operating as expected.

* Preformed a frame order analysis on the CaM free-rotor mode test data set. This is to demonstrate that everything is operating correctly.

* Preformed a frame order analysis on the CaM free-rotor mode test data set with missing data. This is to demonstrate that everything is operating correctly.

* Attempt to speed up the pseudo-elliptic frame order models. The quasi-random numerical integration of the PCS for the pseudo-ellipse has been modified so that the torsion angle check for each Sobol' point is preformed before the [http://www.nmr-relax.com/api/4.0/lib.frame_order.pseudo_ellipse-module.html#tmax_pseudo_ellipse tmax_pseudo_ellipse() function call]. A new check that the tilt angle is less than cone_theta_y, the larger of the two cone angles, has also been added to avoid tmax_pseudo_ellipse() when the θ tilt angle is outside of an isotropic cone defined by cone_theta_y.

* Preformed a frame order analysis on a number of the CaM test data sets. This includes the rotor, isotropic cone, and pseudo-ellipse, and the analyses demonstrate a common bug between all these models.

* Preformed a frame order analysis on the rigid CaM test data set. This is to demonstrate that everything is operating correctly.

* Increased the number of integration points from 50 or 100 to 5000. This is for all CaM frame_order.py test data optimisation scripts. The higher number of points are essential for optimising the frame order models and hence for checking the relax implementation.

* Updated the frame_order.py optimisation script for the small angle CaM rotor frame order test data. This now has the correct rotor torsion angle of 1 degree, and the spherical coordinates are now converted to the axis α parameter.

* Expanded the capabilities of the [http://www.nmr-relax.com/manual/pymol_frame_order.html pymol.frame_order user function]. The isotropic and pseudo-elliptic cones are now represented as they used to be under the [http://www.nmr-relax.com/manual/pymol_cone_pdb.html pymol.cone_pdb user function]. To avoid code duplication, the new [http://www.nmr-relax.com/api/4.0/pipe_control.pymol_control-module.html#represent_cone_axis represent_cone_axis()], [http://www.nmr-relax.com/api/4.0/pipe_control.pymol_control-module.html#represent_cone_object represent_cone_object() ] and [http://www.nmr-relax.com/api/4.0/pipe_control.pymol_control-module.html#represent_rotor_object represent_rotor_object() ] functions have been created to send the commands into PyMOL.

* Increased the precision of all of the CaM frame order system tests by 40 times. The number of Sobol' integration points have been significantly increased while only increasing the frame order system test timings by ~10%. This allows for checking for chi-squared values at the minima much closer to zero, and is much better for demonstrating bugs.

* Optimisation constraints are no longer turned off in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. Constraints are now supported by all frame order models, or automatically turned off for those which do not have parameter constraints.

* Fix for the frame order visualisation script created by the auto-analysis. The call to [http://www.nmr-relax.com/manual/pymol_frame_order.html pymol.frame_order] is now correct for the current version of this user function.

* Removed a terrible hack for handling the frame order analysis without constraints. This is no longer needed as the log-barrier method is now used to constrain the optimisation, so that the torsion angle can no longer be negative.

* Constraints are now implemented in the [http://www.nmr-relax.com/api/4.0/specific_analyses.frame_order.api.Frame_order-class.html#grid_search frame order grid search]. This is useful for the pseudo-elliptic models as the cone θx < θy constraint halves the optimisation space.

* Expanded the CaM rotor test data frame_order.py optimisation script. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower. The results of the new optimisation are included.

* Expanded the CaM pseudo-ellipse test data frame_order.py optimisation script. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower. The results of the new optimisation are included.

* Added one more iteration for the zooming optimisation of the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is to improve the speed of optimisation when all RDC and PCS data is being used. The previous iterations where with [100, 1000, 200000] Sobol' integration points and [1e-2, 1e-3, 1e-4] function tolerances. This has been increased to [100, 1000, 10000, 100000] and [1e-2, 1e-3, 5e-3, 1e-4]. The final number of points has been decreased as that level of accuracy does not appear to be necessary. These are also only default values that the user can change for themselves.

* Updated the CaM frame order data generation base script to print out more information. This is for the first axis system so that the same amount of information as the second system is printed.

* Expanded the CaM isotropic cone test data frame_order.py optimisation script and added the results. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower.

* Updated all of the CaM frame order system test chi-squared values. These have changed slightly due to the rotation and translation changes.

* Added support for the 'pivot_disp' frame order parameter to the grid search. This is required for the double rotor model.

* Changed some of the default values for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The number of Sobol' quasi-random integration points were far too low to obtain any reasonable results.* Simplified the PyMOL visualisation relax script created by the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This now consists of a single [http://www.nmr-relax.com/manual/pymol_frame_order.html pymol.frame_order user function] call. The other pymol user function calls were unnecessary.

* Added the full optimisation results for the large angle double rotor frame order model. This is for the CaM test data using the new frame_order.py optimisation script.

* Added model support for the rotor geometric object. This is the structural object used in the frame order analysis to create PDB representations of rotor motions. The number of atoms created for the rotor is now constant, allowing for models whereby the atom number and connectivity must be preserved between all models.

* Parallelised the frame order grid search to run on clusters or multi-core systems via OpenMPI. This involved the creation of the Frame_order_grid_command class which is the multi-processor Slave_command for performing the grid search. This was created by duplicating the Frame_order_minimise_command class and then differentiating both classes. For the subdivision of the grid search, the new minfx grid.grid_split_array() function is used in the frame order grid() API method. The grid() method no longer calls the minimise() method but instead obtains the processor box itself and adds the subdivided grid slaves to the processor. The relax [http://www.nmr-relax.com/manual/grid_search.html grid_search user function] takes care of the rest.

* Fixes for the parallelised grid search for the frame order analysis. A chi-squared value check was added to the Frame_order_result_command.run() method to check if the value is lower than the current when the result is returned to the master. Without this check, each grid subdivision result will be stored as they are returned rather than storing the results from the global minimum of the entire grid search.

* Added a script for testing out the parameter nesting abilities of the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This script attempts to find the dynamics solution without knowing where the pivot is located. Hence this will be as in the auto-analysis were this pivot point will be used as the base for all other models.

* Sent the verbosity argument to the minfx.grid.grid_split() function for the frame order analysis. This matches the relax trunk changes for the model-free analysis. The minfx function in the next release (1.0.8) will now be more verbose, so this will help with user feedback when running the model-free analysis on a cluster or multi-core system using MPI.

* Improvements for the parallelised grid search for the frame order analysis. As each grid point can take wildly different numbers of CPU cycles to calculate the chi-squared value for, the result of subdividing the grid search was that some subdivisions where incredibly quick while others required much larger amounts of time. To avoid this bad slave management, the grid points are now randomised. This means that the subdivisions will require about the same amount of time to optimise.

* Moved the setup of the target function data structures in the frame order analysis. This is for the [http://www.nmr-relax.com/manual/grid_search.html grid_search] and [http://www.nmr-relax.com/manual/minimise.html minimise] user functions. The target function data setup function has been renamed to target_fn_data_setup(). This is now called before the Frame_order_grid_command and Frame_order_minimise_command multi-processor objects are initialised, and all of the data is now passed into these functions. Although the code is uglier, this has the benefit that the target_fn_data_setup() function will only be called once. This data setup requires a lot of time, so for a large cluster, this can be a large time saving for the grid search.

* Modified the frame_order_free_start.py script to better mimic the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis].

* Updated the frame order optimisation results for the 2nd CaM free rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.

* Updated the frame order optimisation results for the CaM free rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.

* Updated the frame order optimisation results for the CaM torsionless, pseudo-ellipse test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.

* Fix for the Frame_order.test_cam_pseudo_ellipse_free_rotor system test. This is for the change of the X and Y cone opening angles.

* Redesign and expansion of the nested model parameter copying in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The nested parameter protocol used to allow the analysis to complete in under 1,000,000 years was no longer functional due to the switching to the axis α parameter to decrease parameter number and redundancy. The copying of the average domain position for the free rotor models was also incorrect as the dropping of the α Euler angle cause the translation parameters and β and γ angles to change drastically. The new protocol has been split into four methods for the average domain position, the pivot point, the motional eigenframe and the parameters of ordering. These use the fact that the free rotor and torsionless models are the two extrema of the models where the torsion angle is restricted. The pivot copying is a new addition.* Created the Frame_order.test_auto_analysis system test. This will be an extremely quick run through of the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] as this is not currently tested. 1 Sobol' quasi-random integration point will be used for all models for speed. The system test uses the rigid CaM test data to perform a full analysis.

* Alphabetical ordering of the imports in the frame order auto-analysis module.

* Fixes for the backend script of the Frame_order.test_auto_analysis system test. This includes a missing import and the removal of a long ago deleted user function.

* Fix for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] for the call to the grid search user function. This user function has been renamed to [http://www.nmr-relax.com/manual/minimise_grid_search.html minimise.grid_search], however not all parts of the analysis had been converted to the new name.* Created a method in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] to reorder the models. This is needed as the nested model parameter copying protocol requires the simpler models to be optimised first.

* The Frame_order.test_auto_analysis system test now writes all files to the directory of ds.tmpdir. This is to prevent the system test from dumping files in the current directory.

* Modified the specific_analyses.frame_order.parameters.update_model() function. This will no longer set all parameters to 0.0, excluding the pivot point.

* Better handling of unset parameters in the frame order optimisation functions. The specific_analyses.frame_order.optimisation.target_fn_data_setup() and specific_analyses.frame_order.parameters.assemble_param_vector() function both now accept the unset_fail argument. This is set in both the calculate() and minimise() API methods. When set, a RelaxError will be raised in the assemble_param_vector() function when a parameter has not been set yet. This together with previous changes will prevent the frame order analysis from using 0.0 as a starting value for unset parameters.

* Fixes for all of the Frame_order.test_rigid_data_to_*_model system tests. The base script now sets all parameter values so that the [http://www.nmr-relax.com/manual/minimise_calculate.html minimise.calculate user function] can operate. The two free rotor model chi-squared values have been updated as these are sensitive to the motional eigenframe parameter values - these models can never approximate a rigid state.

* Modified the optimisation of the rigid model in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The grid search is now implemented as a zooming grid search.* Updates and fixes for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The custom grid setup now works for the new reduced parameter set models and the double rotor model is now also included. The cone axis α angle to spherical angle conversion has had a bug removed. And some of the printouts are now more detailed.* Redesigned the Frame_order.test_auto_analysis system test. This now uses a hypothetical new Optimisation_settings object from the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis module ] for holding all of the grid search, zooming grid search and minimisation settings. This will allow for far greater user control of the settings and hugely simplify the auto-analysis interface by decreasing the number of input arguments. It should also be less confusing.* Implementation of the Optimisation_settings object in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This object holds all of the grid search, zooming grid search, and minimisation settings. It provides the add_grid() and add_min() methods to allow the user to add successive iterations of optimisation and settings to the object. The loop_grid() and loop_min() methods are used to loop over each iteration of each method. And the get_grid_inc(), get_grid_num_int_pts(), get_grid_zoom_level(), get_min_algor(), get_min_func_tol() and get_min_num_int_pts() methods are used to access the user defined settings. The auto-analysis has been redesigned around this new concept. All of the optimisation arguments have been replaced. Instead there are the opt_rigid, opt_subset, opt_full, and opt_mc arguments which are expected to be instances of the Optimisation_settings object. The optimisation in the auto-analysis is now more advanced in that more user optimisation settings are now available and active.

* Added linear constraints for the pivot and average domain translation frame order parameters. The pivot coordinates are constrained between -999 and 999 Angstrom and the translation between -500 and 500 Angstrom. This allows the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] to operate in the case of failed models - often the free rotors fitting to torsionally restricted data - by preventing the PDB coordinates from being out of the PDB format range. It should also speed up optimisation by stopping the optimisation of failed models earlier.

* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] Optimisation_settings object now handles the maximum iterations. The new max_iter argument has been added to the add_min() method, and the new get_min_max_iter() method added to fetch the value. This is used in the auto-analysis to set the maximum number of optimisation iterations in the [http://www.nmr-relax.com/manual/minimise_execute.html minimise.execute user function] calls. Limiting this will be of greatest benefit for the test suite.

* Speedup of the Frame_order.test_auto_analysis system test. This involves limiting the maximum number of optimisation steps to 20 for most parts (the rigid model excluded so the average domain position is correctly found), and using the PCS subset data for the full data set.

* Updated the full_analysis.py script for the CaM frame order test data. This is for the recent changes to the auto-analysis with the Optimisation_settings object and for the changes of this branch.

* Fixes for the removal of the RDC data checks from the frame order optimisation functions. The specific analysis API method overfit_deselect() has now been created to deselect spins which do not have PCS data or interatomic data containers missing RDC data. The handling of deselected spins and interatomic data containers is now also correctly handled throughout the frame order specific code.

* Enabled pivot optimisation in the full_analysis.py script for the CaM frame order test data.

* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] now calls the [http://www.nmr-relax.com/manual/time.html time user function]. This is used at the start of each model section, as well as at the very start and very end of the analysis. This feedback is needed for the user to be able to optimise the optimisation settings.* Major bugfix for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The algorithm of using a PCS data subset of a few selected residues to find an initial parameter estimate followed by using all PCS data was badly implemented. The use of the PCS subset caused most spin systems to be deselected, however they remained deselected once all data was being used. So the result was that only the spin subset was ever being used in the analysis.

* Fix for the recent lib.period_table and lib.physical_constant module changes.

* Created the model_directory() method for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is used to create the full path for saving model specific files. It replaces spaces with underscores in the path and removes all commas. The commas in the path appear to be fatal for certain PyMOL versions when viewing the frame order representation.* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] results printout has been extended to include the pivot point.* Change to the parameter nesting in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The pivot is now taken from the rotor model for all other models. Taking the pivot point from the isotropic cone model is not a good idea as there are situations where the pivot point optimisation catastrophically fails, sending the point many tens or hundreds of Angstrom away from the molecule.

* Copied a frame order results file for testing axis permutations. This is from the test_suite/shared_data/frame_order/cam/pseudo_ellipse/ directory. The optimisation results were identified to have failed, in that it found the alternative minimum. The pseudo-ellipse model as two minima in the space, and in this case the global minimum was missed.

* Created the Frame_order.test_axis_permutation system test. This is to test the operation of the yet-to-be implemented [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function].

* Extended the Frame_order.test_axis_permutation system test to check frame_order.permute_axes twice. This will check that two calls to the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function] will restore the original parameter values.

* The [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function] can now handle the torsionless pseudo-ellipse. This model does not have the variable cdp.cone_sigma_max set.

* Added support for axis permutations in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is done by copying the data pipe of the already optimised pseudo-elliptic models, permuting the axes, and performing another optimisation using all RDC and PCS data. This allows the second solution for these pseudo-elliptic models to be found. The 2nd pipe is included in the model selection step to allow the best solution for the model to be found.* Fix for the reading of old results files in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The directory name is now processed by the model_directory() method. This will convert the spaces to '_' and remove commas. Without this the already created files could not be found, if the model name contains a space or comma.

* Made the pivot point in the frame order PDB representation fail-proof. If the pivot position was outside of the bounds [-1000, 1000], the PDB file creation would fail as the record would be too long. So now the pivot is shifted to be in these bounds.

* The axis permutation step in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] is now always performed. If an old results file was found, this step was accidentally skipped.

* Added extensive printouts to the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function].

* Redesigned the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function] frontend. Previously only cyclic permutations were considered, however non-cyclic permutations are also allowed when accompanied by an axis inversion. Therefore 3 combinations exist with cone_theta_x ≤ cone_theta_y, or 2 when the current combination is excluded.

* Removed the second permutation from the 6 Frame_order.test_axis_perm_* system tests. A second identical permutation does not necessarily restore the original state.

* Fix for the frame_order.permute_axes for the torsionless pseudo-ellipse model. The data structure cdp.cone_sigma_max does not exist in this model as cone_sigma_max == 0.0.

* Modified the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] axis permutation algorithm to handle both permutations. Instead of creating one additional data pipe for the permutations, two are now created for the permutations 'A' and 'B'. This allows all 3 solutions for the pseudo-elliptic models to be explored and included in the final model selection process.

* Fix for the Frame_order.test_axis_perm_x_le_z_le_y_permB system test. The permuted z-axis needs to be inverted in the test.

* Many fixes for the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function]. The z-axis inversion is now encoded into a 3D numpy array as the index of the new z-axis position needs to be stored. The cone_theta_x, cone_theta_y and cone_sigma_max parameters are now permuted in reverse 'perm' data structure by calling its index() method. And the cone_theta_x - cone_theta_y to y-axis - x-axis switch has been removed (this may need to be reintroduced later).

* Fix for the axis permutation protocol in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The [http://www.nmr-relax.com/manual/pipe_copy.html pipe.copy user function] does not switch pipes, therefore the [http://www.nmr-relax.com/manual/pipe_switch.html pipe.switch user function] is now being called so that the correct pipe is being permuted and optimised.

* Created some test data files for visualising the frame order axis permutation. This uses the CaM frame order synthetic data for the rotor model to visualise the pseudo-ellipse frame order model axis permutations. The initial conversion sets the pseudo-ellipse torsion angle cone_sigma_max to the rotor opening half-angle, and the pseudo-elliptic cone opening to close to zero. Then the axis permutations are performed. All three solutions are optimised. PDB representations before and after optimisation are included to illustrate any problems.

* Bug fix for the new [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function]. The cone and torsion angles were not being correctly permuted. Now the direct permutation array is being used. And the fact that cone_theta_x is a rotation along the y-axis and cone_theta_y along the x-axis is taken into account.

* Created set of files showing the axis permutation problem for the isotopic cone frame order model. This shows that there are two minima. However one has a chi-squared value of ~1, and the other a value of ~150. Nevertheless, the optimisation could be trapped in the non-global minimum so the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function] should be used for the isotopic cones as well, just in case.

* Created the other isotropic cone condition z ≤ x = y. As there are no constraints in this model, this condition should not result in any major differences, just the size of the cone being different and the optimisation having to decrease the cone angle significantly to mimic the rotor.

* Modified the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The axis permutation algorithm is now performed on all isotopic cone and pseudo-ellipse models. This is just in case the non-global minima was found in the original optimisation. The isotropic cone models possess two local minima whereas the pseudo-ellipse models possess three local minima.* Simplified the optimisation in the axis permutation part of the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. Only the last, highest quality setting is used for optimisation.* Fix for the axis permutation protocol in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This would fail if a results file for the permuted model already exists as the [http://www.nmr-relax.com/manual/pipe_copy.html pipe.copy user function] call was being performed too early.

* Created set of files for the axis permutation of the torsionless isotopic cone frame order model.

* Created an initial Frame_order.test_frame_order_pdb_model_ensemble system test. This is to check the operation of the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] when an ensemble of structures is encountered. However as this uses a very minimal number of user functions to set up the system, a number of other minor bugs will probably be uncovered.

* Created the Frame_order.test_frame_order_pdb_model_failed_pivot system test. This simply shows how the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] currently fails if the optimised pivot point is outside of the PDB coordinate limits of "%8.3f".

* The [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] can now properly handle a failed pivot optimisation. This is when the pivot point optimises to a coordinate outside of the PDB limits. Now all calls to specific_analyses.frame_order.data.generate_pivot() from the module specific_analyses.frame_order.geometric set the pdb_limit flag to True. This allows all representation objects to be within the PDB limits. The algorithm in generate_pivot() has been extended to allow higher positive values, as the real PDB limits are [-999.999, 9999.999]. And a RelaxWarning is called when the pivot is outside to tell the user about it.

* Modified the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] to be more fail-safe. Almost all of the protocol is now within a try-finally block so that the execution lock will always be released.

* Fix for the specific_analyses.frame_order.data.pivot_fixed() function. This was recently introduced when the check for PCS data was removed from this function. To fix the problem, instead of calling base_data_types() to see if PCS data is present, the cdp.pcs_ids data structure is checked instead.

* Fix for the model argument for the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function]. The deletion of structural models for the Monte Carlo simulations in the average domain position representation now only happen if more than one model exists.

* Modified the Frame_order.test_frame_order_pdb_model_failed_pivot system test. This is to show that the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] fails if the pivot is close to but still within the PDB coordinate limits.

* Modified the pivot position checking in specific_analyses.frame_order.data.generate_pivot(). Now the pivot is shifted to be within the limits shrunk by 100 Angstrom. This allows any PDB representation created by the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] to be within the PDB limits.

* Fix for the axis permutation protocol in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. If a results file was found for one of the permutations, a return from the function would occur. The result is that the other permutations would not be loaded or optimised.

* Fix for the RelaxError raised by the [http://www.nmr-relax.com/manual/frame_order_select_model.html frame_order.select_model user function]. This is the error if the model name is incorrect.

* Created the Frame_order.test_pseudo_ellipse_zero_cone_angle system test. This is to catch a bug in optimisation when the cone_theta_x is set to zero in the pseudo-ellipse models.

* Updated all the lib.frame_order.*.pcs_numeric_int_*() functions for the new Sobol' point algorithm. The functions now all accept the max_points argument and terminate the loop over the Sobol' points once the maximum number of points has been reached. The calls to numpy.swapaxes() have also been removed as this is now pre-performed by the target function initialisation.

* Changed the default oversampling factor from 100 to 1 in the [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup user function].

* Converted the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] to use the new [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup user function] design. The auto-analysis Optimisation_settings object has also been modified so that all num_int_pts arguments and internal structures have been split into the two new sobol_max_points and sobol_oversample names and objects.

* Fix for the rigid frame order model for the recent [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup user function] changes. For this model, the number of Sobol' points normally is does not exist. This is now correctly handled.

* Created the sobol_setup() method for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is used to correctly handle the new design of the [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup user function] consistently throughout the protocol.

* Updated the Frame_order.test_auto_analysis system test script. This now uses the new auto-analysis Optimisation_settings object design.

* Updated the Frame_order.test_count_sobol_points system test. The call to the frame_order.num_int_pts user function was changed to [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup].

* Editing of the docstring of the [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup user function].

* Fix for the frame order system test optimisation printouts. The cdp.num_int_pts variable is now called cdp.sobol_max_points.

* The starting time of the axis permutation model optimisations is now output. This is in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This call to the [http://www.nmr-relax.com/manual/time.html time user function] occurred for the normal models, so extending it to the permuted axes models makes the output more consistent.

* Simplified the atomic position averaging warning in the frame order analysis. Instead of throwing a warning for each spin, one warning for all spins is now given. This should make the output a lot less verbose.

* The frame order minimise_setup_atomic_pos() function now accepts the verbosity argument. This is used to silence the warnings in user functions such as [http://www.nmr-relax.com/manual/frame_order_sobol_setup.html frame_order.sobol_setup].

* Improvements for the printout from the update_model() frame order function. A list of updated parameters is now created and everything is printed on a single line at the end. The printout is therefore much more compact.

* Spun out part of the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] into the new [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. The new user function arguments required for properly creating the pseudo-Brownian dynamics simulation would have made the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function] too complicated. Therefore this part has been spun out into the new [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. The frame_order.simulate frontend fully describes the algorithm that will be used to simulate the dynamic content of the PCS and RDC data, and warns that not all modes of motion are visible and present.

* Updated the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] to call the new [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. Although not implemented yet, this allows the user function to create the simulation PDB file in the future.

* Small fix for the new [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] backend.

* Updated the base script for the Frame_order.test_cam_* system tests. The [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] is now called directly after the [http://www.nmr-relax.com/manual/frame_order_pdb_model.html frame_order.pdb_model user function].

* Created the backend framework for the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. The backend specific_analyses.frame_order.uf.simulate() function performs all data checks required, prepares the output file object, assembles the frame order parameter values and pivot point, and creates a copy of the structural object object with the ensemble collapsed into a single model. All this data is then passed into the new lib.frame_order.simulation.brownian() function. This initialises all required data structures and the structural object. The main loop of the simulation is also implemented, taking snapshots at every fixed number of steps and terminating the loop once the total number of snapshots are reached. The snapshot consists of copying the original unrotated structural model and rotating it into the new position. The rotation is currently the identity matrix. The old specific_analyses.frame_order.geometric.create_distribution() stub function has been deleted.

* Decreased the time required for the Frame_order.test_cam_* system tests. The [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] now only creates a total of 20 snapshots rather than 1000.

* Added new arguments to the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] for the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. These are the brownian_step_size, brownian_snapshot and brownian_total arguments which are passed directly into the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. This gives the user more control, as well as allowing the test suite to speed up this part of the analysis.

* Huge speedup for the Frame_order.test_auto_analysis system test. The pseudo-Brownian dynamics simulation via the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] has been massively sped up to allow the test to be almost as fast as before.

* Spun out the code for shifting to the average frame order position into a new function. The old code of the create_ave_pos() of the specific_analyses.frame_order.geometric module has been shifted into the new average_position() function. This will allow the code to be reused by other parts of relax to obtain the average frame order structures.

* Changed the χ2 value checks in the Frame_order.test_cam_quad_int_double_rotor* system tests. These were the values for the quasi-random Sobol' integration and needed updating for the SciPy quadratic integration.

* Removed the skip_tests argument for the Frame_order system tests __init__() method. This argument, which was used to manually turn on or off the blacklisted tests, is no longer needed due to the new --no-skip relax command line flag which will enable all blacklisted tests.

* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] Optimisation_settings object now supports the quad_int flag. This is for activating the SciPy quadratic integration. It is accepted as an argument for the add_grid() and add_min() methods, and it returned by the new get_grid_quad_int() and get_min_quad_int() methods.* Added the ability to specify a pre-run directory in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This will be used for refinement purposes. If the new pre_run_dir argument, modelled on the relaxation dispersion auto-analysis, is supplied then results files will be loaded from this directory and the base data pipe copying and PCS subset optimisation steps will be skipped. The model nesting algorithm is also deactivated.* Activated the SciPy quadratic integration in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. If the Optimisation_settings object has been set up with the quad_int flag, then the auto-analysis will skip the sobol_setup() method and instead directly call the [http://www.nmr-relax.com/manual/frame_order_quad_int.html frame_order.quad_int user function]. Optimisation will then use the SciPy quadratic integration rather than the quasi-random Sobol' integration.

* Improvements for the usage of the [http://www.nmr-relax.com/manual/frame_order_quad_int.html frame_order.quad_int user function] in the auto-analysis. The [http://www.nmr-relax.com/manual/frame_order_quad_int.html frame_order.quad_int user function] is now called even when the Optimisation_settings object quad_int flag is False. This allows for switching between the SciPy quadratic integration and the quasi-random Sobol' integration, as the SciPy quadratic integration can now be turned off.

* Additions to the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] documentation.* Incorporated the contents of the summarise.py script into the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis module]. This has been converted into the summarise() function which will generate a results summary table as the analysis is still running.

* Improved logic in the auto_analyses.frame_order.summarise() function. The model names, directories and titles are now being auto-generated from the full list of frame order models in lib.frame_order.variables.MODEL_LIST. To create a common mechanism for determining the model directory name, the Frame_order_analysis.model_directory() method has been converted into a module function.

* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] now calls the summarise() function at the end to create a summary table.* Shifted the final state saving in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] to be within the safety of the try block.

* Turned off the final state saving in the Frame_order.test_auto_analysis system test. This almost halves the time required for the test. A private class variable _final_state has been added to the auto_analyses.frame_order.Frame_order_analysis class which when False will cause the state saving step to be skipped.

* The summarise() function call is now after saving the final state in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is needed because the summarise() function will create a new set of data pipes, loading the results which already exist under a different pipe name in the relax data store. Otherwise the final state file is twice as big as it should be.* Incorporated the contents of count_sobol_points.py into the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis module]. The analysis script has been converted into the count_sobol_points() function which will generate a summary table of the number of quasi-random Sobol' points used for the PCS numerical integration.* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] now calls the count_sobol_points() function at the end. This is to automatically create the Sobol' point summary table.

* Fixes for the auto_analyses.frame_order.summerise() function. If the count_sobol_points() function is called followed by summerise(), a RelaxError will be raised as the data pipe already exists. The summerise() function has been modified to switch to the data pipe if it already exists.

* Expanded the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] documentation. This adds a description for the summarise() and count_sobol_points() functions.

* Elimination of most of the Frame_order.fixme_test_* system tests and associated data. These tests are from a very early stage of the development of the frame order theory back when the base data was the full and reduced alignment tensors for the each domain calculated from the RDC data. They do not fit into the current analysis where the base data is the RDCs and PCSs for the moving domain. There is no point upgrading the tests as it will be far too much effort and it will only duplicate the coverage of the Frame_order.test_cam_* system tests.

* Renamed the Frame_order.fixme_test_opendx_map system test to Frame_order.test_opendx_map to activate it.

* Fix for the frame order specific API calculate() method. This was caught by the Frame_order.test_opendx_map system test. The scaling matrix was not being specified by the [http://www.nmr-relax.com/manual/dx_map.html dx.map user function] backend and this was causing the method to fail. Instead of passing the non-existent scaling matrix into the target function, the argument is simply ignored. The scaling matrix has no effect on the [http://www.nmr-relax.com/manual/minimise_calculate.html minimise.calculate user function] so it is not necessary.

* The verbosity flag is now being respected by the frame order specific API calculate() method. This silences the method when executing the [http://www.nmr-relax.com/manual/dx_map.html dx.map user function]. The χ2 value printout is suppressed and the verbosity argument is being sent into the frame order count_sobol_points() function.

* Added a section printout to the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] when summary tables are created.

* The [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] now defaults to creating a gzipped PDB file. This is to save room, and because most molecular viewers will automatically read gzipped PDB files.

* Fix for the change of the pipe_control.pipes.test() function to check_pipe().

* Small change in the title of the summary table of the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. 'Order parameters' has been replaced by 'Cone half angles' to clarify what the values really are.

* Fix for the frame order optimisation target setup printouts. The 'Numerical integration: ' printout was fixed to 'Quasi-random Sobol' sequence'. This now changes to 'SciPy quadratic integration' if cdp.quad_int is set. The text 'PCS' has also been added for clarification.

* Removed the call to the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] for the rigid model in the auto-analysis. There is no motion to simulate in the rigid model, so the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] has no use.

* Improvements, fixes, and expansion of the results and data visualisation file creation. This is for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The visualisation() method has been renamed to results_output() and its scope expanded. The function previously only called the [http://www.nmr-relax.com/manual/frame_order.pdb_model.html frame_order.pdb_model] and [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate] user functions for creating PDB representations of the frame order motions and performing a pseudo-Brownian frame order dynamics simulate. This has been extended to also call the [http://www.nmr-relax.com/manual/results_write.html results.write user function] for outputting results files and the rdc.corr_plot and pcs_corr_plot for generating correlation plots of the measured vs. back-calculated data. All parts of the auto-analysis were output files are required now call this method. This ensures that all output files are always created, and are placed into the correct directories.* Improvements for the sectioning printouts for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The sections now use the lib.text.formatting subtitle() and subsubtitle() functions to distinguish them from the output of all the user functions, which use the section(), subsection() and subsubsection() functions. New sectioning printouts have been added for clarity.* Possible fixes for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is just in case a user decides to not perform the optimisation starting with a PCS subset. In this case, the analysis will now execute correctly.* Improvements to the summary table for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The rotor and free rotor model motional eigenframe parameter axis_alpha is now being converted into spherical angles and reported in the table. This allows the motional eigenframe of all models to be easily compared in the table.

* Created a directory and base PDB system for testing out the PCS information content. The base PDB system consists of Ad Bax's CaM domain structures superimposed onto the open CaM structure, the N-domain CoM shifted to the origin, and the C-domain CoM shifted to the z-axis.

* Modified the PCS content testing base system. The paramagnetic centre is now shifted to the origin, as this is the real centre of the PCS physics.

* Intermediate optimisation results are now stored by the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The results from each [http://www.nmr-relax.com/manual/minimise_grid_search.html minimise.grid_search] and [http://www.nmr-relax.com/manual/minimise_execute.html minimise.execute] user function call are now stored in specially named directories located in the 'intermediate_results' directory, which itself is located in the auto-analysis results_dir directory. This allows intermediate results to be more easily analysed later on, which can be useful for optimising the optimisation steps. These directories can also be used for the pre_run_dir auto-analysis argument for subsequent refinements from earlier steps in the optimisation. The results stored include everything from the results_output() method and the count_sobol_points() and summarise() functions. To allow this to work, the auto-analysis functions count_sobol_points() and summarise() required modification. Results files are now always loaded into a temporary data pipe, rather switching to the corresponding pipe, and the temporary data pipe is deleted after the data has been extracted. The original data pipe name is also stored and a switch back to that pipe occurs at the end of each function.* The simulation is now turned of for intermediate results in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The intermediate results are only for checking, so for these the full pseudo-Brownian dynamics simulations are not required. The simulation flag has been introduced into the results_output() method of the auto-analysis to control this.* The splitting of the rigid model grid search into rotation and translation parts is now optional. In the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis], the rigid_grid_split argument has been introduced. The alternating algorithm of performing a grid search over the rotational space followed by translation is now optional and turned off by default. The reason is because the global minimum is sometimes missed with this shortcut algorithm.

* Speedup of the Frame_order.test_auto_analysis system test. The splitting of the rigid model grid search into rotation and translation parts has been reactivated.

* Created the Optimisation.has_grid() method for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is used to test if the optimisation settings object has a grid search defined.* The grid search can now be skipped for the rigid model in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. If the input 3D structures are close to the real solution, the grid search over the translational and rotation parameters of the rigid model could be skipped. This speeds up the analysis and can help find the real solution in problematic cases.* The intermediate results storing can now be turned off in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The new store_intermediate Boolean argument has been added to the analysis to allow the storage of these results to be turned on or off.

* The intermediate results are no longer stored in the Frame_order.test_auto_analysis system test. This drops the test timing on one system from ~190 seconds to ~50 seconds.

* The compression level for results files can now be set in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is via the new argument results_compress_type, which is used to set the compress_type argument of the [http://www.nmr-relax.com/manual/results_write.html results.write user function]. The results reading parts of the auto-analysis have been updated to allow uncompressed, bzip2 compressed, and gzip compressed files to be handled.

* Added a printout of the frame order model in the target function setup function. This is printed out when the [http://www.nmr-relax.com/manual/minimise_calculate.html minimise.calculate], [http://www.nmr-relax.com/manual/minimise_grid_search.html minimise.grid_search], or [http://www.nmr-relax.com/manual/minimise_execute.html minimise.execute] user functions are called, and is for better feedback, especially in the auto-analysis where the repetitive optimisations can be confusing.

* Updated the frame order analysis for the structure.load_spins user function changes. The minimise_setup_atomic_pos() function of the specific_analyses.frame_order.optimisation module now handles the mixed type spin.pos variable correctly.

* The data pipe containing a PCS subset is now optional in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is for systems which have so little data that a subset makes no sense.* Redesigned the optimisation steps for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The code has been significantly simplified as the optimisation for the PCS subset and full data set was the same. The code duplication has been eliminated by combining it into the new optimisation() method. The check for the PCS subset has also been expanded so that it is skipped if the subset data pipe is not supplied, even if an optimisation object for the subset has been (this should prevent strange errors when the auto-analysis is incorrectly used). A side effect of this code merger is that the zooming grid search has now been activated for the full PCS data set. This is of great benefit when a PCS subset is not being used.* The [http://www.nmr-relax.com/manual/minimise_execute.html minimise.execute user function] skip_preset flag is now False in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This is for the main model optimisation. Without this flag set, the grid search for the pivot point position for the rotor model was being skipped at the first zoom level.* The pivot point can now be excluded from the grid search in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. If the initial pivot point is known to be reasonable, then it may be possible to skip it in the grid search for the rotor frame order model. This can lead to a speedup of the analysis and can help with stability. The pivot_search argument has been added to the auto-analysis Optimisation.add_grid() method to enable this. The get_grid_pivot_search() method has also been added to allow the auto-analysis to query this and turn it off if desired.

* Updated the description of the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function]. This now includes the isotopic cone.

* Replaced the table in the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function]. The original table was an old and incorrect version. This has been replaced by the correct permutation table.

* Converted the Sobol' rotation matrices to float32 in the frame order target function. This is to conserve huge amounts of memory to allow for more Sobol' points to be used. For example for the models which use 3D Sobol' points (isotropic cone and pseudo-ellipse), a maximum of 50000 Sobol' points requires 50000000 to be created, using about 15 Gb of RAM.

* A few Frame_order system test updates for the float64 to float32 memory saving changes. The chi-squared value of 3 tests was slightly different.

* Bug fix for the activation of quadratic integration in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The calls to the [http://www.nmr-relax.com/manual/frame_order_quad_int.html frame_order.quad_int user function] in the optimisation() method did not supply an argument so the user function was defaulting to False rather than the True value required.* The [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] summary functions are now more robust. If the data pipe already exists for some reason, it is deleted prior to the new one being created.

* Changed the f[http://www.nmr-relax.com/manual/rame_order_quad_int.html rame_order.quad_int user function] argument default to True. This means that calling the user function without arguments will activate the quadratic integration rather than turning it off.

* Added the isotropic cone model frame order matrix simulation graphs for the cone opening angle θx.

* Created two system tests for the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function] for the double rotor model. These are Frame_order.test_simulate_double_rotor_mode1_xz_plane_tilt and Frame_order.test_simulate_double_rotor_mode2_xz_plane_tilt.

* Added relax scripts which match the Frame_order.test_test_simulate_* system tests. These are the tests of the [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. These were used to construct and visually check the Brownian simulation and PDB model representation in a molecular viewer. These could be a useful reference, so have been added to the repository.

* Fix for the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis ] when only the 'rigid' model is optimised. The final summary table printout for the number of Sobol' points used was failing as there were no models in the table. The table is now only printed out if non rigid models are present in the model list.* Introduced the nested_params_ave_dom_pos argument to the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This allows the average domain position to be set to no rotations and translations rather than taking the average position from the rotor or free-rotor model. This can be useful when large motions are present causing the rigid model to have unreasonable domain positions.

* Fix for the [http://www.nmr-relax.com/manual/frame_order_permute_axes.html frame_order.permute_axes user function] description to allow the manual to be compiled. The table caption containing the user function name was causing the LaTeX compilation to fail. Therefore the captions have been rewritten to avoid the user function name.

* Modified the frame order system test check_chi2() method to test the [http://www.nmr-relax.com/manual/statistics_model.html statistics.model user function]. This causes all of the Frame_order.test_cam_* system tests to fail, as the user function backend is not implemented for the frame order analysis.

* Implemented the back-end of the [http://www.nmr-relax.com/manual/frame_order_distribute.html frame_order.distribute user function]. This follows the design of the pseudo-Brownian simulation [http://www.nmr-relax.com/manual/frame_order_simulate.html frame_order.simulate user function]. The specific_analyses.frame_order.uf.distribute() function has been created as a modified copy of the simulate() function of the same module. This simply performs checks and assembles the data, passing into the new lib.frame_order.simulate.uniform_distribution() function, which itself is a modified copy of the brownian() function in the same module.

* Introduced the max_rotations argument into the [http://www.nmr-relax.com/manual/frame_order_distribute.html frame_order.distribute user function]. This is used to prevent the user function from running forever. This happens whenever a cone opening angle or torsion angle is zero, and hence the random sampling of the rotational space will never find rotations within the motional distribution.

* Improved control of the [http://www.nmr-relax.com/manual/frame_order_distribute.html frame_order.distribute user function] in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. The maximum number of rotations can now be set, and the argument for the total states for the distribution has been shortened.

* Speedup of the Frame_order.test_auto_analysis system test. After the introduction of the [http://www.nmr-relax.com/manual/frame_order_distribute.html frame_order.distribute user function] into the auto-analysis, the test was taking far too long to complete. Now the distribution arguments are set to low values to allow the test to pass in under a minute.

* Changed the default relax results compression type to bzip2 in the [http://www.nmr-relax.com/api/4.0/auto_analyses.frame_order-module.html frame order auto-analysis]. This was set to no compression for speeding up some system tests, however the system tests can set this for themselves.

* The Frame_order.test_auto_analysis system test now sets the results file compression type to bzip2.

* Changed the default max_rotations argument value to 100,000 in the [http://www.nmr-relax.com/manual/frame_order_distribute.html frame_order.distribute user function]. This decrease from one million is so the user function completes in a reasonable amount of time.

Bugman

Trusted, Bureaucrats

4,223

edits