Description

This is a major feature release for a new analysis type labelled 'frame order'. The frame order theory aims to unify all rotational molecular physics data sources via a single mechanical model. It is a bridging physics theory for rigid body motions based on the statistical mechanical ordering of reference frames. Its current implementation is for analysing RDC and PCS data from an internal alignment to interpret domain or other rigid body motions within a molecule or molecular complex.

Download

The new relax versions can be downloaded from http://www.nmr-relax.com/download.html. If binary distributions are not yet available for your platform and you manage to compile the binary modules, please consider contributing these to the relax project (described in section 3.6 of the relax manual, http://www.nmr-relax.com/manual/relax_distribution_archives.html).

CHANGES file

Version 4.0.0
(7 October 2015, from /trunk)
http://svn.gna.org/svn/relax/tags/4.0.0

Features

The final, complete, and correct implementation of the frame order theory for studying rigid body motions. This is currently for analysing RDC and PCS data from internally aligned systems.

Changes

Deletion of the frame_order.average_position user function and all of the associated backend code. This user function allowed the user to specify five different types of displacement to the average moving domain position: a pure rotation, with no translation, about the pivot of the motion in the system; a rotation about the pivot of the motion of the system together with a translation; a pure translation with no rotation; a rotation about the centre of mass of the moving domain with no translation; a rotation about the centre of mass of the moving domain together with a translation. Now the last option will be the default and only option. This option is equivalent to the standard superimposition algorithm (the Kabsch algorithm) to a hypothetical structure at the real average position. The other four are due to the history of the development of the theory. These limit the usefulness of the theory and will only cause confusion.
Clean up of the frame order target function code. This matches the previous change of the deletion of the frame_order.average_position user function. The changes include the removal of the translation optimisation flag as this is now always performed, and the removal of the flag which causes the average domain rotation pivot point to match the motional pivot point as these are now permanently decoupled.
Alphabetical ordering of functions in the lib.frame_order.pseudo_ellipse module.
Eliminated all of the 'line' frame order models, as they are not implemented yet. This is just frontend code - the backend does not exist.
Updated the isotropic cone CaM frame order test model optimisation script. Due to all of the changes in the frame order analysis, the old script was no longer functional.
Created a script for the CaM frame order test models for finding the average domain position. As the rotation about a fixed pivot has been eliminated, the shift from 1J7P_1st_NH_rot.pdb to 1J7P_1st_NH.pdb has to be converted into a translation and rotation about the CoM. This script will be used to replace the pivot rotation Euler angles with the translation vector and CoM rotation Euler angles. However the structure.superimpose user function will need to be modified to handle both the standard centroid superimposition as well as a CoM superimposition.
Updated the CaM frame order test model superimposition script. The structure.superimpose user function is now correctly called. The output log file has been added to the repository as it contains the correct translation and Euler rotation information needed for the test models.
Parameter update for the isotropic cone CaM frame order test model optimisation script. The Euler angles for the rotation about the motional pivot have been replaced by the translation vector and Euler angle CoM rotation parameters.
Fix for a number of the frame order models which do not have parameter constraints. The linear_constraint() function was returning A, b = [], [] for these models, but these empty numpy arrays were causing the minfx library (https://gna.org/projects/minfx/) to fail. These values are now caught and the constraint algorithm turned off in the minimise() specific API method.
Increased the precision of all the data in the CaM frame order test data generation base script. These have all been converted from float16 to float64 numpy types.
Fix for the RDC error setting in the CaM frame order test data generation base script. The rdc_err data structure is located in the interatomic data containers, no the spin containers.
Modification of the structure loading part of the CaM frame order data generation base script. The structures are now only loaded if the DIST_PDB flag is set, as they are only used for generating the 3D distribution of structures. This saves a lot of time and computer memory.
Huge speedup of the CaM frame order test data generation base script. By using multidimensional numpy arrays to store the atomic positions and XH unit vectors of all spins, and performing the rotations on these structures using numpy.tensordot(), the calculations are now a factor of 10 times faster. The progress meter had to be changed to show every 1000 rather than 100 iterations. The rotations of the positions and vectors are now performed sequentially, accidentally fixing a bug with the double motion models (i.e. the 'double rotor' model).
Modified the CaM frame order test data generation base script to conserve computer RAM. The XH vector and atomic position data structures for all N rotations are now of the numpy.float32 rather than numpy.float64 type. The main change is to calculate the averaged RDCs and averaged PCSs separately, deleting the N-sized data structures once the data files are written.
Complete redesign of the CaM frame order data generation base script for speed and memory savings. Although the rotated XH bond vector and atomic position code was very fast, the amount of memory needed to store these in the spin containers and interatomic data containers was huge when N > 1e6. The subsequent rdc.back_calc and pcs.back_calc user function calls would also take far too long. Therefore the base script has been redesigned. The _create_distribution() method has been split into four: _calculate_pcs(), _calculate_rdc(), _create_distribution(), and _pipe_setup(). The _pipe_setup() method is called first to set up the data pipe with all required data. Then the _calculate_rdc() and _calculate_pcs() methods, and finally _create_distribution() if the DIST_PDB flag is set. The calls to the rdc.back_calc and pcs.back_calc user functions have been eliminated. Instead the _calculate_rdc() and _calculate_pcs() methods calculate the averaged RDC and PCS themselves as numpy array structures. Rather than storing the huge rotated vectors and atomic positions data structures, the RDCs and PCSs are summed. These are then divided by self.N at the end to average the values. Compared to the old code, when N is set to 20 million the RAM usage drops from ~20 GB to ~65 MB. The total run time is also decreased on one system from a few days to a few hours (an order or two of magnitude).
Changed the progress meter updating for the CaM frame order test data generation base script. The spinner was far too fast, updating every 5 increments, and is now updated every 250. And the total number is now only printed every 10,000 increments.
Improvements to the progress meter for the CaM frame order test data generation base script. Commas are now printed between the thousands and the numbers are now right justified.
Large increase in accuracy of the RDC and PCS averaging. This is for the CaM frame order test data generation base script. By summing the RDCs and PCSs into 1D numpy.float128 arrays (for this, a 64-bit system is required), and then dividing by N at the end, the average value can be calculated with a much higher accuracy. As N becomes larger, the numerical averaging introduces greater and greater amounts of truncation artifacts. So this change alleviates this.
Fix for the RDC and PCS averaging in the CaM frame order test data generation base script. For the double rotor model, or any multiple motional mode model, the averaging was incorrect. Instead of dividing by N, the values should be divided by N**M, where M is the number of motional modes.
Huge increase in precision for the CaM frame order free rotor model test data. The higher precision is because the number structures in the distribution is now twenty million rather than one million, and the much higher precision numpy.float128 averaging of the updated data generation base script has been used. This data should allow for a much better estimate of the beta and gamma average domain position parameter values for the free rotor models which are affected by the collapse of the alpha parameter to zero.
Huge increase in precision for the CaM frame order double rotor model test data. The higher precision is because the number structures in the distribution is now over twenty million (4500**2) rather than a quarter of a million (500**2). And the much higher precision numpy.float128 averaging of the updated data generation base script has been used.
Fix for the constraint deactivation in the frame order minimisation when no constraints are present.
Huge increase in precision for the CaM frame order rotor model test data. The higher precision is because the number structures in the distribution is now 20 million rather than 166,666, and the numpy.float128 data averaging has been used.
Large increase in precision for the 2nd CaM frame order rotor model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1,000,001 and the numpy.float128 data averaging has been used.
Parameter update for the 2nd rotor CaM frame order test model optimisation script. The Euler angles for the rotation about the motional pivot have been replaced by the translation vector and Euler angle CoM rotation parameters.
Large increase in precision for the 2nd CaM frame order free rotor model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 999,999 and the numpy.float128 data averaging has been used.
Updated the CaM frame order test model superimposition script. The Ca2+ atoms are now deleted from the structures before superimposition so that the centroid matches that used in the frame order analysis.
The average domain rotation centroid is printed out when setting up the frame order target functions. This is to help the user understand what is happening in the analysis.
Faster clearing of numpy arrays in the lib.frame_order modules. The x[:] = 0.0 notation is now used to set all elements to zero, rather than nested looping over all dimensions. This however has a negligible effect on the test suite timings.
Large increase in precision for the CaM frame order pseudo-ellipse model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.
Improved the value setting in the optimisation() method of the CaM frame order system tests. This is in the base script used by all scripts in test_suite/system_tests/scripts/frame_order/cam/.
Changed the average domain position parameter values in the CaM frame order system tests. This is in the base script used by all scripts in test_suite/system_tests/scripts/frame_order/cam/. The translation vector coordinates are now set, as well as the CoM Euler angle rotations. These come from the log file of the test_suite/shared_data/frame_order/cam/superimpose.py script, and are needed due to the simplification of the average domain position mechanics now mimicking the Kabsch superimposition algorithm.
The CaM frame order system test mesg_opt_debug() method now prints out the translation vector. This is printed out at the end of all CaM frame order system tests to help with debugging when the test fails.
Change for how the CaM frame order system test scripts handle the average domain position rotation. The trick of pre-rotating the 3D coordinates was used to solve the {a, b, g} -> {0, b', g'} angle conversion problem in the rotor models no longer works now that the average domain position mechanics has been simplified. Instead, high precision optimised b' and g' values are now set, and the ave_pos_alpha value set to None. The high precision parameters were obtained with the frame_order.py script located in the directory test_suite/shared_data/frame_order/cam/free_rotor. The free rotor target function was modified so that the translation vector is hard-coded to [-20.859750185691549, -2.450606987447843, -2.191854570352916] and the axis theta and phi angles to 0.96007997859534299767 and 4.0322755062196229403. These parameters were then commented out for the model in the module specific_analyses.frame_order.parameters so only b' and g' were optimised. Iterative optimisation was used with increasing precision, ending up with high precision using 10,000 Sobol' points.
Updated a number of the CaM frame order system tests for the higher precision data. The new data results in chi-squared values at the real solution to be much closer to zero.
Change for how the CaM frame order free-rotor pseudo-ellipse test script handle the average position.
Added FIXME comments to the 2nd free-rotor CaM frame order model system test scripts. These explain the steps required to obtain the correct b' and g' average domain position rotation angles.
Large increase in precision for the CaM frame order isotropic cone model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.
Large increase in precision for the CaM frame order free-rotor, isotropic cone model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.
Updated the CaM frame order free-rotor model test data set for testing for missing data. This is the data in test_suite/shared_data/frame_order/cam/free_rotor_missing_data. To simplify the copying of data from test_suite/shared_data/frame_order/cam/free_rotor and then the deletion of data, the missing.py script was created to automate the process. The generate_distribution.py script and some of the files it creates were removed from the repository so it is clearer how the data has been created.
Large increase in precision for the 2nd CaM frame order free-rotor, isotropic cone model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.
Large increase in precision for the CaM frame order free-rotor, pseudo-ellipse model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.
Large increase in precision for the CaM frame order pseudo-ellipse model test data set. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used.
Updated a number of the CaM frame order system tests for the higher precision data. The new data results in chi-squared values at the real solution to be much closer to zero. The free-rotor pseudo-ellipse models might need investigation however as the chi-squared values have increased.
Elimination of the error_flag variable from the frame order analysis. This flag is used to activate some old code paths which have now been deleted as they are never used.
Optimisation of the average domain position for the CaM frame order free-rotor models. The log file that shows the optimisation of the average domain position for the free-rotor models has been added to the repository for reference. This is for the simple free-rotor model, but the optimised position holds for the isotropic cone and pseudo-ellipse model data too. To perform the optimisation, the axis_theta and axis_phi parameters were removed from the model and hardcoded into the target function. As the rotor axis is know, this allows the average domain position to be optimised in isolation. Visual inspection of the results confirmed the position to be correct.
Fixes for the 2nd frame order free-rotor system tests. The average domain position parameters are now set to the correct values, matching those in the relax log file frame_order_ave_pos_opt.log in test_suite/shared_data/frame_order/cam/free_rotor2.
Updated the 2nd CaM free-rotor frame order system tests for the correct average domain position. The chi-squared values are now significantly lower.
Increased the precision of the chi-squared value testing in the CaM frame order system tests. The check_chi2 method has been modified so that the chi-squared value is no longer scaled, and the precision has been increased from 1 significant figure to 4. All of the tests have been updated to match.
The minimisation verbosity flag now effects the frame order RelaxWarning about turning constraints off.
Preformed a frame order analysis on the 2nd CaM free-rotor model test data. This is to check that everything is operating as expected.
Small speedup for the frame order target functions for most models. The rotation matrix corresponding to each Sobol' point for the numerical integration is now pre-calculated during target function initialisation rather than once for each function call.
Updates for some of the frame order system tests for the rotation matrix pre-calculation change. As the rotation matrix is being pre-calculated, one consequence is that the Sobol' angles are now full 64-bit precision rather than 32-bit. Therefore this changes the chi-squared value a little, requiring updates to the tests.
Preformed a frame order analysis on the CaM free-rotor mode test data set. This is to demonstrate that everything is operating correctly.
Preformed a frame order analysis on the CaM free-rotor mode test data set with missing data. This is to demonstrate that everything is operating correctly.
Attempt to speed up the pseudo-elliptic frame order models. The quasi-random numerical integration of the PCS for the pseudo-ellipse has been modified so that the torsion angle check for each Sobol' point is preformed before the tmax_pseudo_ellipse() function call. A new check that the tilt angle is less than cone_theta_y, the larger of the two cone angles, has also been added to avoid tmax_pseudo_ellipse() when the theta tilt angle is outside of an isotropic cone defined by cone_theta_y.
Preformed a frame order analysis on a number of the CaM test data sets. This includes the rotor, isotropic cone, and pseudo-ellipse, and the analyses demonstrate a common bug between all these models.
Preformed a frame order analysis on the rigid CaM test data set. This is to demonstrate that everything is operating correctly.
Optimisation of the rotor model to the rigid CaM frame order test data. The optimisation script and all results files have been added to the repository.
Increased the grid search bounds for the frame order average domain translation. Instead of being a 10 Angstrom box centred at {0, 0, 0}, now the translation search has been increased to a 100 Angstrom box.
Proper edge case handling and slight speedup of the frame order PCS integration functions. The case whereby no Sobol' points in the numerical integration lie within the motional distribution is now caught and the rotation matrix set to the motional eigenframe to simulate the rigid state. As the code for averaging the PCS was changed, it was also simplified by removing an unnecessary loop over all spins. This should speed up the PCS integration by a tiny amount.
Created a new CaM frame order test data set. This is for the rotor model with a very small torsion angle of 1 degree, and will be used as a comparison to the rigid model and for testing the performance of the rotor model for an edge case.
Updated the frame order representations in all of the frame_order.py scripts for the CaM test data. All PDB files are now gzipped to save space, the old pymol.cone_pdb user function calls replaced with pymol.frame_order, and an average domain PDB file for the exact solution is now created in all cases.
The minimisation constraints are now turned on for all CaM test data frame_order.py optimisation scripts.
Updated the rotor CaM test data frame_order.py script for the parameter reduction. The rotor axis {theta, phi} polar angles have been replaced by the single axis alpha angle. This now matches the script for the 2nd rotor model.
Updated the parameters in all of the frame_order.py scripts for the CaM test data. The parameters are now specified at the top of the script as variables. All scripts now handle the change to the translation + CoM rotation for the average domain position rather than having a pure rotation about a fixed pivot, which is no longer supported.
The frame_order.num_int_pts user function now throws a RelaxWarning if not enough points are used.
Changed the creation of Sobol' points for numerical integration in the frame order target functions. The points are now all created at once using the i4_sobol_generate() rather than i4_sobol() function from the extern.sobol.sobol_lib module.
Increased the number of integration points from 50 or 100 to 5000. This is for all CaM frame_order.py test data optimisation scripts. The higher number of points are essential for optimising the frame order models and hence for checking the relax implementation.
Updated the frame_order.py optimisation script for the small angle CaM rotor frame order test data. This now has the correct rotor torsion angle of 1 degree, and the spherical coordinates are now converted to the axis alpha parameter.
Expanded the capabilities of the pymol.frame_order user function. The isotropic and pseudo-elliptic cones are now represented as they used to be under the pymol.cone_pdb user function. To avoid code duplication, the new represent_cone_axis(), represent_cone_object() and represent_rotor_object() functions have been created to send the commands into PyMOL.
Increased the precision of all of the CaM frame order system tests by 40 times. The number of Sobol' integration points have been significantly increased while only increasing the frame order system test timings by ~10%. This allows for checking for chi-squared values at the minima much closer to zero, and is much better for demonstrating bugs.
Optimisation constraints are no longer turned off in the frame order auto-analysis. Constraints are now supported by all frame order models, or automatically turned off for those which do not have parameter constraints.
Fix for the frame order visualisation script created by the auto-analysis. The call to pymol.frame_order is now correct for the current version of this user function.
Removed a terrible hack for handling the frame order analysis without constraints. This is no longer needed as the log-barrier method is now used to constrain the optimisation, so that the torsion angle can no longer be negative.
Constraints are now implemented in the frame order grid search. This is useful for the pseudo-elliptic models as the cone theta_x < theta_y constraint halves the optimisation space.
Expanded the CaM rotor test data frame_order.py optimisation script. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower. The results of the new optimisation are included.
Expanded the CaM pseudo-ellipse test data frame_order.py optimisation script. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower. The results of the new optimisation are included.
Added one more iteration for the zooming optimisation of the frame order auto-analysis. This is to improve the speed of optimisation when all RDC and PCS data is being used. The previous iterations where with [100, 1000, 200000] Sobol' integration points and [1e-2, 1e-3, 1e-4] function tolerances. This has been increased to [100, 1000, 10000, 100000] and [1e-2, 1e-3, 5e-3, 1e-4]. The final number of points has been decreased as that level of accuracy does not appear to be necessary. These are also only default values that the user can change for themselves.
Updated the CaM frame order data generation base script to print out more information. This is for the first axis system so that the same amount of information as the second system is printed.
Expanded the CaM isotropic cone test data frame_order.py optimisation script and added the results. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower.
Important fix for the 2nd rotor model of the CaM frame order test data. The tilt angle was not set, and therefore the old data matched the non-tilted 1st rotor model. All PCS and RDC data has been regenerated to the highest quality using 20,000,000 structures.
Updated the 3 Frame_order.test_cam_rotor2* system tests for the higher quality data.
Expanded the 2nd CaM pseudo-ellipse test data frame_order.py optimisation script. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower. The results of the new optimisation have been added to the repository.
Expanded the CaM free-rotor isotropic cone test data frame_order.py optimisation script. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower. The results of the new optimisation have been added to the repository.
Expanded all remaining CaM test data frame_order.py optimisation scripts. The optimisation is now implemented as in the auto-analysis, with an iterative increase in accuracy of the quasi-random numerical integration together with an decrease of the function tolerance cutoff for optimisation. The accuracy of the initial chi-squared calculation is now much higher. And the accuracy of the initial grid search and the Monte Carlo simulations is now much lower.
Updated the CaM 2-site to rotor model frame_order.py optimisation script for the parameter reduction. The rotor frame order model axis spherical angles have now been converted to a single alpha angle.
Fix for a number of the frame order models which do not have parameter constraints. This change to the grid_search() API method is similar to the previous fix for the minimise() method. The linear_constraint() function was returning A, b = [], [] for these models, but these empty numpy arrays were causing the dot product with A to fail in the grid_search() API method. These values are now caught and the constraint algorithm turned off.
Converted the 'free rotor' frame order model to the new axis_alpha parameter system. The axis_theta and axis_phi spherical coordinates are converted to the new reduced parameter set defined by a random point in space (the CoM of all atoms), the pivot point, and a single angle alpha. The alpha parameter defines the rotor axis angle from the xy-plane.
Parameter conversion for all of the CaM free rotor test data frame_order.py optimisation scripts. The rotor axis spherical angles have been replaced by the axis alpha angle defining the rotor with respect to the xy-plane.
Modified the CaM frame order base system test script to catch a bug in the free rotor model. The axis spherical angles are no longer set for the rotor or free rotor models, as they use the alpha angle instead and the lack of the theta and phi parameters triggers the bug. The PDB representation of the frame order motions is also now tested for all frame order models, as it was turned off for the rigid, rotor and free rotor models and this is where the bug lies.
Fix for the failure of the frame_order.pdb_model user function for the free rotor frame order model. This is due to the recent parameter conversion to the axis alpha angle.
Eliminated the average position alpha Euler angle parameter from the free-rotor pseudo-ellipse model. As this frame order model is a free-rotor, the average domain position is therefore undefined and it can freely rotate about the rotor axis. One of the Euler angles for rotating to the average position can therefore be removed, just as in the free rotor and free rotor isotropic cone models.
Eliminated the free rotor psuedo-ellipse model ave_pos_alpha parameter from the target function. The average domain position alpha Euler angle has already been removed from the specific analyses code and this change brings the target function into line with these changes.
Added the full optimisation results for the 2nd rotor frame order model for the CaM test data. This is from the new frame_order.py optimisation script and the results demonstrate the stability of the rotor model.
Added the full optimisation results for the small angle rotor CaM frame order test data. This is from the new frame_order.py optimisation script and the results demonstrate the stability of the rotor model, even when the rotor is as small as 1 degree.
Fix for the free rotor PDB representation created by the frame_order.pdb_model user function. The simulation axes were being incorrectly generated from the theta and phi angles, which no longer exist as they have been replaced by the alpha angle.
Added the full optimisation results for the free rotor pseudo-ellipse frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Added the full optimisation results for the rotor frame order model. This is for the 2-site CaM test data using the new frame_order.py optimisation script.
The CaM frame order data generation base script now uses lib.compat.norm(). This is to allow the test suite to pass on systems with old numpy versions whereby the numpy.linalg.norm() function does not support the new axis argument.
Modified the pymol.cone_pdb and pymol.frame_order user functions to use PyMOL IDs. The PyMOL IDs are used to select individual objects in PyMOL rather than all objects so that the subsequent PyMOL commands will only be applied to that object. This allows for multiple objects to be handled simultaneously.
Added the full optimisation results for the free rotor frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Added the full optimisation results for the 2nd free rotor frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Added the full optimisation results for the free rotor frame order model with missing data. This is for the CaM test data using the new frame_order.py optimisation script.
Added a script for recreating the frame order PDB representation and displaying it in PyMOL. This is for the optimised results.
Fixes for the rotor object created by the frame_order.pdb_model user function. The rotor is now also shown for the free rotor pseudo-ellipse, despite it being a useless model, and the propeller blades are no longer staggered for all the free rotor models so that two circles are no longer produced.
Updated the free rotor and 2nd free rotor PDB representations using the represent_frame_order.py script. This is for the CaM frame order test data.
Reparameterisation of the double rotor frame order model. The two axes defined by spherical angles have been replaced by a full eigenframe and the second pivot has been replaced by a single displacement along the z-axis of the eigenframe.
Removed the 2nd pivot point infrastructure from the frame order analysis. The 2nd pivot is now defined via the pivot_disp parameter.
Added the 2nd rotor axis torsion angle to the list of frame order parameters. This is for the double rotor model.
Comment fixes for the eigenframe reconstruction in the frame order target functions.
Converted the double rotor frame order model target function to use the new parameterisation.
Fix for the PDB representation generated by frame_order.pdb_model for the free rotor pseudo-ellipse.
Fix for the Frame_order.test_rigid_data_to_free_rotor_model system test. As the free rotor has undergone a reparameterisation, the chi-squared value is now higher. The value is reasonable as the free rotor can never model the rigid system.
Removed the structure loading and transformation from the CaM frame order system tests. This was mimicking the old behaviour of the auto-analysis. However as that behaviour has been shifted into the backend of the frame_order.pdb_model user function, which is called by these system tests as well, the code is now redundant and is wasting test suite time.
Removed the setting of the second pivot point in the CaM frame order system tests. The second pivot point has been removed from the double rotor frame order model to eliminate parameter redundancy, so no models now have a conventional second pivot.
Modified the CaM frame order system test base script to test alternative code paths. This pivot point was fixed in all tests, so the code in the target functions behind the pivot_opt flag was not being tested. Now for those system tests whereby the calc rather than minimise user function is called, the pivot is no longer fixed to execute this code.
Simplification and clean up of the RDC and PCS flags in the frame order target functions. The per-alignment flags have been removed and replaced by a global flag for all data. This accidentally fixes a bug when only RDCs are present, as the calc_vectors() method was being called when it should not have been.
Speedup and simplifications for the vector calculations used for the PCS numerical integration. This has a minimal effect on the total speed as the target function calc_vectors() method is not the major bottleneck - the slowest part is the quasi-random numerical integration. However the changes may be useful for speeding up the integration later on. The 3D pivot point, average domain rotation pivot, and paramagnetic centre position arrays are now converted into rank-2 arrays in __init__() where the first dimension corresponds to the spin. Each element is a copy of the 3D array. These are then used for the calculation of the pivot to atom vectors, eliminating the looping over spins. The numpy add() and subtract() ufuncs are used together with the out argument for speed and to avoid temporary data structure creation and deletion. The end result is that the calculated vector structure is transposed, so the first dimension are the spins. The changes required minor updates to a number of system tests. The target functions themselves had to be modified so that the pivot is converted to the larger structure when optimised, or aliased.
Added a script for timing different ways to calculate PCSs and RDCs for multiple vectors. This uses the timeit module rather than profile to demonstrate the speed of 7 different ways to calculate the RDCs or PCSs for an array of vectors using numpy. In the frame order analysis, this is the bottleneck for the quasi-random numerical integration of the PCS. The log file shows a potential 1 order of magnitude speedup between the 1st technique, which is currently used in the frame order analysis, and the 7th and last technique. The first technique loops over each vector, calculating the PCS. The last expands the PCS/RDC equation of the projection of the vector into the alignment tensor, and calculates all PCSs simultaneously.
Added another timing script for RDC and PCS calculation timings. This time, the calculation for multiple alignments is now being timed. An addition set of methods for calculating the values via tensor projections have been added. For 5 alignments and 200 vectors, this demonstrates a potential 20x speedup for this part of the RDC/PCS calculation. Most of this speedup should be obtainable for the numerical PCS integration in the frame order models.
Small speedup for all of the frame order models. The PCS averaging in the quasi-random numerical integration functions now uses the multiply() and divide() numpy methods to eliminate a loop over the alignments. For this, a new dimension over the spins was added to the PCS constant calculated in the target function __init__() method. In one test of the pseudo-ellipse, the time dropped from 191 seconds to 172.
Added another timing script for helping with speeding up the frame order analysis. This is for the part where the rotation matrix for each Sobol' integration point is shifted into the eigenframe.
Python 3 fix for the CaM frame order system test base script.
Added the full optimisation results for the torsionless isotropic cone frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Small speedups for all of the frame order models in the quasi-random numerical PCS integration. These changes result in an ~10% speedup. Testing via the func_pseudo_ellipse() target function using the relax profiling flag, the time for one optimisation decreased from 158 to 146 seconds. The changes consist of pre-calculating all rotations of the rotation matrix into the motional eigenframe in one mathematical operation rather than one operation per Sobol' point rotation, unpacking the Sobol' points into the respective angles prior to looping over the points, and taking the absolute value of the torsion angle and testing if it is out of the bounds rather than checking both the negative and positive values.
Attempt at speeding up the torsionless pseudo-ellipse frame order model. The check if the Sobol' point is outside of an isotropic cone defined by the largest angle theta_y is now performed to avoid many unnecessary calls to the tmax_pseudo_ellipse() function. This however reveals a problem with the test suite data for this model.
Updated all of the CaM frame order system tests for the recent speedup. The speedup switched to the use of numpy.tensordot() for shifting each Sobol' rotation into the eigenframe rather than the previous numpy.dot(). Strangely this affects the precision and hence the chi-squared value calculated for each system test - both increasing and decreasing it randomly.
The frame order target function calc_vectors() method arguments have all been converted to keywords. This is in preparation for handling a second pivot argument for the double rotor model.
Updated the double rotor frame order model to be in a pseudo-functional state. Bugs in the target function method have been removed, the calc_vectors() target function now accepts the pivot2 argument (but does nothing with it yet), and the lib.frame_order.double_rotor module has been updated to match the logic used in all other lib.frame_order modules.
The frame_order.pdb_model user function no longer tries to create a cone object for the double rotor.
Added a timeit script and log file for different ways of checking a binary numpy array.
Modified the rigid_test.py system test script to really be the rigid case. This is used in all of the Frame_order.test_rigid_data_to_*_model system tests. Previously the parameters of the dynamics were set to being close to zero, to catch the cases were a few Sobol' PCS integration points were accepted. But now the case were no Sobol' points can be used is being tested. This checks a code path currently untested in the test suite, demonstrating many failures.
Fix for the frame order matrix calculation for a pseudo-elliptic cone with angles of zero degrees. The lib.frame_order.pseudo_ellipse_torsionless.compile_2nd_matrix_pseudo_ellipse_torsionless() function has been changed to prevent a divide by zero failure. The surface area normalisation factor now defaults to 0.0.
Fixes for all PCS numeric integration for all frame order models in the rigid case. The exact PCS values for the rigid state are now correctly calculated when no Sobol' points lie within the motional model. The identity matrix is used to set the rotation to zero, and the PCS values are now multiplied by the constant.
Updates for the chi-squared value in all the Frame_order.test_rigid_data_to_*_model system tests. This is now much reduced as the true rigid state is now being tested for.
The rigid frame order matrix for the pseudo-ellipse models is now correctly handled. This allows the rigid case RDCs to be correctly calculated for both the pseudo-ellipse and torsionless pseudo-ellipse models. The previous catch of the theta_x cone angle of zero was incorrectly recreating the frame order matrix, which really should be the identity matrix. However truncation artifacts due to the quadratic SciPy integration still cause the model to be ill-conditioned near the rigid case. The rigid case is correctly handled, but a tiny shift of the parameters off zero cause a discontinuity.
Updates for the Frame_order.test_rigid_data_to_pseudo_ellipse*_model system tests. The chi-squared value now matches the rigid model.
Large increase in precision for the CaM frame order torsionless pseudo-ellipse model test data set. In addition, the theta_x and theta_y angles have also been swapped so that the new constraint of 0 <= theta_x <= theta_y <= pi built into the analysis is satisfied. The higher precision is because the number structures in the distribution is now 20 million rather than 1 million and the numpy.float128 data averaging has been used. The algorithm for finding suitable random domain positions within the motional limits has been changed as well by extracting the theta and phi tilt angles from the random rotation, dropping the torsion angle sigma, and reconstructing the rotation from just the tilt angles. This increases the speed of the data generation script by minimally 5 orders of magnitude.
Changed the parameter values for the Frame_order.test_cam_pseudo_ellipse_torsionless* system tests. The theta_x and theta_y angles are now swapped. The chi-squared values are now also lower in the 3 system tests as the data is now of much higher precision.
Speedup for the frame order analyses when only one domain is aligned. When only one domain is aligned, the reverse Ln3+ to spin vectors for the PCS are no longer calculated. For most analyses, this should significantly reduce the number of mathematical operations required for the quasi-random Sobol' point numerical integration.
Support for the 3 vector system for double motions has been added to the frame order analysis. This is used for the quasi-random Sobol' numeric integration of the PCS. The lanthanide to atom vector is the sum of three parts: the 1st pivot to atom vector rotated by the 1st mode of motion; the 2nd pivot to 1st pivot vector rotated by the 2nd mode of motion (together with the rotated 1st pivot to atom vectors); and the lanthanide to second pivot vector. All these vectors are passed into the lib.frame_order.double_rotor.pcs_numeric_int_double_rotor() function, which passes them to the pcs_pivot_motion_double_rotor() function where they are rotated and reconstructed into the Ln3+ to atom vectors.
Fully implemented the double rotor frame order model for PCS data. Sobol' quasi-random points for the numerical integration are now generated separately for both torsion angles, and two separate sets of rotation matrices for both angles for each Sobol' point are now pre-calculated in the create_sobol_data() target function method. The calc_vectors() target function method has also been modified as the lanthanide to pivot vector is to the second pivot in the double rotor model rather than the first. The target function itself has been fixed as the two pivots were mixed up - the 2nd pivot is optimised and the inter-pivot distance along the z-axis gives the position of the 1st pivot. For the lib.frame_order.double_rotor module, the second set of Sobol' point rotation matrices corresponding to sigma2, the rotation about the second pivot, is now passed into the pcs_numeric_int_double_rotor() function. These rotations are frame shifted into the eigenframe of the motion, and then correctly passed into pcs_pivot_motion_double_rotor(). The elimination of Sobol' points outside of the distribution has been fixed in the base pcs_numeric_int_double_rotor() function and now both torsion angles are being checked.
Fix for the unpacking of the double rotor frame order parameters in the target function. This is for when the pivot point is being optimised.
Created a new synthetic CaM data set for the double rotor frame order model. This is the same as the test_suite/shared_data/frame_order/cam/double_rotor data except that the angles have been increased from 11.5 and 10.5 degrees to 85.0 and 55.0 for the two torsion angles. This is to help in debugging the double rotor model as the original test data is too close to the rigid state to notice certain issues.
Corrected the printout from the CaM frame order data generation base script. The number of states used in the distribution of domain positions is now correctly reported for the models with multiple modes of motion.
Created a frame order optimisation script for the CaM double rotor test suite data. This is the script used for testing the implementation, it will not be used in the test suite.
Created the Frame_order.test_rigid_data_to_double_rotor_model system test. This shows that the double rotor model works perfectly when the domains of the molecule are rigid.
Fix for the frame order target functions for when no PCS data is present. In this case, the self.pivot structure was being created as an empty array rather than a rank-2 array with dimensions 1 and 3. This was causing the rotor models to fail, as this pivot is used to recreate the rotation axis.
Fix for the CaM double rotor frame order system tests. The torsion angle cone_sigma_max is a half angle, therefore the full angles from the data generation script are now halved in the system test script.
Created 3 frame order system tests for the new large angle double rotor CaM synthetic data. These are the Frame_order.test_cam_double_rotor_large_angle, Frame_order.test_cam_double_rotor_large_angle_rdc, and Frame_order.test_cam_double_rotor_large_angle_pcs system tests.
Added the full optimisation results for the torsionless pseudo-ellipse frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Added the full optimisation results for the 2nd free rotor isotropic cone frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Small fix for the large angle CaM double rotor frame order model synthetic test data. The way the rotation angle was calculated was slightly out due to integer truncation. The integers are now converted to floats in the generate_distribution.py script and all of the PCS and RDC data averaged over ~20 million states has been recalculated.
Added proper support for the double rotor frame order models to the system test scripts. This is for the CaM synthetic data. The base script can now handle the current parameterisation of the double rotor model with a single pivot, an eigenframe, and the second pivot defined by a displacement along the z-axis. The scripts for the double_rotor and double_rotor_large_angle data sets have been changed to use this parameterisation as well.
Attempt at implementing the 2nd degree frame order matrix for the double rotor model. This is required for the RDC.
The second torsion angle is now printed out for the frame order system tests. This is in the system test class mesg_opt_debug() method and allows for better debugging of the double rotor models.
Fix for the Frame_order.test_cam_double_rotor_large_angle* system tests. The system test script was pointing to the wrong data directory.
The double rotor frame order system tests are no longer blacklisted.
Updated the chi-squared values being checked for the double rotor frame order system tests.
Shifted the frame order geometric representation functions into their own module. This is the new specific_analyses.frame_order.geometric module.
The frame order geometric representation functions are no longer PDB specific. Instead the format argument is accepted. This will allow different formats to be supported in the future. Because of this change, all specific_analyses.frame_order.geometric.pdb_*() functions has been renamed to create_*().
Created an auxiliary function for automatically generating the pivots of the frame order analysis. This is the new specific_analyses.frame_order.data.generate_pivot() function. It will generate the 1st or 2nd pivot, hence supporting both the single motion models and the double motion double rotor model.
Shifted the rotor generation for the frame order geometric representation into its own function. This is the specific_analyses.frame_order.geometric.add_rotors() function which adds the rotors are new structures to a given internal structural object. The code has been extended to add support for the double rotor model.
Fix for the pivots created by the specific_analyses.frame_order.data.generate_pivot() function. This is for the double rotor model where the 1st mode of motion is about the 2nd pivot, and the 2nd mode of motion about the 1st pivot.
Fixes for the cone geometric representation in the internal structural object. The representation can now be created if the given MoleculeContainer object is empty.
Refactored the frame order geometric motional representation code. The code of the specific_analyses.frame_order.geometric.create_geometric_rep() function has been spun out into 3 new functions: add_rotors(), add_axes(), and add_cones(). This is to better isolate the various elements to allow for better control. Each function now adds the atoms for its geometric representation to a separate molecule called 'axes' or 'cones'. The add_rotors() does not create a molecule as the lib.structure.represent.rotor.rotor_pdb() function creates its own. As part of the rafactorisation, the neg_cone flag has been eliminated.
Renamed the residues of the rotor geometric object representation. The rotor axis atoms now belong to the RTX residue and the propeller blades to the RTB residue. The 'RT' at the start represents the rotor and this will allow all the geometric objects to be better isolated.
Improvements to the internal structural object _get_chemical_name() method. This now uses a translation table to convert the hetID or residue name into a description, for example as used in the PDB HETNAM records to give a human readable description of the residue inside the PDB file itself. The new rotor RTX and RTB residue names have been added to the table as well.
Renaming of the residues of the cone geometric representation. The cone apex or centre is now the CNC residue, the cone axis is now CNX and the cone edge is now CNE. These used to be APX, AXE, and EDG respectively. The aim is to make these names 100% specific to the cone object so that they can be more easily selected for manipulating the representation and so that they are more easily identifiable. The internal structural object _get_chemical_name() function now returns a description for each of these. Note that the main cone object is still named CON.
The motional pivots for the frame order models are now labelled in the geometric representation. The pivot points are now added as a new molecule called 'pivots' in the frame_order.pdb_model user function. The atoms all belong to the PIV residue. The pymol.frame_order user function now selects this residue, hides its atoms, and then shows the atom name 'Piv' as the label. For the double rotor model, the atom names 'Piv1' and 'Piv2' are used to differentiate the pivots.
Renamed the lib.structure.represent.rotor.rotor_pdb() function to rotor(). This function is not PDB specific and it just creates a 3D structural representation of a rotor object.
Added support for labels in the rotor geometric object for the internal structural object. The labels are created by the frame_order.pdb_model user function backend. For the double rotor model, these are 'x-ax' and 'y-ax'. For all other models, the label is 'z-ax'. The labels are then sent into the lib.structure.represent.rotor.rotor() function via the new label argument. This function adds two new atoms to the rotor molecule which are 2 Angstrom outside of the rotor span and lying on the rotor axis. These then have their atom name set to the label. The residue name is set to the new RTL name which has been added to the internal structural object _get_chemical_name() method to describe the residue in the PDB file for the user. Finally the pymol.frame_order user function selects these atoms, hides them and then labels them using the atom name (x-ax, y-ax, or z-ax).
Modified the rotor representation generated by the pymol.frame_order user function. This is to make the object less bulky.
Redesign of the axis geometric representation for the frame order motions. This is now much more model dependent to avoid clashes with the rotor objects and other representations: For the torsionless isotropic cone, a single z-axis is created; For the double rotor, a single z-axis is produced connecting the two pivots, from pivot2 to pivot1; For the pseudo-ellipse and free rotor pseudo-ellipse, the x and y-axes are created; For the torsionless pseudo-ellipse, all three x, y and z-axes are created; For all other models, no axis system is produced as this has been made redundant by the rotor objects.
Fixes for the cone geometric object created by the frame_order.pdb_model user function. This was broken by the code refactoring and now works again for the pseudo-ellipse models.
Fix for the pymol.frame_order user function. The representation function for the rotor objects was hiding all parts of the representation, hence the pivot labels where being hidden. To fix this, the hiding of the geometric object now occurs in the base frame_order_geometric() function prior to setting up the representations for the various objects.
Started to redesign the frame_order.pdb_model user function. Instead of having the positive and negative representations in different PDB models, and the Monte Carlo simulations in different molecules, these will now all be shifted into separate files. For this to be possible, the file root rather than file names must now be supplied to the frame_order.pdb_model user function. To allow for different file compression, the compress_type argument is now used. The backend code correctly handles the file root change, but the multiple files are not created yet.
Python 3 fixes using the 2to3 script. Fatal changes to the multi.processor module were reverted.
Improvements to the lib.structure.represent.rotor.rotor() function for handling models. The 'rotor', 'rotor2', or 'rotor3' molecule name determination is now also model specific.
The frame order generate_pivot() function can now return the pivots for Monte Carlo simulations. This is the specific_analyses.frame_order.data.generate_pivot() function. The sim_index argument has been added to the function which will allow the pivots from the Monte Carlo simulations to be returned. If the pivot was fixed, then the original pivot will be returned instead.
Test suite fixes for the recent redesign of the frame_order.pdb_model user function.
Fixes for the frame_order.pdb_model user function for the rotor and free rotor models.
Redesign of the geometric object representation part of the frame_order.pdb_model user function. The positive and negative representations of the frame order motions have been separated out into two PDB files rather than being two models of one PDB file. This will help the user understand that there are two identical representations of the motions, as both will now be displayed rather than having to understand the model concept of PyMOL. The file root is taken, for example 'frame_order', and the files 'frame_order_pos.pdb' and 'frame_order_neg.pdb' are created. If no inverse representation exists for the model, the file 'frame_order.pdb' will be created instead. The Monte Carlo simulations are now also treated differently. Rather than showing multiple vectors in the axes representation component within one molecule in the same file as the frame order representation, these are now in their own file and each simulation is now a different model. If an inverse representation is present, then the positive representation will go into the file 'frame_order_sim_pos.pdb', for example, and the negative representation into the file 'frame_order_sim_neg.pdb'. Otherwise the file 'frame_order_sim.pdb' will be created.
Clean up of the frame_order.pdb_model user function definitions. Some elements were no longer of use, and some descriptions have been updated.
Redesign of the pymol.frame_order user function to match the redesign of frame_order.pdb_model. The file names are no longer given but rather the file root. Then all PDB files matching that file root in the given directory will be loaded into PyMOL.
Updated all of the frame order scripts for the frame_order.pdb_model and pymol.frame_order changes. These are the scripts for the CaM frame order test data.
Redesign of the average domain position part of the frame_order.pdb_model user function. The Monte Carlo simulations are now represented. If the file root is set to the default of 'ave_pos', then these will be placed in the file 'ave_pos.pdb', or a compressed version. Each simulation is in a different model, matching the geometric representation '*_sim.pdb' files. The original structure is copied for each model, and then rotated to the MC simulation average position.
Change all of the domain user function calls in the frame order CaM test data scripts. The domains are now identified by the molecule name rather than the range of residues. This allows non-protein atoms, for example the Ca2+ atoms, to be rotated to the average domain position as well.
The PyMOL disable command is now used by the pymol.frame_order user function. This is to first disable all PyMOL objects prior to loading anything, to hide the original structures and any previous frame order representations, and then to hide all of the Monte Carlo simulation representations. This is to simplify the picture initially presented to the user while still allowing all elements to be easily found.
The pymol.frame_order user function now centers and zooms on all objects.
Simplified the PyMOL view commands in all of the CaM test data optimisation scripts. The pymol.view user function is not necessary as the PyMOL GUI will be launched by the pymol.frame_order user function. And the pymol.command user function call for running the 'hide all' command is also now redundant.
Removed all remaining uncompressed PDB files from the CaM test data directories. These were complicating the debugging of the pymol.frame_order user function, as they were being loaded on top of the compressed versions.
Removed some rotation files from the CaM frame order test data directories. These files are no longer of any use and just take up large amounts of room for nothing.
Added titles to the frame order geometric representation PDB files from frame_order.pdb_model. These are in the form of special Ti atoms placed 40 Angstrom away from the pivot along the z-axis of the system, or shifted 3 more Angstrom for the Monte Carlo simulations. These are used to label the alternative representations or the Monte Carlo simulation representations. The residue type is set to TLE and this has been registered in the internal structural object. The pymol.frame_order user function now calls the represent_titles() function to select these atoms, hide them, and then add a long descriptive title. The atom name is used to distinguish between different titles.
Changed the alternative representation names for the frame order geometric objects. The aim is to put both representations on a more equal footing, as they are identical solutions. Hence the inverted representation might be the correct representation of the domain motions. So instead of calling these 'positive' and 'negative', the 'A' and 'B' notation will be used. This affects the names of the files produced by the frame_order.pdb_model user function as well as the internal titles. Instead of ending the files with "*_pos.*" and "*_neg.*", these have been changed to "*_A.*" and "*_B.*". The atoms used for the titles have also been renamed, and the pymol.frame_order user function now labels the titles using the 'A' and 'B' notation.
Changes to the rotor object in the frame order geometric representations. For the isotropic and pseudo-elliptic cone models, the rotor is now halved. Instead of having two axes radiating from the central pivot and terminating in the propeller blades, now only the positive axis is shown lying in the centre of the cone.
Fixes for the MC simulation rotor objects in the frame order geometric representation. The axes of the Monte Carlo simulation rotors objects were being set to the original values and not to the simulation values.
Fixes for the titles in the frame order geometric representation from frame_order.pdb_model. There were a few bugs for a number of the frame order models preventing this code from working.
Redesign of the geometric representation of the cone structural objects to allow for models. The old representation was not compatible with the PDB model concept whereby each model must have the same number of atoms. To handle this situation, the cone objects have been simplified. Specifically the cone cap. The old behaviour was to remove all points outside of the cone when creating the cone cap, and then to stitch the cap to the cone edge in a subsequent step. Now the behaviour is that all points outside of the distribution are shifted to the cone edge. This avoids the need to stitch the cap to the edge. This behaviour means that all cones with the same inc value will have the same number of atoms. The cones for the pseudo-ellipses are not as nice as the latitudinal lines are not strait at the cone edge, but at least creating multiple models with different cone sizes is now possible.
Bug fix for the y-axis rotation matrix for the double rotor Sobol' integration points. The matrix was inverted.
Updated the frame order system test chi-squared values for the previous fix.
Fixes for the double rotor frame order system tests for the CaM synthetic data. The torsion angles needed to be swapped and the pivot point changed from the C terminal domain CoM to the N domain CoM.
More fixes for the double rotor frame order system tests for the CaM synthetic data. The eigensystem was inverted.
Updated the chi2 check for the large angle double rotor frame order system tests. This is needed for the eigenframe fix.
Updates for the frame order system tests for the float32 to float64 change. Some chi-squared values have slightly changed.
The CaM frame order test data optimisation scripts now save more state files. The state of the true dynamics and the fixed pivot optimisation results are now stored as well. This might be useful for extracting these results without redoing the calculations.
The script for representing the frame order dynamics for the CaM test data has been updated. The domains of the system are now defined.
Changed the CaM frame order test data superimposition values. Because the domains are now defined via the molecule name rather than the residue numbers, the centroid of rotation set to the CoM has been shifted as now the Ca2+ ions are included in the CoM calculation. Therefore the superimpose.py script has been updated to not delete the Ca atoms. All of the frame order optimisation scripts have been updated with the new rotation Euler angles and translation vector. To match this, the system test base script for the CaM frame order test data has also had its rotations and translations updates, and the domain user function call changed to use molecule names.
Updated all of the CaM frame order system test chi-squared values. These have changed slightly due to the rotation and translation changes.
Added support for the 'pivot_disp' frame order parameter to the grid search. This is required for the double rotor model.
Changed some of the default values for the frame order auto-analysis. The number of Sobol' quasi-random integration points were far too low to obtain any reasonable results.
Simplified the PyMOL visualisation relax script created by the frame order auto-analysis. This now consists of a single pymol.frame_order user function call. The other pymol user function calls were unnecessary.
Added the full optimisation results for the large angle double rotor frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Added model support for the rotor geometric object. This is the structural object used in the frame order analysis to create PDB representations of rotor motions. The number of atoms created for the rotor is now constant, allowing for models whereby the atom number and connectivity must be preserved between all models.
Changed the grid search pivot displacement frame order parameter. Instead of searching from 0 to 50 Angstroms, the search is now from 10 to 50. This is to avoid the edge case of pivot_disp = 0.0 from which the optimisation cannot escape.
Speedup of the PCS component of the rigid frame order model. The lanthanide to atom vectors are now being calculated outside of the alignment tensor and spin loops, as well as the inverse vector lengths to the 5th power. This increases the speed by a factor of 1.216 (from 38.133 to 31.368 seconds for 23329 calls of the func_rigid() target function).
Added the full optimisation results for the rigid frame order model. This is for the CaM test data using the new frame_order.py optimisation script.
Numpy <= 1.6 fixes for the frame order PCS code. The numpy.linalg.norm function does not have an axis argument in numpy 1.6, therefore the lib.compat.norm() function is now used instead. This function was created exactly for this axis argument problem.
Created the new specific_analyses.frame_order.variables module. This currently contains variables for all of the frame order model names, as well as various lists of these models. The rest of the frame order specific analysis code as well as the frame order user functions have been converted to use these model variables exclusively rather than having the model name strings hardcoded throughout the codebase.
Added the full optimisation results for the double rotor test data. This is for the CaM frame order test data using the new frame_order.py optimisation script.
Added a script for profiling the target function calls of the pseudo-ellipse frame order model.
Added a timeit script and log file showing how numpy.cos() is 10 times slower than math.cos(). This is for single floats.
Shifted the calculation of the theta_max cone opening for the pseudo-ellipse outside of all loops. This is infrastructure change for potentially eliminating all of the looping for the PCS numeric integration in the future. It however slightly speeds up the pseudo-ellipse frame order model. Using 500 target function calls in the profiling_pseudo_ellipse.py script in test_suite/shared_data/frame_order/timings/, the time spent in the pcs_pivot_motion_full_qrint() function decreases from 20.849 to 20.719 seconds.
Converted the torsionless pseudo-ellipse model to also use the tmax_pseudo_ellipse_array() function. This allows the calculation of the pseudo-elliptic cone opening theta_max to be shifted outside of all loops.
Created a profiling script and log file for the isotropic cone frame order model. This shows where the slow points of the model are, using 2000 target function calls.
Increased the function call number to 500 in the pseudo-ellipse frame order model profiling script. The profiling log file has also been added to show where the slowness is - specifically that the numeric PCS integration takes almost the same amount of time as the RDC frame order matrix construction using the scipy.integrate.quad() function.
Created the specific_analyses.frame_order.checks.check_pivot() function. This is to check that the pivot point has been set.
The frame order grid search is now checking if the pivot point has been set.
Added a profiling script and log file for the free rotor frame order model.
Updated the frame order optimisation results for the CaM isotropic cone test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository and the old ones removed.
Modified the script for recreating the frame order PDB representation and displaying it in PyMOL. The state loading, domain redefinition, and representation creation parts have all been removed, as these will soon all be redundant as the frame order analysis for all models is being redone. All that remains are the pymol.frame_order() function calls for displaying all the representations.
The pivot point parameters in the frame order analysis are no longer scaled by 100. This is to match the average domain position translation which is also not scaled.
The specific_analyses.frame_order.variables module is now used throughout the frame order code. The target function code, auto-analysis, and test suite now all use the variables defined in this module rather than having hardcoded strings. The MODEL_LIST_NONREDUNDANT variable has been created to exclude the redundant free rotor pseudo-ellipse which cannot be optimised, and this is used by the auto-analysis.
Removal of many unused imports in the frame_order_cleanup branch. These were detected using the devel_scripts/find_unused_imports.py script which uses pylint to find all unused imports. The false positives also present in the trunk were ignored. And the unused imports in the dispersion code were also left for clean up the disp_spin_speed branch.
Changed the minimisation in the frame order system tests where optimisation is activated. The number of iterations is now set to 1 for speed testing, and the constraints are turned on.
Turned on the optimisation flag for the Frame_order.test_cam_free_rotor system test. This is to activate code paths currently not tested by the test suite.
Constraints are now properly turned off in the minimise user function for the frame order analysis. The A and b matrices from linear_constraints() are now set to None if they are returned as empty arrays.
Parallelised the frame order optimisation code to run on clusters or multi-core systems via OpenMPI. The optimisation code has been split into the three standard parts of the multi-processor: 1) Frame_order_memo is the new Memo object used to store data on the master for use when data is returned from the slaves. 2) Frame_order_minimise_command is the Slave_command which stored all required data for the optimisation, is pickled and sent to a slave, sets up the target function, and then performs optimisation. 3) Frame_order_result_command is the Result_command initialised by the Slave_command on the slave for pickling and returning results to the master. To avoid pickling the target function class, which is not possible, the store_bc_data() and target_fn_setup() functions of the specific_analyses.frame_order.optimisation module have been redesigned to work with basic data structures rather than the target function class directly. The target_fn_setup() function no longer returns an initialised target function class, but rather all the data assembled prior to the initialisation. And the target function class was itself modified so that pcs_theta and rdc_theta are always defined to allow the store_bc_data() function to be used successfully. This parallelisation currently only allows the Monte Carlo simulations to be run on slave processors.
The frame order linear_constraints() function now returns None if no constraints are present. This allows the code using this to be simplified with respect to turning off the constraints.
Improvements for the printout at the start of optimisation of the frame order models. This is in the target_fn_setup() frame order method. All the printouts are now in one place and they are now better formatted and better controlled.
Parallelised the frame order grid search to run on clusters or multi-core systems via OpenMPI. This involved the creation of the Frame_order_grid_command class which is the multi-processor Slave_command for performing the grid search. This was created by duplicating the Frame_order_minimise_command class and then differentiating both classes. For the subdivision of the grid search, the new minfx grid.grid_split_array() function is used in the frame order grid() API method. The grid() method no longer calls the minimise() method but instead obtains the processor box itself and adds the subdivided grid slaves to the processor. The relax grid_search user function takes care of the rest.
Fixes for the parallelised grid search for the frame order analysis. A chi-squared value check was added to the Frame_order_result_command.run() method to check if the value is lower than the current when the result is returned to the master. Without this check, each grid subdivision result will be stored as they are returned rather than storing the results from the global minimum of the entire grid search.
Added a script for testing out the parameter nesting abilities of the frame order auto-analysis. This script attempts to find the dynamics solution without knowing where the pivot is located. Hence this will be as in the auto-analysis were this pivot point will be used as the base for all other models.
Sent the verbosity argument to the minfx.grid.grid_split() function for the frame order analysis. This matches the relax trunk changes for the model-free analysis. The minfx function in the next release (1.0.8) will now be more verbose, so this will help with user feedback when running the model-free analysis on a cluster or multi-core system using MPI.
Improvements for the parallelised grid search for the frame order analysis. As each grid point can take wildly different numbers of CPU cycles to calculate the chi-squared value for, the result of subdividing the grid search was that some subdivisions where incredibly quick while others required much larger amounts of time. To avoid this bad slave management, the grid points are now randomised. This means that the subdivisions will require about the same amount of time to optimise.
Moved the setup of the target function data structures in the frame order analysis. This is for the grid_search and minimise user functions. The target function data setup function has been renamed to target_fn_data_setup(). This is now called before the Frame_order_grid_command and Frame_order_minimise_command multi-processor objects are initialised, and all of the data is now passed into these functions. Although the code is uglier, this has the benefit that the target_fn_data_setup() function will only be called once. This data setup requires a lot of time, so for a large cluster, this can be a large time saving for the grid search.
Modified the frame_order_free_start.py script to better mimic the frame order auto-analysis.
Updated the frame order optimisation results for the 2nd CaM free rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM free rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM missing data free rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM free rotor isotropic cone test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM small angle rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the 2nd CaM free rotor isotropic cone test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM pseudo-ellipse test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM torsionless isotropic cone test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the 2nd CaM pseudo-elliptic cone test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Some more fixes for the optimisation user function changes.
Removed the parameter scaling for the pivot point frame order parameters. These were already removed from the frame_order_cleanup branch in the assemble_scaling_matrix() function, however they were reintroduced accidentally via the parameter object where this information is now defined. So this removes the scaling a second time.
Fixes for the parameter scaling changes in the trunk. The scaling flag is no longer part of the specific analysis API optimisation methods. Instead the pre-assembled scaling matrices are passed into all three API optimisation methods.
Implemented the frame order specific analysis API method print_model_title(). This is simply aliased from the API common method _print_model_title_global().
Fix for the grid search in the frame order analysis. This is a recently introduced problem due to the changes of the zooming_grid_search branch.
Turned on the optimisation in the Frame_order.test_cam_rigid system test. This is to catch a number of failures in the frame order grid search.
Activated the grid search in the frame order system tests using the CaM synthetic data. This is set to one increment so that the tests can complete in a reasonable time.
Fix for the specific_analyses.frame_order.optimisation.grid_row() function. This can now handle the case of a single grid increment. The change is similar to r163 in the minfx project (https://mail.gna.org/public/minfx-commits/2014-07/msg00015.html).
Converted the frame_order_free_start.py script to use the zooming grid search.
Added lots of calls to the time user function to the frame_order_free_start.py. This will be used to fine tune the frame order analysis on a cluster.
Increased the default grid bounds for the pivot parameters of the frame order models. The pivot point is now searched for in a 50 Angstrom box and the pivot displacement for the double motion models from 10 to 60 Angstroms. These were originally a 20 Angstrom box and 10 to 50 Angstroms. The larger grid is possible when combined with the new zooming grid search.
Updated the frame order optimisation results for the 2-site CaM test data fitting to the rotor model. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the 2nd CaM rotor test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Fixes for the CaM free-rotor pseudo-ellipse frame order model test data set. This is for the constraint 0 <= theta_x <= theta_y <= pi, as the old data was created with theta_x > theta_y. The new data is also of high quality using 20 million structures and numpy.float128 data averaging.
Created the lib.frame_order.rotor_axis.convert_axis_alpha_to_spherical() function. This will convert the axis alpha angle to the equivalent spherical angles theta and phi.
Renamed the lib.frame_order.rotor_axis module to lib.frame_order.conversions. This module will be used for all sorts of frame order parameter conversions.
Added the pipe_name argument to the specific_analyses.frame_order.data.generate_pivot() function. This allows the pivot from data pipes other than the current one to be assembled and returned.
Updated the frame order optimisation results for the CaM free rotor, pseudo-ellipse test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Updated the frame order optimisation results for the CaM torsionless, pseudo-ellipse test data. The optimisation in the frame_order.py is now of higher precision with the number of Sobol' numeric integration points significantly increased, especially for the Monte Carlo simulations. The new frame order representation files have been added to the repository, as well as the intermediate state files.
Fix for the Frame_order.test_cam_pseudo_ellipse_free_rotor system test. This is for the change of the X and Y cone opening angles.
Redesign and expansion of the nested model parameter copying in the frame order auto-analysis. The nested parameter protocol used to allow the analysis to complete in under 1,000,000 years was no longer functional due to the switching to the axis alpha parameter to decrease parameter number and redundancy. The copying of the average domain position for the free rotor models was also incorrect as the dropping of the alpha Euler angle cause the translation parameters and beta and gamma angles to change drastically. The new protocol has been split into four methods for the average domain position, the pivot point, the motional eigenframe and the parameters of ordering. These use the fact that the free rotor and torsionless models are the two extrema of the models where the torsion angle is restricted. The pivot copying is a new addition.
Created the Frame_order.test_auto_analysis system test. This will be an extremely quick run through of the frame order auto-analysis as this is not currently tested. 1 Sobol' quasi-random integration point will be used for all models for speed. The system test uses the rigid CaM test data to perform a full analysis.
Alphabetical ordering of the imports in the frame order auto-analysis module.
Fixes for the backend script of the Frame_order.test_auto_analysis system test. This includes a missing import and the removal of a long ago deleted user function.
Fix for the frame order auto-analysis for the call to the grid search user function. This user function has been renamed to minimise.grid_search, however not all parts of the analysis had been converted to the new name.
Created a method in the frame order auto-analysis to reorder the models. This is needed as the nested model parameter copying protocol requires the simpler models to be optimised first.
The Frame_order.test_auto_analysis system test now writes all files to the directory of ds.tmpdir. This is to prevent the system test from dumping files in the current directory.
Modified the specific_analyses.frame_order.parameters.update_model() function. This will no longer set all parameters to 0.0, excluding the pivot point.
Modified the specific_analyses.frame_order.parameters.assemble_param_vector() function. This can now handle the case of no parameters being present. The corresponding elements of the numpy array will consist of NaN values.
Better handling of unset parameters in the frame order optimisation functions. The specific_analyses.frame_order.optimisation.target_fn_data_setup() and specific_analyses.frame_order.parameters.assemble_param_vector() function both now accept the unset_fail argument. This is set in both the calculate() and minimise() API methods. When set, a RelaxError will be raised in the assemble_param_vector() function when a parameter has not been set yet. This together with previous changes will prevent the frame order analysis from using 0.0 as a starting value for unset parameters.
Fixes for all of the Frame_order.test_rigid_data_to_*_model system tests. The base script now sets all parameter values so that the minimise.calculate user function can operate. The two free rotor model chi-squared values have been updated as these are sensitive to the motional eigenframe parameter values - these models can never approximate a rigid state.
Modified the optimisation of the rigid model in the frame order auto-analysis. The grid search is now implemented as a zooming grid search.
Updates and fixes for the frame order auto-analysis. The custom grid setup now works for the new reduced parameter set models and the double rotor model is now also included. The cone axis alpha angle to spherical angle conversion has had a bug removed. And some of the printouts are now more detailed.
Redesigned the Frame_order.test_auto_analysis system test. This now uses a hypothetical new Optimisation_settings object from the frame order auto-analysis module for holding all of the grid search, zooming grid search and minimisation settings. This will allow for far greater user control of the settings and hugely simplify the auto-analysis interface by decreasing the number of input arguments. It should also be less confusing.
Implementation of the Optimisation_settings object in the frame order auto-analysis. This object holds all of the grid search, zooming grid search, and minimisation settings. It provides the add_grid() and add_min() methods to allow the user to add successive iterations of optimisation and settings to the object. The loop_grid() and loop_min() methods are used to loop over each iteration of each method. And the get_grid_inc(), get_grid_num_int_pts(), get_grid_zoom_level(), get_min_algor(), get_min_func_tol() and get_min_num_int_pts() methods are used to access the user defined settings. The auto-analysis has been redesigned around this new concept. All of the optimisation arguments have been replaced. Instead there are the opt_rigid, opt_subset, opt_full, and opt_mc arguments which are expected to be instances of the Optimisation_settings object. The optimisation in the auto-analysis is now more advanced in that more user optimisation settings are now available and active.
Added linear constraints for the pivot and average domain translation frame order parameters. The pivot coordinates are constrained between -999 and 999 Angstrom and the translation between -500 and 500 Angstrom. This allows the frame_order.pdb_model user function to operate in the case of failed models - often the free rotors fitting to torsionally restricted data - by preventing the PDB coordinates from being out of the PDB format range. It should also speed up optimisation by stopping the optimisation of failed models earlier.
The frame order auto-analysis Optimisation_settings object now handles the maximum iterations. The new max_iter argument has been added to the add_min() method, and the new get_min_max_iter() method added to fetch the value. This is used in the auto-analysis to set the maximum number of optimisation iterations in the minimise.execute user function calls. Limiting this will be of greatest benefit for the test suite.
Speedup of the Frame_order.test_auto_analysis system test. This involves limiting the maximum number of optimisation steps to 20 for most parts (the rigid model excluded so the average domain position is correctly found), and using the PCS subset data for the full data set.
Updated the full_analysis.py script for the CaM frame order test data. This is for the recent changes to the auto-analysis with the Optimisation_settings object and for the changes of this branch.
Removed the RDC data checks from the frame order optimisation. This is in the minimise_setup_rdcs() and store_bc_data() functions of the specific_analyses.frame_order.optimisation module, called before and after all optimisation. The reason was identified by profiling - this check was adding significant amounts of time to the setup and results unpacking parts of the optimisation. Specifically the interatomic_loop() function was identified via profiling as the function requiring the most amount of cumulative time in the Frame_order.test_auto_analysis system test (17 seconds out of a total of ~60 seconds).
Fixes for the removal of the RDC data checks from the frame order optimisation functions. The specific analysis API method overfit_deselect() has now been created to deselect spins which do not have PCS data or interatomic data containers missing RDC data. The handling of deselected spins and interatomic data containers is now also correctly handled throughout the frame order specific code.
Enabled pivot optimisation in the full_analysis.py script for the CaM frame order test data.
The frame order auto-analysis now calls the time() user function. This is used at the start of each model section, as well as at the very start and very end of the analysis. This feedback is needed for the user to be able to optimise the optimisation settings.
Major bugfix for the frame order auto-analysis. The algorithm of using a PCS data subset of a few selected residues to find an initial parameter estimate followed by using all PCS data was badly implemented. The use of the PCS subset caused most spin systems to be deselected, however they remained deselected once all data was being used. So the result was that only the spin subset was ever being used in the analysis.
Fix for the recent lib.period_table and lib.physical_constant module changes.
Created the model_directory() method for the frame order auto-analysis. This is used to create the full path for saving model specific files. It replaces spaces with underscores in the path and removes all commas. The commas in the path appear to be fatal for certain PyMOL versions when viewing the frame order representation.
The frame order auto-analysis results printout has been extended to include the pivot point.
Change to the parameter nesting in the frame order auto-analysis. The pivot is now taken from the rotor model for all other models. Taking the pivot point from the isotropic cone model is not a good idea as there are situations where the pivot point optimisation catastrophically fails, sending the point many tens or hundreds of Angstrom away from the molecule.
Copied a frame order results file for testing axis permutations. This is from the test_suite/shared_data/frame_order/cam/pseudo_ellipse/ directory. The optimisation results were identified to have failed, in that it found the alternative minimum. The pseudo-ellipse model as two minima in the space, and in this case the global minimum was missed.
Created the Frame_order.test_axis_permutation system test. This is to test the operation of the yet-to-be implemented frame_order.permute_axes user function.
Implemented the frame_order.permute_axes user function. This is used to switch between local minima in the pseudo-elliptic frame order models.
Fix for the Frame_order.test_axis_permutation system test. The motional eigenframe in the old log file was not exactly correct and did not correspond exactly to the Euler angles in the cam_pseudo_ellipse.bz2 results file in test_suite/shared_data/frame_order/axis_permutations/.
Extended the Frame_order.test_axis_permutation system test to check frame_order.permute_axes twice. This will check that two calls to the frame_order.permute_axes user function will restore the original parameter values.
The frame_order.permute_axes user function can now handle the torsionless pseudo-ellipse. This model does not have the variable cdp.cone_sigma_max set.
Added support for axis permutations in the frame order auto-analysis. This is done by copying the data pipe of the already optimised pseudo-elliptic models, permuting the axes, and performing another optimisation using all RDC and PCS data. This allows the second solution for these pseudo-elliptic models to be found. The 2nd pipe is included in the model selection step to allow the best solution for the model to be found.
Fix for the reading of old results files in the frame order auto-analysis. The directory name is now processed by the model_directory() method. This will convert the spaces to '_' and remove commas. Without this the already created files could not be found, if the model name contains a space or comma.
Made the pivot point in the frame order PDB representation fail-proof. If the pivot position was outside of the bounds [-1000, 1000], the PDB file creation would fail as the record would be too long. So now the pivot is shifted to be in these bounds.
The axis permutation step in the frame order auto-analysis is now always performed. If an old results file was found, this step was accidentally skipped.
Added extensive printouts to the frame_order.permute_axes user function.
Redesigned the frame_order.permute_axes user function frontend. Previously only cyclic permutations were considered, however non-cyclic permutations are also allowed when accompanied by an axis inversion. Therefore 3 combinations exist with cone_theta_x <= cone_theta_y, or 2 when the current combination is excluded.
Created 6 system tests for the frame_order.permute_axes user function. This covers the 3 starting conditions (x<y<z, x<z<y, z<x<y) and the two permutations ('A' and 'B') for each of these which do not include the starting permutation. They replace the original Frame_order.test_axis_permutation system test with the tests Frame_order.test_axis_perm_x_le_y_le_z_permA, Frame_order.test_axis_perm_x_le_y_le_z_permB, Frame_order.test_axis_perm_x_le_z_le_y_permA, Frame_order.test_axis_perm_x_le_z_le_y_permB, Frame_order.test_axis_perm_z_le_x_le_y_permA, and Frame_order.test_axis_perm_z_le_x_le_y_permB.
Implemented the new frame_order.permute_axes backend. The 3 starting conditions x<y<z, x<z<y, and z<x<y and the two permutations 'A' and 'B' (for each of these which do not include the starting permutation) are now supported. For these 6 combinations, the axis and order parameter permutation and the z-axis inversion are selected and applied to the current system.
Removed the second permutation from the 6 Frame_order.test_axis_perm_* system tests. A second identical permutation does not necessarily restore the original state.
Fix for the frame_order.permute_axes for the torsionless pseudo-ellipse model. The data structure cdp.cone_sigma_max does not exist in this model as cone_sigma_max == 0.0.
Modified the frame order auto-analysis axis permutation algorithm to handle both permutations. Instead of creating one additional data pipe for the permutations, two are now created for the permutations 'A' and 'B'. This allows all 3 solutions for the pseudo-elliptic models to be explored and included in the final model selection process.
Fix for the Frame_order.test_axis_perm_x_le_z_le_y_permB system test. The permuted z-axis needs to be inverted in the test.
Many fixes for the frame_order.permute_axes user function. The z-axis inversion is now encoded into a 3D numpy array as the index of the new z-axis position needs to be stored. The cone_theta_x, cone_theta_y and cone_sigma_max parameters are now permuted in reverse 'perm' data structure by calling its index() method. And the cone_theta_x - cone_theta_y to y-axis - x-axis switch has been removed (this may need to be reintroduced later).
Fix for the axis permutation protocol in the frame order auto-analysis. The pipe.copy user function does not switch pipes, therefore the pipe.switch user function is now being called so that the correct pipe is being permuted and optimised.
Created some test data files for visualising the frame order axis permutation. This uses the CaM frame order synthetic data for the rotor model to visualise the pseudo-ellipse frame order model axis permutations. The initial conversion sets the pseudo-ellipse torsion angle cone_sigma_max to the rotor opening half-angle, and the pseudo-elliptic cone opening to close to zero. Then the axis permutations are performed. All three solutions are optimised. PDB representations before and after optimisation are included to illustrate any problems.
Bug fix for the new frame_order.permute_axes user function. The cone and torsion angles were not being correctly permuted. Now the direct permutation array is being used. And the fact that cone_theta_x is a rotation along the y-axis and cone_theta_y along the x-axis is taken into account.
Redesign of the axis permutation algorithm of the frame_order.permute_axes user function. Instead of tracking the fact that cone_theta_x is a rotation around the y-axis and cone_theta_y is about the x-axis, now two permutation arrays are created - one for the three angles and one for the axes. The permutation array values have also been completely changed as previously the incorrect inverse permutation was coded into the algorithm.
Updated the frame order pseudo-ellipse motion permutation test data. This is for the CaM frame order rotor model synthetic data. The correct axis and cone angle permutations of the frame_order.permute_axes user function are now being used and optimised.
Renamed the pseudo-ellipse permutation directory to perm_pseudo_ellipse_x_le_y_le_z. This is for the CaM frame order rotor model synthetic data.
Fix for the frame_order.permute_axes user function. One of the 6 permutations had the x and y axes switched (the x <= z <= y condition, permutation A).
Visualisation files for all of the pseudo-ellipse permutations by frame_order.permute_axes. This includes the x <= z <= y and z <= x <= y conditions (the previous files were for x <= y <= z). In all permutation combinations, optimisation has been performed to demonstrate that these are all local minima. These all approximate the rotor when using the CaM frame order rotor model synthetic data.
Added support for the isotopic cone models to the frame_order.permute_axes user function. This is a simpler setup, but it uses the same permutation algorithm as derived for the pseudo-ellipse models. Instead of setting the x and y cone angles separately, they are instead averaged. And as the cone axis is undefined in the xy plane, the axis has been randomly selected as being the axis perpendicular to both the z-axis and the reference frame x-axis.
Created set of files showing the axis permutation problem for the isotopic cone frame order model. This shows that there are two minima. However one has a chi-squared value of ~1, and the other a value of ~150. Nevertheless, the optimisation could be trapped in the non-global minimum so the frame_order.permute_axes user function should be used for the isotopic cones as well, just in case.
Created the other isotropic cone condition z <= x = y. As there are no constraints in this model, this condition should not result in any major differences, just the size of the cone being different and the optimisation having to decrease the cone angle significantly to mimic the rotor.
Modified the frame order auto-analysis. The axis permutation algorithm is now performed on all isotopic cone and pseudo-ellipse models. This is just in case the non-global minima was found in the original optimisation. The isotropic cone models possess two local minima whereas the pseudo-ellipse models possess three local minima.
Simplified the optimisation in the axis permutation part of the frame order auto-analysis. Only the last, highest quality setting is used for optimisation.
Fix for the axis permutation protocol in the frame order auto-analysis. This would fail if a results file for the permuted model already exists as the pipe.copy user function call was being performed too early.
Created set of files for the axis permutation of the torsionless isotopic cone frame order model.
Created an initial Frame_order.test_frame_order_pdb_model_ensemble system test. This is to check the operation of the frame_order.pdb_model user function when an ensemble of structures is encountered. However as this uses a very minimal number of user functions to set up the system, a number of other minor bugs will probably be uncovered.
Added printouts to the specific_analyses.frame_order.parameters.update_model() function. This is to make it easier to understand why certain things fail due to the system not being fully set up.
Simplified the operation of the frame_order.select_model user function. This is by removing the check of PCS data from the specific_analyses.frame_order.data.pivot_fixed() function using the base_data_types() function call. This allows the model to be set up more easily.
Modified the frame order check_pivot() function to operate on any data pipe. The function now accepts the pipe_name argument so that checks can happen on any data pipe.
Missing imports in the specific_analyses.frame_order.checks module. This is from the recent pipe_name argument addition in the check_pivot() function.
The frame order generate_pivot() function can now handle no pivot being present. At the start of this specific_analyses.frame_order.data module function, the check_pivot() function is being called to make sure that a pivot is present.
Modified the Frame_order.test_frame_order_pdb_model_ensemble system test so it is set up correctly. The pivot point and moving domain are now specified.
Added Monte Carlo simulations to the Frame_order.test_frame_order_pdb_model_ensemble system test. This is only setting up Monte Carlo simulation data structures via the monte_carlo.setup user function. This demonstrates a failure of the frame_order.pdb_model user function when an ensemble of structures is present with Monte Carlo simulations.
Added support for the model argument for the frame_order.pdb_model user function. This argument is used to specify which of the models in an ensemble will be used to represent the average domain position Monte Carlo simulations, as each simulation is encoded as a model, as well as for the distribution of structures simulating the motion of the system. The argument is therefore passed into the create_ave_pos() and create_distribution() functions of the specific_analyses.frame_order.geometric module. To handle all models being used in the non Monte Carlo simulation PDB file and only one in this file, the internal structural object is copied twice. The second copy for the MC sims has all but the chosen model deleted out of it.
Fix for the Frame_order.test_frame_order_pdb_model_ensemble system test. More needed to be done to set up the Monte Carlo simulations - the monte_carlo.initial_values user function call was required.
Modified the frame order sim_init_values() API method to handle missing optimisation data. The monte_carlo.initial_values user function was failing if optimisation had not been performed. This is now caught and handled correctly.
Created the Frame_order.test_frame_order_pdb_model_failed_pivot system test. This simply shows how the frame_order.pdb_model user function currently fails if the optimised pivot point is outside of the PDB coordinate limits of "%8.3f".
The frame_order.pdb_model user function can now properly handle a failed pivot optimisation. This is when the pivot point optimises to a coordinate outside of the PDB limits. Now all calls to specific_analyses.frame_order.data.generate_pivot() from the module specific_analyses.frame_order.geometric set the pdb_limit flag to True. This allows all representation objects to be within the PDB limits. The algorithm in generate_pivot() has been extended to allow higher positive values, as the real PDB limits are [-999.999, 9999.999]. And a RelaxWarning is called when the pivot is outside to tell the user about it.
Modified the frame order auto-analysis to be more fail-safe. Almost all of the protocol is now within a try-finally block so that the execution lock will always be released.
Fix for the specific_analyses.frame_order.data.pivot_fixed() function. This was recently introduced when the check for PCS data was removed from this function. To fix the problem, instead of calling base_data_types() to see if PCS data is present, the cdp.pcs_ids data structure is checked instead.
Fix for the model argument for the frame_order.pdb_model user function. The deletion of structural models for the Monte Carlo simulations in the average domain position representation now only happen if more than one model exists.
Modified the Frame_order.test_frame_order_pdb_model_failed_pivot system test. This is to show that the frame_order.pdb_model user function fails if the pivot is close to but still within the PDB coordinate limits.
Modified the pivot position checking in specific_analyses.frame_order.data.generate_pivot(). Now the pivot is shifted to be within the limits shrunk by 100 Angstrom. This allows any PDB representation created by the frame_order.pdb_model user function to be within the PDB limits.
Fix for the axis permutation protocol in the frame order auto-analysis. If a results file was found for one of the permutations, a return from the function would occur. The result is that the other permutations would not be loaded or optimised.
Fix for the RelaxError raised by the frame_order.select_model user function. This is the error if the model name is incorrect.
Created the Frame_order.test_pseudo_ellipse_zero_cone_angle system test. This is to catch a bug in optimisation when the cone_theta_x is set to zero in the pseudo-ellipse models.
Bug fix for the lib.frame_order.pseudo_ellipse.tmax_pseudo_ellipse_array() function. The problem was that when theta_x or theta_y were zero, the floating point value of 0.0 would be returned. This is the incorrect behaviour as the returned value must be an array matching the dimensions of the phi angle array argument.
Fix for the Pseudo_elliptic cone object for when the cone angles are zero. The Pseudo_elliptic.phi_max() method now avoids a divide by zero error.
Updates for all of the Frame_order.test_axis_perm_* system tests. The axis permutations and angle permutations are now performed correctly within the tests themselves. This allows the tests to pass.
Modified the Frame_order.test_pseudo_ellipse_zero_cone_angle system test to be quick. Now that the test passes, the optimisation needs to be short. So a maximum of two iterations are now set. Otherwise the test would take hours to complete.
Small speedup of the Frame_order.test_auto_analysis system test.
Alphabetical ordering of most of the Frame_order system tests.
Created the very simple Frame_order.test_num_int_points system test. This simply creates a data pipe and calls the frame_order.num_int_pts user function to test its operation. This is to increase the test suite coverage of this user function.
Created the Frame_order.test_num_int_pts2 system test. This checks the operation of the frame_order.num_int_pts user function when only the model has been chosen.
Renamed the Frame_order.test_num_int_points system test to Frame_order.test_num_int_pts.
Created the check_domain() function for the frame order analysis. This is in the specific_analyses.frame_order.checks module. The function checks that the reference domain has been specified.
Created the check_model() function for the frame order analysis. This is in the specific_analyses.frame_order.checks module. The function checks that the frame order model has been selected via the frame_order.select_model user function.
The frame_order.ref_domain user function backend now uses the check_domain() function.
Created the check_parameters() function for the frame order analysis. This is in the specific_analyses.frame_order.checks module. The function checks that the frame order parameters have been set up and have values.
Created the Frame_order.test_num_int_pts3 system test. This checks the operation of the frame_order.num_int_pts user function when the model has been and the frame order parameters have been set up.
Created the Frame_order.test_count_sobol_points system test. This will test that the frame_order.num_int_pts user function can correctly count the number of Sobol' integration points used for the current set of parameter values. This frame_order.num_int_pts functionality does not exist yet.
Implementation of the specific_analyses.frame_order.optimisation.count_sobol_points() function. This is used by the frame_order.num_int_pts user function to provide a printout of the number of Sobol' integration points used for the current parameter values. This is to provide user feedback so that it is know if enough Sobol' points have been used.
Modified the Frame_order.test_count_sobol_points system test. The number of points has been massively decreased as generating Sobol' points takes a long time, and the check for the number of used Sobol' points has been set to the real value.
Created the Frame_order.test_count_sobol_points2 system test. This checks the operation of the frame_order.count_sobol_points user function. As this user function has not been implemented yet, the test currently fails.
Created the frame_order.count_sobol_points user function. This is simply a frontend to the new specific_analyses.frame_order.optimisation.count_sobol_points() function.
Updated the Frame_order.test_count_sobol_points2 system test for the correct number of Sobol' points.
Created the Frame_order.test_count_sobol_points_rigid system test. This is to demonstrate a failure of the frame_order.test_count_sobol_points user function when applied to the rigid frame order model.
Fix for the frame_order.count_sobol_points user function for the rigid model. This model is now caught at the start, a message printed out, and the function exited.
Fix for the Frame_order.test_count_sobol_points_rigid system test. This now checks that cdp.used_sobol_points does not exist for the rigid frame order model after a call to the frame_order.count_sobol_points user function.
Created the Frame_order.test_count_sobol_points_rotor system test. This is to test the frame_order.count_sobol_points user function for the rotor model.
Fix for the frame_order.count_sobol_points user function for the rotor model. The sigma angles unpacking required a dimensionality collapse in the Sobol' angle data structure.
Updated the number of points to allow the Frame_order.test_count_sobol_points_rotor system test to pass.
The frame order count_sobol_points() function is now being called by all of minimise user functions. This occurs at the end of the minimise.calculate, minimise.grid_search, and minimise.execute user function backends to provide more feedback to the user as to the quality of the optimisation. To avoid initialising the target function twice, the count_sobol_points() function now accepts the initialised target function as an optional argument.
Created the Frame_order.test_count_sobol_points_free_rotor system test. This is to demonstrate that the frame_order.count_sobol_points user function currently fails for the free-rotor model.
Fix for the frame_order.count_sobol_points user function for the free-rotor models. The torsion angle is now correctly handled as the 3 free-rotor models do not have cdp.cone_sigma_max set.
Updated the number of points in the Frame_order.test_count_sobol_points_free_rotor system test. This is to allow the test to pass.
Fix for the frame order count_sobol_points() function. The checks for the model, parameter and domain set up must come first, before cdp.model is accessed. Otherwise the frame_order.num_int_pts user function will often fail.
Fix for the frame order count_sobol_points() function. The free-rotor isotropic cone model was incorrectly handled, as the cone parameter is 'cone_s1' and not 'cone_theta'. The order parameter is now converted to an angle before checking if the Sobol' point is outside of the cone or not.
More fixes for the frame order count_sobol_points() function. The torsion angle for the torsionless models is no longer accessed, and the cone_theta parameter is only accessed for models with this parameter.
Created the Frame_order.test_count_sobol_points_iso_cone_free_rotor system test. This is to test the frame_order.count_sobol_points user function for the free-rotor isotropic cone model.
Fix for the frame order count_sobol_points() function. The torsion angle ranges from -pi to pi, so the absolute value needs to be checked, just as in the lib.frame_order modules.
Updates for the number of Sobol' points in the Frame_order.test_count_sobol_points_* system tests. This is simply to allow all Frame_order system tests to pass.
Redesigned the frame_order.num_int_pts user function frontend for the oversampling idea. The use of the quasi-random Sobol' sequence for numerical PCS integration will be modified to use the concept of oversampling. Instead of specifying the exact number of points in the Sobol' sequence and then removing points outside of the current parameter values, the algorithm will oversample as N * Ov * 10**M, where N is the maximum number of Sobol' points to be used for the integration, Ov is the oversampling factor, and M is the number of dimensions or torsion-tilt angles used in the system. The aim is to try to use the maximum number of points N for all frame order models and all ranges of dynamics.
Renamed the frame_order.num_int_pts user function to frame_order.sobol_setup. The user function no longer specifies the number of integration points. Instead it now specifies the maximum number of points N and oversampling factor Ov used to generate the oversampled Sobol' sequence.
Implemented the Sobol' sequence oversampling in the frame order target function class.
Converted all of the specific_analyses.frame_order package to the Sobol' point oversampling design. The correct values are now sent into the target function and all references to cdp.num_int_pts has been replaced with the cdp.sobol_max_points and cdp.sobol_oversample pair of variables. The frame_order.count_sobol_points user function backend has also been updated to show the total number of oversampling points and the number of points used.
The frame_order.count_sobol_points user function now shows more information. The maximum number and oversampling factors are now also printed out for maximum user feedback.
Improved the printout formatting for the count_sobol_points() frame order function.
The frame order target function now passes the maximum number of Sobol' points to the relax library. The value is being passed into the lib.frame_order.*.pcs_numeric_int_*() functions, though it is not used set.
Fix for the percentage calculation for the frame order count_sobol_points() function.
Changed the creation of the Sobol' points in the frame order target function. For increased accuracy of the numerical PCS integration, the first 1000 points of the Sobol' sequence are now skipped to avoid any bias. For speed, the axis order of the Sobol' torsion-tilt angles has been swapped so that the numpy.swapaxes() function call is no longer required in the lib.frame_order.*.pcs_numeric_int_*() functions.
Updated the frame order count_sobol_points() function to handle the swapped axis order.
Huge speedup for the generation of the Sobol' sequence data in the frame order target function. The new Sobol_data class has been created and is instantiated in the module namespace as target_function.frame_order.sobol_data. This is used to store all of the Sobol' sequence associated data, including the torsion-tilt angles and all corresponding rotation matrices. When initialising the target function, if the Sobol_data container holds the data for the same model and same total number of Sobol' points, then the pre-existing data will be used rather than regenerating all the data. This can save a huge amount of time.
Updated the frame order count_sobol_points() function to use the new Sobol_data container. The Sobol' sequence data generated by the target function is now located at target_functions.frame_order.sobol_data.
Updated all the lib.frame_order.*.pcs_numeric_int_*() functions for the new Sobol' point algorithm. The functions now all accept the max_points argument and terminate the loop over the Sobol' points once the maximum number of points has been reached. The calls to numpy.swapaxes() have also been removed as this is now pre-performed by the target function initialisation.
Changed the default oversampling factor from 100 to 1 in the frame_order.sobol_setup user function.
Converted the frame order auto-analysis to use the new frame_order.sobol_setup user function design. The auto-analysis Optimisation_settings object has also been modified so that all num_int_pts arguments and internal structures have been split into the two new sobol_max_points and sobol_oversample names and objects.
Fix for the rigid frame order model for the recent frame_order.sobol_setup user function changes. For this model, the number of Sobol' points normally is does not exist. This is now correctly handled.
Created the sobol_setup() method for the frame order auto-analysis. This is used to correctly handle the new design of the frame_order.sobol_setup user function consistently throughout the protocol.
Updated the Frame_order.test_auto_analysis system test script. This now uses the new auto-analysis Optimisation_settings object design.
Updated the Frame_order.test_count_sobol_points system test. The call to the frame_order.num_int_pts user function was changed to frame_order.sobol_setup.
Fixes for the Frame_order.test_count_sobol_points2 system test. The test_suite/shared_data/frame_order/axis_permutations/cam_pseudo_ellipse.bz2 relax state file has been manual edited to change the num_int_pts data pipe structure to sobol_max_points and to add the new sobol_oversample variable.
Added a backwards compatibility hook for state and results files for the Sobol' sequence changes. The data pipe num_int_pts variable is now renamed to sobol_max_points when present, and the sobol_oversample variable is created and set to 1.
Updates to all of the Frame_order.test_count_sobol_points_* system tests. The frame_order.sobol_setup user function is used to set a small maximum number of points (20) to allow the tests to be fast. The value of 20 is also checked for to allow the tests to pass.
Renamed the cdp.used_sobol_points variable to sobol_points_used. This is created by the count_sobol_points() frame order function. The name change is to match the sobol_max_points and sobol_oversample variable names.
Renamed all the Frame_order.test_num_int_pts* system tests to Frame_order.test_sobol_setup*. These system tests where for checking the operation of the old frame_order.num_int_pts user function. But this is now the frame_order.sobol_setup user function.
Fix for all of the Frame_order.test_rigid_data_to_*_model system tests. The frame_order.num_int_pts user function call was changed to frame_order.sobol_setup.
Updated the chi2 check in the Frame_order.test_rigid_data_to_free_rotor_model system test. This value has changed due to the first 1000 points of the Sobol' sequence being skipped.
Fixes for all of the lib.frame_order.*.pcs_numeric_int_*_qrint() functions. The loop over the Sobol' points was broken. As numpy.swapaxes() has been applied to the points argument already, the loop needs to be over the second dimension of the points data structure.
Updates for all of the Frame_order.test_cam_* system tests. The NUM_INT_PTS variable in the system tests scripts is now passed into the frame_order.sobol_setup user function as the max_num argument. This number has also been changed so that the tests take a reasonable amount of time. All chi2 value checks were updated. These were validated by increasing the number of integration points and watching the chi2 value of the Frame_order.test_cam_*_pcs version of the system tests head to zero.
Another update for the chi2 check in the Frame_order.test_rigid_data_to_free_rotor_model system test. The previous commit used an incorrect value for the chi2. This new value is now much closer to the original.
Turned down the verbosity of the update_model() frame order function. The verbosity flag is now accepted and set to zero by the get_param_names() API method and specific_analyses.frame_order.parameters.param_num() function. This removes a lot of useless printouts from many different user functions.
Introduced the verbosity argument to the count_sobol_points() frame order function. This is used to turn the printouts on or off. The optimisation code now calls this function with the verbosity argument sent into the minimise.grid_search and minimise.execute user functions. Hence the printouts are suppressed for Monte Carlo simulations.
Removed the axis system printout from the frame_order.pdb_model user function. This is for the geometric representation of the frame order dynamics. The axis system is printed out as the rotation matrix used for the lib.structure.geometric.generate_vector_residues() function later on anyway. The change is to simplify the printouts.
Editing of the docstring of the frame_order.sobol_setup user function.
Fix for the frame order system test optimisation printouts. The cdp.num_int_pts variable is now called cdp.sobol_max_points.
The starting time of the axis permutation model optimisations is now output. This is in the frame order auto-analysis. This call to the time user function occurred for the normal models, so extending it to the permuted axes models makes the output more consistent.
Simplified the atomic position averaging warning in the frame order analysis. Instead of throwing a warning for each spin, one warning for all spins is now given. This should make the output a lot less verbose.
The frame order minimise_setup_atomic_pos() function now accepts the verbosity argument. This is used to silence the warnings in user functions such as frame_order.sobol_setup.
Improvements for the frame order overfit_deselect() API method. Three changes have been made: The print statements have been converted to RelaxWarnings; The spin IDs or spin ID pairs are now stored in a list and one RelaxWarning for the missing PCS data and one for the missing RDC data is now given; And the verbose flag is now used to determine if a RelaxWarning will be given.
Change to the position averaging warning in the minimise_setup_atomic_pos() frame order function.
Improvements for the printout from the update_model() frame order function. A list of updated parameters is now created and everything is printed on a single line at the end. The printout is therefore much more compact.
Spun out part of the frame_order.pdb_model user function into the new frame_order.simulate user function. The new user function arguments required for properly creating the pseudo-Brownian dynamics simulation would have made the frame_order.pdb_model user function too complicated. Therefore this part has been spun out into the new frame_order.simulate user function. The frame_order.simulate frontend fully describes the algorithm that will be used to simulate the dynamic content of the PCS and RDC data, and warns that not all modes of motion are visible and present.
Updated the frame order auto-analysis to call the new frame_order.simulate user function. Although not implemented yet, this allows the user function to create the simulation PDB file in the future.
Small fix for the new frame_order.simulate user function backend.
Updated the base script for the Frame_order.test_cam_* system tests. The frame_order.simulate user function is now called directly after the frame_order.pdb_model user function.
Created the backend framework for the frame_order.simulate user function. The backend specific_analyses.frame_order.uf.simulate() function performs all data checks required, prepares the output file object, assembles the frame order parameter values and pivot point, and creates a copy of the structural object object with the ensemble collapsed into a single model. All this data is then passed into the new lib.frame_order.simulation.brownian() function. This initialises all required data structures and the structural object. The main loop of the simulation is also implemented, taking snapshots at every fixed number of steps and terminating the loop once the total number of snapshots are reached. The snapshot consists of copying the original unrotated structural model and rotating it into the new position. The rotation is currently the identity matrix. The old specific_analyses.frame_order.geometric.create_distribution() stub function has been deleted.
Decreased the time required for the Frame_order.test_cam_* system tests. The frame_order.simulate user function now only creates a total of 20 snapshots rather than 1000.
Added new arguments to the frame order auto-analysis for the frame_order.simulate user function. These are the brownian_step_size, brownian_snapshot and brownian_total arguments which are passed directly into the frame_order.simulate user function. This gives the user more control, as well as allowing the test suite to speed up this part of the analysis.
Huge speedup for the Frame_order.test_auto_analysis user function. The pseudo-Brownian dynamics simulation via the frame_order.simulate user function has been massively sped up to allow the test to be almost as fast as before.
Spun out the code for shifting to the average frame order position into a new function. The old code of the create_ave_pos() of the specific_analyses.frame_order.geometric module has been shifted into the new average_position() function. This will allow the code to be reused by other parts of relax to obtain the average frame order structures.
Implemented the shifting to the average position for the frame_order.simulate user function backend. This simply sends the structural object into the new average_position() function of the specific_analyses.frame_order.geometric module.
Improvements for the frame_order.simulate user function. The rigid model is now skipped, the PDB file closed, and some printouts for better user feedback have been added.
Changed the default PDB file name for the frame_order.simulate user function to 'simulate.pdb'. The '*.bz2' extension has been dropped so that the file is quicker to create and does not need to be decompressed for loading into molecular viewers.
Created the specific_analyses.frame_order.geometric.generate_axis_system() function. This is now used by most parts of the frame order analysis to generate the full 3D eigenframe of the motions. Previously this was implemented each time the frame or major axis was required. This replicated and highly inconsistent code has been eliminated.
Fix for the new specific_analyses.frame_order.geometric.generate_axis_system() function. The rotor and free rotor models were not correctly handled and the returned eigenframe was the zero matrix.
Implemented the pseudo-Brownian frame order dynamics simulation for the single motion models. This uses the same logic as in the test_suite/shared_data/frame_order/cam/*/generate_distribution.py scripts which were used to generate all of the test suite data. However rather than using a random rotation matrix, a random 3D vector is used to rotate a fixed angle. And the rotation is used to rotate the current state to state i+1. The rotation for the state is decomposed into torsion-tilt angles once shifted into the motional eigenframe, the violations checked for as the state shifted to the boundary, then the new state reconstructed from the corrected torsion-tilt angles, and then it is shifted from the motional eigenframe to the PDB frame.
Shifted the specific_analyses.frame_order.variables module into the lib.frame_order package. This is both to minimise circular dependencies, as previously the specific_analyses.frame_order modules import from target_functions.frame_order and vice-versa, and to allow the relax library functions to have access to these variables.
Implemented the frame_order.simulate user function backend for the double rotor frame order model. This involved extending the algorithm to loop over N states, where N=2 for the double rotor and N=1 for all other models. To handle the rotations being about the x and y-axes, an axis permutation algorithm is used to shift these axes to z prior to decomposing to the torsion-tilt angles. The reverse permutation is used to shift the axes back after correcting for being outside of the allowed angles.
Fixes for the specific_analyses.frame_order.geometric.average_position() function. The recent trunk changes with the structural object Internal_selection class required a change in this function.
Updated the lib.frame_order.simulation.brownian() function. This now uses the internal structural object selection object logic - the selection() method is called to obtain the Internal_selection object, and this is then passed into the rotation() method.
The quad_int argument for the frame order target function class now defaults to False. This is so that quasi-random Sobol' numerical integration will be used by default.
The cdp.quad_int flag is now passed into the target function for the frame order calculate() method. This is for the minimise.calculate user function backend.
Fixes for the missing cdp.quad_int flag. If the cdp.quad_int flag is missing, this is now set to False before setting up the target function class. The previous behaviour was that the frame_order.quad_int user function must be called prior to optimisation. Now it is optional for turning this flag on and off.
The RDC only optimisation now defaults to the *_qrint() frame order target functions. This restores the earlier behaviour prior to the restoration of the SciPy quadratic integration.
Clean up for the frame order target function aliasing. The Scipy quadratic integration and the quasi-random Sobol' integration target functions are now aliased using the getattr() Python method to programmatically choose one or the other. The rigid model has been removed from the list as it is not a numeric model, and the func_double_rotor() target function has been renamed to func_double_rotor_qrint() to make it consistent with the naming of the other target functions.
Renaming of all the frame order target functions and PCS integration functions. For consistency, all quasi-random Sobol' integration functions now use the 'qr_int' tag whereas the SciPy quadratic integration functions use the 'quad_int' tag. This is not only in the target function names but also the PCS integration functions in lib.frame_order.
Duplicated all Frame_order.test_cam_* system tests for testing the SciPy quadratic integration. The Frame_order.test_cam_* system tests have all been renamed to Frame_order.test_cam_qr_int_*. These have been duplicated and renamed to Frame_order.test_cam_quad_int_*. The flag() system test method has been extended to include the quad_int flag which is then stored in the status object and used in the base CaM frame order system test script to activate the frame_order.quad_int user function.
Activated the quad_int flag for a number of the Frame_order.test_cam_quad_int_* system tests. The quad_int argument for the flags() test suite method had been missed for a few of these tests.
Updated the chi2 check in the Frame_order.test_cam_qr_int_pseudo_ellipse_free_rotor_rdc system test. This test is not normally run as it blacklisted and duplicates the coverage of other tests. However its chi-squared value check had not been updated for a while and hence the test fails when explicitly run.
The Sobol' point counting is now turned off for the frame order optimisation functions if none exist. If the cdp.quad_int flag is set, then there will be no Sobol' points to count. This count_sobol_point() user feedback function will therefore not be called by the minimise.calculate, minimise.grid_search and minimise.execute user functions.
Turned off optimisation for all of the Frame_order.test_cam_quad_int_* system tests. The SciPy quadratic integration is far too slow to be used in the test suite. The simple call to the minimise.calculate user function is sufficient for checking these target functions.
Updated all of the Scipy quadratic integration frame order target functions. A number of the data structures in the target function class have been redesigned since these target functions were deleted. All of the func_*_quad_int*() target functions have been updated for these changes.
Updated all of the chi2 value checks for the Frame_order.test_cam_quad_int_* system tests. This is only for those tests which use PCS data - the RDC only test chi2 values are the same as in the Frame_order.test_cam_qr_int_* system tests. In all cases, the chi2 value is lower for the more accurate SciPy quadratic integration as compared to the quasi-random Sobol' integration, as expected.
Implemented the SciPy quadratic integration target function for the double rotor frame order model. This simply follows from what all the other quadratic integration target functions and lib.frame_order module functions do.
Changed the chi2 value checks in the Frame_order.test_cam_quad_int_double_rotor* system tests. These were the values for the quasi-random Sobol' integration and needed updating for the SciPy quadratic integration.
Removed the skip_tests argument for the Frame_order system tests __init__() method. This argument, which was used to manually turn on or off the blacklisted tests, is no longer needed due to the new --no-skip relax command line flag which will enable all blacklisted tests.
The frame order auto-analysis Optimisation_settings object now supports the quad_int flag. This is for activating the SciPy quadratic integration. It is accepted as an argument for the add_grid() and add_min() methods, and it returned by the new get_grid_quad_int() and get_min_quad_int() methods.
Added the ability to specify a pre-run directory in the frame order auto-analysis. This will be used for refinement purposes. If the new pre_run_dir argument, modelled on the relaxation dispersion auto-analysis, is supplied then results files will be loaded from this directory and the base data pipe copying and PCS subset optimisation steps will be skipped. The model nesting algorithm is also deactivated.
Activated the SciPy quadratic integration in the frame order auto-analysis. If the Optimisation_settings object has been set up with the quad_int flag, then the auto-analysis will skip the sobol_setup() method and instead directly call the frame_order.quad_int user function. Optimisation will then use the SciPy quadratic integration rather than the quasi-random Sobol' integration.
Improvements for the usage of the frame_order.quad_int user function in the auto-analysis. The frame_order.quad_int user function is now called even when the Optimisation_settings object quad_int flag is False. This allows for switching between the SciPy quadratic integration and the quasi-random Sobol' integration, as the SciPy quadratic integration can now be turned off.
Additions to the frame order auto-analysis documentation.
Incorporated the contents of the summarise.py script into the frame order auto-analysis module. This has been converted into the summarise() function which will generate a results summary table as the analysis is still running.
Improved logic in the auto_analyses.frame_order.summarise() function. The model names, directories and titles are now being auto-generated from the full list of frame order models in lib.frame_order.variables.MODEL_LIST. To create a common mechanism for determining the model directory name, the Frame_order_analysis.model_directory() method has been converted into a module function.
The frame order auto-analysis now calls the summarise() function at the end to create a summary table.
Shifted the final state saving in the frame order auto-analysis to be within the safety of the try block.
Turned off the final state saving in the Frame_order.test_auto_analysis system test. This almost halves the time required for the test. A private class variable _final_state has been added to the auto_analyses.frame_order.Frame_order_analysis class which when False will cause the state saving step to be skipped.
The summarise() function call is now after saving the final state in the frame order auto-analysis. This is needed because the summarise() function will create a new set of data pipes, loading the results which already exist under a different pipe name in the relax data store. Otherwise the final state file is twice as big as it should be.
Incorporated the contents of count_sobol_points.py into the frame order auto-analysis module. The analysis script has been converted into the count_sobol_points() function which will generate a summary table of the number of quasi-random Sobol' points used for the PCS numerical integration.
The frame order auto-analysis now calls the count_sobol_points() function at the end. This is to automatically create the Sobol' point summary table.
Fixes for the auto_analyses.frame_order.summerise() function. If the count_sobol_points() function is called followed by summerise(), a RelaxError will be raised as the data pipe already exists. The summerise() function has been modified to switch to the data pipe if it already exists.
Expanded the frame order auto-analysis documentation. This adds a description for the summarise() and count_sobol_points() functions.
Elimination of most of the Frame_order.fixme_test_* system tests and associated data. These tests are from a very early stage of the development of the frame order theory back when the base data was the full and reduced alignment tensors for the each domain calculated from the RDC data. They do not fit into the current analysis where the base data is the RDCs and PCSs for the moving domain. There is no point upgrading the tests as it will be far too much effort and it will only duplicate the coverage of the Frame_order.test_cam_* system tests.
Renamed the Frame_order.fixme_test_opendx_map system test to Frame_order.test_opendx_map to activate it.
Upgraded the Frame_order.test_opendx_map system test. To upgrade from the ancient design to the current design so that the test is functional and relevant, this now uses the same setup as the Frame_order.test_cam_qr_int_rigid user function. Instead of performing optimisation, the test calls the dx.map user function.
Fix for the frame order specific API calculate() method. This was caught by the Frame_order.test_opendx_map system test. The scaling matrix was not being specified by the dx.map user function backend and this was causing the method to fail. Instead of passing the non-existent scaling matrix into the target function, the argument is simply ignored. The scaling matrix has no effect on the minimise.calculate user function so it is not necessary.
The verbosity flag is now being respected by the frame order specific API calculate() method. This silences the method when executing the dx.map user function. The chi2 value printout is suppressed and the verbosity argument is being sent into the frame order count_sobol_points() function.
Added a section printout to the frame order auto-analysis when summary tables are created.
The frame_order.simulate user function now defaults to creating a gzipped PDB file. This is to save room, and because most molecular viewers will automatically read gzipped PDB files.
Fix for the change of the pipe_control.pipes.test() function to check_pipe().
Small change in the title of the summary table of the frame order auto-analysis. 'Order parameters' has been replaced by 'Cone half angles' to clarify what the values really are.
Fix for the frame order optimisation target setup printouts. The 'Numerical integration: ' printout was fixed to 'Quasi-random Sobol' sequence'. This now changes to 'SciPy quadratic integration' if cdp.quad_int is set. The text 'PCS' has also been added for clarification.
Removed the call to the frame_order.simulate user function for the rigid model in the auto-analysis. There is no motion to simulate in the rigid model, so the frame_order.simulate user function has no use.
Improvements, fixes, and expansion of the results and data visualisation file creation. This is for the frame order auto-analysis. The visualisation() method has been renamed to results_output() and its scope expanded. The function previously only called the frame_order.pdb_model and frame_order.simulation user functions for creating PDB representations of the frame order motions and performing a pseudo-Brownian frame order dynamics simulate. This has been extended to also call the results.write user function for outputting results files and the rdc.corr_plot and pcs_corr_plot for generating correlation plots of the measured vs. back-calculated data. All parts of the auto-analysis were output files are required now call this method. This ensures that all output files are always created, and are placed into the correct directories.
Improvements for the sectioning printouts for the frame order auto-analysis. The sections now use the lib.text.formatting subtitle() and subsubtitle() functions to distinguish them from the output of all the user functions, which use the section(), subsection() and subsubsection() functions. New sectioning printouts have been added for clarity.
Possible fixes for the frame order auto-analysis. This is just in case a user decides to not perform the optimisation starting with a PCS subset. In this case, the analysis will now execute correctly.
Improvements to the summary table for the frame order auto-analysis. The rotor and free rotor model motional eigenframe parameter axis_alpha is now being converted into spherical angles and reported in the table. This allows the motional eigenframe of all models to be easily compared in the table.
Created a directory and base PDB system for testing out the PCS information content. The base PDB system consists of Ad Bax's CaM domain structures superimposed onto the open CaM structure, the N-domain CoM shifted to the origin, and the C-domain CoM shifted to the z-axis.
Modified the PCS content testing base system. The paramagnetic centre is now shifted to the origin, as this is the real centre of the PCS physics.
Intermediate optimisation results are now stored by the frame order auto-analysis. The results from each minimise.grid_search and minimise.execute user function call are now stored in specially named directories located in the 'intermediate_results' directory, which itself is located in the auto-analysis results_dir directory. This allows intermediate results to be more easily analysed later on, which can be useful for optimising the optimisation steps. These directories can also be used for the pre_run_dir auto-analysis argument for subsequent refinements from earlier steps in the optimisation. The results stored include everything from the results_output() method and the count_sobol_points() and summarise() functions. To allow this to work, the auto-analysis functions count_sobol_points() and summarise() required modification. Results files are now always loaded into a temporary data pipe, rather switching to the corresponding pipe, and the temporary data pipe is deleted after the data has been extracted. The original data pipe name is also stored and a switch back to that pipe occurs at the end of each function.
The simulation is now turned of for intermediate results in the frame order auto-analysis. The intermediate results are only for checking, so for these the full pseudo-Brownian dynamics simulations are not required. The simulation flag has been introduced into the results_output() method of the auto-analysis to control this.
The splitting of the rigid model grid search into rotation and translation parts is now optional. In the frame order auto-analysis, the rigid_grid_split argument has been introduced. The alternating algorithm of performing a grid search over the rotational space followed by translation is now optional and turned off by default. The reason is because the global minimum is sometimes missed with this shortcut algorithm.
Speedup of the Frame_order.test_auto_analysis system test. The splitting of the rigid model grid search into rotation and translation parts has been reactivated.
Created the Optimisation.has_grid() method for the frame order auto-analysis. This is used to test if the optimisation settings object has a grid search defined.
The grid search can now be skipped for the rigid model in the frame order auto-analysis. If the input 3D structures are close to the real solution, the grid search over the translational and rotation parameters of the rigid model could be skipped. This speeds up the analysis and can help find the real solution in problematic cases.
The intermediate results storing can now be turned off in the frame order auto-analysis. The new store_intermediate Boolean argument has been added to the analysis to allow the storage of these results to be turned on or off.
The intermediate results are no longer stored in the Frame_order.test_auto_analysis system test. This drops the test timing on one system from ~190 seconds to ~50 seconds.
The compression level for results files can now be set in the frame order auto-analysis. This is via the new argument results_compress_type, which is used to set the compress_type argument of the results.write user function. The results reading parts of the auto-analysis have been updated to allow uncompressed, bzip2 compressed, and gzip compressed files to be handled.
Added a printout of the frame order model in the target function setup function. This is printed out when the minimise.calculate, minimise.grid_search, or minimise.execute user functions are called, and is for better feedback, especially in the auto-analysis where the repetitive optimisations can be confusing.
Updated the frame order analysis for the structure.load_spins user function changes. The minimise_setup_atomic_pos() function of the specific_analyses.frame_order.optimisation module now handles the mixed type spin.pos variable correctly.
The data pipe containing a PCS subset is now optional in the frame order auto-analysis. This is for systems which have so little data that a subset makes no sense.
Redesigned the optimisation steps for the frame order auto-analysis. The code has been significantly simplified as the optimisation for the PCS subset and full data set was the same. The code duplication has been eliminated by combining it into the new optimisation() method. The check for the PCS subset has also been expanded so that it is skipped if the subset data pipe is not supplied, even if an optimisation object for the subset has been (this should prevent strange errors when the auto-analysis is incorrectly used). A side effect of this code merger is that the zooming grid search has now been activated for the full PCS data set. This is of great benefit when a PCS subset is not being used.
The minimise.execute user function skip_preset flag is now False in the frame order auto-analysis. This is for the main model optimisation. Without this flag set, the grid search for the pivot point position for the rotor model was being skipped at the first zoom level.
The pivot point can now be excluded from the grid search in the frame order auto-analysis. If the initial pivot point is known to be reasonable, then it may be possible to skip it in the grid search for the rotor frame order model. This can lead to a speedup of the analysis and can help with stability. The pivot_search argument has been added to the auto-analysis Optimisation.add_grid() method to enable this. The get_grid_pivot_search() method has also been added to allow the auto-analysis to query this and turn it off if desired.
Updated the description of the frame_order.permute_axes user function. This now includes the isotopic cone.
Replaced the table in the frame_order.permute_axes user function. The original table was an old and incorrect version. This has been replaced by the correct permutation table.
Added some old relax scripts for both simulating and predicting the frame order matrix elements. These were used for the initial implementation of the pseudo-ellipse frame order model back in July 2010. The scripts will be extended for all frame order models. The simulated values could then be used in unit tests of the frame order matrix code in lib.frame_order.
Updated the frame_order_simulate.py script for simulating frame order matrix elements. The MODEL variable has been added in preparation for supporting all model types, and this is now added to the file name. The Grace header is now also being automatically generated.
Improvements for the Grace files produced by the frame_order_simulate.py script. The model name is now set as a variable and is used for the subheading.
Updated the frame_order_solution.py script for directly calculating the frame order matrix elements. The MODEL variable has been added in preparation for supporting all model types, and this is now added to the file name. The Grace header is now also being automatically generated and this matches that for the frame_order_simulation.py script.
Zero values can now be handled in the pseudo-ellipse 1st degree frame order matrix function. This is in lib.frame_order.pseudo_ellipse.compile_1st_matrix_pseudo_ellipse().
Removed some unused code in the pseudo-ellipse 2nd degree frame order matrix function. This is the compile_2nd_matrix_pseudo_ellipse() function in the lib.frame_order.pseudo_ellipse module. The change should make the RDC part of the frame order analysis for the pseudo-ellipse model slightly faster.
Modified the rotate_daeg() function as this is independent of the degree of the frame order matrix. This is the lib.frame_order.matrix_ops.rotate_daeg() function.
Fix for the compile_1st_matrix_pseudo_ellipse() function. This function of the lib.frame_order.pseudo_ellipse module now can rotate the 1st degree frame order matrix out of its eigenframe and into the PDB frame.
Created an executable Python script for mass converting the frame order matrix Grace graphs. The script converts the *.agr files to EPS and PNG files.
Modified the frame order matrix Grace graph to EPS/PNG format conversion script. The binary being called is now 'grace' rather than 'xmgrace'. This allows different Grace versions to be used.
Modified the frame order matrix Grace graph to EPS/PNG format conversion script. Grace is now used to create a PostScript file and then the ps2eps program is called to convert to EPS. This produces much better EPS files for inclusion into LaTeX documents.
Redesign of the frame_order_solution.py script for calculating the frame order matrix elements. This script now loops over all models, all motional frame orientations, and all order parameters to generate the Grace graphs of all 1st and 2nd degree frame order matrix elements. Therefore the script only needs to be executed once. The script also now calculates a point at zero (slightly shifted to 0.01 to avoid artifacts).
Added all of the Grace graphs produced by the frame_order_solution.py script. These are the graphs of the 1st and 2nd degree frame order matrix elements, calculated using the functions in lib.frame_order.
Updated frame_order_simulate.py to be much faster in simulating the frame order matrix elements. The script also matches the Grace file output of the frame_order_solution.py script. The inside() method has been renamed for the pseudo-ellipse and the infrastructure for adding support for the other frame order models has been added. By shifting calculations outside of the loops, the script is now many orders of magnitude faster.
Implemented the compile_1st_matrix_rotor() function. This is for the lib.frame_order.rotor module. The function will calculate the 1st degree in-frame frame order matrix for the rotor model.
Created the Grace graphs for the rotor model 1st degree frame order matrix elements. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Implemented the compile_1st_matrix_free_rotor() function. This is for the lib.frame_order.free_rotor module. The function will calculate the 1st degree in-frame frame order matrix for the free rotor model.
Created the Grace graphs for the free rotor model 1st degree frame order matrix elements. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Implemented the compile_1st_matrix_iso_cone() function. This is for the lib.frame_order.iso_cone module. The function will calculate the 1st degree in-frame frame order matrix for the isotropic cone model.
Created the Grace graphs for the isotropic cone model 1st degree frame order matrix elements. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Implemented the compile_1st_matrix_iso_cone_torsionless() function. This is for the lib.frame_order.iso_cone_torsionless module. The function will calculate the 1st degree in-frame frame order matrix for the torsionless isotropic cone model.
Created the Grace graphs for the torsionless isotropic cone 1st degree frame order matrix elements. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Implemented the compile_1st_matrix_iso_cone_free_rotor() function. This is for the lib.frame_order.iso_cone_free_rotor module. The function will calculate the 1st degree in-frame frame order matrix for the free rotor isotropic cone model.
Created the Grace graphs for the free rotor isotropic cone 1st degree frame order matrix elements. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Docstring fixes for the new compile_1st_matrix_iso_cone() function.
A minor speedup for the frame_order_simulate.py script. The angles are now being calculated at the very start prior to the main loop, removing repetitive calculations.
The frame_order_simulate.py script now uses lib.text.progress.progress_meter(). This script for simulating the frame order matrix elements now uses the standard progress meter in relax to simplify the script. This should also speed up the calculations as the progress printouts were slowing down the calculations.
Simulation of the pseudo-ellipse frame order matrix elements. This is for a simulation of 1,000,000 states for each angle increment, and includes in-frame and out-of-frame and varying of theta X, Y, and Z. The resultant Grace graphs have been added to the repository.
The frame order matrix element simulation script now uses the Kronecker outer product. This allows the frame order matrix to be in the same notation as that used internally in relax. It will cause the colours of the Sijkl_* curves to match between the simulation and solution scripts.
Added the rotor model to the frame order matrix element simulation script. The generated in-frame and out-of-frame Grace graphs containing the matrix values for 1,000,000 simulation values have been added to the repository. The script was modified so that the rotation is generated by special rotation_*() methods which are aliased depending on the model.
Added the free rotor model to the frame order matrix element simulation script. The generated in-frame and out-of-frame Grace graphs containing the matrix values for 1,000,000 simulation values have been added to the repository. The inside_free_rotor() method has been added to always return True for the rotation generated by rotation_z_axis().
Simplifications and fixes for the 1st degree frame order matrix calculation for the pseudo-ellipse. The compile_1st_matrix_pseudo_ellipse() function of the lib.frame_order.pseudo_ellipse module has been significantly simplified by shifting a lot of maths outside of the quadratic integration.
Updated all the calculated 1st degree frame order matrix graphs for the pseudo-ellipse. The changes are due to the fixes in the lib.frame_order.pseudo_ellipse module.
Simplifications for all of the torsionless pseudo-ellipse frame order matrix equations.
Implemented the compile_1st_matrix_pseudo_ellipse_torsionless() function. This is for the lib.frame_order.pseudo_ellipse_torsionless module. The function will calculate the 1st degree in-frame frame order matrix for the torsionless pseudo-ellipse model.
Created the Grace graphs for the torsionless pseudo-ellipse model 1st degree frame order matrix. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Added the isotropic cone model to the frame order matrix element simulation script. The generated in-frame and out-of-frame Grace graphs for the torsion angle cone_sigma_max, containing the matrix values for 1,000,000 simulation values have been added to the repository. The inside_iso_cone() method has been created to check for the theta_x and theta_z angle violations from the rotation_hypersphere() method.
Simplifications for the inside_*() methods of the frame order matrix element simulation script. The limit() method is now called only once outside of these methods and the maximum cone half-angles passed into the inside_*() methods. Although only slightly faster, this is mainly to simplify the code.
Alphabetical ordering of methods in the frame order matrix element simulation script.
Simplification of some of the pseudo-ellipse 2nd degree frame order matrix equations.
More simplifications of the pseudo-ellipse 2nd degree frame order matrix equations.
Integer to float conversions in part_int_daeg2_pseudo_ellipse_13(). This avoid integer to float conversion during execution, saving a little time for the pseudo-ellipse 2nd degree frame order matrix compilation.
Removal of many repetitive calculations in the pseudo-ellipse 2nd degree frame order matrix equations.
Simplifications of pseudo-ellipse 1st degree frame order matrix functions. The xx, yy, and zz have been renamed to 00, 11, and 22 for consistency. And all sigma_max arguments have been dropped as they are not used.
Small numerical changes for the pseudo-ellipse 2nd degree frame order matrix graphs. These are only for the first point close to zero and the changes are minimal, caused by the recent simplifications of the code.
Created the Grace graphs for the free rotor pseudo-ellipse model 1st degree frame order matrix. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Implemented the compile_1st_matrix_pseudo_ellipse_free_rotor() function. This is for the lib.frame_order.pseudo_ellipse_free_rotor module. The function will calculate the 1st degree in-frame frame order matrix for the free_rotor pseudo-ellipse model.
Speedups and simplifications of the free rotor pseudo-ellipse 2nd degree frame order matrix equations.
Added the torsionless isotropic cone model to the frame order matrix element simulation script.
Implemented the compile_1st_matrix_double_rotor() function. This is for the lib.frame_order.double_rotor module. The function will calculate the 1st degree frame order matrix for the double_rotor model.
Created the Grace graphs for the double rotor model 1st degree frame order matrix. These are the values calculated directly from the lib.frame_order modules. The graphs were previously all zeros.
Recreated all of the simulated pseudo-ellipse frame order matrix element graphs. These are now in the Kronecker product notation so that they will match the graphs calculated using the relax lib.frame_order.pseudo_ellipse module.
Fix for the pseudo-ellipse 1st degree frame order matrix Daeg22 element.
Updated all of the pseudo-ellipse 1st degree frame order matrix graphs for the recent fix.
Converted the Sobol' rotation matrices to float32 in the frame order target function. This is to conserve huge amounts of memory to allow for more Sobol' points to be used. For example for the models which use 3D Sobol' points (isotropic cone and pseudo-ellipse), a maximum of 50000 Sobol' points requires 50000000 to be created, using about 15 Gb of RAM.
A few Frame_order system test updates for the float64 to float32 memory saving changes. The chi-squared value of 3 tests was slightly different.
Bug fix for the activation of quadratic integration in the frame order auto-analysis. The calls to the frame_order.quad_int user function in the optimisation() method did not supply an argument so the user function was defaulting to False rather than the True value required.
The frame order auto-analysis summary functions are now more robust. If the data pipe already exists for some reason, it is deleted prior to the new one being created.
Changed the frame_order.quad_int user function argument default to True. This means that calling the user function without arguments will activate the quadratic integration rather than turning it off.
Added the isotropic cone model frame order matrix simulation graphs for the cone opening angle theta_x.
Created and added all of the torsionless isotropic cone simulated frame order matrix element graphs.
Added the free rotor isotropic cone model to the frame order matrix element simulation script. The generated Grace graphs containing the matrix values for 1,000,000 simulation values have been added to the repository. The self.torsion_check variable has been created to allow the inside_iso_cone() method to skip the torsion angle check when its value is False.
Added the torsionless pseudo-ellipse model to the frame order matrix element simulation script. The generated Grace graphs containing the matrix values for 1,000,000 simulation values have been added to the repository. The rotations are generated by the rotation_hypersphere_torsionless() method and the angle violations checked using the inside_pseudo_ellipse() method.
Bug fix for the torsionless pseudo-ellipse 1st degree frame order matrix. The 11 element was of the wrong sign.
Fixes for the torsionless pseudo-ellipse 1st degree frame order matrix element graphs.
Added the free rotor pseudo-ellipse model to the frame order matrix element simulation script. This only required the self.torsion_check variable to be set to False. The model uses the inside_pseudo_ellipse() and rotation_hypersphere() methods.
Fixes for free rotor isotropic cone 1st degree frame order matrix graphs calculated using relax. The 1st degree function accepts the cone opening angle theta rather than the order parameter S.
Added the frame order matrix element graphs for the in-frame free rotor pseudo-ellipse model.
Added the frame order matrix element graphs for the out-of-frame free rotor pseudo-ellipse model.
Added support for the double rotor model to the frame order matrix element simulation script. The double rotation is constructed in the new rotation_double_xy_axes() method, and the checks for the violation of the two torsion angles in the inside_double_rotor() method. In the main loop, the theta, phi and sigma angles correspond to sigma1, sigma2, and nothing.
Fixes for all of the calculated double rotor model frame order matrix graphs. The X and Y angles were mixed up. The first torsion half-angle sigma1 corresponds to a y-axis rotation and the second sigma2 corresponds to a x-axis rotation.
Added the frame order matrix element graphs for the double rotor model.
A divide by zero fix for the torsionless pseudo-ellipse. This is in the compile_2nd_matrix_pseudo_ellipse_torsionless() relax library function.
A divide by zero fix for the free rotor pseudo-ellipse. This is in the compile_2nd_matrix_pseudo_ellipse_free_rotor() relax library function.
The 1st angle for the calculated frame order matrix graphs is 0 for all non pseudo-ellipse models. This is for the frame_order_solution.py script. Only the pseudo-ellipse models where numerical integration is required fail for the angle of 0.0. Therefore the changing of the first angle from 0.0 to 0.01 only occurs for the pseudo-ellipse models. All graphs have been updated.
The 1st pseudo-ellipse torsion angle value in the frame order matrix graphs is now 0.0. Only the cone opening angles set to 0.0 cause a failure in the pseudo-ellipse models, so the torsion angle is now allowed to start at exactly zero.
Clean up of the frame order matrix element simulation script.
Redesign of the free rotor isotropic cone frame order model - the order parameter has been replaced. From the frame order matrix element graphs in test_suite/shared_data/frame_order/sim_vs_pred_matrix, specifically Sijkl_iso_cone_free_rotor_in_frame_theta_x_calc.agr, Sijkl_iso_cone_free_rotor_axis2_1_3_theta_x_calc.agr, and Sijkl_iso_cone_free_rotor_out_of_frame_theta_x_calc.agr, it is clear that the symmetry of the order parameter after 120 degrees causes the 2nd degree frame order matrix to be incorrectly estimated. Therefore the S1 order parameter has been replaced with the original cone opening angle cone_theta. All parts of relax have been updated for this large conversion.
Updated the frame order matrix element graphs for the free rotor isotropic cone fixes. The cone S1 parameter has been converted back to the original cone theta opening half-angle, allowing the 2nd degree frame order matrix elements to be properly calculated for all motions.
Eliminated the lib.frame_order.iso_cone.populate_*() functions. The populate_1st_eigenframe_iso_cone() function was unused and incorrect, so it was deleted. The contents of the populate_2nd_eigenframe_iso_cone() function have been shifted compile_2nd_matrix_iso_cone() as a separate function is unnecessary. This now matches all the other lib.frame_order modules.
Bug fix for the frame_order.simulate user function. The incorrect model number was being specified and hence the simulation was not starting from the optimised average domain position but rather the arbitrary position of the original structure.
Manual Python 3 fixes for the dict.key() function which returns a list or iterator in Python 2 or 3. This matches r26519 in trunk.
Python 3 fixes via 2to3 - the "while 1" construct has been replaces with "while True". The command used was: 2to3 -j 4 -w -f idioms .
Python 3 fixes via 2to3 - the spacing around commas has been fixed. The command used was: 2to3 -j 4 -w -f ws_comma .
Python 3 fixes via 2to3 - the xrange() function has been replaced by range(). The command used was: 2to3 -j 4 -w -f xrange .
Started to create the Frame_order.test_pdb_model_rotor system test. This will be used to check that the PDB representations of the frame order motions are correct.
Modified the frame_order.pdb_model user function backend to handle missing structural data. The create_ave_pos() function of the specific_analyses.frame_order.geometric module now checks that cdp.structure exists, and if not a warning is given and the PDB file creating is skipped.
Fixes for the frame_order.pdb_model user function backend for when no data is present. The pipe_centre_of_mass() function of pipe_control.structure.mass module is now called with the missing_error flag set to False so that the PDB generation can continue with the CoM set to [0, 0, 0].
The geometric representation part of the frame_order.pdb_model user function now checks parameters. This calls the specific_analyses.frame_order.checks.check_parameters Check object to make sure that all necessary parameters for the model exist.
Completed the Frame_order.test_pdb_model_rotor system test. This now sets the rotor axis to the z-axis (with a printout to be sure), sets the torsion angle to zero for simplicity, creates a new data pipe and loads the PDB representation file, then checks all of the key atom coordinates.
Fixes for the unit tests of the lib.frame_order_matrix_ops module for the free rotor isotropic cone. The S1 order parameter has been eliminated due to angles > pi/2.0 causing the frame order matrix to be incorrectly predicted. Therefore all unit tests have been converted to use the cone opening angle theta instead. In addition, the test_compile_2nd_matrix_iso_cone_free_rotor_disorder had been modified to pass with the incorrect frame order matrix by comparing to the half cone frame order matrix rather than the identity frame order matrix.
Fix for inverted axes in the new Frame_order.test_pdb_model_rotor system test.
Huge bug fix for the frame_order.pdb_model user function - the single axis direction was incorrect. In the PDB representation of the frame order motion for the rotor and isotropic cone models (rotor, free rotor, isotropic cone, free rotor isotropic cone, and torsionless isotropic cone), the X and Z axes were swapped. This is because the eigenframe of the motion was being incorrectly constructed via the lib.geometry.rotations.two_vect_to_R() function. For better control, the specific_analyses.frame_order.geometric.frame_from_axis() function has been created. This constructs a full motional eigenframe from the Z-axis. The problem was detected via the new Frame_order.test_pdb_model_rotor system test.
Size fix for the rotor representation from the frame_order.pdb_model user function. The size problem was detected via the Frame_order.test_pdb_model_rotor system test. The rotors in the PDB representation were all fixed in size, and ignored the 'size' argument of the frame_order.pdb_model user function. The size argument is now passed into the add_rotors() function of the specific_analyses.frame_order.geometric and passed on to the rotor() function of the lib.structure.represent.rotor module.
Created the Frame_order.test_pdb_model_rotor2 system test to check for an offset pivot. The pivot is set to [1, 0, 1] so that the rotor axis is tilted -45 degrees in the xz-plane. And the size of the geometric object is set to 100 Angstrom for better testing of the sizes of the elements.
Simplification of the Frame_order.test_pdb_model_rotor system test. The size is now programatically handled.
Created the Frame_order.test_pdb_model_iso_cone system test. This is for checking the PDB representation of the isotropic cone frame order model created by the frame_order.pdb_model user function. It checks both A and B representations.
Fix for the cone sized created by the frame_order.pdb_model user function. The 'size' argument was not being used at all for the cone size. It is now passed into the lib.structure.represent.cone.cone() function as the 'scale' argument.
Small fix for the Frame_order.test_pdb_model_iso_cone system test for the 'B' representation.
Fix for the representation label positions created by the frame_order.pdb_model user function. The 'size' argument was not being used at all for the representation title atoms. It is now passed into the add_titles() function as the displacement argument + 10 Angstrom.
Printout fix for the axis in the Frame_order.test_pdb_model_iso_cone system test.
Created the Frame_order.test_pdb_model_iso_cone_xz_plane_tilt system test. This checks the PDB file from the frame_order.pdb_model user function for the isotropic cone model with a xz-plane tilt.
Renamed all of the Frame_order.test_pdb_model_* system tests to be more descriptive.
Improvements for all of the Frame_order.test_pdb_model_* system tests. The rotate_from_Z() method has been introduced to simplify the determination of the 3D coordinates expected for the PDB file. This will allow for more advanced testing of the PDB for the cone models.
Fixes for the printouts from the Frame_order.test_pdb_model_rotor_* system tests.
Alphabetical ordering of the Frame_order system test methods.
Fixes for all of the Frame_order system tests - the temporary directories are now being deleted. The system test base class tearDown() method is now being called to properly clean up after the tests.
Created the Frame_order.test_pdb_model_pseudo_ellipse_z_axis system test. This demonstrates the correct atom coordinates in the PDB file created by the frame_order.pdb_model user function for the pseudo-ellipse model along the z-axis.
Fixes for the checks in the Frame_order.test_pdb_model_* system tests. Atomic positions are now checked with self.assertAlmostEqual() to 3 places, and the residue and atom names and numbers are checked with self.assertEqual().
Created the Frame_order.test_pdb_model_pseudo_ellipse_xz_plane_tilt system test. This checks the PDB file created by the frame_order.pdb_model user function for the pseudo-ellipse model with a xz-plane tilt. To properly construct the coordinates, the rotate_from_Z() method was modified to accept a rotation matrix argument to allow the geometric shape to be rotated.
Modified the Frame_order.test_pdb_model_iso_cone_xz_plane_tilt system test to have a cone angle. The cone opening half-angle was previously 0.0. The test now checks the geometric object in the PDB file for a cone opening half-angle of 2.0.
Modified the Frame_order.test_pdb_model_iso_cone_z_axis system test to have a cone angle. The cone opening half-angle was previously 0.0. The test now checks the geometric object in the PDB file for a cone opening half-angle of 2.0.
Created two new system tests for the free rotor PDB representation file. This is the file from the frame_order.pdb_model user function. The two new unit tests are Frame_order.test_pdb_model_free_rotor_z_axis and Frame_order.test_pdb_model_free_rotor_xz_plane_tilt.
Created two new frame order system tests for the free rotor isotropic cone PDB representation file. This is the two PDB files from the frame_order.pdb_model user function. The two new system tests are Frame_order.test_pdb_model_iso_cone_free_rotor_z_axis and Frame_order.test_pdb_model_iso_cone_free_rotor_xz_plane_tilt.
Created two new frame order system tests for the torsionless isotropic cone PDB representation file. This is the two PDB files from the frame_order.pdb_model user function. The two new system tests are Frame_order.test_pdb_model_iso_cone_torsionless_z_axis and Frame_order.test_pdb_model_iso_cone_torsionless_xz_plane_tilt.
Created two new frame order system tests for the free rotor pseudo-ellipse PDB representation file. This is the two PDB files from the frame_order.pdb_model user function. The two new system tests are Frame_order.test_pdb_model_pseudo_ellipse_free_rotor_z_axis and Frame_order.test_pdb_model_pseudo_ellipse_free_rotor_xz_plane_tilt.
Created two new frame order system tests for the torsionless pseudo-ellipse PDB representation file. This is the two PDB files from the frame_order.pdb_model user function. The two new system tests are Frame_order.test_pdb_model_pseudo_ellipse_torsionless_z_axis and Frame_order.test_pdb_model_pseudo_ellipse_torsionless_xz_plane_tilt.
Created two new frame order system tests for the double rotor PDB representation file. This is the two PDB files from the frame_order.pdb_model user function. The two new system tests are Frame_order.test_pdb_model_double_rotor_z_axis and Frame_order.test_pdb_model_double_rotor_xz_plane_tilt.
Added relax scripts and PDB files which match the Frame_order.test_test_pdb_model_* system tests. These were used to construct and visually check the tests in a molecular viewer. These could be a useful reference, so have been added to the repository.
Simplified all of the Frame_order.test_pdb_model_* system tests. The atom, residue and 3D coordinate checking in all these methods has been shifted into the common check_pdb_model_representation() method. This dramatically decreases the amount of code in the system test file.
Simplification for all of the Frame_order.test_pdb_model_* system tests. The model setup in all of these tests has been merged into the common setup_model() method. This not only removes a large quantity of repetitive code, but the new method can also be used for constructing future tests, for example for checking the frame_order.simulate user function.
Created an initial version of the Frame_order.test_simulate_rotor_z_axis system test. This is to check the frame_order.simulate user function rotor model along the z-axis. It currently fails due to a bug in the user function.
Fixes for the Frame_order.test_simulate_rotor_z_axis system test. Now 6 atoms are being created at X, -X, Y, -Y, Z, and -Z, 100 Angstrom from the origin. This is required so that the CoM is at the origin, to allow the CoM-pivot vector to be unchanged at [1, 0, 0] so that the axis alpha angle of pi/2 creates an axis parallel to Z. The origin to atom distance check has also been loosened due to the PDB truncation artifact.
Fix for the Frame_order.test_pdb_model_free_rotor_xz_plane_tilt system test. This was broken while implementing the Frame_order.test_simulate_rotor_z_axis system test. Instead of shifting the 6 atom structure so its CoM is the pivot of the motion when creating the atoms, now the Frame_order.test_simulate_rotor_z_axis system test sets the average domain translation vector to the pivot to achieve the same result. This preserves the z-axis orientation of the rotor models.
Created the Frame_order.test_simulate_free_rotor_z_axis system test. This is to check the frame_order.simulate user function for the free rotor model along the z-axis.
Created the Frame_order.test_simulate_iso_cone_z_axis system test. This is to check the frame_order.simulate user function for the isotropic cone model along the z-axis.
Created the Frame_order.test_simulate_iso_cone_free_rotor_z_axis system test. This is to check the frame_order.simulate user function for the free rotor isotropic cone model along the z-axis.
Created the Frame_order.test_simulate_iso_cone_torsionless_z_axis system test. This is to check the frame_order.simulate user function for the torsionless isotropic cone model along the z-axis.
Created the Frame_order.test_simulate_pseudo_ellipse_z_axis system test. This is to check the frame_order.simulate user function for the pseudo-ellipse model along the z-axis.
Created the Frame_order.test_simulate_iso_cone_xz_plane_tilt system test. This is to check the frame_order.simulate user function for the torsionless isotropic cone model with a xz-plane tilt.
Created the Frame_order.test_simulate_pseudo_ellipse_free_rotor_z_axis system test. This is to check the frame_order.simulate user function for the free rotor pseudo-ellipse model along the z-axis.
Created the Frame_order.test_simulate_pseudo_ellipse_xy_plane_tilt system test. This is to check the frame_order.simulate user function for the pseudo-ellipse model with a xz-plane tilt.
Created the Frame_order.test_simulate_pseudo_ellipse_torsionless_z_axis system test. This is to check the frame_order.simulate user function for the torsionless pseudo-ellipse model along the z-axis.
Fix for the Frame_order.test_simulate_pseudo_ellipse_xz_plane_tilt system test name. This was mislabelled as Frame_order.test_simulate_pseudo_ellipse_xy_plane_tilt.
Redesign of the pymol.frame_order user function. This user function was still fitting to the old design in the relax trunk. It has been updated for the frame_order_cleanup branch whereby the frame_order.pdb_model user function has been split up and the positional distribution has been replaced by the Brownian simulation user function frame_order.simulate.
Better checking for the non-moving domain setup. The frame_order.pdb_model user function will now raise a RelaxError if the frame_order.ref_domain user function has not been called to set up the non-moving domain.
Updated the frame_order.ref_domain user function for the current branch design. This user function was quite out of date. The alignment tensor checks have been removed, to allow this to be used in the absence of base data. And the user function description has been updated.
Updated all frame order system tests for the frame_order.ref_domain user function requirement.
Expanded all of the Frame_order.test_simulate_* system tests. Two atoms have been added to the origin [0, 0, 0], one in the moving domain, the other in the reference non-moving domain. The positions of these atoms are checked to make sure that the domain systems are correctly handled.
Expanded the double rotor model description in the frame_order.select_model user function.
Added the pipe_name argument to the frame order check_model() function. This is for the specific_analyses.frame_order.checks module.
Converted the specific_analyses.frame_order.checks module to the new Check object design. This follows from http://wiki.nmr-relax.com/Relax_source_design#The_check_.2A.28.29_functions and the changes significantly simplify the checking objects.
Improved checking for the frame order generate_pivot() function. The check_model() checking object is now called to make sure the frame order model has been specified, as this is essential for this function.
Created two system tests for the frame_order.simulate user function for the double rotor model. These are Frame_order.test_simulate_double_rotor_mode1_z_axis and Frame_order.test_simulate_double_rotor_mode2_z_axis.
Created two system tests for the frame_order.simulate user function for the double rotor model. These are Frame_order.test_simulate_double_rotor_mode1_xz_plane_tilt and Frame_order.test_simulate_double_rotor_mode2_xz_plane_tilt.
Added relax scripts which match the Frame_order.test_test_simulate_* system tests. These are the tests of the frame_order.simulate user function. These were used to construct and visually check the Brownian simulation and PDB model representation in a molecular viewer. These could be a useful reference, so have been added to the repository.
Fix for the frame order auto-analysis when only the 'rigid' model is optimised. The final summary table printout for the number of Sobol' points used was failing as there were no models in the table. The table is now only printed out if non rigid models are present in the model list.
Introduced the nested_params_ave_dom_pos argument to the frame order auto-analysis. This allows the average domain position to be set to no rotations and translations rather than taking the average position from the rotor or free-rotor model. This can be useful when large motions are present causing the rigid model to have unreasonable domain positions.
Fix for the frame_order.permute_axes user function description to allow the manual to be compiled. The table caption containing the user function name was causing the LaTeX compilation to fail. Therefore the captions have been rewritten to avoid the user function name.
Modified the frame order system test check_chi2() method to test the statistics.model user function. This causes all of the Frame_order.test_cam_* system tests to fail, as the user function backend is not implemented for the frame order analysis.
Implemented the frame order analysis backend for the statistics.model and statistics.aic user functions. This simply required aliasing the specific analysis API common _get_model_container_cdp() method to get_model_container().
Bug fix for the frame order specific analysis API base_data_loop() method. This was looping over non-existent PCS and RDC data. Now the alignment ID is checked for in the interatomic data container 'rdc' data structure and the spin container 'pcs' data structure, as well as values of None, before yielding the data.
Created a large set of system tests for implementing the frame_order.distribute user function. This user function will be similar to frame_order.simulate. However instead of creating a PDB file with models from a pseudo-Brownian simulation, the frame_order.distribute user function will generate a PDB file of models forming a uniform distribution of structures covering the full frame order motional space. The new system tests are: Frame_order.test_distribute_double_rotor_mode1_xz_plane_tilt, Frame_order.test_distribute_double_rotor_mode1_z_axis, Frame_order.test_distribute_double_rotor_mode2_xz_plane_tilt, Frame_order.test_distribute_double_rotor_mode2_z_axis, Frame_order.test_distribute_free_rotor_z_axis, Frame_order.test_distribute_iso_cone_z_axis, Frame_order.test_distribute_iso_cone_xz_plane_tilt, Frame_order.test_distribute_iso_cone_torsionless_z_axis, Frame_order.test_distribute_pseudo_ellipse_xz_plane_tilt, Frame_order.test_distribute_pseudo_ellipse_z_axis, Frame_order.test_distribute_pseudo_ellipse_free_rotor_z_axis, Frame_order.test_distribute_pseudo_ellipse_torsionless_z_axis, Frame_order.test_distribute_rotor_z_axis. These are aliases for the equivalent Frame_order.test_simulate_* system tests which have had the 'type' keyword argument added, defaulting to 'sim', which allows to switch between the frame_order.simulate and frame_order.distribute user functions. The concept behind these system tests are the same for both user functions, so the code is shared.
Created the front-end of the frame_order.distribute user function. This is a copy and modification of the frame_order.simulate user function, as the concepts are similar.
Small modification of the frame_order.simulate user function. The GUI file opening dialog wildcard selectors are now set to all PDB file types (plain text, bzip2 compressed, and gzip compressed).
Added the frame_order.distribute user function to the auto-analysis results output. This will allow both the pseudo-Brownian simulation and uniform distribution PDB files to be available to the user in all results directories (excluding the intermediate results for speed).
Implemented the back-end of the frame_order.distribute user function. This follows the design of the pseudo-Brownian simulation frame_order.simulate user function. The specific_analyses.frame_order.uf.distribute() function has been created as a modified copy of the simulate() function of the same module. This simply performs checks and assembles the data, passing into the new lib.frame_order.simulate.uniform_distribution() function, which itself is a modified copy of the brownian() function in the same module.
Introduced the max_rotations argument into the frame_order.distribute user function. This is used to prevent the user function from running forever. This happens whenever a cone opening angle or torsion angle is zero, and hence the random sampling of the rotational space will never find rotations within the motional distribution.
Improved control of the frame_order.distribute user function in the frame order auto-analysis. The maximum number of rotations can now be set, and the argument for the total states for the distribution has been shortened.
Speedup of the Frame_order.test_auto_analysis system test. After the introduction of the frame_order.distribute user function into the auto-analysis, the test was taking far too long to complete. Now the distribution arguments are set to low values to allow the test to pass in under a minute.
Changed the default relax results compression type to bzip2 in the frame order auto-analysis. This was set to no compression for speeding up some system tests, however the system tests can set this for themselves.
The Frame_order.test_auto_analysis system test now sets the results file compression type to bzip2.
Changed the default max_rotations argument value to 100,000 in the frame_order.distribute user function. This decrease from one million is so the user function completes in a reasonable amount of time.
The frame_order.distribute user function now warns when the maximum rotations are reached.
Deleted a number of Frame_order.test_distribute* system tests. These are the four double rotor model tests. The frame_order.distribute user function cannot operate on these test cases as one of the two torsion angles are set to zero in the tests.
Fix to allow Monte Carlo simulations to be repeated in the frame order analysis. The code for checking for pre-existing Monte Carlo simulation data structures and raising a RelaxError if anything is found has been deleted.
Fix of a fatal bug preventing the frame order analysis to be run on a multi-processor system. The multi-processor code was calling the count_sobol_points() function of the specific_analyses.frame_order.optimisation module to give feedback when calling the minimise.execute or minimise.calculate user functions. However this was run in the slave command run() method, hence would be executed on the slave. The problem is that count_sobol_points() performs a number of checks on the current data pipe, however the slaves do not have any data pipes set up.
Added the new 'atom_id' argument to the frame_order.distribute user function. This uses the new inverse selection functionality recently introduced into the trunk to delete all structural data not matching the atom_id from the copy of the loaded structural data string prior to generating the distribution of structures.
Bug fix for the frame order target function (introduced recently). The copy.deepcopy() function is now used for all numpy input data to avoid the data from being modified between function calls. This is important for missing RDC and PCS data which is sent in as NaN values. In the target function __init__() method, the NaN values are replaced by 0.0 after the self.missing_rdc and self.missing_pcs structures have been by checking for NaN values. However the recent specific_analyses.frame_order.optimisation change in the Frame_order_minimise_command slave command to printout the number of integration points resulted in the target function being initialised twice, causing all NaN values to be 0.0 in the second initialisation. Hence all missing data was being treated as real data with values of 0.0.
Created a new skeleton chapter in the relax manual for the frame order analysis.
Added a theory section to the new frame order chapter. This is taken from an in-preparation supplement.
Rearrangement of the frame order chapter in the manual. The theory section has been spun out into its own frame_order_theory.tex LaTeX file for better organisation.
Added two more sections to the frame order chapter of the manual. This includes a frame order modelling section and PCS numerical integration section. Both are from a supplement from an in-preparation manuscript.
Added a DOI and ISBN number to the bibliography.
Moved the frame_order_theory.tex LaTeX file into the frame_order directory.
Shifted the frame order model derivations into their own 'Advanced topics' chapter.
Added the frame order sample scripts used in the CaM-IQ analysis.
Added an introduction for the frame order chapter of the manual.
Added a 'Data analysis' section to the frame order chapter of the manual. This includes the N-state and frame order analysis scripts required to perform a full analysis.
Editing of the data analysis section of the frame order chapter of the manual. A PCS structural error figure has been added, all the text improved, and the scripts made to match those in sample_scripts/frame_order/.
Added a section to the end of the frame order chapter about the long computation times.
The 'scons clean' target now removes all LaTeX *.aux files. The docs/latex/frame_order/ directory is now also being checked for *.aux files.
Removed many unnecessary references to relax.
Removed lots of useless comments about book references.
Added some images missing from the frame order chapter of the manual.
Avoided a doubly defined label in the manual.
Removed some duplicated text in the frame order models chapter of the manual. This is duplicated from the frame order analysis chapter.
Indentation fix for allowing the API documentation to be properly compiled.
Added a patch file for fixing Epydoc version 3.0.1. This is needed to allow the dot graph files names to be unique (by no longer truncating to 30 characters), and to allow epydoc to handle newer Graphvis versions.
Improvements for the release checklist document. The backporting of the CHANGES file to trunk is now more obvious, and instructions for fixing Epydoc have been added.
Clean up of some of the release instructions (for using vim).
Added error catching to the find_unused_imports.py developer script.
Fix for the error catching in the find_unused_imports.py developer script. The numerous pylint warnings are also sent to STDERR.
Removed the printout of pylint STDERR messages in the find_unused_imports.py developer script.
Elimination of a number of wildcard imports from some frame order timing scripts. This is to avoid excessive function imports.
Removal of an unused import from the user_functions.frame_order module.
Removal of unused imports from the test_suite/shared_data/frame_order/simulation scripts.
Updated some unused frame order scripts to use the new minimise user function design.
Unused import clean up in the test_suite/shared_data/curve_fitting/numeric_topology directory. All the scripts in this directory have been cleaned up to remove unused imports. In one case, commented out code was replaced with an 'if 0:' statement to silence the unused import warnings from the devel_scripts/find_unused_imports.py script.
Unused import clean up in the test_suite/shared_data/curve_fitting/profiling directory. The scripts in this directory have been cleaned up to remove unused imports.
Added an exception system to the find_unused_imports.py developer script. Sometimes pylint will give an "Unused import" warning for imports that are needed by the module. Therefore an exception list of the file name and module has been created to skip these warnings. The list covers the dep_check module and all of the profiling_*.py scripts in the directory test_suite/shared_data/dispersion/profiling/.
Added a copyright notice to the find_unused_imports.py development script. This is mainly to indicate how out of date the script will be in the future.
A directory can now be supplied on the command line for the find_unused_imports.py devel script.
Changed the imports in the test_monte_carlo_mean.py script. This inconsequential change is to avoid false positives from the find_unused_imports.py devel script.
Modifications of the test suite script for calculating synthetic CPMG data. The imports in cpmg_synthetic.py are now all used, rather than being commented out. This allows the find_unused_imports.py devel script to pass.
Unused import cleanup of all scripts in the test_suite/shared_data/dispersion/ directories. This both removes unused imports, or uncomments but deactivates temporarily unused code.
Removed unused imports from the scripts in the test_suite/shared_data/frame_order subdirectories.
Removed unused imports from the Spectrum system test base module.
Removed unused imports from the Relax_disp system test base module.
Clean up of all unused imports in the system test scripts.
Removed unused imports from the Structure system test base module.
Changed how the import of lib.regex in the Test_regex unit tests is used. The module is no longer stored in the TestCase class namespace, but is rather called directly within the unit test.
Changed the import of pipe_control.state in the test_state unit test module.
Removed unused imports from the unit tests.
Added another exception to the find_unused_imports.py devel script. This is for the test_suite.unit_tests._lib._geometry.test_rotations module which programatically obtains the imports using globals().
Added a workaround or hack for exceptions for circular imports in the find_unused_imports.py script. This is currently for the test_suite.unit_tests._lib.test___init__ and test_suite.unit_tests._lib._geometry.test___init__ modules.
Removal of unused imports from the GUI test modules.
Removed all unused imports from the pipe_control package.
Added import exceptions for the lib.compat module in the find_unused_imports.py devel script.
Added import exceptions for the lib.xml module in the find_unused_imports.py devel script. These are needed because of eval() function calls on XML stored Python data structures.
Removed all unused imports from the relax library package.
Removed all unused imports from the target_functions package.
Removed unused imports from the developer scripts.
Removed all unused imports from the specific_analyses package.
Removed all unused imports from the auto_analyses package.
Removed all unused imports from the numdifftools extern package.
Removal of the last unused import from the target_functions package.
Fix for the PCS system tests on old Python versions. The self.assertAlmostEqual() function cannot compare None values in earlier Python versions.
MS Windows fix for the Frame_order.test_generate_rotor2_distribution system test. The locale.setlocale() function call for correctly setting up a spinning progress meter was failing on MS Windows. The error is now caught and the local setting skipped.
Added Python 3.5 to the manual C module compilation script.
Added Python 3.5 to the Python multiversion test suite script.
Changes to the introduction of the frame order theory chapter of the manual.

Bugfixes

Fix for the alignment tensor MC simulation objects in the data store for Python 3.1. The sim_indices object was sometimes created with the range() method, however the returned iterator does not possess an index() function in Python 3.1. Therefore it was converted to a standard list.
Cosmetic bug fix for the running of the test suite in the GUI. The list of skipped tests in the status object was not being reinitialised for each run of the test suite. This only affects the GUI where the tests can be run multiple times. The result was that the list of skipped tests was always being printed out, even if no tests were skipped.
Fix for the numpy version number checking in the dep_check module. The version_comparison() function is now being used to compare numbers, replacing the previous hack.

Links

For reference, the announcement for this release can also be found at following links:

Softpedia also has information about the newest relax releases:

Announcements

If you would like to receive announcements about new relax versions, please subscribe to the relax announcement mailing list. This list only receives ~10 emails per year. It is archived at the SourceForge archives and in The Mail Archive.

relax 4.0.0