Trusted, Bureaucrats
4,228
edits
Changes
Jump to navigation
Jump to search
→Changes: Copied the changes list from the CHANGES file.
<section begin=changes/>
* Created the N_state_model.test_statistics system test. This system test will be used to implement the new statistics user function class consisting of the structure.model and structure.aic user functions for calculating and storing the [chi2, n, k] parameters and Akaike's Information Criterion statistic respectively.
* Added the structure.align user function to the renaming translation table. This is so relax identifies structure.align user functions in scripts to raise an error saying that the structure.superimpose user function should be used instead.
* Added the office-chart-pie set of Oxygen icons for use in the new statistics user function class.
* Created the empty statistics user function class. This adds the infrastructure for creating the statistics user functions.
* Small fix for the structure.add_model user function description.
* Created the frontend for the statistics.model user function.
* Created a wizard graphic for the statistics user functions. This is based on a number of Oxygen icons, as labelled in the SVG layer names.
* The statistics.model user function now uses the new statistics wizard graphic.
* Created the empty pipe_control.statistics module. This will be used for the backend of all of the statistics user functions.
* Fixes for the EPS versions of some Oxygen icons used in the relax manual. This is the actions.document-preview-archive and actions.office-chart-pie Oxygen icons used for the user function icons. The files were not created correctly in the Gimp. The export to EPS requires the width and height to be both set to 6 mm, and the X and Y offsets to zero. This allows the icon bounding boxes and sizes to match the other EPS icons.
* Implemented the backend of the statistics.model user function. The implementation heavily uses the specific analysis API, calling the calculate(), model_loop(), print_model_title(), model_statistics() and get_model_container() methods to do all of the work. The last of these API methods is yet to be implemented.
* Fix for the statistics.model user function backend. The API methods are now called with the model_info argument set to a keyword argument so that it is always passed in as the correct argument.
* Fix for the specific analysis API _print_model_title_global() common method. This method was horribly broken, as it was never used. The new statistics.model user function together with the N-state model uncovers this breakage.
* Defined the get_model_container() specific analysis API method. This base method raises a RelaxImplementError, therefore each analysis type must implement its own method (or use an API common method).
* Implemented the specific analysis API _get_model_container_cdp() commmon method. This is to be used for the get_model_container() for returning the current data pipe object as the model container. This is for the global models where the model information is stored in the pipe object rather than in spin containers.
* The N-state model now uses the _get_model_container_cdp() method. This is aliased as the get_model_container() specific analysis API method.
* Fix for the N_state_model.test_statistics system test - the probabilities were missing from k.
* Expanded the printouts from the statistics.model user function to include the statistics.
* Updated the N-state model num_data_points() function to use more modern integer incrementation.
* Fix for the N_state_model.test_statistics system test. The deselected spins and interatomic data containers are now taken into account for the RDC and PCS data point counts.
* Implementation of the statistics.aic user function. This is very similar to the statistics.model user function - the code was copied and only slightly modified. The new user function will calculate the current chi-squared value per model, obtain the model statistics, calculate the AIC value per model, and store the AIC value, chi-squared value and number of parameters in the appropriate location for the model in the relax data store.
* Created the empty lib.plotting.veusz module for graphing using Veusz (http://home.gna.org/veusz/).
* Shifted the lib.software.grace module to lib.plotting.grace. This follows from http://thread.gmane.org/gmane.science.nmr.relax.devel/7532 and http://thread.gmane.org/gmane.science.nmr.relax.devel/7536.
* Created XY-data functions for the plotting API of the relax library. These are currently copies of the heads of the lib.plotting.grace functions write_xy_data() and write_xy_header(). These lib.plotting.api functions (write_xy_data() and write_xy_header()) are set up to use the grace functions.
* Converted all of the Grace plotting in relax to use the plotting API of the relax library.
* Shifted the pipe_control.grace.write() function. This is now the format independent pipe_control.plotting.write_xy() function. The format argument has been added and this defaults to 'grace'. The grace.write user function has been updated to use the new backend.
* Updated the pcs.structural_noise user function to use the relax library plotting API.
* Fixes for the new pipe_control.plotting.write_xy() function. This includes missing imports which should have moved from pipe_control.grace, as well as shifting the axis_setup() function from the pipe_control.grace module into the pipe_control.plotting module.
* The rdc.corr_plot user function backend now uses the relax library plotting API. The write_xy_data() and write_xy_header() functions from lib.plotting.api are now uses instead of the equivalent pipe_control.grace functions which no longer exist.
* More import fixes for the new pipe_control.plotting.write_xy() function.
* Fix for the backend of the relax_disp.plot_disp_curves user function. The lib.plotting.api functions write_xy_data() and write_xy_header() require the format argument.
* Updated the relative stereochemistry auto-analysis to use the relax library plotting API.
* Huge speed up for the assembly of atomic coordinates from a large number of structures. The internal structural object validate_models() method was being called once for each structure when assembling the atomic coordinates. This resulted in the _translate() internal structural object method, which converts all input data to formatted strings, being called hundreds of millions of times. The problem was in lib.structure.internal.coordinates.assemble_atomic_coordinates(), in that the one_letter_codes() method, which calls validate_models(), was called for each molecule encountered. The solution was not to validate models in one_letter_codes().
* Huge speed up of the internal structural object validate_models() method. The string formatting to create pseudo-PDB records and the large number of calls to the _translate() method for atomic information string formatting has been shifted to only be called when atomic information does not match. Instead the structural information is directly compared within a large if-else statement.
* Created the Structure.test_atomic_fluctuations_no_match system test. This demonstrates a failure in the operation of the structure.atomic_fluctuations user function when the supplied atom ID matches no atoms.
* Fix for the Structure.test_atomic_fluctuations_no_match system test. The structure.atomic_fluctuations user function will now raise a RelaxError when no data corresponding to the atom ID can be found, so the test now checks for this.
* Created the unit test infrastructure for the lib.structure.internal.object module.
* Created the Test_object.test_add_atom_sort unit test. This is from the _lib._structure._internal.test_object unit test module. The test will be used to implement the sorting of input data by residue number in the add_atom() internal structural object method. This will mean that added atoms will be placed in residue sequence order, so that output PDB files are correctly ordered.
* Implementation of methods for sorting sequence data in the internal structural object. The information is sorted in the molecule container level using the new MolContainer._sort() private method. This uses the _sort_key() helper method which determines what the new order should be. This is used as the 'key' argument for the Python sort() method. Instead of list shuffling, new lists in the correct order are created. Although not memory efficient, this might be faster than shuffling.
* The loading of structural data now sorts the data if the merge flag is True. The pack_structs() method for sorting the data will now call the new MolContainer._sort() function is the data is being merged. This is to ensure that the final structural data is correctly ordered.
* Fixes for a number of Structure system tests for the sorted structural data changes.
* Modified the structure.read_pdb user function backend to skip water molecules. All residues with the name 'HOH' are now skipped when loading PDB files. This is implemented in the MolContainer.fill_object_from_pdb() method, and a RelaxWarning is printed listing the residue numbers of all skipped waters.
* Modified the Structure.test_read_pdb_1UBQ system test for the new water skipping feature. As the structure.read_pdb user function will now skip waters, the last atom in the structural object will now be the last ubiquitin atom and not the last water atom.
* Modified the Test_object.test_add_atom_sort unit test to check atom connectivities. This is from the _lib._structure._internal.test_object unit test module. The problem is that the MolContainer._sort() method for sorting the structural data currently does not correctly update the bonded data structure.
* Completed the implementation of the sorting of structural data in the internal structural object. The MolContainer._sort() private method now changes the connect atom indices in the bonded data structure to the new sorted indices.
* Created new system tests for implementing new functionality for the structure.mean user function. This includes the Structure.test_mean_models and Structure.test_mean_molecules. The idea is to convert the user function to the new pipes/models/molecules/atom_id design. This will allow molecules with non-identical sequences and atomic compositions to be averaged. The set_mol_name and set_model_num arguments from the structure.read_pdb, structure.read_gaussian, and structure.read_xyz user functions will also be implemented to allow the mean structure to be stored alongside the other molecules.
* Some fixes for the checks in the Structure.test_mean_molecules system test.
* Fix for the structure.mean user function call in the Structure.test_mean_models system test.
* Expanded the checking in all the Structure.test_mean* system tests to cover all atomic information. This includes the Structure.test_mean, Structure.test_mean_models, and Structure.test_mean_molecules system tests. All structural data is now carefully checked to make sure that the structure.mean user function operates correctly.
* Converted the structure.mean user function to the new pipe/model/molecule/atom_id design. This allows the average structure calculation to work on atomic coordinates from different data pipes, different structural models, and different molecules. The user function backend uses the new pipe_control.structure.main.assemble_structural_coordinates() function to assemble the common atom coordinates, molecule names, residue names, residue numbers, atom names and elements. All this information is then used to construct a new molecule container for storing the average structure in the internal structural object. To allow for the averaged structural data to be stored, the internal structural object method add_coordinates() has been created. This is modelled on the PDB, Gaussian, and XYZ format loading methods. The internal structural object mean() method is no longer used, but remains for anyone who might have interest in the future (though as it is untested, bit-rot will be a problem).
* Small correction for the structure.read_pdb user function description.
* Created the Structure.test_read_merge_simultaneous system test. This is to demonstrate a failure in the structure.read_pdb user function when merging multiple molecules from one file into one molecule simultaneously with a single user function call.
* Added some error checking for the monte_carlo.setup user function. A RelaxError is now raised if the number of simulations is less than 3. This prevents Python errors when later calling the monte_carlo.error_analysis user function.
* Test suite fixes for the error checking in the monte_carlo.setup user function. The number of simulations has been increased from either 1 or 2 in all tests to the minimal number of simulations (3).
* Created the Structure.test_bug_23293_missing_hetatm system test. This is to catch bug #23293 (https://gna.org/bugs/?23293), the PDB HETATM loading error whereby the last HETATM record is sometimes not read from the PDB file.
* Small fix for the chain IDs in the Structure.test_bug_23293_missing_hetatm system test.
* Created the Structure.test_multi_model_and_multi_molecule system test. This is used to check the loading and writing of a multi-model and multi-molecule PDB file. The test shows that this functions correctly.
* Modified the Structure.test_multi_model_and_multi_molecule test to check for model consistency. This is just for better test suite coverage of the handling of PDB structural data.
* Created the Structure.test_bug_23294_multi_mol_automerge system test. This is used to catch bug #2329 (https://gna.org/bugs/?23294), the automatic merging of PDB molecules resulting in an IndexError. It reads in the 'in.pdb' PDB file attached to the bug report, now named 'bug_23294_multi_mol_automerge.pdb', to show the IndexError. The test also checks the structure.write_pdb user function to make sure that the output PDB file contains a single merged molecule.
* Added the PDB file to the repository for the Structure.test_bug_23294_multi_mol_automerge system test.
* Fix for the Structure.test_bug_23294_multi_mol_automerge system test. The MASTER PDB record has been added to the data to check for, as this will be produced by the structure.write_pdb user function.
* Improved the RelaxWarning for missing atom numbers in the PDB CONECT records. This is for the structure.read_pdb user function. Now only one warning is given for the entire PDB file listing all of the missing atom numbers rather than one warning per missing atom. This can significantly compact the warnings, removing a lot of repetition.
* Improved the quality of the printouts from the structure.read_pdb user function. This also affects the structure.read_gaussian and structure.read_xyz user functions. The messages about adding new molecules or merging with existing molecules has been significantly improved. The text with the model information is now only printed if the model number is present in the PDB file or has been supplied by the user.
* Fixes for all of the PDB documentation HTML links in the docstrings. The PDB have shifted their documentation from http://www.wwpdb.org/documentation/format33/v3.3.html to http://www.wwpdb.org/documentation/file-format/format33/v3.3.html, stupidly without redirects. This will create dead links in the relax API documentation at http://www.nmr-relax.com/api/3.3/, as well as the older API documentation (http://www.nmr-relax.com/api/2.2/, http://www.nmr-relax.com/api/3.0/, http://www.nmr-relax.com/api/3.1/, http://www.nmr-relax.com/api/3.2/).
* Created the Structure.test_bug_23295_ss_metadata_merge system test. This is to catch bug #23295 (https://gna.org/bugs/?23295), the PDB secondary structure HELIX and SHEET records not updated when merging molecules. This uses the '2BE6_secondary_structure.pdb' structure file and 'test.py' relax script contents as the test, checking the HELIX and SHEET records.
* Added one more check to the Structure.test_bug_23295_ss_metadata_merge system test. The test would pass if no HELIX or SHEET records were to be written to the PDB file.
* Fix for the Structure.test_bug_23295_ss_metadata_merge system test and additional printouts.
* Fix for the Structure.test_pdb_combined_secondary_structure system test. The SHEET PDB record check was incorrect and was checking for the improperly formatted atom name field, which has now been fixed in relax.
* Large speed up of the structure.web_of_motion user function. With the introduction of the _sort() internal structural object method and it being called by the add_atom(), the structure.web_of_motion user function was now painfully slow. As sorting the structural data is unnecessary for the backend of this user function, the add_atom() boolean argument 'sort' has been added to turn the sorting on and off, and the structure.web_of_motion backend now sets this to False.
* Fix for the internal structural object unit test Test_object.test_add_atom_sort. This test of the _lib._structure._internal.test_object unit test module now requires the sort argument set to True when calling the add_atom() method.
* Improvement for a RelaxError message when assembling structural data but no coordinates can be found.
* Created a series of unit tests for implementing a new internal structural object feature. These tests check a new 'inv' argument for the selection() structural object method for allowing all atoms not matching the atom ID string to be selected.
* Implemented the new 'inv' argument for the selection() structural object method. This allows for all atoms not matching the atom ID string to be selected. The unit tests for this argument now all pass, validating the implementation.
* Improvement for the structure.mean user function. This can now be used to store an averaged structure in an empty data pipe. Previously structural data needed to be present in the current data pipe for the user function to work.
* Created a system test to show a limitation of the rdc.copy user function. Currently, it cannot work when spin systems in two data pipes are different. The system test will be used to implement the support.
* Simplification of the new Rdc.test_rdc_copy_different_spins system test. This no longer tests the deletion of interatomic data containers by the spin.delete user function, something which is not implemented.
* Some more fixes for the Rdc.test_rdc_copy_different_spins system test. The residue.delete and not spin.delete user function is required to delete the sequence data.
* Another small fix for the new Rdc.test_rdc_copy_different_spins system test. The rdc.copy user function requires the pipe_to argument to be supplied in this case.
* Expansion of the Rdc.test_rdc_copy_different_spins system test. The interatomic data containers are now defined via the interatom.define user function, which requires the spin.element user function to set up the element information. A printout has also been added to demonstrate a failure in the pipe_control.interatomic.interatomic_loop() function in handling the correct data pipe.
* Some more modifications for the Rdc.test_rdc_copy_different_spins system test. One of the interatomic data containers does not have RDC data, as it is not present in the original data pipe, hence this is checked for. And the printouts have more formatting.
* Expanded the functionality of the rdc.copy user function. The user function will now operate on two data pipes with different spin sequences. If the interatomic data container is missing from the target data pipe, a warning is given. And if the interatomic data container is not present in the source data pipe, nothing will be copied.
* Modified the rdc.copy user function to printout all copied RDC values and errors.
* Created the Rdc.test_rdc_copy_back_calc system test. This will be used to implement the back_calc Boolean argument for the rdc.copy user function to allow not only measured, but also back-calculated RDC values to be copied.
* Modified the rdc.copy printout of RDCs to occur for each alignment ID.
* Implemented the back_calc argument for the rdc.copy user function. This allows the back-calculated RDCs to be additionally copied together with the real value and error.
* Small formatting change for the rdc.copy user function printouts.
* Created the Pcs.test_pcs_copy_different_spins system test. This will be used to show a limitation of the pcs.copy user function in that it cannot copy data between two data pipes with different molecule, residue, and spin sequences.
* Added a printout of the alignment ID for the pcs.copy user function. This is to match the rdc.copy user function.
* Created the Pcs.test_pcs_copy_back_calc system test. This will be used to implement the back_calc Boolean argument for the pcs.copy user function to allow not only measured, but also back-calculated PCS values to be copied. It matches the equivalent Rdc.test_rdc_copy_back_calc system test.
* Implemented the back_calc argument for the pcs.copy user function. This allows the back-calculated PCSs to be additionally copied together with the real value and error. The implementation simply copies that of the rdc.copy user function.
* Added full per-alignment data printouts to the pcs.copy user function to match rdc.copy. The feedback is important to know what was actually copied.
* Modified the pcs.copy user function to handle different spin sequence between data pipes.
* Fixes for the Pcs.test_pcs_copy_different_spins and Pcs.test_pcs_copy_back_calc system tests.
* Fix for the pcs.copy user function for a recently introduced problem. The data pipe for the spin_loop() function must be supplied.
* The pcs.copy user function now skips deselected spins.
* Modified the N_state_model.test_data_copying system test to skip deselected spins.
* Added more checks to the three Pcs.test_pcs_copy* system tests.
* Added more checks to the three Rdc.test_rdc_copy* system tests.
* Created the Rdc.test_calc_q_factors_no_tensor system test. This is to demonstrate a failure in the rdc.calc_q_factors user function when no alignment tensor is present. In addition, the test is also triggering an earlier problem of spin isotope information being missing. However the isotope is not required if the tensor is absent.
* The Rdc.test_rdc_copy_* system tests now check for the 'rdc_data_types' data structure. This is in the Rdc.test_rdc_copy_different_spins and Rdc.test_rdc_copy_back_calc system tests and shows that the rdc.copy user function fails to duplicate this information.
* The Rdc.test_rdc_copy_* system tests now check for the 'absolute_rdc' data structure. This is in the Rdc.test_rdc_copy_different_spins and Rdc.test_rdc_copy_back_calc system tests and shows that the rdc.copy user function fails to duplicate this information as well.
* Expanded the rdc.copy user function to copy the RDC data type and absolute RDC flag information.
* Created the Rdc.test_corr_plot system test to check the rdc.corr_plot user function. This shows that this poorly tested function works correctly.
* Created the Pcs.test_corr_plot system test to check the pcs.corr_plot user function. This user function is poorly tested, and this test triggers a series of bugs.
* Added the 'title' and 'subtitle' arguments to the pcs.corr_plot user function. This problem was detected by the new Pcs.test_corr_plot system test. The pcs.corr_plot user function now matches the rdc.corr_plot user function in terms of arguments.
* Completed the Pcs.test_corr_plot system test. The file contents are now known and have been carefully checking in Grace.
* Clarification of the RDC and PCS Q factors. This affects the rdc.calc_q_factors and pcs.calc_q_factors user functions, as well as all other operations involving the calculation of Q factors. The printouts have been modified to clarify if the normalisation is via the tensor size (2Da^2(4 + 3R)/5) or via the sum of data squared, and the separation of the two is now clearer. This allows for better RDC vs. PCS comparisons. In addition, the data pipe variable names have been updated to reflect the normalisation, so it is instantly known when looking at the XML contents of results or save files which was used. The backwards compatibility hooks have been modified to support the data pipe variable name changes.
* The align_tensor.copy user function 'tensor_from' argument can now be None. This is to enable the copying of all alignment tensors from one data pipe to another.
* Created the Align_tensor.test_copy_pipes system test. This is to show a problem in the align_tensor.copy user function when copying all tensors between data pipes.
* Modified the pipe_control.align_tensor.align_data_exists() function to handle no tensor IDs. If no tensor ID is supplied, this will then return True if any alignment data exists.
* Improvement for the align_tensor.copy user function. The user function has been modified to allow all alignment tensors to be copied between two data pipes. This allows the Align_tensor.test_copy_pipes system test to pass.
* Fixes for the align_tensor.copy user function argument unit tests. The tensor_from and tensor_to arguments can now be None.
* Created the Align_tensor.test_copy_pipes_sims system test. This demonstrates a failure of the align_tensor.copy user function when Monte Carlo simulated tensors are present.
* Deleted the data_store.align_tensor.AlignTensorSimList.append() method. This replacement list method was proving fatal when copy.deepcopy() is called on the alignment tensor object. The change allows the Align_tensor.test_copy_pipes_sims system test to pass.
* Huge speed up for loading results and state files with Monte Carlo simulation alignment tensors. The reading of the alignment tensor component of XML formatted results and state files has been modified. Previously the data_store.align_tensor.AlignTensorData._update_object() method for updating the alignment tensor object (for values, errors, simulations) was being called once for each Monte Carlo simulation. Now is it called only once for all simulations. In one test, the reading of the save file with 500 simulations dropped from 253.7 to 10.0 seconds.
* Added an extra check for the assembly of RDC data. This is in the pipe_control.rdc.return_rdc_data() function and the check is for any unit vectors set to None, which is a fatal condition.
* Improved the RelaxError message from the RDC assembly function when unit vectors are None.
* Added a new warning to the interatom.unit_vectors user function if data is missing. This is to aid in detecting problems earlier before unit vectors of None are encountered by other parts of relax.
* Modified the rdc.corr_plot user function to skip deselected interatomic data containers. This would normally happen as no back-calculated data is normally present. However, if data has been copied from elsewhere, this may not always be the case.
* Created the Sequence.test_bug_23372_read_csv system test. This is to catch bug #23372 (https://gna.org/bugs/?23372), the sequence.read failure with CSV files. It uses a truncated version of the CSV data file attached to sr #3219 (https://gna.org/support/?3219).
* Converted the lib.sequence.validate_sequence() to the checking function design. This is the checking function design documented at http://wiki.nmr-relax.com/Relax_source_design#The_check_.2A.28.29_functions. The validate_sequence() function has been renamed to check_sequence_func() and the checking object is called check_sequence. It removes the string processing hack to convert RelaxErrors to RelaxWarnings in the lib.sequence.read_spin_data() function, avoiding strange messages such at "RelaxWarning: ror: The sequence data in the line..." as seen in the Sequence.test_bug_23372_read_csv system test.
* Small typo fix for the Sequence.test_bug_23372_read_csv system test.
* Added the raise_flag argument to the lib.sequence.read_spin_data() function. This is to allow the missing data RelaxError to be deactivated.
* Modified the spectrum.read_intensities user function backend to be more robust. This affects the generic formatted peak lists, via the lib.spectrum.peak_list.intensity_generic() function. The peak list reading will now continue reading the file after corrupted lines have been encountered.
* Python 3 improvement for the rdc.corr_plot and pcs.corr_plot user functions. The world view is now set in floating point numbers. In Python 2, the math.ceil() and math.floor() functions return floats, whereas in Python 3 these functions return integers. The behaviour is now consistent in both Python versions, fixing a few system tests.
* Modified the internal formatting of the data section of the Grace 2D graph files. This affects the lib.plotting.grace.write_xy_data() function. The formatting is now more consistent, with the X value now set to a fixed number of decimal places, and hence will no longer change between Python 2 and 3. The data is now all right justified as well, for easier reading. All affected system tests have been updated for the new format.
* Epydoc documentation fix for the lib.structure.pdb_write_handle_atom_name() function.
<section end=changes/>
