Trusted, Bureaucrats
4,228
edits
Changes
User function HTML manual links.
<section begin=description/>
This is a minor feature and bugfix release. It includes the addition of the new [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] which can use the 'Central Star' multiple sequence alignment algorithm or align based on residue numbers, saving the results in the relax data store. The assembly of structural coordinates used by the [http://www.nmr-relax.com/manual/structure_align.html structure.align], [http://www.nmr-relax.com/manual/structure_atomic_fluctuations.html structure.atomic_fluctuations], [http://www.nmr-relax.com/manual/structure_com.html structure.com], [http://www.nmr-relax.com/manual/structure_displacement.html structure.displacement], [http://www.nmr-relax.com/manual/structure_find_pivot.html structure.find_pivot], [http://www.nmr-relax.com/manual/structure_mean.html structure.mean], [http://www.nmr-relax.com/manual/structure_rmsd.html structure.rmsd], [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] and [http://www.nmr-relax.com/manual/structure_web_of_motion.html structure.web_of_motion ] user functions has been redesigned around this new user function. It will use any pre-existing sequence alignments for the molecules of interest, but default to a residue number based alignment if the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] has not been used. This version also fixes a system test failure on Mac OS X and Iinf parameter text files and Grace graphs are produced by the relaxation curve-fitting auto-analysis for the inversion recovery and saturation recovery experiment types.
<section end=description/>
* Sequence alignments are now saved in the relax data store.
* Important formatting improvement for the description in the GUI user function windows, removing excess empty lines after lists.
* Creation of the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function]. The MSA algorithm can be set to either 'Central Star' or 'residue number', the pairwise sequence alignment algorithm to 'NW70' for the Needleman-Wunsch algorithm, and the substitution matrix to one of 'BLOSUM62', 'PAM250', or 'NUC 4.4'.
* More advanced support for different numpy number types in the lib.xml relax library module. This allows numpy int16, int32, float32, and float64 objects to be saved in the relax data store and retrieved from relax XML save and results files.
* Merger of structure.align into the [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose user function].* The assembly of common atomic coordinates by the structure user functions now takes sequence alignments into account. The logic is to first use a sequence alignment from the relax data store if present, use no sequence alignment if coordinates only come from structural models, or fall back to a residue number based alignment. This affects the [http://www.nmr-relax.com/manual/structure_align.html structure.align], [http://www.nmr-relax.com/manual/structure_atomic_fluctuations.html structure.atomic_fluctuations], [http://www.nmr-relax.com/manual/structure_com.html structure.com], [http://www.nmr-relax.com/manual/structure_displacement.html structure.displacement], [http://www.nmr-relax.com/manual/structure_find_pivot.html structure.find_pivot], [http://www.nmr-relax.com/manual/structure_mean.html structure.mean], [http://www.nmr-relax.com/manual/structure_rmsd.html structure.rmsd], [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] and [http://www.nmr-relax.com/manual/structure_web_of_motion.html structure.web_of_motion ] user functions.
* Large improvements in the memory management for all parts of the GUI.
<section end=features/>
<section begin=changes/>
* Spelling fixes for the CHANGES document.
* Created the Structure.test_align_molecules2 system test. This is to demonstrate a failure condition in the [http://www.nmr-relax.com/manual/structure_align.html structure.align user function].
* Large simplification of the atomic coordinate assembly code in the internal structural object. This is in the lib.structure.internal.coordinates.assemble_coord_array() function. The logic of the function has recently changed due to the introduction of the pairwise sequence alignments. This caused a lot of code to now be redundant, and also incorrect in certain cases. This simplification fixes the problem caught by the Structure.test_align_molecules2 system test.
* Fix for the Structure.test_displacement system test - the molecule IDs needed updating.
* Created the Structure.test_align_molecules_end_truncation system test. This is to demonstrate a failure of the common residue detection algorithm using multiple pairwise alignments in the backend of the [http://www.nmr-relax.com/manual/structure_align.html structure.align ] and other multiple structure based user functions.
* Created empty unit test infrastructure for testing the lib.structure.internal.coordinates module.
* Created the Test_coordinates.test_common_residues unit test. This is from the _lib._structure._internal.test_coordinates unit test module. The test shows that the lib.structure.internal.coordinates.common_residues() function is working correctly. However the printout, which is not caught by the test, is incorrect.
* Fixes for the unit tests of the _lib._sequence_alignment.test_align_protein module. The Test_align_protein.test_align_pairwise_PAM250 unit test was accidentally duplicated due to a copy and paste error. And the lib.sequence_alignment.align_protein.align_pairwise() function now also returns the alignment score.
* Fixes for the unit tests of the _lib._sequence_alignment.test_needleman_wunsch module. The lib.sequence_alignment.needleman_wunsch.needleman_wunsch_align() function now returns the alignment score.
* The assemble_coord_array() function is now using the central star multiple sequence alignment. This is the function from the lib.structure.internal.coordinates module used to assemble common atomic coordinate information, used by the [http://www.nmr-relax.com/manual/structure_align.html structure.align], [http://www.nmr-relax.com/manual/structure_atomic_fluctuations.html structure.atomic_fluctuations], [http://www.nmr-relax.com/manual/structure_com.html structure.com], [http://www.nmr-relax.com/manual/structure.displacement.html structure.displacement], [http://www.nmr-relax.com/manual/structure_find_pivot.html structure.find_pivot], [http://www.nmr-relax.com/manual/structure_mean.html structure.mean], [http://www.nmr-relax.com/manual/structure_rmsd.html structure.rmsd], [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] and [http://www.nmr-relax.com/manual/structure_web_of_motion.html structure.web_of_motion ] user functions. The non-functional lib.structure.internal.coordinates.common_residues() function has been removed as the lib.sequence_alignment.msa.central_star() function performs this functionality correctly.
* Deleted the Test_coordinates.test_common_residues unit test. This is from the _lib._structure._internal.test_coordinates unit test module. The lib.structure.internal.coordinates.common_residues() function no longer exists.
* Alphabetical ordering of all Structure system tests.
* Fixes for the Structure.test_align_molecules_end_truncation system test. This system test had only been partly converted from the old Structure.test_align_molecules2 system test it had been copied from.
* Created the Internal_selection.count_atoms() internal structural object selection method. This counts the number of atoms in the current selection.
* Added final printouts to the [http://www.nmr-relax.com/manual/structure_rotate.html structure.rotate ] and [http://www.nmr-relax.com/manual/structure_translate.html structure.translate ] user function backends. This is to give feedback to the user as to how many atoms were translated or rotated, to aid in solving problems with the structure user functions. These backend functions are also used by the [http://www.nmr-relax.com/manual/structure_align.html structure.align ] and [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] user functions.
* Corrections for the Structure.test_align_CaM_BLOSUM62 system test. The CaM N and C domains can not be aligned together in a global MSA as they would align very well to themselves, causing the atomic coordinate assembly function to fail.
* Improvement for the lib.sequence_alignment.msa.central_star() function. The strings and gap matrix returned by the function have been reordered to match the input sequences.
* Modified the Structure.test_align_molecules_end_truncation system test. The calmodulin bound calciums are now deleted prior to the [http://www.nmr-relax.com/manual/structure_align.html structure.align user function ] call. This prevents these being labelled as '*' residues and aligning with real amino acids via the central star multiple sequence alignment (MSA) algorithm.
* Large speed up of the mol-res-spin selection object. The Selection.contains_mol(), Selection.contains_res() and Selection.contains_spin() methods of the lib.selection module have been redesigned for speed. Instead of setting a number of flags and performing bit operations at the end of the method to return the correct Boolean value, each of the multiple checks now simply returns a Boolean value, avoiding all subsequent checks. The check list order has also been rearranged so that the least expensive checks are to the top and the most time intensive checks are last.
* Created the new relax data store object for saving sequence alignments. This is in the new data_store.seq_align module via the Seqence_alignments object, subclassed from RelaxListType, for holding all alignments and the Alignment Element object, subclassed from Element, for holding each individual alignment. The objects are currently unused.
* Shifted the data_store.seq_align.Alignment.generate_id() method into the relax library. It has been converted into the lib.structure.internal.coordinates.generate_id() function to allow for greater reuse.
* Created the Sequence.test_align_molecules system test. This will be used to implement the sequence.align user function which will be used for performing sequence alignments on structural data within the relax data store and storing the data in the data pipe independent sequence_alignments data store object (which will be an instance of data_store.seq_align.Sequence_alignments). The system test also checks the XML saving and loading of the ds.sequence_alignments data structure.
* Renamed the Sequence.test_align_molecules system test to Structure.test_sequence_alignment_molecules. As the sequence alignment is dependent on the structural data in the relax data store, the user function for sequence alignment would be better named as [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment]. The [http://www.nmr-relax.com/manual/sequence_align.html sequence.align user function ] is not appropriate as all other sequence user functions relate to the molecule, residue, and spin data structure of each data pipe rather than to the structural data.* Modified the Structure.test_sequence_alignment_molecules system test. Changed and expanded the arguments to the yet to be implemented [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function].
* Important formatting improvement for the description in the GUI user function windows. Previously lists, item lists, and prompt items were spaced with one empty line at the top and two at the bottom. The two empty lines at the bottom was an accident caused by how the list text elements were built up. Now the final newline character is stripped so that the top and bottom of the lists only consist of one empty line. The change can give a lot more room in the GUI window.
* Created the frontend for the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function]. This is based on the [http://www.nmr-relax.com/manual/structure_align.html structure.align user function ] with the 3D superimposition arguments removed and new arguments added for selecting the MSA algorithm and the pairwise alignment algorithm (despite only NW70 being currently implemented).
* Modified the assemble_coordinates() function of the pipe_control.structure.main module. The function has been renamed to assemble_structural_objects(). The call to the lib.structure.internal.coordinates.assemble_coord_array() function has also been shifted out of assemble_structural_objects() to simplify the logic and decrease the amount of arguments passed around.
* Spun out the atomic assembly code of the assemble_coord_array() function. The code from the lib.structure.internal.coordinates.assemble_coord_array() function has been shifted to the new assemble_atomic_coordinates(). This is to simplify assemble_coord_array() as well as to isolate the individual functionality for reuse.
* Implemented the backend of the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function]. This checks some of the input parameters, assembles the structural objects then the atomic coordinate information, performs the multiple sequence alignment, and then stores the results.
* Fixes for the sequence alignment objects for the relax data store. The Sequence_alignments(RelaxListType) and Alignment(Element) classes were not being set up correctly. The container names and descriptions were missing.
* The data store ds.sequence_alignment object is now being treated as special and is blacklisted. The object is now explicitly recreated in the data store from_xml() method.
* Implemented the data store Sequence_alignments.from_xml() method. This method is required for being able to read RelaxListType objects from the XML file.
* Modified the data returned by lib.structure.internal.coordinates.assemble_atomic_coordinates(). The function will now assemble simple lists of object IDs, model numbers and molecule names with each list element corresponding to a different structural model. This will be very useful for converting from the complicated pipes, models, and molecules user function arguments into relax data store independent flat lists.
* Updates for the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function]. This is for the changes to the lib.structure.internal.coordinates.assemble_atomic_coordinates() function return values. The new object ID, model, and molecule flat lists are used directly for storing the alignment results in the relax data store.* Updates for the Structure.test_sequence_alignment_molecules system test. This is required due to the changes in the backend of the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function].* Merger of the [http://www.nmr-relax.com/manual/structure_align.html structure.align ] and [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] user functions. The final user function is called [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose]. As the sequence alignment component of the [http://www.nmr-relax.com/manual/structure_align.html structure.align user function ] has been shifted into the new [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] and the information is now stored in ds.sequence_alignments relax data store object, the functionality of structure.align and structure.superimpose are now essentially the same. The sequence alignment arguments and documentation has also been eliminated. And the documentation has been updated to say that sequence alignments from structure.sequence_alignment will be used for superimposing the structures.* Updated the Structure system tests for the [http://www.nmr-relax.com/manual/structure_align.html structure.align ] and [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] user function merger.* Fix for the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function]. The alignment data should be stored in ds.sequence_alignments rather than ds.sequence_alignment.
* Sequence alignments can now be retrieved without supplying the algorithm settings. This is in the data_store.seq_align.Sequence_alignments.find_alignment() method. The change allows for the retrieval of pre-existing sequence alignments at any stage.
* Added a function for assemble the common atomic coordinates taking sequence alignments into account. This is the new pipe_control.structure.main.assemble_structural_coordinates() function. It takes the sequence alignment logic out of the lib.structure.internal.coordinates.assemble_coord_array() function so that sequence alignments in the relax data store can be used. The logic has also been redefined as: 1, use a sequence alignment from the relax data store if present; 2, use no sequence alignment if coordinates only come from structural models; 3, fall back to a residue number based alignment. The residue number based alignment is yet to be implemented. As a consequence, the lib.structure.internal.coordinates.assemble_coord_array() function has been greatly simplified. It no longer handles sequence alignments, but instead expects the residue skipping data structure, built from the alignment, as an argument. The seq_info_flag argument has also been eliminated in this function as well as the pipe_control.structure.main module.
* Updated the [http://www.nmr-relax.com/manual/structure_displacement.html structure.displacement user function ] for the changed atomic assembly logic. This now uses the assemble_structural_coordinates() function of the pipe_control.structure.main module to obtain the common coordinates based on pre-existing sequence alignments, no-alignment, or the default of a residue number based alignment.* Updated the [http://www.nmr-relax.com/manual/structure_find_pivot.html structure.find_pivot user function ] for the changed atomic assembly logic. This now uses the assemble_structural_coordinates() function of the pipe_control.structure.main module to obtain the common coordinates based on pre-existing sequence alignments, no-alignment, or the default of a residue number based alignment.* Updated the [http://www.nmr-relax.com/manual/structure_atomic_fluctuations.html structure.atomic_fluctuations user function ] for the changed atomic assembly logic. This now uses the assemble_structural_coordinates() function of the pipe_control.structure.main module to obtain the common coordinates based on pre-existing sequence alignments, no-alignment, or the default of a residue number based alignment.* Updated the [http://www.nmr-relax.com/manual/structure_rmsd.html structure.rmsd user function ] for the changed atomic assembly logic. This now uses the assemble_structural_coordinates() function of the pipe_control.structure.main module to obtain the common coordinates based on pre-existing sequence alignments, no-alignment, or the default of a residue number based alignment.* Updated the [http://www.nmr-relax.com/manual/structure_web_of_motion.html structure.web_of_motion user function ] for the changed atomic assembly logic. This now uses the assemble_structural_coordinates() function of the pipe_control.structure.main module to obtain the common coordinates based on pre-existing sequence alignments, no-alignment, or the default of a residue number based alignment.* Fix for the [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose user function ] if no data pipes are supplied. This reintroduces the pipes list construction.
* Fix for the new pipe_control.structure.main.assemble_structural_coordinates() function. The atom_id argument is now passed into the assemble_atomic_coordinates() function of the lib.structure.internal.coordinates module so that atom subsets are once again recognised.
* Another fix for the new pipe_control.structure.main.assemble_structural_coordinates() function. The logic for determining if only models will be superimposed was incorrect.
* Renamed the Structure.test_sequence_alignment_molecules system test. The new name is Structure.test_sequence_alignment_central_star_nw70_blosum62, to better reflect what the test is doing.
* Modified the Structure.test_sequence_alignment_central_star_nw70_blosum62 system test. Some residues are now deleted so that the sequences are not identical.
* Created the Structure.test_sequence_alignment_residue_number system test. This will be used to test the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] together with the 'residue number' MSA algorithm. This is simply a copy of the Structure.test_sequence_alignment_central_star_nw70_blosum62 system test with a few small changes.
* Corrections and simplifications for the Structure.test_sequence_alignment_residue_number system test.
* Modified the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] arguments. The pairwise_algorithm and matrix arguments can no be None, and they default to None.* Updated the Structure.test_align_CaM_BLOSUM62 system test script. The MSA algorithm and pairwise alignment algorithms are now specified in the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] calls.* Creation of the lib.sequence_alignment.msa.msa_general() function. This consists of code from the [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] backend function pipe_control.structure.main.sequence_alignment() for selecting between the different sequence alignment methods.* The [http://www.nmr-relax.com/manual/structure_sequence_alignment.html structure.sequence_alignment user function ] now sets some arguments to None before storage. This is for all arguments not used in the sequence alignment. For example the residue number based alignment does not use the gap penalties, pairwise alignment algorithm or the substitution matrices.
* Fix for the lib.sequence_alignment.msa.msa_residue_skipping() function. The sequences argument for passing in the one letter codes has been removed. The per molecule loop should be over the alignment strings rather than one letter codes, otherwise the loop will be too short.
* Fix for the internal structural object atomic coordinate assembly function. This is the pipe_control.structure.main.assemble_structural_coordinates() function. The case of no sequence alignment being required as only models are being handled is now functional. The strings and gaps data structures passed into the lib.sequence_alignment.msa.msa_residue_skipping() function for generating the residue skipping data structure are now set to the one letter codes and an empty structure of zeros respectively.
<section begin=bugfixes/>
* Bug fix for the [http://www.nmr-relax.com/manual/structure_align.html structure.align user function]. The addition of the molecule name to the displacement ID is now correctly performed.
* Fix for the new Internal_selection.count_atoms() internal structural object selection method. The method was previously returning the total number of molecules, not the total number of atoms in the selection.
* Printout fix for the backend of the [http://www.nmr-relax.com/manual/structure_translate.html structure.translate ] and [http://www.nmr-relax.com/manual/structure_rotate.html structure.rotate ] user functions. Model numbers of zero were not correctly identified. This also affects the [http://www.nmr-relax.com/manual/structure_align.html structure.align ] and [http://www.nmr-relax.com/manual/structure_superimpose.html structure.superimpose ] user functions which uses this backend code.
* Another fix for the Internal_selection.count_atoms() internal structural object selection method.
* Small fix for the lib.structure.internal.coordinates.assemble_coord_array() function. The termination condition for determining the residues in common between all structures was incorrect.