Difference between revisions of "Tutorial for model free SBiNLab"

From relax wiki
Jump to navigation Jump to search
 
(180 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Background ==
+
= Background =
 
This is a tutorial for Lau and Kaare in SBiNLab, and hopefully others.
 
This is a tutorial for Lau and Kaare in SBiNLab, and hopefully others.
  
 
To get inspiration of example scripts files and '''see''' how the protocol is performed, have a look here:
 
To get inspiration of example scripts files and '''see''' how the protocol is performed, have a look here:
* nmr-relax-code/test_suite/system_tests/scripts/model_free/dauvergne_protocol.py
+
* [https://github.com/nmr-relax/relax/blob/master/auto_analyses/dauvergne_protocol.py nmr-relax-code/auto_analyses/dauvergne_protocol.py]
* nmr-relax-code/auto_analyses/dauvergne_protocol.py
 
  
== Scripts ==
+
For references, see [http://www.nmr-relax.com/refs.shtml relax references]:
To get the protocol to work, we need to
+
* [[Model-free_analysis_single_field#Protocol|See this description of the protocol by Edward]] and image [http://www.nmr-relax.com/manual/The_diffusion_seeded_paradigm.html The diffusion seeded paradigm]
 +
* [http://www.nmr-relax.com/manual/Model_free_analysis.html Link to the manual]
 +
* [http://www.nmr-relax.com/manual/The_model_free_models.html Summary of model-free models]
 +
* [http://www.nmr-relax.com/manual/molmol_macro_apply.html#SECTION081284600000000000000 Summary of parameter meaning and value to pymol visualization]
 +
* d'Auvergne, E. J. and Gooley, P. R. (2008). [http://dx.doi.org/10.1007/s10858-007-9214-2 Optimisation of NMR dynamic models I. Minimisation algorithms and their performance within the model-free and Brownian rotational diffusion spaces. J. Biomol. NMR, 40(2), 107-119.]
 +
* d'Auvergne, E. J. and Gooley, P. R. (2008). [http://dx.doi.org/10.1007/s10858-007-9213-3 Optimisation of NMR dynamic models II. A new methodology for the dual optimisation of the model-free parameters and the Brownian rotational diffusion tensor. J. Biomol. NMR, 40(2), 121-133.]
  
* Load a PDB structure
+
= Script inspiration =
* Assign the "data structure" in relax through spin-assignments
+
== model-free : Script inspiration for setup and analysis ==
* Assign necessary "information" as isotope information to each spin-assignment
+
The distribution of relax includes a folder '''sample_scripts/model_free''' which contain
* Read "R1, R2 and NOE" for different magnet field strengths
+
a folder with scripts for analysis.
* Calculate some properties
 
* Check the data
 
* Run the protocol
 
  
To work most efficiently, it is important to perform each step 1 by 1,
+
It can be seen here: https://github.com/nmr-relax/relax/tree/master/sample_scripts/model_free
and closely inspect the log for any errors.
 
  
For similar tutorial, have a look at: [[Tutorial_for_model-free_analysis_sam_mahdi|Tutorial for model-free analysis sam mahdi]]
+
Here is the current list
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/back_calculate.py back_calculate.py]. Back-calculate and save relaxation data starting from a saved model-free results file.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/bmrb_deposition.py bmrb_deposition.py] Script for creating a NMR-STAR 3.1 formatted file for BMRB deposition of model-free results.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/cv.py cv.py] Script for model-free analysis using cross-validation model selection.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/dasha.py dasha.py] Script for model-free analysis using the program Dasha.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/dauvergne_protocol.py dauvergne_protocol.py] Script for black-box model-free analysis.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/diff_min.py diff_min.py] Demonstration script for diffusion tensor optimisation in a model-free analysis.]
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/final_data_extraction.py final_data_extraction.py] Extract Data to Table
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/generate_ri.py generate_ri.py] Script for back-calculating the relaxation data.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/grace_S2_vs_te.py grace_S2_vs_te.py] Script for creating a grace plot of the simulated order parameters vs. simulated correlation times.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/ grace_ri_data_correlation.py] Script for creating correlations plots of experimental verses back calculated relaxation data.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/map.py map.py] Script for mapping the model-free space for OpenDX visualisation.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/mf_multimodel.py mf_multimodel.py] This script performs a model-free analysis for the models 'm0' to 'm9' (or 'tm0' to 'tm9').
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/modsel.py modsel.py] Script for model-free model selection.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/molmol_plot.py molmol_plot.py] Script for generating Molmol macros for highlighting model-free motions
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/palmer.py palmer.py] Script for model-free analysis using Art Palmer's program 'Modelfree4'. Download from http://comdnmr.nysbc.org/comd-nmr-dissem/comd-nmr-software
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/remap.py remap.py] Script for mapping the model-free space.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/single_model.py single_model.py] This script performs a model-free analysis for the single model 'm4'.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/table_csv.py table_csv.py] Script for converting the model-free results into a CSV table.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/model_free/table_latex.py table_latex.py] Script for converting the model-free results into a LaTeX table.
  
=== 01 - Test load of PDB ===
+
== Other script inspiration for checking ==
First we just want to test to read the PDB file.
+
The distribution of relax includes a folder '''sample_scripts/''' which contain a folder with scripts for analysis.
  
'''01_read_pdb.py'''
+
It can be seen here: https://github.com/nmr-relax/relax/tree/master/sample_scripts
<source lang="python">
 
# Python module imports.
 
from time import asctime, localtime
 
import os
 
  
# relax module imports.
+
'''R1 / R2 Calculation'''
from auto_analyses.dauvergne_protocol import dAuvergne_protocol
+
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/relax_fit.py relax_fit.py] Script for relaxation curve fitting.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/relax_curve_diff.py relax_curve_diff.py] Script for creating a Grace plot of peak intensity differences.
 +
The resultant plot is useful for finding bad points or bad spectra when fitting exponential curves determine the R1 and R2 relaxation rates.  If the averages deviate systematically from zero, bias in the spectra or fitting will be clearly revealed. To use this script, R1 or R2 exponential curve fitting must have previously have been carried out the program state saved to the file 'rx.save' (either with or without the .gz or .bz2 ).  The file name of the saved state can be changed at the top of this script.
  
# Set up the data pipe.
+
'''NOE calculation'''
#######################
+
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/noe.py noe.py] Script for calculating NOEs.
  
# The following sequence of user function calls can be changed as needed.
+
'''Test data'''
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/jw_mapping.py jw_mapping.py] Script for reduced spectral density mapping.
 +
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/consistency_tests.py consistency_tests.py] Script for consistency testing.
 +
Severe artifacts can be introduced if model-free analysis is performed from inconsistent multiple magnetic field datasets. The use of simple tests as validation tools for the consistency assessment can help avoid such problems in order to extract more reliable information from spin relaxation experiments. In particular, these tests are useful for detecting inconsistencies arising from R2 data. Since such inconsistencies can yield artifactual Rex parameters within model-free analysis, these tests should be use routinely prior to any analysis such as model-free calculations.
 +
This script will allow one to calculate values for the three consistency tests J(0), F_eta and F_R2. Once this is done, qualitative analysis can be performed by comparing values obtained at different magnetic fields. Correlation plots and histograms are useful tools for such comparison, such as presented in Morin & Gagne (2009a) J. Biomol. NMR, 45: 361-372.
  
# Create the data pipe.
+
'''Other representations'''
bundle_name = "mf (%s)" % asctime(localtime())
+
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/angles.py angles.py] Script for calculating the protein NH bond vector angles with respect to the diffusion tensor.
name = "origin"
+
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/xh_vector_dist.py xh_vector_dist.py] Script for creating a PDB representation of the distribution of XH bond vectors.
pipe.create(name, 'mf', bundle=bundle_name)
+
* [https://github.com/nmr-relax/relax/blob/master/sample_scripts/diff_tensor_pdb.py diff_tensor_pdb.py] Script for creating a PDB representation of the Brownian rotational diffusion tensor.
  
# Load the PDB file.
+
= Scripts - Part 2 =
structure.read_pdb('energy_1.pdb', set_mol_name='ArcCALD', read_model=1)
+
We now try to setup things a little more efficient.
  
# Set up the 15N and 1H spins (both backbone and Trp indole sidechains).
+
Relax is able to read previous results file, so let us divide the task up into:
structure.load_spins('@N', ave_pos=True)
 
structure.load_spins('@NE1', ave_pos=True)
 
structure.load_spins('@H', ave_pos=True)
 
structure.load_spins('@HE1', ave_pos=True)
 
  
# Assign isotopes
+
* 1: Load the data and save as state file. Inspect in GUI before running.
spin.isotope('15N', spin_id='@N*')
+
* 2: Run the Model 1: local_tm.
spin.isotope('1H', spin_id='@H*')
+
* 3: Here make 4 scripts. Each of them only depends on Model 1:
 +
** Model 2: sphere
 +
** Model 3: prolate
 +
** Model 4: oblate
 +
** Model 5: ellipsoid
 +
* 4: Make an intermediate 'final' model script. This will automatically detect files from above.
 +
 
 +
== Prepare data ==
 +
We make a new folder and try.
 +
 
 +
{| class="mw-collapsible mw-collapsed wikitable"
 +
! See commands
 +
|-
 +
|
 +
<source lang="bash">
 +
mkdir 20171010_model_free_2_HADDOCK
 +
cp 20171010_model_free/*.dat 20171010_model_free_2_HADDOCK
 +
cp 20171010_model_free/*.pdb 20171010_model_free_2_HADDOCK
 +
 
 +
# Get scripts
 +
cd 20171010_model_free_2_HADDOCK
 +
git init
 +
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
 +
git fetch
 +
git checkout -t origin/master
 
</source>
 
</source>
 +
|}
  
Run with
+
And a new one, changing the NOE error
 +
{| class="mw-collapsible mw-collapsed wikitable"
 +
! See commands
 +
|-
 +
|
 +
<source lang="bash">
 +
mkdir 20171010_model_free_3_HADDOCK
 +
cp 20171010_model_free/*.dat 20171010_model_free_3_HADDOCK
 +
cp 20171010_model_free/*.pdb 20171010_model_free_3_HADDOCK
 +
 
 +
# Get scripts
 +
cd 20171010_model_free_3_HADDOCK
 +
git init
 +
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
 +
git fetch
 +
git checkout -t origin/master
 +
 
 +
# Change NOE error
 +
sed -i 's/0.1*$/0.05/' NOE_600MHz_new.dat
 +
sed -i 's/0.1*$/0.05/' NOE_750MHz.dat
 +
</source>
 +
|}
 +
 
 +
And a new one, changing the NOE error, and deselecting N-terminal.<br>
 +
Consistency test, found that this stretch contained outliers.
 +
{| class="mw-collapsible mw-collapsed wikitable"
 +
! See commands
 +
|-
 +
|
 +
<source lang="bash">
 +
mkdir 20171010_model_free_4_HADDOCK
 +
cp 20171010_model_free/*.dat 20171010_model_free_4_HADDOCK
 +
cp 20171010_model_free/*.pdb 20171010_model_free_4_HADDOCK
 +
 
 +
# Get scripts
 +
cd 20171010_model_free_4_HADDOCK
 +
git init
 +
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
 +
git fetch
 +
git checkout -t origin/master
 +
 
 +
# Change NOE error
 +
sed -i 's/0.1*$/0.05/' NOE_600MHz_new.dat
 +
sed -i 's/0.1*$/0.05/' NOE_750MHz.dat
 +
 
 +
# Make deselection
 +
echo "#" > deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t151" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t152" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t153" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t154" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t155" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t156" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt
 +
</source>
 +
|}
 +
 
 +
And a new one, changing the NOE error, and deselecting spins found from consistency test.<br>
 +
{| class="mw-collapsible mw-collapsed wikitable"
 +
! See commands
 +
|-
 +
|
 
<source lang="bash">
 
<source lang="bash">
relax 01_read_pdb.py -t 01_read_pdb.log
+
mkdir 20171010_model_free_5_HADDOCK
 +
cp 20171010_model_free/*.dat 20171010_model_free_5_HADDOCK
 +
cp 20171010_model_free/*.pdb 20171010_model_free_5_HADDOCK
 +
 
 +
# Get scripts
 +
cd 20171010_model_free_5_HADDOCK
 +
git init
 +
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
 +
git fetch
 +
git checkout -t origin/master
 +
 
 +
# Change NOE error
 +
sed -i 's/0.1*$/0.05/' NOE_600MHz_new.dat
 +
sed -i 's/0.1*$/0.05/' NOE_750MHz.dat
 +
 
 +
# Make deselection
 +
echo "#" > deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t17" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t120" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t59" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t98" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t49" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t76" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t155" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t156" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t48" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t154" >> deselect.txt
 
</source>
 
</source>
 +
|}
  
 +
And a new one, without changing the NOE error, and deselecting spins found from consistency test.<br>
 
{| class="mw-collapsible mw-collapsed wikitable"
 
{| class="mw-collapsible mw-collapsed wikitable"
! Output from logfile
+
! See commands
 
|-
 
|-
 
|
 
|
 
<source lang="bash">
 
<source lang="bash">
 +
mkdir 20171010_model_free_6_HADDOCK
 +
cp 20171010_model_free/*.dat 20171010_model_free_6_HADDOCK
 +
cp 20171010_model_free/*.pdb 20171010_model_free_6_HADDOCK
 +
 +
# Get scripts
 +
cd 20171010_model_free_6_HADDOCK
 +
git init
 +
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
 +
git fetch
 +
git checkout -t origin/master
 +
 +
# Make deselection
 +
echo "#" > deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t17" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt
 +
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t59" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t98" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t76" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t155" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t156" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t120" >> deselect.txt
 +
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t49" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t48" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t154" >> deselect.txt
 +
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t33" >> deselect.txt
 +
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t67" >> deselect.txt
 +
</source>
 +
|}
 +
 +
== 11_read_data_GUI_inspect.py - Read data GUI inspect ==
 +
This will read the data and save as a state.
 +
 +
The GUI can be a good place to inspect the setup and files.
 +
 +
See content of:
 +
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/11_read_data_GUI_inspect.py 11_read_data_GUI_inspect.py]
 +
 +
Run with
 +
<source lang="bash">
 +
relax 11_read_data_GUI_inspect.py -t 11_read_data_GUI_inspect.log
 +
</source>
 +
 +
To check in GUI
 +
* relax -g
 +
* File -> Open relax state
 +
* In folder "result_10" open "result_10_ini.bz2"
 +
* View -> Data pipe editor
 +
* Right click on pipe, and select "Associate with a new auto-analysis"
 +
 +
=== relax 11_test_consistency.py - Consistency test of our data ===
 +
Before running the analysis, it is wise to run a script for [[Tutorial_for_model_free_SBiNLab#Other_script_inspiration_for_checking|consistency testing]].
 +
 +
See here:
 +
* Morin & Gagne (2009a) [http://dx.doi.org/10.1007/s10858-009-9381-4 Simple tests for the validation of multiple field spin relaxation data. J. Biomol. NMR, 45: 361-372.]
 +
 +
Highlights:
 +
* Comparing results obtained at different magnetic fields should, in the case of perfect consistency and assuming the absence of conformational exchange, yield equal values independently of the magnetic field.
 +
* avoid the potential extraction of erroneous information as well as the waste of time associated to dissecting inconsistent datasets using numerous long model-free minimisations with different subsets of data.
 +
* The authors prefer the use of the spectral density at zero frequency J(0) alone since it '''does not rely''' on an estimation of the global correlation time '''tc/tm''', neither on a measure of theta, the angle between the 15N–1H vector and the principal axis of the 15N chemical shift tensor. Hence, J(0) is less likely to be affected by incorrect parameterisation of input parameters.
 +
 +
See content of:
 +
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/11_test_consistency.py 11_test_consistency.py]
 +
<source lang="bash">
 +
relax 11_test_consistency.py -t 11_test_consistency.py.log
 +
 +
#Afterwards, go into the folder at plot data.
 +
python plot_txt_files.py
 +
./grace2images.py
 +
</source>
 +
 +
== 12_Model_1_I_local_tm.py - Only run local_tm ==
 +
Now we only run '''Model 1'''.
 +
 +
* DIFF_MODEL = ['local_tm']
 +
* GRID_INC = 11 # This is the standard
 +
* MC_NUM = 0 # This has no influence in Model 1-5
 +
* MAX_ITER = 20 # Stop if it has not converged in 20 rounds
 +
 +
Normally between 8 to 15 multiple rounds of optimisation of the are required for the proper execution of this script.<br>
 +
This is can also be see here in Figure 2.
 +
* d'Auvergne, E. J. and Gooley, P. R. (2008). [http://dx.doi.org/10.1007/s10858-007-9213-3 Optimisation of NMR dynamic models II. A new methodology for the dual optimisation of the model-free parameters and the Brownian rotational diffusion tensor. J. Biomol. NMR, 40(2), 121-133.]
 +
 +
Relax should stop calculation, if a model does not converge.
 +
 +
See content of:
 +
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/12_Model_1_I_local_tm.py 12_Model_1_I_local_tm.py]
 +
 +
We use [http://www.dayid.org/comp/tm.html tmux] to make a terminal-session, we can get back to,
 +
if our own terminal connection get closed.
 +
 +
Run with
 +
<source lang="bash">
 +
# Make terminal-session
 +
tmux new -s m1
 +
 +
relax 12_Model_1_I_local_tm.py -t 12_Model_1_I_local_tm.log
 +
 +
# or
 +
tmux new -s m1
 +
mpirun -np 22 relax --multi='mpi4py' 12_Model_1_I_local_tm.py -t 12_Model_1_I_local_tm.log
 +
</source>
 +
 +
You can then in another terminal follow the logfile by
 +
<source lang="bash">
 +
less +F 12_Model_I_local_tm.log
 +
</source>
 +
 +
* To scroll up and down, use keyboard: '''Ctrl+c'''
 +
* To return to follow mode, use keyboard: '''Shift+f'''
 +
* To exit, use keyboard: '''Ctrl+c'''    and then: '''q'''
 +
 +
== 13_Model_2-5 - Run Model 2 to 5 ==
 +
When Model 1 is completed, then make 4 terminal windows and run them at the
 +
same time.
 +
 +
These scripts do:
 +
* Read the state file from before with setup
 +
* Change DIFF_MODEL accordingly
 +
 +
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_2_II_sphere.py 13_Model_2_II_sphere.py]
 +
<source lang="bash">
 +
tmux new -s m2
 +
relax 13_Model_2_II_sphere.py -t 13_Model_2_II_sphere.log
 +
# Or
 +
mpirun -np 5 relax --multi='mpi4py' 13_Model_2_II_sphere.py -t 13_Model_2_II_sphere.log
  
 +
# When relax is running, push: Ctrl+b and then d, to disconnect without exit
 +
</source>
  
 +
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_3_III_prolate.py 13_Model_3_III_prolate.py]
 +
<source lang="bash">
 +
tmux new -s m3
 +
relax 13_Model_3_III_prolate.py -t 13_Model_3_III_prolate.log
 +
# Or
 +
mpirun -np 5 relax --multi='mpi4py' 13_Model_3_III_prolate.py -t 13_Model_3_III_prolate.log
 +
</source>
  
                                    relax repository checkout
+
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_4_IV_oblate.py 13_Model_4_IV_oblate.py]
                              c6b38290b22c033c2756abde2b19bddf37b79f7a
+
<source lang="bash">
                          https://git.code.sf.net/p/nmr-relax/code (fetch)
+
tmux new -s m4
                          https://git.code.sf.net/p/nmr-relax/code (push)
+
relax 13_Model_4_IV_oblate.py -t 13_Model_4_IV_oblate.log
 +
# Or
 +
mpirun -np 5 relax --multi='mpi4py' 13_Model_4_IV_oblate.py -t 13_Model_4_IV_oblate.log
 +
</source>
  
                              Molecular dynamics by NMR data analysis
+
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_5_V_ellipsoid.py 13_Model_5_V_ellipsoid.py]
 +
<source lang="bash">
 +
tmux new -s m5
 +
relax 13_Model_5_V_ellipsoid.py -t 13_Model_5_V_ellipsoid.log
 +
# Or
 +
mpirun -np 5 relax --multi='mpi4py' 13_Model_5_V_ellipsoid.py -t 13_Model_5_V_ellipsoid.log
 +
</source>
  
                            Copyright (C) 2001-2006 Edward d'Auvergne
+
To join session
                        Copyright (C) 2006-2017 the relax development team
+
<source lang="bash">
 +
# List
 +
tmux list-s
  
This is free software which you are welcome to modify and redistribute under the conditions of the
+
# Join either
GNU General Public License (GPL).  This program, including all modules, is licensed under the GPL
+
tmux a -t m1
and comes with absolutely no warranty.  For details type 'GPL' within the relax prompt.
+
tmux a -t m2
 +
tmux a -t m3
 +
tmux a -t m4
 +
tmux a -t m5
 +
</source>
  
Assistance in using the relax prompt and scripting interface can be accessed by typing 'help' within
+
== 14_intermediate_final.py - Inspection during model optimization ==
the prompt.
+
During running of model 2-5, the current results can be inspected with
 +
this nifty scripts.
  
Processor fabric:  Uni-processor.
+
The script will ask for input of MC numbers. So just run it.
  
script = '01_read_pdb.py'
+
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/14_intermediate_final.py 14_intermediate_final.py]
----------------------------------------------------------------------------------------------------
+
<source lang="bash">
# Python module imports.
+
tmux new -s final
from time import asctime, localtime
+
relax 14_intermediate_final.py -t 14_intermediate_final.log
import os
+
</source>
  
# relax module imports.
+
This does:
from auto_analyses.dauvergne_protocol import dAuvergne_protocol
+
* Option: Collect current best result from Model 2-5, and make MC simulations, and finalize to get current results files
 +
** [http://comdnmr.nysbc.org/comd-nmr-dissem/comd-nmr-software Make analysis script for palmer Modelfree4]
 +
** Get more spin information
 +
* Make a pymol file, that collects all of relax pymol command files into 1 pymol session
 +
* Option: Collect all chi2 and number of params k, for each iteration per model
 +
** Make a python plot file for plotting this results
  
# Set up the data pipe.
+
=== Per iteration get: chi2, k, tm ===
#######################
+
Afterwards, plot the data.
 +
<source lang="bash">
 +
python results_collected.py
 +
</source>
  
# The following sequence of user function calls can be changed as needed.
+
=== Pymol macro ===
 +
You also get a pymol folder.
  
# Create the data pipe.
+
See here for info how the macro is applied
bundle_name = "mf (%s)" % asctime(localtime())
+
* [http://www.nmr-relax.com/manual/molmol_macro_apply.html#SECTION081284600000000000000 Summary of parameter meaning and value to pymol visualization]
name = "origin"
 
pipe.create(name, 'mf', bundle=bundle_name)
 
  
# Load the PDB file.
+
Run with
structure.read_pdb('energy_1.pdb', set_mol_name='ArcCALD', read_model=1)
+
<source lang="bash">
 +
pymol 0_0_apply_all_pymol_commands.pml
 +
</source>
  
# Set up the 15N and 1H spins (both backbone and Trp indole sidechains).
+
= To run on Haddock =
structure.load_spins('@N', ave_pos=True)
+
Have a look here, how to get standalone python [[Anaconda_linux_mac|Anaconda linux]].
structure.load_spins('@NE1', ave_pos=True)
+
Also have a look here [[OpenMPI]].
structure.load_spins('@H', ave_pos=True)
 
structure.load_spins('@HE1', ave_pos=True)
 
  
# Assign isotopes
+
<source lang="bash">
spin.isotope('15N', spin_id='@N*')
+
# SSH in
spin.isotope('1H', spin_id='@H*')
+
ssh haddock
  
----------------------------------------------------------------------------------------------------
+
# Test with shell
 +
mpirun -np 6 echo "hello world"
  
relax> pipe.create(pipe_name='origin', pipe_type='mf', bundle='mf (Fri Oct 13 17:44:18 2017)')
+
# Test with python
 +
mpirun -np 6 python -m mpi4py helloworld
  
relax> structure.read_pdb(file='energy_1.pdb', dir=None, read_mol=None, set_mol_name='ArcCALD', read_model=1, set_model_num=None, alt_loc=None, verbosity=1, merge=False)
+
# Test with relax
 +
mpirun -np 6 relax --multi='mpi4py'
 +
# Look for: Processor fabric:  MPI 2.2 running via mpi4py with 5 slave processors & 1 master.  Using MPICH2 1.4.1.
 +
</source>
  
Internal relax PDB parser.
+
Now we run '''04_run_default_with_tolerance_lim.py''' with more power!<br>
Opening the file 'energy_1.pdb' for reading.
+
We use [http://www.dayid.org/comp/tm.html tmux] to make a terminal-session, we can get back to,
RelaxWarning: Cannot determine the element associated with atom 'X'.
+
if our own terminal connection get closed.
RelaxWarning: Cannot determine the element associated with atom 'Z'.
 
RelaxWarning: Cannot determine the element associated with atom 'OO'.
 
RelaxWarning: Cannot determine the element associated with atom 'OO2'.
 
Adding molecule 'ArcCALD' to model 1 (from the original molecule number 1 of model 1).
 
  
relax> structure.load_spins(spin_id='@N', from_mols=None, mol_name_target=None, ave_pos=True, spin_num=True)
+
* start a new session: '''tmux'''
Adding the following spins to the relax data store.
+
* re-attach a detached session: '''tmux attach'''
  
# mol_name    res_num    res_name    spin_num    spin_name   
+
<source lang="bash">
ArcCALD      1          SER        1          N           
+
# Make terminal-session
ArcCALD      2          PRO        14          N           
+
tmux
ArcCALD      3          GLY        28          N           
 
ArcCALD      4          LEU        35          N           
 
ArcCALD      5          ASP        54          N           
 
ArcCALD      6          THR        66          N           
 
ArcCALD      7          GLN        80          N           
 
ArcCALD      8          ILE        97          N           
 
ArcCALD      9          PHE        116        N           
 
ArcCALD      10        GLU        136        N           
 
ArcCALD      11        ASP        151        N           
 
ArcCALD      12        PRO        163        N           
 
ArcCALD      13        ARG        177        N           
 
ArcCALD      14        GLU        201        N           
 
ArcCALD      15        PHE        216        N           
 
ArcCALD      16        LEU        236        N           
 
ArcCALD      17        SER        255        N           
 
ArcCALD      18        HIS        266        N           
 
ArcCALD      19        LEU        284        N           
 
ArcCALD      20        GLU        303        N           
 
ArcCALD      21        GLU        318        N           
 
ArcCALD      22        TYR        333        N           
 
ArcCALD      23        LEU        354        N           
 
ArcCALD      24        ARG        373        N           
 
ArcCALD      25        GLN        397        N           
 
ArcCALD      26        VAL        414        N           
 
ArcCALD      27        GLY        430        N           
 
ArcCALD      28        GLY        437        N           
 
ArcCALD      29        SER        444        N           
 
ArcCALD      30        GLU        455        N           
 
ArcCALD      31        GLU        470        N           
 
ArcCALD      32        TYR        485        N           
 
ArcCALD      33        TRP        506        N           
 
ArcCALD      34        LEU        530        N           
 
ArcCALD      35        SER        549        N           
 
ArcCALD      36        GLN        560        N           
 
ArcCALD      37        ILE        577        N           
 
ArcCALD      38        GLN        596        N           
 
ArcCALD      39        ASN        613        N           
 
ArcCALD      40        HIS        627        N           
 
ArcCALD      41        MET        645        N           
 
ArcCALD      42        ASN        662        N           
 
ArcCALD      43        GLY        676        N           
 
ArcCALD      44        PRO        683        N           
 
ArcCALD      45        ALA        697        N           
 
ArcCALD      46        LYS        707        N           
 
ArcCALD      47        LYS        729        N           
 
ArcCALD      48        TRP        751        N           
 
ArcCALD      49        TRP        775        N           
 
ArcCALD      50        GLU        799        N           
 
ArcCALD      51        PHE        814        N           
 
ArcCALD      52        LYS        834        N           
 
ArcCALD      53        GLN        856        N           
 
ArcCALD      54        GLY        873        N           
 
ArcCALD      55        SER        880        N           
 
ArcCALD      56        VAL        891        N           
 
ArcCALD      57        LYS        907        N           
 
ArcCALD      58        ASN        929        N           
 
ArcCALD      59        TRP        943        N           
 
ArcCALD      60        VAL        967        N           
 
ArcCALD      61        GLU        983        N           
 
ArcCALD      62        PHE        998        N           
 
ArcCALD      63        LYS        1018        N           
 
ArcCALD      64        LYS        1040        N           
 
ArcCALD      65        GLU        1062        N           
 
ArcCALD      66        PHE        1077        N           
 
ArcCALD      67        LEU        1097        N           
 
ArcCALD      68        GLN        1116        N           
 
ArcCALD      69        TYR        1133        N           
 
ArcCALD      70        SER        1154        N           
 
ArcCALD      71        GLU        1165        N           
 
ArcCALD      72        GLY        1180        N           
 
ArcCALD      73        THR        1187        N           
 
ArcCALD      74        LEU        1201        N           
 
ArcCALD      75        SER        1220        N           
 
ArcCALD      76        ARG        1231        N           
 
ArcCALD      77        GLU        1255        N           
 
ArcCALD      78        ALA        1270        N           
 
ArcCALD      79        ILE        1280        N           
 
ArcCALD      80        GLN        1299        N           
 
ArcCALD      81        ARG        1316        N           
 
ArcCALD      82        GLU        1340        N           
 
ArcCALD      83        LEU        1355        N           
 
ArcCALD      84        ASP        1374        N           
 
ArcCALD      85        LEU        1386        N           
 
ArcCALD      86        PRO        1405        N           
 
ArcCALD      87        GLN        1419        N           
 
ArcCALD      88        LYS        1436        N           
 
ArcCALD      89        GLN        1458        N           
 
ArcCALD      90        GLY        1475        N           
 
ArcCALD      91        GLU        1482        N           
 
ArcCALD      92        PRO        1497        N           
 
ArcCALD      93        LEU        1511        N           
 
ArcCALD      94        ASP        1530        N           
 
ArcCALD      95        GLN        1542        N           
 
ArcCALD      96        PHE        1559        N           
 
ArcCALD      97        LEU        1579        N           
 
ArcCALD      98        TRP        1598        N           
 
ArcCALD      99        ARG        1622        N           
 
ArcCALD      100        LYS        1646        N           
 
ArcCALD      101        ARG        1668        N           
 
ArcCALD      102        ASP        1692        N           
 
ArcCALD      103        LEU        1704        N           
 
ArcCALD      104        TYR        1723        N           
 
ArcCALD      105        GLN        1744        N           
 
ArcCALD      106        THR        1761        N           
 
ArcCALD      107        LEU        1775        N           
 
ArcCALD      108        TYR        1794        N           
 
ArcCALD      109        VAL        1815        N           
 
ArcCALD      110        ASP        1831        N           
 
ArcCALD      111        ALA        1843        N           
 
ArcCALD      112        GLU        1853        N           
 
ArcCALD      113        GLU        1868        N           
 
ArcCALD      114        GLU        1883        N           
 
ArcCALD      115        GLU        1898        N           
 
ArcCALD      116        ILE        1913        N           
 
ArcCALD      117        ILE        1932        N           
 
ArcCALD      118        GLN        1951        N           
 
ArcCALD      119        TYR        1968        N           
 
ArcCALD      120        VAL        1989        N           
 
ArcCALD      121        VAL        2005        N           
 
ArcCALD      122        GLY        2021        N           
 
ArcCALD      123        THR        2028        N           
 
ArcCALD      124        LEU        2042        N           
 
ArcCALD      125        GLN        2061        N           
 
ArcCALD      126        PRO        2078        N           
 
ArcCALD      127        LYS        2092        N           
 
ArcCALD      128        PHE        2114        N           
 
ArcCALD      129        LYS        2134        N           
 
ArcCALD      130        ARG        2156        N           
 
ArcCALD      131        PHE        2180        N           
 
ArcCALD      132        LEU        2200        N           
 
ArcCALD      133        ARG        2219        N           
 
ArcCALD      134        HIS        2243        N           
 
ArcCALD      135        PRO        2261        N           
 
ArcCALD      136        LEU        2275        N           
 
ArcCALD      137        PRO        2294        N           
 
ArcCALD      138        LYS        2308        N           
 
ArcCALD      139        THR        2330        N           
 
ArcCALD      140        LEU        2344        N           
 
ArcCALD      141        GLU        2363        N           
 
ArcCALD      142        GLN        2378        N           
 
ArcCALD      143        LEU        2395        N           
 
ArcCALD      144        ILE        2414        N           
 
ArcCALD      145        GLN        2433        N           
 
ArcCALD      146        ARG        2450        N           
 
ArcCALD      147        GLY        2474        N           
 
ArcCALD      148        MET        2481        N           
 
ArcCALD      149        GLU        2498        N           
 
ArcCALD      150        VAL        2513        N           
 
ArcCALD      151        GLN        2529        N           
 
ArcCALD      152        ASP        2546        N           
 
ArcCALD      153        GLY        2558        N           
 
ArcCALD      154        LEU        2565        N           
 
ArcCALD      155        GLU        2584        N           
 
ArcCALD      156        GLN        2599        N           
 
ArcCALD      157        ALA        2616        N           
 
ArcCALD      158        ALA        2626        N           
 
ArcCALD      159        GLU        2636        N           
 
  
relax> structure.load_spins(spin_id='@NE1', from_mols=None, mol_name_target=None, ave_pos=True, spin_num=True)
+
# Start relax
Adding the following spins to the relax data store.
+
mpirun -np 20 relax --multi='mpi4py' 04_run_default_with_tolerance_lim.py -t 04_run_default_with_tolerance_lim.log
 +
</source>
  
# mol_name    res_num    res_name    spin_num    spin_name   
+
= Useful commands to log file =
ArcCALD      33        TRP        517        NE1         
 
ArcCALD      48        TRP        762        NE1         
 
ArcCALD      49        TRP        786        NE1         
 
ArcCALD      59        TRP        954        NE1         
 
ArcCALD      98        TRP        1609        NE1         
 
  
relax> structure.load_spins(spin_id='@H', from_mols=None, mol_name_target=None, ave_pos=True, spin_num=True)
+
While the analysis is running, these commands could be used to check the logfile for errors
Adding the following spins to the relax data store.
+
<source lang="bash">
 +
### Check convergence
 +
# For chi2
 +
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Chi-squared test:"
  
# mol_name    res_num    res_name    spin_num    spin_name   
+
# For other tests
ArcCALD      3          GLY        29          H           
+
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical "
ArcCALD      4          LEU        36          H           
+
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical model-free models test:"
ArcCALD      5          ASP        55          H           
+
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical diffusion tensor parameter test:"
ArcCALD      6          THR        67          H           
+
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical model-free parameter test:"
ArcCALD      7          GLN        81          H           
 
ArcCALD      8          ILE        98          H           
 
ArcCALD      9          PHE        117        H           
 
ArcCALD      10         GLU        137        H           
 
ArcCALD      11        ASP        152        H           
 
ArcCALD      13        ARG        178        H           
 
ArcCALD      14        GLU        202        H           
 
ArcCALD      15        PHE        217        H           
 
ArcCALD      16        LEU        237        H           
 
ArcCALD      17        SER        256        H           
 
ArcCALD      18        HIS        267        H           
 
ArcCALD      19        LEU        285        H           
 
ArcCALD      20        GLU        304        H           
 
ArcCALD      21        GLU        319        H           
 
ArcCALD      22        TYR        334        H           
 
ArcCALD      23        LEU        355        H           
 
ArcCALD      24        ARG        374        H           
 
ArcCALD      25        GLN        398        H           
 
ArcCALD      26        VAL        415        H           
 
ArcCALD      27        GLY        431        H           
 
ArcCALD      28        GLY        438        H           
 
ArcCALD      29        SER        445        H           
 
ArcCALD      30        GLU        456        H           
 
ArcCALD      31        GLU        471        H           
 
ArcCALD      32        TYR        486        H           
 
ArcCALD      33        TRP        507        H           
 
ArcCALD      34        LEU        531        H           
 
ArcCALD      35        SER        550        H           
 
ArcCALD      36        GLN        561        H           
 
ArcCALD      37        ILE        578        H           
 
ArcCALD      38        GLN        597        H           
 
ArcCALD      39        ASN        614        H           
 
ArcCALD      40        HIS        628        H           
 
ArcCALD      41        MET        646        H           
 
ArcCALD      42        ASN        663        H           
 
ArcCALD      43        GLY        677        H           
 
ArcCALD      45        ALA        698        H           
 
ArcCALD      46        LYS        708        H           
 
ArcCALD      47        LYS        730        H           
 
ArcCALD      48        TRP        752        H           
 
ArcCALD      49        TRP        776        H           
 
ArcCALD      50        GLU        800        H           
 
ArcCALD      51        PHE        815        H           
 
ArcCALD      52        LYS        835        H           
 
ArcCALD      53        GLN        857        H           
 
ArcCALD      54        GLY        874        H           
 
ArcCALD      55        SER        881        H           
 
ArcCALD      56        VAL        892        H           
 
ArcCALD      57        LYS        908        H           
 
ArcCALD      58        ASN        930        H           
 
ArcCALD      59        TRP        944        H           
 
ArcCALD      60        VAL        968        H           
 
ArcCALD      61        GLU        984        H           
 
ArcCALD      62        PHE        999        H           
 
ArcCALD      63        LYS        1019        H           
 
ArcCALD      64        LYS        1041        H           
 
ArcCALD      65        GLU        1063        H           
 
ArcCALD      66        PHE        1078        H           
 
ArcCALD      67        LEU        1098        H           
 
ArcCALD      68        GLN        1117        H           
 
ArcCALD      69        TYR        1134        H           
 
ArcCALD      70        SER        1155        H           
 
ArcCALD      71        GLU        1166        H           
 
ArcCALD      72        GLY        1181        H           
 
ArcCALD      73        THR        1188        H           
 
ArcCALD      74        LEU        1202        H           
 
ArcCALD      75        SER        1221        H           
 
ArcCALD      76        ARG        1232        H           
 
ArcCALD      77        GLU        1256        H           
 
ArcCALD      78        ALA        1271        H           
 
ArcCALD      79        ILE        1281        H           
 
ArcCALD      80        GLN        1300        H           
 
ArcCALD      81        ARG        1317        H           
 
ArcCALD      82        GLU        1341        H           
 
ArcCALD      83        LEU        1356        H           
 
ArcCALD      84        ASP        1375        H           
 
ArcCALD      85        LEU        1387        H           
 
ArcCALD      87        GLN        1420        H           
 
ArcCALD      88        LYS        1437        H           
 
ArcCALD      89        GLN        1459        H           
 
ArcCALD      90        GLY        1476        H           
 
ArcCALD      91        GLU        1483        H           
 
ArcCALD      93        LEU        1512        H           
 
ArcCALD      94        ASP        1531        H           
 
ArcCALD      95        GLN        1543        H           
 
ArcCALD      96        PHE        1560        H           
 
ArcCALD      97        LEU        1580        H           
 
ArcCALD      98        TRP        1599        H           
 
ArcCALD      99        ARG        1623        H           
 
ArcCALD      100        LYS        1647        H           
 
ArcCALD      101        ARG        1669        H           
 
ArcCALD      102        ASP        1693        H           
 
ArcCALD      103        LEU        1705        H           
 
ArcCALD      104        TYR        1724        H           
 
ArcCALD      105        GLN        1745        H           
 
ArcCALD      106        THR        1762        H           
 
ArcCALD      107        LEU        1776        H           
 
ArcCALD      108        TYR        1795        H           
 
ArcCALD      109        VAL        1816        H           
 
ArcCALD      110        ASP        1832        H           
 
ArcCALD      111        ALA        1844        H           
 
ArcCALD      112        GLU        1854        H           
 
ArcCALD      113        GLU        1869        H           
 
ArcCALD      114        GLU        1884        H           
 
ArcCALD      115        GLU        1899        H           
 
ArcCALD      116        ILE        1914        H           
 
ArcCALD      117        ILE        1933        H           
 
ArcCALD      118        GLN        1952        H           
 
ArcCALD      119        TYR        1969        H           
 
ArcCALD      120        VAL        1990        H           
 
ArcCALD      121        VAL        2006        H           
 
ArcCALD      122        GLY        2022        H           
 
ArcCALD      123        THR        2029        H           
 
ArcCALD      124        LEU        2043        H           
 
ArcCALD      125        GLN        2062        H           
 
ArcCALD      127        LYS        2093        H           
 
ArcCALD      128        PHE        2115        H           
 
ArcCALD      129        LYS        2135        H           
 
ArcCALD      130        ARG        2157        H           
 
ArcCALD      131        PHE        2181        H           
 
ArcCALD      132        LEU        2201        H           
 
ArcCALD      133        ARG        2220        H           
 
ArcCALD      134        HIS        2244        H           
 
ArcCALD      136        LEU        2276        H           
 
ArcCALD      138        LYS        2309        H           
 
ArcCALD      139        THR        2331        H           
 
ArcCALD      140        LEU        2345        H           
 
ArcCALD      141        GLU        2364        H           
 
ArcCALD      142        GLN        2379        H           
 
ArcCALD      143        LEU        2396        H           
 
ArcCALD      144        ILE        2415        H           
 
ArcCALD      145        GLN        2434        H           
 
ArcCALD      146        ARG        2451        H           
 
ArcCALD      147        GLY        2475        H           
 
ArcCALD      148        MET        2482        H           
 
ArcCALD      149        GLU        2499        H           
 
ArcCALD      150        VAL        2514        H           
 
ArcCALD      151        GLN        2530        H           
 
ArcCALD      152        ASP        2547        H           
 
ArcCALD      153        GLY        2559        H           
 
ArcCALD      154        LEU        2566        H           
 
ArcCALD      155        GLU        2585        H           
 
ArcCALD      156        GLN        2600        H           
 
ArcCALD      157        ALA        2617        H           
 
ArcCALD      158        ALA        2627        H           
 
ArcCALD      159        GLU        2637        H           
 
  
relax> structure.load_spins(spin_id='@HE1', from_mols=None, mol_name_target=None, ave_pos=True, spin_num=True)
+
# To look for not converged errors
Adding the following spins to the relax data store.
+
# For chi2
 +
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The chi-squared value has not converged."
  
# mol_name    res_num    res_name    spin_num    spin_name   
+
# For other tests
ArcCALD      9          PHE        129        HE1         
+
cat 04_run_default_with_tolerance_lim.log | grep -B 7 " have not converged."
ArcCALD      15        PHE        229        HE1         
+
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The model-free models have not converged."
ArcCALD      18        HIS        279        HE1         
+
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The diffusion parameters have not converged."
ArcCALD      22        TYR        346        HE1         
+
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The model-free parameters have not converged."
ArcCALD      32        TYR        498        HE1         
+
</source>
ArcCALD      33        TRP        518        HE1         
 
ArcCALD      40        HIS        640        HE1         
 
ArcCALD      41        MET        657        HE1         
 
ArcCALD      46        LYS        721        HE1         
 
ArcCALD      47        LYS        743        HE1         
 
ArcCALD      48        TRP        763        HE1         
 
ArcCALD      49        TRP        787        HE1         
 
ArcCALD      51        PHE        827        HE1         
 
ArcCALD      52        LYS        848        HE1         
 
ArcCALD      57        LYS        921        HE1         
 
ArcCALD      59        TRP        955        HE1         
 
ArcCALD      62        PHE        1011        HE1         
 
ArcCALD      63        LYS        1032        HE1         
 
ArcCALD      64        LYS        1054        HE1         
 
ArcCALD      66        PHE        1090        HE1         
 
ArcCALD      69        TYR        1146        HE1         
 
ArcCALD      88        LYS        1450        HE1         
 
ArcCALD      96        PHE        1572        HE1         
 
ArcCALD      98        TRP        1610        HE1         
 
ArcCALD      100        LYS        1660        HE1         
 
ArcCALD      104        TYR        1736        HE1         
 
ArcCALD      108        TYR        1807        HE1         
 
ArcCALD      119        TYR        1981        HE1         
 
ArcCALD      127        LYS        2106        HE1         
 
ArcCALD      128        PHE        2127        HE1         
 
ArcCALD      129        LYS        2148        HE1         
 
ArcCALD      131        PHE        2193        HE1         
 
ArcCALD      134        HIS        2256        HE1         
 
ArcCALD      138        LYS        2322        HE1         
 
ArcCALD      148        MET        2493        HE1         
 
  
relax> spin.isotope(isotope='15N', spin_id='@N*', force=False)
+
You can then inspect the logfile by '''less''': [http://www.thegeekstuff.com/2010/02/unix-less-command-10-tips-for-effective-navigation 10-tips for less]
  
relax> spin.isotope(isotope='1H', spin_id='@H*', force=False)
+
<source lang="bash">
 +
less 04_run_default_with_tolerance_lim.log
 +
</source>
  
 +
'''To find pattern:''' We have to escape with '''\''' for special character like: ()[] etc.
 +
<source lang="bash">
 +
# Search forward
 +
/Value \(iter 14\)
 +
/The chi-squared value has not converged
 +
</source>
 +
'''n''' or '''N''' – for next match in forward / previous match in backward
 +
 +
* To return to follow mode, use keyboard: '''Shift+f'''
 +
* To exit, use keyboard: '''Ctrl+c'''    and then: '''q'''
 +
 +
= rsync files =
 +
== rsync files after completion to Sauron ==
 +
When a run is completed, then sync files to Sauron file server.
 +
 +
Make a '''rsync_to_sbinlab.sh''' file with content
 +
{| class="mw-collapsible mw-collapsed wikitable"
 +
! See file content
 +
|-
 +
|
 +
<source lang="bash">
 +
#!/bin/bash
 +
 +
read -p "Username on sauron :" -r
 +
 +
RUSER=$REPLY
 +
SAURON=10.61.4.60
 +
PROJ=`basename "$PWD"`
 +
 +
FROM=${PWD}
 +
TO=${RUSER}@${SAURON}:/data/sbinlab2/${RUSER}/Downloads
 +
 +
# -a: "archive"- archive mode; equals -rlptgoD (no -H,-A,-X). syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
 +
# We want to remove the -o and -g options:
 +
# -o, --owner                preserve owner (super-user only)
 +
# -g, --group                preserve group
 +
# -rlptD : Instead or
 +
# -a --no-o --no-g 
 +
# -z: Compression over network
 +
# -P: It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
 +
# -h, Output numbers in a more human-readable format.
 +
 +
# Always double-check your arguments before executing an rsync command.
 +
# -n
 +
 +
echo "I will now do a DRY RUN, which does not move files"
 +
read -p "Are you sure? y/n :" -n 1 -r
 +
echo ""
 +
 +
if [[ $REPLY =~ ^[Yy]$ ]]; then
 +
  rsync -rlptDPzh -n ${FROM} ${TO}
 +
else
 +
  echo "Not doing DRY RUN"
 +
fi
 +
 +
echo ""
 +
 +
echo "I will now do the sync of files"
 +
read -p "Are you sure? y/n :" -n 1 -r
 +
echo ""
 +
 +
if [[ $REPLY =~ ^[Yy]$ ]]; then
 +
  rsync -rlptDPzh ${FROM} ${TO}
 +
else
 +
  echo "Not doing anything"
 +
fi
 +
</source>
 +
 +
Make it executable and run
 +
<source lang="bash">
 +
chmod +x rsync_to_sbinlab.sh
 +
 +
#run
 +
./rsync_to_sbinlab2.sh
 +
</source>
 +
|}
 +
 +
== rsync files from BIO to home mac ==
 +
To inspect from home mac.
 +
 +
Make a '''rsync_from_bio_to_home.sh''' file with content
 +
{| class="mw-collapsible mw-collapsed wikitable"
 +
! See file content
 +
|-
 +
|
 +
<source lang="bash">
 +
#!/bin/bash
 +
 +
read -p "Username on bio:" -r
 +
 +
RUSER=$REPLY
 +
BIO=ssh-bio.science.ku.dk
 +
 +
#PROJ=Desktop/kaare_relax
 +
PROJ=Desktop/kaare_relax/20171010_model_free_HADDOCK
 +
PROJDIR=`basename "$PROJ"`
 +
 +
FROM=${RUSER}@${BIO}:/home/${RUSER}/${PROJ}
 +
TO=${PWD}/${PROJDIR}
 +
 +
# -a: "archive"- archive mode; equals -rlptgoD (no -H,-A,-X). syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
 +
# We want to remove the -o and -g options:
 +
# -o, --owner                preserve owner (super-user only)
 +
# -g, --group                preserve group
 +
# -rlptD : Instead or
 +
# -a --no-o --no-g 
 +
# -z: Compression over network
 +
# -P: It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
 +
# -h, Output numbers in a more human-readable format.
 +
 +
# Always double-check your arguments before executing an rsync command.
 +
# -n
 +
 +
echo "I will now do a DRY RUN, which does not move files"
 +
read -p "Are you sure? y/n :" -n 1 -r
 +
echo ""
 +
 +
if [[ $REPLY =~ ^[Yy]$ ]]; then
 +
  rsync -rlptDPzh -n ${FROM} ${TO}
 +
else
 +
  echo "Not doing DRY RUN"
 +
fi
 +
 +
echo ""
 +
 +
echo "I will now do the sync of files"
 +
read -p "Are you sure? y/n :" -n 1 -r
 +
echo ""
 +
 +
if [[ $REPLY =~ ^[Yy]$ ]]; then
 +
  rsync -rlptDPzh ${FROM} ${TO}
 +
else
 +
  echo "Not doing anything"
 +
fi
 +
</source>
 +
 +
Make it executable and run
 +
<source lang="bash">
 +
chmod +x rsync_from_bio_to_home.sh
 +
 +
#run
 +
./rsync_from_bio_to_home.sh
 
</source>
 
</source>
 
|}
 
|}
  
== See also ==
+
= About the protocol =
 +
 
 +
'''Model I -  'local_tm''''<br>
 +
This will optimise the diffusion model whereby all spin of the molecule have a local tm value, i.e. there is no global diffusion tensor.  This model needs to be optimised prior to optimising any of the other diffusion models.  Each spin is fitted to the multiple model-free models separately, where the parameter tm is included in each model.
 +
 
 +
'''Model II - 'sphere''''<br>
 +
This will optimise the isotropic diffusion model.  Multiple steps are required, an initial optimisation of the diffusion tensor, followed by a repetitive optimisation until convergence of the diffusion tensor.  In the relax script UI each of these steps requires this script to be rerun, unless the conv_loop flag is True.  In the GUI (graphical user interface), the procedure is repeated automatically until convergence.  For the initial optimisation, which will be placed in the directory './sphere/init/', the following steps are used:
 +
* The model-free models and parameter values for each spin are set to those of diffusion model MI.
 +
* The local tm parameter is removed from the models.
 +
* The model-free parameters are fixed and a global spherical diffusion tensor is minimised
 +
* For the repetitive optimisation, each minimisation is named from 'round_1' onwards.  The initial 'round_1' optimisation will extract the diffusion tensor from the results file in './sphere/init/', and the results will be placed in the directory './sphere/round_1/'.  Each successive round will take the diffusion tensor from the previous round.  The following steps are used:
 +
** The global diffusion tensor is fixed and the multiple model-free models are fitted to each spin.
 +
** AIC model selection is used to select the models for each spin.
 +
** All model-free and diffusion parameters are allowed to vary and a global optimisation of all parameters is carried out.
 +
 
 +
'''Model III - 'prolate''''<br>
 +
The methods used are identical to those of diffusion model MII, except that an axially symmetric diffusion tensor with Da >= 0 is used.  The base directory containing all the results is './prolate/'.
 +
 
 +
'''Model IV -'oblate''''<br>
 +
The methods used are identical to those of diffusion model MII, except that an axially symmetric diffusion tensor with Da <= 0 is used.  The base directory containing all the results is './oblate/'.
 +
 
 +
'''Model V - 'ellipsoid''''<br>
 +
The methods used are identical to those of diffusion model MII, except that a fully anisotropic diffusion tensor is used (also known as rhombic or asymmetric diffusion).  The base directory is './ellipsoid/'
 +
 
 +
''''final''''<br>
 +
Once all the diffusion models have converged, the final run can be executed.  This is done by setting the variable diff_model to 'final'.  This consists of two steps, diffusion tensor model selection, and Monte Carlo simulations.  Firstly AIC model selection is used to select between the diffusion tensor models.  Monte Carlo simulations are then run solely on this selected diffusion model.  Minimisation of the model is bypassed as it is assumed that the model is already fully optimised (if this is not the case the final run is not yet appropriate).
 +
The final black-box model-free results will be placed in the file 'final/results'.
 +
 
 +
= See also =
 
[[Category:Tutorials]]
 
[[Category:Tutorials]]
 
[[Category:Model-free_analysis]]
 
[[Category:Model-free_analysis]]

Latest revision as of 10:40, 22 October 2017

Background

This is a tutorial for Lau and Kaare in SBiNLab, and hopefully others.

To get inspiration of example scripts files and see how the protocol is performed, have a look here:

For references, see relax references:

Script inspiration

model-free : Script inspiration for setup and analysis

The distribution of relax includes a folder sample_scripts/model_free which contain a folder with scripts for analysis.

It can be seen here: https://github.com/nmr-relax/relax/tree/master/sample_scripts/model_free

Here is the current list

  • back_calculate.py. Back-calculate and save relaxation data starting from a saved model-free results file.
  • bmrb_deposition.py Script for creating a NMR-STAR 3.1 formatted file for BMRB deposition of model-free results.
  • cv.py Script for model-free analysis using cross-validation model selection.
  • dasha.py Script for model-free analysis using the program Dasha.
  • dauvergne_protocol.py Script for black-box model-free analysis.
  • diff_min.py Demonstration script for diffusion tensor optimisation in a model-free analysis.]
  • final_data_extraction.py Extract Data to Table
  • generate_ri.py Script for back-calculating the relaxation data.
  • grace_S2_vs_te.py Script for creating a grace plot of the simulated order parameters vs. simulated correlation times.
  • grace_ri_data_correlation.py Script for creating correlations plots of experimental verses back calculated relaxation data.
  • map.py Script for mapping the model-free space for OpenDX visualisation.
  • mf_multimodel.py This script performs a model-free analysis for the models 'm0' to 'm9' (or 'tm0' to 'tm9').
  • modsel.py Script for model-free model selection.
  • molmol_plot.py Script for generating Molmol macros for highlighting model-free motions
  • palmer.py Script for model-free analysis using Art Palmer's program 'Modelfree4'. Download from http://comdnmr.nysbc.org/comd-nmr-dissem/comd-nmr-software
  • remap.py Script for mapping the model-free space.
  • single_model.py This script performs a model-free analysis for the single model 'm4'.
  • table_csv.py Script for converting the model-free results into a CSV table.
  • table_latex.py Script for converting the model-free results into a LaTeX table.

Other script inspiration for checking

The distribution of relax includes a folder sample_scripts/ which contain a folder with scripts for analysis.

It can be seen here: https://github.com/nmr-relax/relax/tree/master/sample_scripts

R1 / R2 Calculation

The resultant plot is useful for finding bad points or bad spectra when fitting exponential curves determine the R1 and R2 relaxation rates. If the averages deviate systematically from zero, bias in the spectra or fitting will be clearly revealed. To use this script, R1 or R2 exponential curve fitting must have previously have been carried out the program state saved to the file 'rx.save' (either with or without the .gz or .bz2 ). The file name of the saved state can be changed at the top of this script.

NOE calculation

  • noe.py Script for calculating NOEs.

Test data

Severe artifacts can be introduced if model-free analysis is performed from inconsistent multiple magnetic field datasets. The use of simple tests as validation tools for the consistency assessment can help avoid such problems in order to extract more reliable information from spin relaxation experiments. In particular, these tests are useful for detecting inconsistencies arising from R2 data. Since such inconsistencies can yield artifactual Rex parameters within model-free analysis, these tests should be use routinely prior to any analysis such as model-free calculations. This script will allow one to calculate values for the three consistency tests J(0), F_eta and F_R2. Once this is done, qualitative analysis can be performed by comparing values obtained at different magnetic fields. Correlation plots and histograms are useful tools for such comparison, such as presented in Morin & Gagne (2009a) J. Biomol. NMR, 45: 361-372.

Other representations

  • angles.py Script for calculating the protein NH bond vector angles with respect to the diffusion tensor.
  • xh_vector_dist.py Script for creating a PDB representation of the distribution of XH bond vectors.
  • diff_tensor_pdb.py Script for creating a PDB representation of the Brownian rotational diffusion tensor.

Scripts - Part 2

We now try to setup things a little more efficient.

Relax is able to read previous results file, so let us divide the task up into:

  • 1: Load the data and save as state file. Inspect in GUI before running.
  • 2: Run the Model 1: local_tm.
  • 3: Here make 4 scripts. Each of them only depends on Model 1:
    • Model 2: sphere
    • Model 3: prolate
    • Model 4: oblate
    • Model 5: ellipsoid
  • 4: Make an intermediate 'final' model script. This will automatically detect files from above.

Prepare data

We make a new folder and try.

See commands
mkdir 20171010_model_free_2_HADDOCK
cp 20171010_model_free/*.dat 20171010_model_free_2_HADDOCK
cp 20171010_model_free/*.pdb 20171010_model_free_2_HADDOCK

# Get scripts
cd 20171010_model_free_2_HADDOCK
git init
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
git fetch
git checkout -t origin/master

And a new one, changing the NOE error

See commands
mkdir 20171010_model_free_3_HADDOCK
cp 20171010_model_free/*.dat 20171010_model_free_3_HADDOCK
cp 20171010_model_free/*.pdb 20171010_model_free_3_HADDOCK

# Get scripts
cd 20171010_model_free_3_HADDOCK
git init
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
git fetch
git checkout -t origin/master

# Change NOE error
sed -i 's/0.1*$/0.05/' NOE_600MHz_new.dat
sed -i 's/0.1*$/0.05/' NOE_750MHz.dat

And a new one, changing the NOE error, and deselecting N-terminal.
Consistency test, found that this stretch contained outliers.

See commands
mkdir 20171010_model_free_4_HADDOCK
cp 20171010_model_free/*.dat 20171010_model_free_4_HADDOCK
cp 20171010_model_free/*.pdb 20171010_model_free_4_HADDOCK

# Get scripts
cd 20171010_model_free_4_HADDOCK
git init
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
git fetch
git checkout -t origin/master

# Change NOE error
sed -i 's/0.1*$/0.05/' NOE_600MHz_new.dat
sed -i 's/0.1*$/0.05/' NOE_750MHz.dat

# Make deselection
echo "#" > deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t151" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t152" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t153" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t154" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t155" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t156" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt

And a new one, changing the NOE error, and deselecting spins found from consistency test.

See commands
mkdir 20171010_model_free_5_HADDOCK
cp 20171010_model_free/*.dat 20171010_model_free_5_HADDOCK
cp 20171010_model_free/*.pdb 20171010_model_free_5_HADDOCK

# Get scripts
cd 20171010_model_free_5_HADDOCK
git init
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
git fetch
git checkout -t origin/master

# Change NOE error
sed -i 's/0.1*$/0.05/' NOE_600MHz_new.dat
sed -i 's/0.1*$/0.05/' NOE_750MHz.dat

# Make deselection
echo "#" > deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t17" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t120" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t59" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t98" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t49" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t76" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t155" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t156" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t48" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t154" >> deselect.txt

And a new one, without changing the NOE error, and deselecting spins found from consistency test.

See commands
mkdir 20171010_model_free_6_HADDOCK
cp 20171010_model_free/*.dat 20171010_model_free_6_HADDOCK
cp 20171010_model_free/*.pdb 20171010_model_free_6_HADDOCK

# Get scripts
cd 20171010_model_free_6_HADDOCK
git init
git remote add origin git@github.com:tlinnet/relax_modelfree_scripts.git
git fetch
git checkout -t origin/master

# Make deselection
echo "#" > deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t17" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt

cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t59" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t98" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t76" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t155" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t156" >> deselect.txt 
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t120" >> deselect.txt

cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t49" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t48" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t154" >> deselect.txt

cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t33" >> deselect.txt
cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t67" >> deselect.txt

11_read_data_GUI_inspect.py - Read data GUI inspect

This will read the data and save as a state.

The GUI can be a good place to inspect the setup and files.

See content of: 11_read_data_GUI_inspect.py

Run with

relax 11_read_data_GUI_inspect.py -t 11_read_data_GUI_inspect.log

To check in GUI

  • relax -g
  • File -> Open relax state
  • In folder "result_10" open "result_10_ini.bz2"
  • View -> Data pipe editor
  • Right click on pipe, and select "Associate with a new auto-analysis"

relax 11_test_consistency.py - Consistency test of our data

Before running the analysis, it is wise to run a script for consistency testing.

See here:

Highlights:

  • Comparing results obtained at different magnetic fields should, in the case of perfect consistency and assuming the absence of conformational exchange, yield equal values independently of the magnetic field.
  • avoid the potential extraction of erroneous information as well as the waste of time associated to dissecting inconsistent datasets using numerous long model-free minimisations with different subsets of data.
  • The authors prefer the use of the spectral density at zero frequency J(0) alone since it does not rely on an estimation of the global correlation time tc/tm, neither on a measure of theta, the angle between the 15N–1H vector and the principal axis of the 15N chemical shift tensor. Hence, J(0) is less likely to be affected by incorrect parameterisation of input parameters.

See content of: 11_test_consistency.py

relax 11_test_consistency.py -t 11_test_consistency.py.log

#Afterwards, go into the folder at plot data.
python plot_txt_files.py
./grace2images.py

12_Model_1_I_local_tm.py - Only run local_tm

Now we only run Model 1.

  • DIFF_MODEL = ['local_tm']
  • GRID_INC = 11 # This is the standard
  • MC_NUM = 0 # This has no influence in Model 1-5
  • MAX_ITER = 20 # Stop if it has not converged in 20 rounds

Normally between 8 to 15 multiple rounds of optimisation of the are required for the proper execution of this script.
This is can also be see here in Figure 2.

Relax should stop calculation, if a model does not converge.

See content of: 12_Model_1_I_local_tm.py

We use tmux to make a terminal-session, we can get back to, if our own terminal connection get closed.

Run with

# Make terminal-session
tmux new -s m1

relax 12_Model_1_I_local_tm.py -t 12_Model_1_I_local_tm.log

# or
tmux new -s m1
mpirun -np 22 relax --multi='mpi4py' 12_Model_1_I_local_tm.py -t 12_Model_1_I_local_tm.log

You can then in another terminal follow the logfile by

less +F 12_Model_I_local_tm.log
  • To scroll up and down, use keyboard: Ctrl+c
  • To return to follow mode, use keyboard: Shift+f
  • To exit, use keyboard: Ctrl+c and then: q

13_Model_2-5 - Run Model 2 to 5

When Model 1 is completed, then make 4 terminal windows and run them at the same time.

These scripts do:

  • Read the state file from before with setup
  • Change DIFF_MODEL accordingly

13_Model_2_II_sphere.py

tmux new -s m2
relax 13_Model_2_II_sphere.py -t 13_Model_2_II_sphere.log
# Or
mpirun -np 5 relax --multi='mpi4py' 13_Model_2_II_sphere.py -t 13_Model_2_II_sphere.log

# When relax is running, push: Ctrl+b and then d, to disconnect without exit

13_Model_3_III_prolate.py

tmux new -s m3
relax 13_Model_3_III_prolate.py -t 13_Model_3_III_prolate.log
# Or
mpirun -np 5 relax --multi='mpi4py' 13_Model_3_III_prolate.py -t 13_Model_3_III_prolate.log

13_Model_4_IV_oblate.py

tmux new -s m4
relax 13_Model_4_IV_oblate.py -t 13_Model_4_IV_oblate.log
# Or
mpirun -np 5 relax --multi='mpi4py' 13_Model_4_IV_oblate.py -t 13_Model_4_IV_oblate.log

13_Model_5_V_ellipsoid.py

tmux new -s m5
relax 13_Model_5_V_ellipsoid.py -t 13_Model_5_V_ellipsoid.log
# Or
mpirun -np 5 relax --multi='mpi4py' 13_Model_5_V_ellipsoid.py -t 13_Model_5_V_ellipsoid.log

To join session

# List
tmux list-s

# Join either
tmux a -t m1
tmux a -t m2
tmux a -t m3
tmux a -t m4
tmux a -t m5

14_intermediate_final.py - Inspection during model optimization

During running of model 2-5, the current results can be inspected with this nifty scripts.

The script will ask for input of MC numbers. So just run it.

14_intermediate_final.py

tmux new -s final
relax 14_intermediate_final.py -t 14_intermediate_final.log

This does:

  • Option: Collect current best result from Model 2-5, and make MC simulations, and finalize to get current results files
  • Make a pymol file, that collects all of relax pymol command files into 1 pymol session
  • Option: Collect all chi2 and number of params k, for each iteration per model
    • Make a python plot file for plotting this results

Per iteration get: chi2, k, tm

Afterwards, plot the data.

python results_collected.py

Pymol macro

You also get a pymol folder.

See here for info how the macro is applied

Run with

pymol 0_0_apply_all_pymol_commands.pml

To run on Haddock

Have a look here, how to get standalone python Anaconda linux. Also have a look here OpenMPI.

# SSH in
ssh haddock

# Test with shell
mpirun -np 6 echo "hello world"

# Test with python
mpirun -np 6 python -m mpi4py helloworld

# Test with relax
mpirun -np 6 relax --multi='mpi4py'
# Look for: Processor fabric:  MPI 2.2 running via mpi4py with 5 slave processors & 1 master.  Using MPICH2 1.4.1.

Now we run 04_run_default_with_tolerance_lim.py with more power!
We use tmux to make a terminal-session, we can get back to, if our own terminal connection get closed.

  • start a new session: tmux
  • re-attach a detached session: tmux attach
# Make terminal-session
tmux

# Start relax
mpirun -np 20 relax --multi='mpi4py' 04_run_default_with_tolerance_lim.py -t 04_run_default_with_tolerance_lim.log

Useful commands to log file

While the analysis is running, these commands could be used to check the logfile for errors

### Check convergence 
# For chi2
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Chi-squared test:"

# For other tests
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical "
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical model-free models test:"
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical diffusion tensor parameter test:"
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical model-free parameter test:"

# To look for not converged errors
# For chi2
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The chi-squared value has not converged."

# For other tests
cat 04_run_default_with_tolerance_lim.log | grep -B 7 " have not converged."
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The model-free models have not converged."
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The diffusion parameters have not converged."
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The model-free parameters have not converged."

You can then inspect the logfile by less: 10-tips for less

less 04_run_default_with_tolerance_lim.log

To find pattern: We have to escape with \ for special character like: ()[] etc.

# Search forward
/Value \(iter 14\)
/The chi-squared value has not converged

n or N – for next match in forward / previous match in backward

  • To return to follow mode, use keyboard: Shift+f
  • To exit, use keyboard: Ctrl+c and then: q

rsync files

rsync files after completion to Sauron

When a run is completed, then sync files to Sauron file server.

Make a rsync_to_sbinlab.sh file with content

See file content
#!/bin/bash

read -p "Username on sauron :" -r

RUSER=$REPLY
SAURON=10.61.4.60
PROJ=`basename "$PWD"`

FROM=${PWD}
TO=${RUSER}@${SAURON}:/data/sbinlab2/${RUSER}/Downloads

# -a: "archive"- archive mode; equals -rlptgoD (no -H,-A,-X). syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
# We want to remove the -o and -g options:
# -o, --owner                 preserve owner (super-user only)
# -g, --group                 preserve group
# -rlptD : Instead or
# -a --no-o --no-g  
# -z: Compression over network
# -P: It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
# -h, Output numbers in a more human-readable format.

# Always double-check your arguments before executing an rsync command.
# -n 

echo "I will now do a DRY RUN, which does not move files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""

if [[ $REPLY =~ ^[Yy]$ ]]; then
  rsync -rlptDPzh -n ${FROM} ${TO} 
else
  echo "Not doing DRY RUN"
fi

echo ""

echo "I will now do the sync of files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""

if [[ $REPLY =~ ^[Yy]$ ]]; then
  rsync -rlptDPzh ${FROM} ${TO}
else
  echo "Not doing anything"
fi

Make it executable and run

chmod +x rsync_to_sbinlab.sh

#run
./rsync_to_sbinlab2.sh

rsync files from BIO to home mac

To inspect from home mac.

Make a rsync_from_bio_to_home.sh file with content

See file content
#!/bin/bash
 
read -p "Username on bio:" -r
 
RUSER=$REPLY
BIO=ssh-bio.science.ku.dk

#PROJ=Desktop/kaare_relax
PROJ=Desktop/kaare_relax/20171010_model_free_HADDOCK
PROJDIR=`basename "$PROJ"`

FROM=${RUSER}@${BIO}:/home/${RUSER}/${PROJ} 
TO=${PWD}/${PROJDIR}

# -a: "archive"- archive mode; equals -rlptgoD (no -H,-A,-X). syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
# We want to remove the -o and -g options:
# -o, --owner                 preserve owner (super-user only)
# -g, --group                 preserve group
# -rlptD : Instead or
# -a --no-o --no-g  
# -z: Compression over network
# -P: It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
# -h, Output numbers in a more human-readable format.
 
# Always double-check your arguments before executing an rsync command.
# -n 
 
echo "I will now do a DRY RUN, which does not move files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""
 
if [[ $REPLY =~ ^[Yy]$ ]]; then
  rsync -rlptDPzh -n ${FROM} ${TO} 
else
  echo "Not doing DRY RUN"
fi
 
echo ""
 
echo "I will now do the sync of files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""
 
if [[ $REPLY =~ ^[Yy]$ ]]; then
  rsync -rlptDPzh ${FROM} ${TO}
else
  echo "Not doing anything"
fi

Make it executable and run

chmod +x rsync_from_bio_to_home.sh

#run
./rsync_from_bio_to_home.sh

About the protocol

Model I - 'local_tm'
This will optimise the diffusion model whereby all spin of the molecule have a local tm value, i.e. there is no global diffusion tensor. This model needs to be optimised prior to optimising any of the other diffusion models. Each spin is fitted to the multiple model-free models separately, where the parameter tm is included in each model.

Model II - 'sphere'
This will optimise the isotropic diffusion model. Multiple steps are required, an initial optimisation of the diffusion tensor, followed by a repetitive optimisation until convergence of the diffusion tensor. In the relax script UI each of these steps requires this script to be rerun, unless the conv_loop flag is True. In the GUI (graphical user interface), the procedure is repeated automatically until convergence. For the initial optimisation, which will be placed in the directory './sphere/init/', the following steps are used:

  • The model-free models and parameter values for each spin are set to those of diffusion model MI.
  • The local tm parameter is removed from the models.
  • The model-free parameters are fixed and a global spherical diffusion tensor is minimised
  • For the repetitive optimisation, each minimisation is named from 'round_1' onwards. The initial 'round_1' optimisation will extract the diffusion tensor from the results file in './sphere/init/', and the results will be placed in the directory './sphere/round_1/'. Each successive round will take the diffusion tensor from the previous round. The following steps are used:
    • The global diffusion tensor is fixed and the multiple model-free models are fitted to each spin.
    • AIC model selection is used to select the models for each spin.
    • All model-free and diffusion parameters are allowed to vary and a global optimisation of all parameters is carried out.

Model III - 'prolate'
The methods used are identical to those of diffusion model MII, except that an axially symmetric diffusion tensor with Da >= 0 is used. The base directory containing all the results is './prolate/'.

Model IV -'oblate'
The methods used are identical to those of diffusion model MII, except that an axially symmetric diffusion tensor with Da <= 0 is used. The base directory containing all the results is './oblate/'.

Model V - 'ellipsoid'
The methods used are identical to those of diffusion model MII, except that a fully anisotropic diffusion tensor is used (also known as rhombic or asymmetric diffusion). The base directory is './ellipsoid/'

'final'
Once all the diffusion models have converged, the final run can be executed. This is done by setting the variable diff_model to 'final'. This consists of two steps, diffusion tensor model selection, and Monte Carlo simulations. Firstly AIC model selection is used to select between the diffusion tensor models. Monte Carlo simulations are then run solely on this selected diffusion model. Minimisation of the model is bypassed as it is assumed that the model is already fully optimised (if this is not the case the final run is not yet appropriate). The final black-box model-free results will be placed in the file 'final/results'.

See also