Trusted, Bureaucrats
1,382
edits
Changes
This is a tutorial for Lau and Kaare in SBiNLab, and hopefully others.
To get inspiration of example scripts files and '''see''' how the protocol is performed, have a look here:
* [https://github.com/nmr-relax-code/test_suiterelax/system_testsblob/scriptsmaster/model_freeauto_analyses/dauvergne_protocol.py* nmr-relax-code/auto_analyses/dauvergne_protocol.py]
=== 01 - Test load of PDB =Other script inspiration for checking ==First we just want to test to read the PDB fileThe distribution of relax includes a folder '''sample_scripts/''' which contain a folder with scripts for analysis.
</source>
|}
<source lang="bash">
</source>
|}
And a new one, without changing the NOE error, and deselecting spins found from consistency test.<br>
{| class="mw-collapsible mw-collapsed wikitable"
! Output from logfileSee commands
|-
|
<source lang="bash">
# relax module importsGet scriptscd 20171010_model_free_6_HADDOCKgit initgit remote add origin git@github.com:tlinnet/relax_modelfree_scripts.gitfrom auto_analyses.dauvergne_protocol import dAuvergne_protocolgit fetchgit checkout -t origin/master
# Set up the data pipeMake deselectionecho "#" > deselect.txtcat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t158" >> deselect.txtcat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t157" >> deselect.txtcat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t17" >> deselect.txt#######################cat R1_600MHz_new_model_free.dat | grep -P "ArcCALD\t159" >> deselect.txt
Run with<source lang="bash">relax11_read_data_GUI_inspect.py -t 11_read_data_GUI_inspect.log</source> pipe.create(pipe_name='origin', pipe_type='mf', bundle='mf (Fri Oct 13 17:44:18 2017)')
To check in GUI* relax-g* File -> structureOpen relax state* In folder "result_10" open "result_10_ini.read_pdb(file='energy_1.pdb', dir=None, read_mol=None, set_mol_name='ArcCALD', read_model=1, set_model_num=None, alt_loc=None, verbosity=1bz2"* View -> Data pipe editor* Right click on pipe, merge=False)and select "Associate with a new auto-analysis"
# mol_name res_num res_name spin_num spin_name Afterwards, go into the folder at plot data.ArcCALD 33 TRP 517 NE1 python plot_txt_files.pyArcCALD 48 TRP 762 NE1 ./grace2images.py ArcCALD 49 TRP 786 NE1 ArcCALD 59 TRP 954 NE1 ArcCALD 98 TRP 1609 NE1 </source>
* DIFF_MODEL = ['local_tm']* GRID_INC = 11 # mol_name res_num res_name spin_num spin_name This is the standardArcCALD 3 GLY 29 H ArcCALD 4 LEU 36 H ArcCALD * MC_NUM = 0 # This has no influence in Model 1-5 ASP 55 H ArcCALD 6 THR 67 H ArcCALD 7 GLN 81 H ArcCALD 8 ILE 98 H ArcCALD 9 PHE 117 H ArcCALD 10 GLU 137 H ArcCALD 11 ASP 152 H ArcCALD 13 ARG 178 H ArcCALD 14 GLU 202 H ArcCALD 15 PHE 217 H ArcCALD 16 LEU 237 H ArcCALD 17 SER 256 H ArcCALD 18 HIS 267 H ArcCALD 19 LEU 285 H ArcCALD * MAX_ITER = 20 # Stop if it has not converged in 20 GLU 304 H ArcCALD 21 GLU 319 H ArcCALD 22 TYR 334 H ArcCALD 23 LEU 355 H ArcCALD 24 ARG 374 H ArcCALD 25 GLN 398 H ArcCALD 26 VAL 415 H ArcCALD 27 GLY 431 H ArcCALD 28 GLY 438 H ArcCALD 29 SER 445 H ArcCALD 30 GLU 456 H ArcCALD 31 GLU 471 H ArcCALD 32 TYR 486 H ArcCALD 33 TRP 507 H ArcCALD 34 LEU 531 H ArcCALD 35 SER 550 H ArcCALD 36 GLN 561 H ArcCALD 37 ILE 578 H ArcCALD 38 GLN 597 H ArcCALD 39 ASN 614 H ArcCALD 40 HIS 628 H ArcCALD 41 MET 646 H ArcCALD 42 ASN 663 H ArcCALD 43 GLY 677 H ArcCALD 45 ALA 698 H ArcCALD 46 LYS 708 H ArcCALD 47 LYS 730 H ArcCALD 48 TRP 752 H ArcCALD 49 TRP 776 H ArcCALD 50 GLU 800 H ArcCALD 51 PHE 815 H ArcCALD 52 LYS 835 H ArcCALD 53 GLN 857 H ArcCALD 54 GLY 874 H ArcCALD 55 SER 881 H ArcCALD 56 VAL 892 H ArcCALD 57 LYS 908 H ArcCALD 58 ASN 930 H ArcCALD 59 TRP 944 H ArcCALD 60 VAL 968 H ArcCALD 61 GLU 984 H ArcCALD 62 PHE 999 H ArcCALD 63 LYS 1019 H ArcCALD 64 LYS 1041 H ArcCALD 65 GLU 1063 H ArcCALD 66 PHE 1078 H ArcCALD 67 LEU 1098 H ArcCALD 68 GLN 1117 H ArcCALD 69 TYR 1134 H ArcCALD 70 SER 1155 H ArcCALD 71 GLU 1166 H ArcCALD 72 GLY 1181 H ArcCALD 73 THR 1188 H ArcCALD 74 LEU 1202 H ArcCALD 75 SER 1221 H ArcCALD 76 ARG 1232 H ArcCALD 77 GLU 1256 H ArcCALD 78 ALA 1271 H ArcCALD 79 ILE 1281 H ArcCALD 80 GLN 1300 H ArcCALD 81 ARG 1317 H ArcCALD 82 GLU 1341 H ArcCALD 83 LEU 1356 H ArcCALD 84 ASP 1375 H ArcCALD 85 LEU 1387 H ArcCALD 87 GLN 1420 H ArcCALD 88 LYS 1437 H ArcCALD 89 GLN 1459 H ArcCALD 90 GLY 1476 H ArcCALD 91 GLU 1483 H ArcCALD 93 LEU 1512 H ArcCALD 94 ASP 1531 H ArcCALD 95 GLN 1543 H ArcCALD 96 PHE 1560 H ArcCALD 97 LEU 1580 H ArcCALD 98 TRP 1599 H ArcCALD 99 ARG 1623 H ArcCALD 100 LYS 1647 H ArcCALD 101 ARG 1669 H ArcCALD 102 ASP 1693 H ArcCALD 103 LEU 1705 H ArcCALD 104 TYR 1724 H ArcCALD 105 GLN 1745 H ArcCALD 106 THR 1762 H ArcCALD 107 LEU 1776 H ArcCALD 108 TYR 1795 H ArcCALD 109 VAL 1816 H ArcCALD 110 ASP 1832 H ArcCALD 111 ALA 1844 H ArcCALD 112 GLU 1854 H ArcCALD 113 GLU 1869 H ArcCALD 114 GLU 1884 H ArcCALD 115 GLU 1899 H ArcCALD 116 ILE 1914 H ArcCALD 117 ILE 1933 H ArcCALD 118 GLN 1952 H ArcCALD 119 TYR 1969 H ArcCALD 120 VAL 1990 H ArcCALD 121 VAL 2006 H ArcCALD 122 GLY 2022 H ArcCALD 123 THR 2029 H ArcCALD 124 LEU 2043 H ArcCALD 125 GLN 2062 H ArcCALD 127 LYS 2093 H ArcCALD 128 PHE 2115 H ArcCALD 129 LYS 2135 H ArcCALD 130 ARG 2157 H ArcCALD 131 PHE 2181 H ArcCALD 132 LEU 2201 H ArcCALD 133 ARG 2220 H ArcCALD 134 HIS 2244 H ArcCALD 136 LEU 2276 H ArcCALD 138 LYS 2309 H ArcCALD 139 THR 2331 H ArcCALD 140 LEU 2345 H ArcCALD 141 GLU 2364 H ArcCALD 142 GLN 2379 H ArcCALD 143 LEU 2396 H ArcCALD 144 ILE 2415 H ArcCALD 145 GLN 2434 H ArcCALD 146 ARG 2451 H ArcCALD 147 GLY 2475 H ArcCALD 148 MET 2482 H ArcCALD 149 GLU 2499 H ArcCALD 150 VAL 2514 H ArcCALD 151 GLN 2530 H ArcCALD 152 ASP 2547 H ArcCALD 153 GLY 2559 H ArcCALD 154 LEU 2566 H ArcCALD 155 GLU 2585 H ArcCALD 156 GLN 2600 H ArcCALD 157 ALA 2617 H ArcCALD 158 ALA 2627 H ArcCALD 159 GLU 2637 H rounds
Run with
<source lang="bash">
# Make terminal-session
tmux new -s m1
relax 12_Model_1_I_local_tm.py -t 12_Model_1_I_local_tm.log
# or
tmux new -s m1
mpirun -np 22 relax --multi='mpi4py' 12_Model_1_I_local_tm.py -t 12_Model_1_I_local_tm.log
</source>
You can then in another terminal follow the logfile by
<source lang="bash">
less +F 12_Model_I_local_tm.log
</source>
* To scroll up and down, use keyboard: '''Ctrl+c'''* To return to follow mode, use keyboard: '''Shift+f'''* To exit, use keyboard: '''Ctrl+c''' and then: '''q''' === 01 13_Model_2-5 - Test load of data Run Model 2 to 5 ==When Model 1 is completed, then make 4 terminal windows and run them at the same time. These scripts do:* Read the state file from before with setup* Change DIFF_MODEL accordingly [https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_2_II_sphere.py 13_Model_2_II_sphere.py]<source lang="bash">tmux new -s m2relax 13_Model_2_II_sphere.py -t 13_Model_2_II_sphere.log# OrThat looked to go fine, so let us try to just load datampirun -np 5 relax --multi='mpi4py' 13_Model_2_II_sphere.py -t 13_Model_2_II_sphere.log
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_3_III_prolate.py 13_Model_3_III_prolate.py]<source lang="bash">tmux new -s m3relax 13_Model_3_III_prolate.py -t 13_Model_3_III_prolate.log# Ormpirun -np 5 relax module imports--multi='mpi4py' 13_Model_3_III_prolate.py -t 13_Model_3_III_prolate.logfrom auto_analyses.dauvergne_protocol import dAuvergne_protocol</source>
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_4_IV_oblate.py 13_Model_4_IV_oblate.py]<source lang="bash">tmux new -s m4relax 13_Model_4_IV_oblate.py -t 13_Model_4_IV_oblate.log# Set up the data pipeOrmpirun -np 5 relax --multi='mpi4py' 13_Model_4_IV_oblate.py -t 13_Model_4_IV_oblate.log#######################</source>
[https://github.com/tlinnet/relax_modelfree_scripts/blob/master/13_Model_5_V_ellipsoid.py 13_Model_5_V_ellipsoid.py]<source lang="bash">tmux new -s m5relax 13_Model_5_V_ellipsoid.py -t 13_Model_5_V_ellipsoid.log# The following sequence of user function calls can be changed as neededOrmpirun -np 5 relax --multi='mpi4py' 13_Model_5_V_ellipsoid.py -t 13_Model_5_V_ellipsoid.log</source>
# Load the PDB file.Join eitherstructure.read_pdb('energy_1.pdb', set_mol_name='ArcCALD', read_model=1)tmux a -t m1tmux a -t m2tmux a -t m3tmux a -t m4tmux a -t m5</source>
</source>
=== Pymol macro ===
You also get a pymol folder.
See here for info how the macro is applied
* [http://www.nmr-relax.com/manual/molmol_macro_apply.html#SECTION081284600000000000000 Summary of parameter meaning and value to pymol visualization]
Run with
<source lang="bash">
</source>
= To run on Haddock =
Have a look here, how to get standalone python [[Anaconda_linux_mac|Anaconda linux]].
Also have a look here [[OpenMPI]].
<source lang="bash">
# SSH in
ssh haddock
# Test with shell
mpirun -np 6 echo "hello world"
# Test with python
mpirun -np 6 python -m mpi4py helloworld
# Test with relax
mpirun -np 6 relax --multi='mpi4py'
# Look for: Processor fabric: MPI 2.2 running via mpi4py with 5 slave processors & 1 master. Using MPICH2 1.4.1.
</source>
Now we run '''04_run_default_with_tolerance_lim.py''' with more power!<br>
We use [http://www.dayid.org/comp/tm.html tmux] to make a terminal-session, we can get back to,
if our own terminal connection get closed.
* start a new session: '''tmux'''
* re-attach a detached session: '''tmux attach'''
<source lang="bash">
# Make terminal-session
tmux
# Start relax
mpirun -np 20 relax --multi='mpi4py' 04_run_default_with_tolerance_lim.py -t 04_run_default_with_tolerance_lim.log
</source>
= Useful commands to log file =
While the analysis is running, these commands could be used to check the logfile for errors
<source lang="bash">
### Check convergence
# For chi2
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Chi-squared test:"
# For other tests
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical "
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical model-free models test:"
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical diffusion tensor parameter test:"
cat 04_run_default_with_tolerance_lim.log | grep -A 10 "Identical model-free parameter test:"
# To look for not converged errors
# For chi2
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The chi-squared value has not converged."
# For other tests
cat 04_run_default_with_tolerance_lim.log | grep -B 7 " have not converged."
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The model-free models have not converged."
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The diffusion parameters have not converged."
cat 04_run_default_with_tolerance_lim.log | grep -B 7 "The model-free parameters have not converged."
</source>
You can then inspect the logfile by '''less''': [http://www.thegeekstuff.com/2010/02/unix-less-command-10-tips-for-effective-navigation 10-tips for less]
<source lang="bash">
less 04_run_default_with_tolerance_lim.log
</source>
'''To find pattern:''' We have to escape with '''\''' for special character like: ()[] etc.
<source lang="bash">
# Search forward
/Value \(iter 14\)
/The chi-squared value has not converged
</source>
'''n''' or '''N''' – for next match in forward / previous match in backward
* To return to follow mode, use keyboard: '''Shift+f'''
* To exit, use keyboard: '''Ctrl+c''' and then: '''q'''
= rsync files =
== rsync files after completion to Sauron ==
When a run is completed, then sync files to Sauron file server.
Make a '''rsync_to_sbinlab.sh''' file with content
{| class="mw-collapsible mw-collapsed wikitable"
! Output from logfileSee file content
|-
|
<source lang="bash">
#!/bin/bash
read -p "Username on sauron :" -r
RUSER=$REPLY
SAURON=10.61.4.60
PROJ=`basename "$PWD"`
FROM=${PWD}
TO=${RUSER}@${SAURON}:/data/sbinlab2/${RUSER}/Downloads
# -a: "archive"- archive mode; equals -rlptgoD (no -H,-A,-X). syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
# We want to remove the -o and -g options:
# -o, --owner preserve owner (super-user only)
# -g, --group preserve group
# -rlptD : Instead or
# -a --no-o --no-g
# -z: Compression over network
# -P: It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
# -h, Output numbers in a more human-readable format.
# Always double-check your arguments before executing an rsync command.
# -n
echo "I will now do a DRY RUN, which does not move files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""
if [[ $REPLY =~ ^[Yy]$ ]]; then
rsync -rlptDPzh -n ${FROM} ${TO}
else
echo "Not doing DRY RUN"
fi
echo ""
echo "I will now do the sync of files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""
if [[ $REPLY =~ ^[Yy]$ ]]; then
rsync -rlptDPzh ${FROM} ${TO}
else
echo "Not doing anything"
fi
</source>
Make it executable and run
<source lang="bash">
chmod +x rsync_to_sbinlab.sh
#run
./rsync_to_sbinlab2.sh
</source>
|}
== rsync files from BIO to home mac ==
To inspect from home mac.
Make a '''rsync_from_bio_to_home.sh''' file with content
{| class="mw-collapsible mw-collapsed wikitable"
! See file content
|-
|
<source lang="bash">
#!/bin/bash
read -p "Username on bio:" -r
RUSER=$REPLY
BIO=ssh-bio.science.ku.dk
#PROJ=Desktop/kaare_relax
PROJ=Desktop/kaare_relax/20171010_model_free_HADDOCK
PROJDIR=`basename "$PROJ"`
FROM=${RUSER}@${BIO}:/home/${RUSER}/${PROJ}
TO=${PWD}/${PROJDIR}
# -a: "archive"- archive mode; equals -rlptgoD (no -H,-A,-X). syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.
# We want to remove the -o and -g options:
# -o, --owner preserve owner (super-user only)
# -g, --group preserve group
# -rlptD : Instead or
# -a --no-o --no-g
# -z: Compression over network
# -P: It combines the flags --progress and --partial. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:
# -h, Output numbers in a more human-readable format.
# Always double-check your arguments before executing an rsync command.
# -n
echo "I will now do a DRY RUN, which does not move files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""
if [[ $REPLY =~ ^[Yy]$ ]]; then
rsync -rlptDPzh -n ${FROM} ${TO}
else
echo "Not doing DRY RUN"
fi
echo ""
echo "I will now do the sync of files"
read -p "Are you sure? y/n :" -n 1 -r
echo ""
if [[ $REPLY =~ ^[Yy]$ ]]; then
rsync -rlptDPzh ${FROM} ${TO}
else
echo "Not doing anything"
fi
</source>
Make it executable and run
<source lang="bash">
chmod +x rsync_from_bio_to_home.sh
#run
./rsync_from_bio_to_home.sh
</source>
|}
=About the protocol = '''Model I - 'local_tm''''<br>This will optimise the diffusion model whereby all spin of the molecule have a local tm value, i.e. there is no global diffusion tensor. This model needs to be optimised prior to optimising any of the other diffusion models. Each spin is fitted to the multiple model-free models separately, where the parameter tm is included in each model. '''Model II - 'sphere''''<br>This will optimise the isotropic diffusion model. Multiple steps are required, an initial optimisation of the diffusion tensor, followed by a repetitive optimisation until convergence of the diffusion tensor. In the relax script UI each of these steps requires this script to be rerun, unless the conv_loop flag is True. In the GUI (graphical user interface), the procedure is repeated automatically until convergence. For the initial optimisation, which will be placed in the directory './sphere/init/', the following steps are used:* The model-free models and parameter values for each spin are set to those of diffusion model MI.* The local tm parameter is removed from the models.* The model-free parameters are fixed and a global spherical diffusion tensor is minimised* For the repetitive optimisation, each minimisation is named from 'round_1' onwards. The initial 'round_1' optimisation will extract the diffusion tensor from the results file in './sphere/init/', and the results will be placed in the directory './sphere/round_1/'. Each successive round will take the diffusion tensor from the previous round. The following steps are used:** The global diffusion tensor is fixed and the multiple model-free models are fitted to each spin.** AIC model selection is used to select the models for each spin.** All model-free and diffusion parameters are allowed to vary and a global optimisation of all parameters is carried out. '''Model III - 'prolate''''<br>The methods used are identical to those of diffusion model MII, except that an axially symmetric diffusion tensor with Da >= 0 is used. The base directory containing all the results is './prolate/'. '''Model IV -'oblate''''<br>The methods used are identical to those of diffusion model MII, except that an axially symmetric diffusion tensor with Da <= 0 is used. The base directory containing all the results is './oblate/'. '''Model V - 'ellipsoid''''<br>The methods used are identical to those of diffusion model MII, except that a fully anisotropic diffusion tensor is used (also known as rhombic or asymmetric diffusion). The base directory is './ellipsoid/' ''''final''''<br>Once all the diffusion models have converged, the final run can be executed. This is done by setting the variable diff_model to 'final'. This consists of two steps, diffusion tensor model selection, and Monte Carlo simulations. Firstly AIC model selection is used to select between the diffusion tensor models. Monte Carlo simulations are then run solely on this selected diffusion model. Minimisation of the model is bypassed as it is assumed that the model is already fully optimised (if this is not the case the final run is not yet appropriate).The final black-box model-free results will be placed in the file 'final/results'. = See also ==
[[Category:Tutorials]]
[[Category:Model-free_analysis]]