Trusted, Bureaucrats
4,228
edits
Changes
Jump to navigation
Jump to search
== Script Problems with "the cloud"* Very time consuming to install =="run an online" cloud.* The benefits of cloud.sagemath.com are well explained here: http://www.sagemath.org/library-why.html
python 1_prepare_dataPython 2.7.9 (default, Apr 2 2015, 15:33:21)[GCC 4.9.2] on linux2Type "help", "copyright", "credits" or "license" for more information.py
The script isPaste in:<source lang="python">try: import wx print wx.__file__ print wx.version() print wx.__version__except ImportError: print "no wx module" try: import bmrblib print bmrblib.__file__ print bmrblib.__version__except ImportError: print "no bmrblib module" try: import minfx print minfx.__file__ print minfx.__version__except ImportError: print "no minfx module"
{{collapsible script
| type = Python 2/3 script
| title = The <code>1_prepare_data.py</code> script.
| lang = python
| script =
→Is subversion available: Switched from svn to git.
The [https://cloud.sagemath.com/ SageMathCloud] is a cloud computing platform designed for computational mathematics that is a part of the [https://en.wikipedia.org/wiki/SageMath Sage project].
== Purpose ==
Why ?
* The first installation of '''relax''' can be an energy barrier
* It could be of interest to make relax available in "the cloud" for the first analysis, so time is not used on solving software dependencies
Other info
* The pricing is here: https://cloud.sagemath.com/policies/pricing.html
* There is also the Bio-NMR initiative: http://www.bio-nmr.net
* And a web service with all NMR software - WeNMR: http://www.wenmr.eu
== Possibilities with cloud.sagemath.com ==
cloud.sagemath.com currently does not provide admin account or dedicated SMC servers. <br>
A subscription can be purchased for $49 per month (or $499 per year), one can apply the quotas listed here to any projects:
<source lang="text">
* 16 upgrades Member hosting
* 40 upgrades Network access
* 8 days Idle timeout
* 24 GB Memory
* 40 GB Disk space
* 4 shares CPU shares
* 2 cores CPU cores
</source>
'''The procedure should be'''
* A '''new interested scientist''' would go to SMC, create a project.
* Add '''relax''' as a collaborator to their free project, and '''we''' would click a button to upgrade their project to '''member hosting'''.
* This add '''network access''', increase the cpu shares, etc.
* "We" have a page which lists, for each project, what upgrades "we" have applied
* "We" can also see if the project is being used.
* "We" can remove upgrades at any time (in any granularity).
'''The value of upgrading to "member hosting" is:'''
* Computation is '''much''' faster. The free pool of computers are typically '''massively''' loaded during the week.
* The "free servers" work fine, but they can easily be 15% of the speed of the members-only machines.
* With the $49/month subscription, "we" get to move 16 projects to members-only machines.
=== Installation and analysis in the cloud ===
'''Installation of relax in cloud.sagemath.com'''
* Instead of making "relax" available for all users on cloud.sagemath.com, each user should install their local copy of relax
* When using svn to checkout relax, there will be a: install_relax_cloud_sagemath_com.sh bash script
* The user now have '''full control''' over any version of relax
* The user can potentially "develop" online with subversion
'''To do Analysis'''
* Use ssh to send data to the server
** Supported right now, even for free projects. See the button "ssh to your project" in project settings.
* The uploaded data for the researcher is private:
** Even for free projects, data in projects is only visible to collaborators on that project and nobody else.
** SMC is not like github, where free = public by default.
** With SMC everything is private by default, even for free users.
** However, people can also explicitly share any file or directory tree, and that makes it publicly available
** Could be a way to share data related to a paper or research project
* Use relax in the terminal in the server
** Fine for free projects.
== The test project at cloud.sagemath.com ==
This audit was performed: '''2015/11/24'''
===Project location===
<source lang="bash">
https://cloud.sagemath.com/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/files/test%20ipyth.ipynb
</source>
===Project resources===
<source lang="bash">
Disk space: 3000 MB disk space available - 59 MB used 3000 MB given by free project
Memory: 1000 MB RAM memory available - 133 MB used 1000 MB given by free project
CPU cores:1 core 1 core given by free project
CPU shares: 1 share 1 share given by free project
Idle timeout: 1 hour of non-interactive use before project stops 1 hour given by free project
Network access: Yes 1 upgrade given by free project
Member hosting: No 0 upgrades given by free project
</source>
==Software at cloud.sagemath.com==
===Is git available===
<source lang="bash">
~$ git --version
git version 2.7.6
</source>
===Is scons available===
<source lang="bash">
~$ which scons
/usr/bin/scons
~$ scons --version
SCons by Steven Knight et al.:
script: v2.3.1, 2014/03/02 14:18:15, by garyo on lubuntu
engine: v2.3.1, 2014/03/02 14:18:15, by garyo on lubuntu
engine path: ['/usr/lib/scons/SCons']
</source>
===Is xmgrace available===
<source lang="bash">
~$ which xmgrace
/usr/bin/xmgrace
</source>
===Is mpirun available===
<source lang="bash">
~$ which mpicc
/usr/bin/mpicc
~$ which mpiexec
/usr/bin/mpiexec
~$ which mpirun
/usr/bin/mpirun
~$ mpirun --version
mpirun (Open MPI) 1.6.5
Report bugs to http://www.open-mpi.org/community/help/
~$ mpirun --report-bindings -np 4 echo "hello world"
hello world
hello world
hello world
hello world
</source>
===Server ===
<source lang="bash">
~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping: 0
CPU MHz: 2299.972
BogoMIPS: 4599.94
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 46080K
NUMA node0 CPU(s): 0-3
</source>
==Python packages at cloud.sagemath.com==
Start Python:
<source lang="bash">
python
</source>
The current Python version:
<source lang="bash">
</source>
try:
import mpi4py
print mpi4py.__file__
print mpi4py.__version__
except ImportError:
print "no mpi4py module"
</source>
Output:
<source lang="python">
>>> try:
... import wx
... print wx.__file__
... print wx.version()
... print wx.__version__
... except ImportError:
... print "no wx module"
...
no wx module
>>> try:
... import bmrblib
... print bmrblib.__file__
... print bmrblib.__version__
... except ImportError:
... print "no bmrblib module"
...
no bmrblib module
>>> try:
... import minfx
... print minfx.__file__
... print minfx.__version__
... except ImportError:
... print "no minfx module"
...
no minfx module
>>> try:
... import mpi4py
... print mpi4py.__file__
... print mpi4py.__version__
... except ImportError:
... print "no mpi4py module"
...
no mpi4py module
</source>
== Install Python modules ==
===Bmrblib===
See [[Bmrblib]]
{{#tag:source|
v={{current version bmrblib}}
mkdir -p $HOME/Downloads
cd $HOME/Downloads
curl http://download.gna.org/bmrblib/bmrblib-$v.tar.gz -o bmrblib-$v.tar.gz
tar -xzf bmrblib-$v.tar.gz
cd bmrblib-$v
pip install .
cd ..
|lang="bash"
}}
===Minfx===
See [[Minfx]]
{{#tag:source|
v={{current version minfx}}
mkdir -p $HOME/Downloads
cd $HOME/Downloads
curl http://download.gna.org/minfx/minfx-$v.tar.gz -o minfx-$v.tar.gz
tar -xzf minfx-$v.tar.gz
cd minfx-$v
pip install .
cd ..
|lang="bash"
}}
===Mpi4py===
See [[OpenMPI#Install_mpi4py]]
{{#tag:source|
v={{current version mpi4py}}
mkdir -p $HOME/Downloads
cd $HOME/Downloads
wget https://bitbucket.org/mpi4py/mpi4py/downloads/mpi4py-$v.tar.gz
tar -xzf mpi4py-$v.tar.gz
cd mpi4py-$v
pip install .
cd ..
|lang="bash"
}}
===Test Python packages===
Paste in:
<source lang="python">
try:
import bmrblib
print bmrblib.__file__
print bmrblib.__version__
except ImportError:
print "no bmrblib module"
try:
import minfx
print minfx.__file__
print minfx.__version__
except ImportError:
print "no minfx module"
try:
import mpi4py
print mpi4py.__file__
print mpi4py.__version__
except ImportError:
print "no mpi4py module"
</source>
Output:
<source lang="python">
>>> try:
... import bmrblib
... print bmrblib.__file__
... print bmrblib.__version__
... except ImportError:
... print "no bmrblib module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/bmrblib/__init__.pyc
1.0.4
>>> try:
... import minfx
... print minfx.__file__
... print minfx.__version__
... except ImportError:
... print "no minfx module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/minfx/__init__.pyc
1.0.12
>>> try:
... import mpi4py
... print mpi4py.__file__
... print mpi4py.__version__
... except ImportError:
... print "no mpi4py module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/mpi4py/__init__.pyc
2.0.0
</source>
==Checkout relax and build==
{{#tag: source|
cd $HOME
{{relax clone}}
# Build
cd relax
scons
# Link to executable
cd $HOME
mkdir -p $HOME/bin
ln -s $HOME/relax/relax $HOME/bin
# Test all of relax with unittests and systemtests
relax --time -u
relax --time -s
| lang="bash"
}}
'''relax works!'''
== Test of OpenMPI ==
See [[OpenMPI#Linux_and_Mac]]
<source lang="bash">
mpirun --report-bindings -np 4 echo "hello world"
mpirun --report-bindings -np 2 relax --multi='mpi4py'
Traceback (most recent call last):
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
relax.start(profile_flag=False)
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
object = clazz(callback=callback, processor_size=processor_size)
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
raise Too_few_slaves_exception()
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
Traceback (most recent call last):
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
relax.start(profile_flag=False)
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
object = clazz(callback=callback, processor_size=processor_size)
File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
raise Too_few_slaves_exception()
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
</source>
== See also ==
[[Category:Devel tutorial]]
