Difference between revisions of "Install relax cloud.sagemath.com"

From relax wiki
Jump to navigation Jump to search
(→‎Is subversion available: Switched from svn to git.)
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
The [https://cloud.sagemath.com/ SageMathCloud] is a cloud computing platform designed for computational mathematics that is a part of the [https://en.wikipedia.org/wiki/SageMath Sage project].
 +
 
== Purpose ==
 
== Purpose ==
 
Why ?
 
Why ?
Line 9: Line 11:
  
 
Other info
 
Other info
* The prizing is here: https://cloud.sagemath.com/policies/pricing.html
+
* The pricing is here: https://cloud.sagemath.com/policies/pricing.html
 
* There is also the Bio-NMR initiative: http://www.bio-nmr.net
 
* There is also the Bio-NMR initiative: http://www.bio-nmr.net
 
* And a web service with all NMR software - WeNMR: http://www.wenmr.eu
 
* And a web service with all NMR software - WeNMR: http://www.wenmr.eu
Line 15: Line 17:
 
== Possibilities with cloud.sagemath.com ==
 
== Possibilities with cloud.sagemath.com ==
  
cloud.sagemath.com currently don't  provide admin account or dedicated SMC servers. <br>
+
cloud.sagemath.com currently does not provide admin account or dedicated SMC servers. <br>
A purchase for a subscription for $ 49 / month (or $499/year), one can apply the quotas listed here to any projects:
+
A subscription can be purchased for $49 per month (or $499 per year), one can apply the quotas listed here to any projects:
  
 
<source lang="text">
 
<source lang="text">
Line 80: Line 82:
  
 
==Software at cloud.sagemath.com==
 
==Software at cloud.sagemath.com==
===Is subversion available===
+
===Is git available===
 
<source lang="bash">
 
<source lang="bash">
~$ svn --version
+
~$ git --version
svn, version 1.8.10 (r1615264)
+
git version 2.7.6
 
</source>
 
</source>
  
Line 100: Line 102:
 
===Is xmgrace available===
 
===Is xmgrace available===
 
<source lang="bash">
 
<source lang="bash">
~$ xmgrace
+
~$ which xmgrace
The program 'xmgrace' is currently not installed. To run 'xmgrace' please ask your administrator to install the package 'grace'
+
/usr/bin/xmgrace
~$ grace
 
The program 'grace' is currently not installed. To run 'grace' please ask your administrator to install the package 'grace'
 
 
</source>
 
</source>
  
Line 129: Line 129:
 
</source>
 
</source>
  
==python packages at cloud.sagemath.com==
+
===Server ===
Start python
+
<source lang="bash">
 +
~$ lscpu
 +
Architecture:          x86_64
 +
CPU op-mode(s):        32-bit, 64-bit
 +
Byte Order:            Little Endian
 +
CPU(s):                4
 +
On-line CPU(s) list:  0-3
 +
Thread(s) per core:    2
 +
Core(s) per socket:    2
 +
Socket(s):            1
 +
NUMA node(s):          1
 +
Vendor ID:            GenuineIntel
 +
CPU family:            6
 +
Model:                63
 +
Model name:            Intel(R) Xeon(R) CPU @ 2.30GHz
 +
Stepping:              0
 +
CPU MHz:              2299.972
 +
BogoMIPS:              4599.94
 +
Hypervisor vendor:    KVM
 +
Virtualization type:  full
 +
L1d cache:            32K
 +
L1i cache:            32K
 +
L2 cache:              256K
 +
L3 cache:              46080K
 +
NUMA node0 CPU(s):    0-3
 +
</source>
 +
 
 +
==Python packages at cloud.sagemath.com==
 +
Start Python:
 
<source lang="bash">
 
<source lang="bash">
 
python
 
python
 
</source>
 
</source>
  
Version of python
+
The current Python version:
 
<source lang="bash">
 
<source lang="bash">
 
Python 2.7.9 (default, Apr  2 2015, 15:33:21)
 
Python 2.7.9 (default, Apr  2 2015, 15:33:21)
Line 142: Line 170:
 
</source>
 
</source>
  
Paste in
+
Paste in:
 
<source lang="python">
 
<source lang="python">
 
try:
 
try:
Line 177: Line 205:
 
</source>
 
</source>
  
Output
+
Output:
 
<source lang="python">
 
<source lang="python">
 
>>> try:
 
>>> try:
Line 214: Line 242:
 
</source>
 
</source>
  
== Install python modules ==
+
== Install Python modules ==
  
 
===Bmrblib===
 
===Bmrblib===
 
See [[Bmrblib]]
 
See [[Bmrblib]]
<source lang="bash">
+
{{#tag:source|
v=1.0.4
+
v={{current version bmrblib}}
  
 
mkdir -p $HOME/Downloads
 
mkdir -p $HOME/Downloads
Line 228: Line 256:
 
pip install .
 
pip install .
 
cd ..
 
cd ..
</source>
+
|lang="bash"
 +
}}
  
===minfx===
+
===Minfx===
 
See [[Minfx]]
 
See [[Minfx]]
<source lang="bash">
+
{{#tag:source|
v=1.0.12
+
v={{current version minfx}}
  
 
mkdir -p $HOME/Downloads
 
mkdir -p $HOME/Downloads
Line 242: Line 271:
 
pip install .
 
pip install .
 
cd ..
 
cd ..
</source>
+
|lang="bash"
 +
}}
  
===mpi4py===
+
===Mpi4py===
 
See [[OpenMPI#Install_mpi4py]]
 
See [[OpenMPI#Install_mpi4py]]
<source lang="bash">
+
{{#tag:source|
v=2.0.0
+
v={{current version mpi4py}}
  
 
mkdir -p $HOME/Downloads
 
mkdir -p $HOME/Downloads
Line 256: Line 286:
 
pip install .
 
pip install .
 
cd ..
 
cd ..
</source>
+
|lang="bash"
 +
}}
  
===Test python packages===
+
===Test Python packages===
Paste in
+
Paste in:
 
<source lang="python">
 
<source lang="python">
 
try:
 
try:
Line 285: Line 316:
 
</source>
 
</source>
  
Output
+
Output:
 
<source lang="python">
 
<source lang="python">
 
>>> try:
 
>>> try:
Line 317: Line 348:
  
 
==Checkout relax and build==
 
==Checkout relax and build==
<source lang="bash">
+
{{#tag: source|
 
cd $HOME
 
cd $HOME
svn co svn://svn.gna.org/svn/relax/trunk relax_trunk
+
{{relax clone}}
  
 
# Build
 
# Build
cd relax_trunk
+
cd relax
 
scons
 
scons
  
Line 328: Line 359:
 
cd $HOME
 
cd $HOME
 
mkdir -p $HOME/bin
 
mkdir -p $HOME/bin
ln -s $HOME/relax_trunk/relax $HOME/bin
+
ln -s $HOME/relax/relax $HOME/bin
  
 
# Test all of relax with unittests and systemtests
 
# Test all of relax with unittests and systemtests
 
relax --time -u
 
relax --time -u
 
relax --time -s
 
relax --time -s
</source>
+
| lang="bash"
 +
}}
  
 
'''relax works!'''
 
'''relax works!'''
 +
 +
== Test of OpenMPI ==
 +
See [[OpenMPI#Linux_and_Mac]]
 +
<source lang="bash">
 +
mpirun --report-bindings -np 4 echo "hello world"
 +
 +
mpirun --report-bindings -np 2 relax --multi='mpi4py'
 +
 +
Traceback (most recent call last):
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
 +
    relax.start(profile_flag=False)
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
 +
    processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
 +
    object = clazz(callback=callback, processor_size=processor_size)
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
 +
    raise Too_few_slaves_exception()
 +
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
 +
Traceback (most recent call last):
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
 +
    relax.start(profile_flag=False)
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
 +
    processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
 +
    object = clazz(callback=callback, processor_size=processor_size)
 +
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
 +
    raise Too_few_slaves_exception()
 +
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
 +
--------------------------------------------------------------------------
 +
mpirun noticed that the job aborted, but has no info as to the process
 +
that caused that situation.
 +
--------------------------------------------------------------------------
 +
</source>
  
 
== See also ==
 
== See also ==
 
[[Category:Devel tutorial]]
 
[[Category:Devel tutorial]]

Latest revision as of 11:59, 27 October 2017

The SageMathCloud is a cloud computing platform designed for computational mathematics that is a part of the Sage project.

Purpose

Why ?

  • The first installation of relax can be an energy barrier
  • It could be of interest to make relax available in "the cloud" for the first analysis, so time is not used on solving software dependencies

Problems with "the cloud"

Other info

Possibilities with cloud.sagemath.com

cloud.sagemath.com currently does not provide admin account or dedicated SMC servers.
A subscription can be purchased for $49 per month (or $499 per year), one can apply the quotas listed here to any projects:

* 16 upgrades Member hosting
* 40 upgrades Network access
* 8 days Idle timeout
* 24 GB Memory
* 40 GB Disk space
* 4 shares CPU shares
* 2 cores CPU cores

The procedure should be

  • A new interested scientist would go to SMC, create a project.
  • Add relax as a collaborator to their free project, and we would click a button to upgrade their project to member hosting.
  • This add network access, increase the cpu shares, etc.
  • "We" have a page which lists, for each project, what upgrades "we" have applied
  • "We" can also see if the project is being used.
  • "We" can remove upgrades at any time (in any granularity).

The value of upgrading to "member hosting" is:

  • Computation is much faster. The free pool of computers are typically massively loaded during the week.
  • The "free servers" work fine, but they can easily be 15% of the speed of the members-only machines.
  • With the $49/month subscription, "we" get to move 16 projects to members-only machines.

Installation and analysis in the cloud

Installation of relax in cloud.sagemath.com

  • Instead of making "relax" available for all users on cloud.sagemath.com, each user should install their local copy of relax
  • When using svn to checkout relax, there will be a: install_relax_cloud_sagemath_com.sh bash script
  • The user now have full control over any version of relax
  • The user can potentially "develop" online with subversion

To do Analysis

  • Use ssh to send data to the server
    • Supported right now, even for free projects. See the button "ssh to your project" in project settings.
  • The uploaded data for the researcher is private:
    • Even for free projects, data in projects is only visible to collaborators on that project and nobody else.
    • SMC is not like github, where free = public by default.
    • With SMC everything is private by default, even for free users.
    • However, people can also explicitly share any file or directory tree, and that makes it publicly available
    • Could be a way to share data related to a paper or research project
  • Use relax in the terminal in the server
    • Fine for free projects.

The test project at cloud.sagemath.com

This audit was performed: 2015/11/24

Project location

https://cloud.sagemath.com/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/files/test%20ipyth.ipynb

Project resources

Disk space: 3000 MB disk space available - 59 MB used    3000 MB given by free project
Memory: 1000 MB RAM memory available - 133 MB used    1000 MB given by free project
CPU cores:1 core    1 core given by free project
CPU shares: 1 share    1 share given by free project
Idle timeout: 1 hour of non-interactive use before project stops    1 hour given by free project
Network access: Yes    1 upgrade given by free project
Member hosting: No    0 upgrades given by free project

Software at cloud.sagemath.com

Is git available

~$ git --version
git version 2.7.6

Is scons available

~$ which scons
/usr/bin/scons

~$ scons --version
SCons by Steven Knight et al.:
        script: v2.3.1, 2014/03/02 14:18:15, by garyo on lubuntu
        engine: v2.3.1, 2014/03/02 14:18:15, by garyo on lubuntu
        engine path: ['/usr/lib/scons/SCons']

Is xmgrace available

~$ which xmgrace
/usr/bin/xmgrace

Is mpirun available

~$ which mpicc
/usr/bin/mpicc

~$ which mpiexec
/usr/bin/mpiexec

~$ which mpirun
/usr/bin/mpirun

~$ mpirun --version
mpirun (Open MPI) 1.6.5
 
Report bugs to http://www.open-mpi.org/community/help/

~$ mpirun --report-bindings -np 4 echo "hello world"
hello world
hello world
hello world
hello world

Server

~$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Model name:            Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping:              0
CPU MHz:               2299.972
BogoMIPS:              4599.94
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              46080K
NUMA node0 CPU(s):     0-3

Python packages at cloud.sagemath.com

Start Python:

python

The current Python version:

Python 2.7.9 (default, Apr  2 2015, 15:33:21)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Paste in:

try:
    import wx
    print wx.__file__
    print wx.version()
    print wx.__version__
except ImportError:
    print "no wx module"


try:
    import bmrblib
    print bmrblib.__file__
    print bmrblib.__version__
except ImportError:
    print "no bmrblib module"


try:
    import minfx
    print minfx.__file__
    print minfx.__version__
except ImportError:
    print "no minfx module"


try:
    import mpi4py
    print mpi4py.__file__
    print mpi4py.__version__
except ImportError:
    print "no mpi4py module"

Output:

>>> try:
...     import wx
...     print wx.__file__
...     print wx.version()
...     print wx.__version__
... except ImportError:
...     print "no wx module"
...
no wx module
>>> try:
...     import bmrblib
...     print bmrblib.__file__
...     print bmrblib.__version__
... except ImportError:
...     print "no bmrblib module"
...
no bmrblib module
>>> try:
...     import minfx
...     print minfx.__file__
...     print minfx.__version__
... except ImportError:
...     print "no minfx module"
...
no minfx module
>>> try:
...     import mpi4py
...     print mpi4py.__file__
...     print mpi4py.__version__
... except ImportError:
...     print "no mpi4py module"
...
no mpi4py module

Install Python modules

Bmrblib

See Bmrblib

v=1.0.4

mkdir -p $HOME/Downloads
cd $HOME/Downloads
curl http://download.gna.org/bmrblib/bmrblib-$v.tar.gz -o bmrblib-$v.tar.gz
tar -xzf bmrblib-$v.tar.gz
cd bmrblib-$v
pip install .
cd ..

Minfx

See Minfx

v=1.0.12

mkdir -p $HOME/Downloads
cd $HOME/Downloads
curl http://download.gna.org/minfx/minfx-$v.tar.gz -o minfx-$v.tar.gz
tar -xzf minfx-$v.tar.gz
cd minfx-$v
pip install .
cd ..

Mpi4py

See OpenMPI#Install_mpi4py

v=2.0.0 

mkdir -p $HOME/Downloads
cd $HOME/Downloads
wget https://bitbucket.org/mpi4py/mpi4py/downloads/mpi4py-$v.tar.gz
tar -xzf mpi4py-$v.tar.gz
cd mpi4py-$v
pip install .
cd ..

Test Python packages

Paste in:

try:
    import bmrblib
    print bmrblib.__file__
    print bmrblib.__version__
except ImportError:
    print "no bmrblib module"


try:
    import minfx
    print minfx.__file__
    print minfx.__version__
except ImportError:
    print "no minfx module"


try:
    import mpi4py
    print mpi4py.__file__
    print mpi4py.__version__
except ImportError:
    print "no mpi4py module"

Output:

>>> try:
...     import bmrblib
...     print bmrblib.__file__
...     print bmrblib.__version__
... except ImportError:
...     print "no bmrblib module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/bmrblib/__init__.pyc
1.0.4
>>> try:
...     import minfx
...     print minfx.__file__
...     print minfx.__version__
... except ImportError:
...     print "no minfx module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/minfx/__init__.pyc
1.0.12
>>> try:
...     import mpi4py
...     print mpi4py.__file__
...     print mpi4py.__version__
... except ImportError:
...     print "no mpi4py module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/mpi4py/__init__.pyc
2.0.0

Checkout relax and build

cd $HOME
git clone git://git.code.sf.net/p/nmr-relax/code/ relax

# Build
cd relax
scons

# Link to executable
cd $HOME
mkdir -p $HOME/bin
ln -s $HOME/relax/relax $HOME/bin

# Test all of relax with unittests and systemtests
relax --time -u
relax --time -s

relax works!

Test of OpenMPI

See OpenMPI#Linux_and_Mac

mpirun --report-bindings -np 4 echo "hello world"

mpirun --report-bindings -np 2 relax --multi='mpi4py'

Traceback (most recent call last):
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
    relax.start(profile_flag=False)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
    processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
    object = clazz(callback=callback, processor_size=processor_size)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
    raise Too_few_slaves_exception()
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
Traceback (most recent call last):
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
    relax.start(profile_flag=False)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
    processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
    object = clazz(callback=callback, processor_size=processor_size)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
    raise Too_few_slaves_exception()
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------

See also