Install relax cloud.sagemath.com

From relax wiki
Revision as of 09:51, 25 November 2015 by Bugman (talk | contribs) (Added an introductory sentence - this is essential for the wiki main page "Did you know..." section.)
Jump to navigation Jump to search

The SageMathCloud is a cloud computing platform designed for computational mathematics that is a part of the Sage project.

Purpose

Why ?

  • The first installation of relax can be an energy barrier
  • It could be of interest to make relax available in "the cloud" for the first analysis, so time is not used on solving software dependencies

Problems with "the cloud"

Other info

Possibilities with cloud.sagemath.com

cloud.sagemath.com currently does not provide admin account or dedicated SMC servers.
A subscription can be purchased for $49 per month (or $499 per year), one can apply the quotas listed here to any projects: 

* 16 upgrades Member hosting
* 40 upgrades Network access
* 8 days Idle timeout
* 24 GB Memory
* 40 GB Disk space
* 4 shares CPU shares
* 2 cores CPU cores

The procedure should be

  • A new interested scientist would go to SMC, create a project.
  • Add relax as a collaborator to their free project, and we would click a button to upgrade their project to member hosting.
  • This add network access, increase the cpu shares, etc.
  • "We" have a page which lists, for each project, what upgrades "we" have applied
  • "We" can also see if the project is being used.
  • "We" can remove upgrades at any time (in any granularity).

The value of upgrading to "member hosting" is:

  • Computation is much faster. The free pool of computers are typically massively loaded during the week.
  • The "free servers" work fine, but they can easily be 15% of the speed of the members-only machines.
  • With the $49/month subscription, "we" get to move 16 projects to members-only machines.

Installation and analysis in the cloud

Installation of relax in cloud.sagemath.com

  • Instead of making "relax" available for all users on cloud.sagemath.com, each user should install their local copy of relax
  • When using svn to checkout relax, there will be a: install_relax_cloud_sagemath_com.sh bash script
  • The user now have full control over any version of relax
  • The user can potentially "develop" online with subversion

To do Analysis

  • Use ssh to send data to the server
    • Supported right now, even for free projects. See the button "ssh to your project" in project settings.
  • The uploaded data for the researcher is private:
    • Even for free projects, data in projects is only visible to collaborators on that project and nobody else.
    • SMC is not like github, where free = public by default.
    • With SMC everything is private by default, even for free users.
    • However, people can also explicitly share any file or directory tree, and that makes it publicly available
    • Could be a way to share data related to a paper or research project
  • Use relax in the terminal in the server
    • Fine for free projects.

The test project at cloud.sagemath.com

This audit was performed: 2015/11/24

Project location

https://cloud.sagemath.com/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/files/test%20ipyth.ipynb

Project resources

Disk space: 3000 MB disk space available - 59 MB used    3000 MB given by free project
Memory: 1000 MB RAM memory available - 133 MB used    1000 MB given by free project
CPU cores:1 core    1 core given by free project
CPU shares: 1 share    1 share given by free project
Idle timeout: 1 hour of non-interactive use before project stops    1 hour given by free project
Network access: Yes    1 upgrade given by free project
Member hosting: No    0 upgrades given by free project

Software at cloud.sagemath.com

Is subversion available

~$ svn --version
svn, version 1.8.10 (r1615264)

Is scons available

~$ which scons
/usr/bin/scons

~$ scons --version
SCons by Steven Knight et al.:
        script: v2.3.1, 2014/03/02 14:18:15, by garyo on lubuntu
        engine: v2.3.1, 2014/03/02 14:18:15, by garyo on lubuntu
        engine path: ['/usr/lib/scons/SCons']

Is xmgrace available

~$ xmgrace
The program 'xmgrace' is currently not installed. To run 'xmgrace' please ask your administrator to install the package 'grace'
~$ grace
The program 'grace' is currently not installed. To run 'grace' please ask your administrator to install the package 'grace'

Is mpirun available

~$ which mpicc
/usr/bin/mpicc

~$ which mpiexec
/usr/bin/mpiexec

~$ which mpirun
/usr/bin/mpirun

~$ mpirun --version
mpirun (Open MPI) 1.6.5
 
Report bugs to http://www.open-mpi.org/community/help/

~$ mpirun --report-bindings -np 4 echo "hello world"
hello world
hello world
hello world
hello world

Server

~$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Model name:            Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping:              0
CPU MHz:               2299.972
BogoMIPS:              4599.94
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              46080K
NUMA node0 CPU(s):     0-3

Python packages at cloud.sagemath.com

Start Python: 

python

The current Python version: 

Python 2.7.9 (default, Apr  2 2015, 15:33:21)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Paste in: 

try:
    import wx
    print wx.__file__
    print wx.version()
    print wx.__version__
except ImportError:
    print "no wx module"


try:
    import bmrblib
    print bmrblib.__file__
    print bmrblib.__version__
except ImportError:
    print "no bmrblib module"


try:
    import minfx
    print minfx.__file__
    print minfx.__version__
except ImportError:
    print "no minfx module"


try:
    import mpi4py
    print mpi4py.__file__
    print mpi4py.__version__
except ImportError:
    print "no mpi4py module"

Output: 

>>> try:
...     import wx
...     print wx.__file__
...     print wx.version()
...     print wx.__version__
... except ImportError:
...     print "no wx module"
...
no wx module
>>> try:
...     import bmrblib
...     print bmrblib.__file__
...     print bmrblib.__version__
... except ImportError:
...     print "no bmrblib module"
...
no bmrblib module
>>> try:
...     import minfx
...     print minfx.__file__
...     print minfx.__version__
... except ImportError:
...     print "no minfx module"
...
no minfx module
>>> try:
...     import mpi4py
...     print mpi4py.__file__
...     print mpi4py.__version__
... except ImportError:
...     print "no mpi4py module"
...
no mpi4py module

Install Python modules

Bmrblib

See Bmrblib 

v=1.0.4

mkdir -p $HOME/Downloads
cd $HOME/Downloads
curl http://download.gna.org/bmrblib/bmrblib-$v.tar.gz -o bmrblib-$v.tar.gz
tar -xzf bmrblib-$v.tar.gz
cd bmrblib-$v
pip install .
cd ..

Minfx

See Minfx 

v=1.0.12

mkdir -p $HOME/Downloads
cd $HOME/Downloads
curl http://download.gna.org/minfx/minfx-$v.tar.gz -o minfx-$v.tar.gz
tar -xzf minfx-$v.tar.gz
cd minfx-$v
pip install .
cd ..

Mpi4py

See OpenMPI#Install_mpi4py 

v=2.0.0 

mkdir -p $HOME/Downloads
cd $HOME/Downloads
wget https://bitbucket.org/mpi4py/mpi4py/downloads/mpi4py-$v.tar.gz
tar -xzf mpi4py-$v.tar.gz
cd mpi4py-$v
pip install .
cd ..

Test Python packages

Paste in: 

try:
    import bmrblib
    print bmrblib.__file__
    print bmrblib.__version__
except ImportError:
    print "no bmrblib module"


try:
    import minfx
    print minfx.__file__
    print minfx.__version__
except ImportError:
    print "no minfx module"


try:
    import mpi4py
    print mpi4py.__file__
    print mpi4py.__version__
except ImportError:
    print "no mpi4py module"

Output: 

>>> try:
...     import bmrblib
...     print bmrblib.__file__
...     print bmrblib.__version__
... except ImportError:
...     print "no bmrblib module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/bmrblib/__init__.pyc
1.0.4
>>> try:
...     import minfx
...     print minfx.__file__
...     print minfx.__version__
... except ImportError:
...     print "no minfx module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/minfx/__init__.pyc
1.0.12
>>> try:
...     import mpi4py
...     print mpi4py.__file__
...     print mpi4py.__version__
... except ImportError:
...     print "no mpi4py module"
...
/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/.local/lib/python2.7/site-packages/mpi4py/__init__.pyc
2.0.0

Checkout relax and build

cd $HOME
svn co svn://svn.gna.org/svn/relax/trunk relax_trunk

# Build
cd relax_trunk
scons

# Link to executable
cd $HOME
mkdir -p $HOME/bin
ln -s $HOME/relax_trunk/relax $HOME/bin

# Test all of relax with unittests and systemtests
relax --time -u
relax --time -s

relax works!

Test of OpenMPI

See OpenMPI#Linux_and_Mac 

mpirun --report-bindings -np 4 echo "hello world"

mpirun --report-bindings -np 2 relax --multi='mpi4py'

Traceback (most recent call last):
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
    relax.start(profile_flag=False)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
    processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
    object = clazz(callback=callback, processor_size=processor_size)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
    raise Too_few_slaves_exception()
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
Traceback (most recent call last):
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/bin/relax", line 7, in <module>
    relax.start(profile_flag=False)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/relax.py", line 114, in start
    processor = load_multiprocessor(relax.multiprocessor_type, callbacks, processor_size=relax.n_processors, verbosity=verbosity)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/__init__.py", line 189, in load_multiprocessor
    object = clazz(callback=callback, processor_size=processor_size)
  File "/projects/9cd9d431-3eb2-4cde-a03a-4b9c6b49a1c2/relax_trunk/multi/mpi4py_processor.py", line 56, in __init__
    raise Too_few_slaves_exception()
multi.multi_processor_base.Too_few_slaves_exception: master slave processing requires at least 2 processors to run you only provided 1, exiting....
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------

See also

Retrieved from "http://wiki.nmr-relax.com/wiki/index.php?title=Install_relax_cloud.sagemath.com&oldid=6440"