Trusted, Bureaucrats
4,223
edits
Changes
Switch to the {{relax source}} and {{relax url}} templates for the code URLs.
The [https://en.wikipedia.org/wiki/Google_Cloud_Platform Google Cloud Platform] is a cloud computing platform that can be used for cheaply running relax on a cluster.
== Background ==
This follows the same idea as from [[Run_relax_at_cloud.sagemath.com|Run relax at cloud.sagemath.com]]
* Instance type of (HIGH CPU) '''n1-highcpu-8 (vCPUs: 8, RAM: 7.20 GB)'''
* 0 '''SSD'''
* Datacenter location '''EuropeUnited States'''
* Average days per month each server is running: '''1''' days, per month.
This '''1 day''' price was of November 2015, '''$8.06'''. Constantly running is $171.70 per month. <br>
* You should already have a project called '''My First Project'''.
* '''Create a new project''' in top right corner of the browser under the dropdown of '''My First Project'''.
* Call the new project name '''relax''' with , Project ID '''relax-1''', '''App Engine location''' at '''europeus-westcentral'''. NOTE: App engine HAS TO BE located in united states, if datalab should be enabled later.
== Create a Virtual Machine instance ==
* Click then at '''Try Compute Engine''' or go Go to https://console.developers.google.com/compute* Click '''Create instance'''* '''Name:''' relax* '''Zone:''' Don't change this. As of Nov 2015, it seems that if you change this, you cannot select more than 2 vCPU'shome* '''Machine type:''' From In the very bottomtop right corner, create '''8 vCPUs'''. 32 cores need an upgrade there is a link to you account, [https://cloud.google.com/free-trial/#/faq and the free trial only allows for 8 vCPU's.]* '''Boot disk:Activate Google Cloud Shell''' 10 GB of Ubuntu 14.04 LTS* '''Zone:''' Now change Clicking this to: europe-west1-c* Let should start a terminal in half the rest be default and click '''Create'''window.
<source lang== Get relax and unpack =="bash"># List all compute instances in the projectgcloud compute instances listgcloud compute zones listgcloud compute machine-types list | grep us-central1-fgcloud compute machine-types list | grep us-central1-f | sort -k3 -ngcloud compute images list
# Get relaxOrcurl http://download.gna.org/relax/gcloud compute instances create relax-$v.GNU-Linux.x86_64.tar.bz2 zone us-central1-f --machine-type n1-highcpu-4 --image rhel-6-v20160921 --o relaxboot-$v.GNUdisk-Linux.x86_64.tar.bz2size 10GB
See the list of deploy scripts here* {{relax url|path== Inspect server ==devel_scripts/deploy_scripts/}}
# Then check serverset download URLURL={{#tag:sourcerelax url|uptimewhoamilscpuhtop -u `whoami`path=devel_scripts/deploy_scripts/$VERS|langview="bash"raw}}
}}
{{#tag:source|
|lang="bash"
}}
==Test openmpi =install Minfx===See [[Minfx]]{{#tag:source|v={{current version minfx}}
# Test relaxSource functionsrelax_trunk -isource openmpi_test_install_bash.shtestopenmpi|lang="bash"
}}
For TCSH shell<br>{{relax source|path=devel_scripts/openmpi_test_install_tcsh.sh|view=raw|text=install Bmrblib===See [[Bmrblib]]the script here}}{{#tag:source|v=alias get 'set noglob; wget {{current version bmrblibrelax url|path=devel_scripts/openmpi_test_install_tcsh.sh|view=raw}}; mv openmpi_test_install_tcsh.sh* openmpi_test_install_tcsh.sh;unset noglob'get
}}
==Transfer files back and forth to the server==
* https://cloud.google.com/compute/docs/tutorials/transfer-files
=== Install gcloudon your local computer===
See https://cloud.google.com/sdk/
On your own computer
<source lang="pythonbash">
cd $HOME/Downloads
curl https://sdk.cloud.google.com | bash
This will start a browser to authenticate you. When it ask for a ID, just write "test" and say no to cloning.
=== ssh-config with gcloud ===
<source lang="bash">
PROJ=relax-1ZONE=us-central1-fPRID=relaxGCC=gccgcloud compute --project "relax-1143" $PROJ config-ssh
cat $HOME/.ssh/config
# Use sed to change value of your local computes ssh config file
ls -l $HOME/.ssh/config*
grep 'gcc_relax' "$GCC" $HOME/.ssh/configssh gcc_relax$GCCexit
</source>
{{#tag:source|
#First fast check
cd $HOME/Downloads
touch test2.txt
scp test2.txt gcc_relax$GCC:ssh gcc_relax$GCC
ls
rm test2.txt
exit
rm test2.txt
# Or use rsync
mkdir dirtestdeletethisdirrsync --progress -avzhe ssh ./dirtest gcc_relaxdeletethisdir $GCC:ssh gcc_relax$GCC
ls
rmdir dirtestdeletethisdir
exit
rmdir deletethisdir
|lang="bash"
}}
== Use relax on a test dataset from a tutorial ==
This follows some example data from the tutorial here [[Tutorial_for_sorting_data_stored_as_numpy_to_on-resonance_R1rho_analysis|Tutorial for sorting data stored as numpy to on resonance R1rho_analysis]]
{{relax source|path=test_suite/shared_data/dispersion/Paul_Schanda_2015_Nov|text=Overview of directory}}
Copy data to local dir
{{#tag:source|
cd $HOME
cp -r $HOME/relax_trunk/test_suite/shared_data/dispersion/Paul_Schanda_2015_Nov .
cd Paul_Schanda_2015_Nov
|lang="bash"
}}
{{relax source|path=test_suite/shared_data/dispersion/Paul_Schanda_2015_Nov/1_tutorial_prepare_data.py|text=Prepare data}}
{{#tag:source|
cat 1_tutorial_prepare_data.py
python 1_tutorial_prepare_data.py
ls -latr
|lang="bash"
}}
{{relax source|path=test_suite/shared_data/dispersion/Paul_Schanda_2015_Nov/2_tutorial_load_data.py|text=Run analysis}}
{{#tag:source|
cat 2_tutorial_load_data.py
mpirun --np 2 relax_trunk --multi='mpi4py' 2_tutorial_load_data.py
ls -latr
|lang="bash"
}}
{{relax source|path=test_suite/shared_data/dispersion/Paul_Schanda_2015_Nov/3_tutorial_clustered.py|text=Run clustered analysis}}
{{#tag:source|
cat 3_tutorial_clustered.py
mpirun --np 2 relax_trunk --multi='mpi4py' 3_tutorial_clustered.py
ls -latr
|lang="bash"
}}
See results
{{#tag:source|
cd results_cluster/final
ls -latr
cat phi_ex.out
cat kex.out
|lang="bash"
}}
===By terminal===
<source lang="bash">
PROJ=relax-1
ZONE=us-central1-f
PRID=relax
GCC=gcc
# Get at list of projects
gcloud compute --project "relax-1143" $PROJ instances list
# Start it, it takes 30 seconds.
gcloud compute --project "relax-1143" $PROJ instances start --zone "europe-west1-c" "relax"$ZONE $PRIDssh gcc_relax$GCC
exit
# Stop it through terminal
gcloud compute --project "relax-1143" $PROJ instances stop --zone "europe-west1-c" "relax"$ZONE $PRID
# This should now not work
ssh gcc_relax$GCC
</source>
See
* https://cloud.google.com/free-trial/docs/upgrade-paid-account
* https://console.developers.google.com/compute/quotas
* https://cloud.google.com/free-trial/docs/free-trial-quotas
==Deploy google datalab to your server==
{{warning|This does not work}}
Read more here: http://techcrunch.com/2015/10/13/google-launches-cloud-datalab-an-interactive-tool-for-exploring-and-visualizing-data
Note: '''Cloud Datalab uses AppEngine and Managed VMs beta, which is available in the US region only.'''<br>
'''You cannot change the location after the project has been created.'''<br>
See Location at: https://appengine.google.com/
Stop your server:
<source lang="bash">
gcloud compute --project $PROJ instances stop --zone $ZONE $PRID
</source>
Cloud Datalab is deployed as an Google App Engine application module in a Google Developers Console project. <br>
[https://console.developers.google.com/flows/enableapi?apiid=compute_component&redirect=https://console.developers.google.com/&_ga=1.74861778.1584957856.1448496848 Click here to → Enable the Google Compute Engine API.]
Visit:
* https://cloud.google.com/datalab
* https://cloud.google.com/datalab/getting-started
Do the following
* Launch datalab
* Sign in
* Deploy to your project.
== See also ==
[[Category:Installation]]