New release with Batch processing

[This article was first published on cloudnumbers.com » R-project, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

This week we rolled out a new release at cloudnumbers.com which implements two new main features:

  • cloudnumbers.com now supports Batch processing.
  • Due to some changes in the architecture we were able to reduce our system requirements. In detail, we do not need that much open ports in your firewall. Please check our updated System Requirements document.

Batch processing example:
Batch processing is the execution of a job on a computer cluster without manual intervention. As soon as the job is completed, you will get an email and if the corresponding flag is set the computer cluster in the Cloud will be switched of automatically.
To start batch processing at cloudnumbers.com please start your computer cluster in the Cloud as usual. As soon as you can connect to the master node you can test your code for example in a small scenario. After this you can submit any executable shell script into batch mode. For example, to submit your R job please use the following command:

batchJob -s /home/user/workspace/Rscript.sh

Rscript.sh is a simple shell script which executes “Rscript” and passes files or expressions to R. The attached Rscript.sh file is available in the sample workspace, too. You can now close your browser or external terminal. Do not stop the session! As soon as the job is finished, you will get an email and the computer cluster will be switched off (-s). You can now open your workspace from the dashboard or via ftps to check the logfiles and to download your result files (this step is for free!).

Python scripts you can start directly with the command

batchJob -s /home/user/workspace/helloworld.py

To get more information about all parameters please type batchJob -help or read more in our knowledge base.

To leave a comment for the author, please follow the link and comment on their blog: cloudnumbers.com » R-project.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)