Thursday, December 29, 2011

Globus Provision

I tested globus-provision which is an easy way to have a globus+condor cluster ready for number crunching on Amazon’s EC2 cloud.
My front end was my laptop which runs Windows. Python on Windows does not behave exactly as on Linux when it comes to SIGINT and therefore os.fork() generated an error message. Thanks to the support of Borja Sotomayor the SIGINT issues has resolved and the solution was to by-pass the SIGINT so I could proceed with the test. You can read more about this issue from here:
I tried the example from:
I decided not to use globusonline for the eBooks transfer and I used wget instead.
Below are a few screen captures that show the test.

My configuration file:
 Preparing the instance:

after a minute or two:

checking the instance:

Watching my cluster at the AWS management console from my browser:
Configuring Putty for SSH:

Connecting to one of the nodes:
Submitting a condor task (multiple jobs):

(unfortunately this specific test didn’t reached to the final destination due to an NFS failure of the /nfs/scratch/ mount point which I think is not directly related to this post…)

Monday, December 19, 2011

"Big Data" - The New Big Kid in Town

A year ago almost no one knew this term. With a doubling time of less than 6 months "Big Data" is becoming a hot hype, as can be seen below (by Google Trends)


Sunday, November 27, 2011

The End of Supercomputers As We Know Them Today

About four years ago I predicted in this blog the fall of Grid Computing and the rise of Cloud Computing ( )
Now I want to claim that Supercomputers as we know them today will not be the kind we will see in the future.
I think that within a couple of years:
1. Most supercomputers will shift to the cloud (HPC Cloud).
2. The governance of X86 is not guaranteed: We already see the rise of GPGPUs and we will see more families of technologies being involved, e.g. Tegra and ARM, Reconfigureable Computing (RC) i.e. FPGAs, etc'.
3. Supercomputers won't be anymore General Purpose machines. They will be reconstructed as a Cloud Service (Supercomputing as a Service) from all the available technologies (see previous section) per single mission (program) in order to maximize the performance and minimize the consumed energy and cost. I decided to use the word mission because the word task is usually reserved for a thread inside a program. Also the network topology will be ad-hoc (fully non-blocking, partially blocking, hyper-cube,....) and the user will be able to select from a few options the interconnect kind (Infiniband, 10GigE, ...).

Sunday, November 06, 2011

Cloud Computing turns 15!

Congratulations! Cloud Computing turns 15, don't believe? check this link. The original article is by Technology Review, check here

Monday, October 10, 2011

Computational Science and Engineering

You are welcome to join my Meetup group about Computational Science and Engineering (we are the leftmost site but the exact location is not exactly as appears on the map :-( )

Wednesday, September 28, 2011


Tuesday, September 06, 2011

Gmail is becoming too smart

I replied to someone's email who was supposed to send me attached files which were absent so I wrote back:
"No files are attached". When I pressed the "send" button Gmail analyzed my text and immediately opened this pop up window:
The Big Brother at its best.

Friday, July 08, 2011

A first trial with DotCloud

DotCloud claims to be One Platform - Any Stack Cloud solution
I decided to test it myself, read below.

1. Registration
Very easy and quick!

2. Setup
I used a Linux desktop (dream linux). I needed to install first Python´s ¨easy-install¨ which was missing. 

2.1 Installing easy_install
sudo apt-get update
sudo apt-get install python-setuptools

2.2 Dotcloud API installation
telzur@dream:~/Downloads/dotcloud$ sudo easy_install pip && sudo pip install dotcloud
Searching for pip
Best match: pip 1.0.1
Processing pip-1.0.1.tar.gz
Running pip-1.0.1/ -q bdist_egg --dist-dir /tmp/easy_install-XXryzJ/pip-1.0.1/egg-dist-tmp-f2uO_w
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
no previously-included directories found matching 'docs/_build/_sources'
Adding pip 1.0.1 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip-2.6 script to /usr/local/bin

Installed /usr/local/lib/python2.6/dist-packages/pip-1.0.1-py2.6.egg
Processing dependencies for pip
Finished processing dependencies for pip
Downloading/unpacking dotcloud
  Downloading dotcloud-0.4.1.tar.gz
  Running egg_info for package dotcloud
Downloading/unpacking dotcloud.cli==0.4.1 (from dotcloud)
  Downloading dotcloud.cli-0.4.1.tar.gz
  Running egg_info for package dotcloud.cli
Installing collected packages: dotcloud, dotcloud.cli
  Running install for dotcloud
  Running install for dotcloud.cli
    changing mode of build/scripts-2.6/dotcloud from 644 to 755
    changing mode of build/scripts-2.6/__dotcloud_git_ssh from 644 to 755
    changing mode of /usr/local/bin/__dotcloud_git_ssh to 755
    changing mode of /usr/local/bin/dotcloud to 755
Successfully installed dotcloud dotcloud.cli
Cleaning up...

3. Enter API key:
telzur@dream:~/Downloads/dotcloud$ dotcloud setup

4. Test
I tried the easy example from dotcloud website
about deploying a basic WWW server.
I will not repeat here the code insertion since it is described in detail at the website therefore I will only enclose below the output so you will be able  to verify the process:

telzur@dream:~/Downloads/dotcloud$ mkdir simpleapp
telzur@dream:~/Downloads/dotcloud$ cd simpleapp/
telzur@dream:~/Downloads/dotcloud/simpleapp$ vi index.html
telzur@dream:~/Downloads/dotcloud/simpleapp$ dotcloud create helloworldapp
Created repos "helloworldapp"
telzur@dream:~/Downloads/dotcloud/simpleapp$ vi dotcloud.yml
telzur@dream:~/Downloads/dotcloud/simpleapp$ dotcloud push helloworldapp
# upload /home/telzur/Downloads/dotcloud/simpleapp ssh://
# rsync
Pseudo-terminal will not be allocated because stdin is not a terminal.
Warning: Permanently added '[]:21122,[]:21122' (RSA) to the list of known hosts.
building file list ... done

sent 265 bytes  received 53 bytes  57.82 bytes/sec
total size is 122  speedup is 0.38
Deployment for "helloworldapp" triggered. Will be available in a few seconds.
2011-07-07 18:41:50 [api] Waiting for the build. (It may take a few minutes)
2011-07-07 18:41:50 [www.0] Deploying...
2011-07-07 18:42:04 [www.0] Service booted
2011-07-07 18:42:04 [www.0] The build started
2011-07-07 18:42:05 [www.0] Fetched code revision rsync-1310064109.13
2011-07-07 18:42:08 [www.0] Reloading nginx configuration: nginx.
2011-07-07 18:42:08 [www.0] The build finished successfully
2011-07-07 18:42:08 [api] Deploy finished

Deployment finished. Your application is available at the following URLs

5. Verification
I opened the URL and got, as expected, the ¨Hello World!¨ from the cloud virtual web server. If you own a domain name you can forward it to point to this URL and you are done.

6. Conclusions
6.1 Very cool and a nice game for the afternoon to experience the power of the cloud.
6.2 I confirmed how easy it is to deploy a webserver as dotcloud had promised
6.3 There are still many open questions about how mature is this solution for  real heavy, scalable and secured web services.
6.4 It will be interesting to follow dotcloud and see how they evolve.

Monday, May 02, 2011

The Chernobyl accident of Cloud Computing

    When introducing Cloud Computing (CC) people usually like to emphasize the similarities between Electric Power Grids (EPG) and CC.
    These analogies include: On demand service, Pay-Per-Use (PPU) model, the Elasticity of the resources, load balancing and even water cooling,
    After the Amazon EC2 service disruption on April 21st, 2011, we can identify more similarities between EPG and CC in disasters related issues. These similarities include:
1. A global damage (with immediate and delayed components).
2. Uncontrolled behavior of the resources (neutron population vs. loss of connectivity or fuel rods meltdown vs. failing servers).
3. During the crisis, similar announcements to the public which minimize the catastrophe.
4. After the crisis, publication of the event investigation in length.
5. Loss of confidence in the technology by the users.

Sunday, February 13, 2011

e-Science: Are we there yet?

Distinguished lecture, Prof. David Abramson 
You are invited to attend!


Tuesday, February 08, 2011

Supercomputing trends: Peta mug vs.Tera mug