Guy Tel-Zur's Blog

I tested globus-provision which is an easy way to have a globus+condor cluster ready for number crunching on Amazon’s EC2 cloud. My front end was my laptop which runs Windows. Python on Windows does not behave exactly as on Linux when it comes to SIGINT and therefore os.fork() generated an error message. Thanks to the support of  Borja Sotomayor the SIGINT issues has resolved and the solution was to by-pass the SIGINT so I could proceed with the test. You can read more about this issue from here: http://jira.globus.org/browse/GP-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel I tried the example from: http://globus.org/provision/guide_compute_go.html#guide-compute-go I decided not to use globusonline for the eBooks transfer and I used wget instead. Below are a few screen captures that show the test. My configuration file:  Preparing the instance: after a minute or two: checking the instance: Watching my cluster at the AWS management ...

    When introducing Cloud Computing (CC) people usually like to emphasize the similarities between Electric Power Grids (EPG) and CC.     These analogies include: On demand service, Pay-Per-Use (PPU) model, the Elasticity of the resources, load balancing and even water cooling,     After the Amazon EC2 service disruption on April 21 st , 2011, we can identify more similarities between EPG and CC in disasters related issues. These similarities include: 1. A global damage (with immediate and delayed components). 2. Uncontrolled behavior of the resources (neutron population vs. loss of connectivity or fuel rods meltdown vs. failing servers). 3. During the crisis, similar announcements to the public which minimize the catastrophe. 4. After the crisis, publication of the event investigation in length. 5. Loss of confidence in the technology by the users.

Prof. Miron Livny The Head of the Condor project: Open Source for High Throughput Computing (HTC) http://www.cs.wisc.edu/condor/ Condor and the Cloud - The Challenges and the Roadmap of Condor The goal of the Condor® Project is to develop, implement, deploy, and evaluate mechanisms and policies that support High Throughput Computing (HTC) on large collections of distributively owned computing resources. Today, Condor is being used as a service in private clouds, Amazon EC2 and is embedded in Redhat Linux. Dhruba Borthakur Software Engineer at Facebook and Project Lead for Apache Hadoop File System Hadoop Distributed File System (including Hive) & Condor I discuss the design and architecture of the Hadoop Distributed File system. I then talk about configuration details about the private Hadoop Cloud used by Facebook to process petabytes of data. Date: July 15th, 2009 14:30-17:00 Location: Microsoft R&D Center, Hertzelia To reserve your place, please send your ...

Next time you will need computing resources, e.g. a web server, get it from the cloud, it is simple and easy! This post is not meant to be a full tutorial but only a brief description of my own personal experience playing with Amazon EC2. For more complete information about working with Amazon EC2 check this or this references or browse here for additional documentation Step 1: Create an Amazon Elastic Compute Cloud (EC2) account. Pricing and additional information is available here . Step 2: Download the Amazon command line API tools and/or install the Elasticfox - Firefox plug-in . Step 3: Set the security keys to allow SSH. Step 4: Configure the Elasticfox (for example open port 22 for SSH and port 80 for http in my case). Step 5. I used one of the pre-configured Amazon Machine Images (AMI), therefore starting the machine was immediate. Step 6: Enjoy the new virtual machine - see screen shots below: Figure 1: The Amazon Elasticfox GUI. Figure 2: Connecting to the machine via S...