Guy Tel-Zur's Blog

The slides of my talk at the  HPC Advisory Council Israel Supercomputing Conference 2012 that took place on February 7th, 2012 at Tel-Aviv university are available from here . The Importance of High-Performance and High-Throughput Computing in Israel

I tested globus-provision which is an easy way to have a globus+condor cluster ready for number crunching on Amazon’s EC2 cloud. My front end was my laptop which runs Windows. Python on Windows does not behave exactly as on Linux when it comes to SIGINT and therefore os.fork() generated an error message. Thanks to the support of  Borja Sotomayor the SIGINT issues has resolved and the solution was to by-pass the SIGINT so I could proceed with the test. You can read more about this issue from here: http://jira.globus.org/browse/GP-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel I tried the example from: http://globus.org/provision/guide_compute_go.html#guide-compute-go I decided not to use globusonline for the eBooks transfer and I used wget instead. Below are a few screen captures that show the test. My configuration file:  Preparing the instance: after a minute or two: checking the instance: Watching my cluster at the AWS management ...

Miron Livny talking at the IGT event, July 15th, 2009. The presentation of Dhruba Borthakur about Hadoop and Condor is available here .

Prof. Miron Livny The Head of the Condor project: Open Source for High Throughput Computing (HTC) http://www.cs.wisc.edu/condor/ Condor and the Cloud - The Challenges and the Roadmap of Condor The goal of the Condor® Project is to develop, implement, deploy, and evaluate mechanisms and policies that support High Throughput Computing (HTC) on large collections of distributively owned computing resources. Today, Condor is being used as a service in private clouds, Amazon EC2 and is embedded in Redhat Linux. Dhruba Borthakur Software Engineer at Facebook and Project Lead for Apache Hadoop File System Hadoop Distributed File System (including Hive) & Condor I discuss the design and architecture of the Hadoop Distributed File system. I then talk about configuration details about the private Hadoop Cloud used by Facebook to process petabytes of data. Date: July 15th, 2009 14:30-17:00 Location: Microsoft R&D Center, Hertzelia To reserve your place, please send your ...

I left Madison today after a visit of a week. A great city, a great university and great people. I Hope to visit there again soon.

My post on Cellular Automata from January 12 was not put there by mistake. I want to use it as a starting point for a couple of exercises in my Parallel Processing course. In that post I gave a few drawings that differ only by the generating rule number. Today, I am going to show how using the Condor High-Throughput Computing system allows to handle in a very simple way large volume of computations. I used this simple Condor submit file : universe = vanilla executable = nks.py Error = err.$(Process) Output= out.$(Process) Log = log.$(Process) Arguments = $(Process) Queue 256 And was able to compute the whole set of 256 rules (jobs) with the same effort of computing a single rule. I submitted the task to my Personal Condor on my laptop and was not disappointed; After a while I got all the outputs happily waiting for post processing.