IGT Event: Condor and the Cloud with Prof. Miron Livny & a Facebook IT Case Study

-

Prof. Miron Livny
The Head of the Condor project: Open Source for High Throughput Computing (HTC)
http://www.cs.wisc.edu/condor/

Condor and the Cloud - The Challenges and the Roadmap of Condor
The goal of the Condor® Project is to develop, implement, deploy, and evaluate mechanisms and policies that support High Throughput Computing (HTC) on large collections of distributively owned computing resources.
Today, Condor is being used as a service in private clouds, Amazon EC2 and is embedded in Redhat Linux.


Dhruba Borthakur
Software Engineer at Facebook and Project Lead for Apache Hadoop File System

Hadoop Distributed File System (including Hive) & Condor
I discuss the design and architecture of the Hadoop Distributed File system.
I then talk about configuration details about the private Hadoop Cloud used by Facebook to process petabytes of data.


Date: July 15th, 2009 14:30-17:00
Location: Microsoft R&D Center, Hertzelia

To reserve your place, please send your contact details to: info@grid.org.il
Date Jul 15, 2009 14:30 17:00
Location Microsoft R&D Center, Shenkar 13 St. Gav-Yam Building No 5, Hertzelia

Comments

Post a Comment

Popular posts from this blog

The End of Grid Computing?

-
Image
In the year 2003 the MIT Technology review ranked "Grid Computing" among the 10 Emerging Technologies That Will Change the World [1]. We are now four years later and something is not going well with "Grid Computing". An indication that there is a problem can easily be seen by looking at the "Google Trends" plot for the term "Grid Computing": (click on the image to get the current trend). This finding can be compared with another buzz word, "Virtualization", which is older than "Grid Computing" and yet is gaining more and more momentum: There is however one exception. The Academic Grid is still having lot's of glory thanks to the huge heavily funded European (EGEE) and other US projects. When LHC data will start to be taken at CERN it will reach it's top importance. But, it seems that for other scientific projects Grid Computing is not going to be such a success. It will remain as "Nice to have" but will never...
Read more

Parallel Debugging with a Serial Debugger

-
Image
Parallel Debuggers are important tools when trying to eliminate bugs in parallel programs. Commercial parallel debuggers are expensive. In this blog post I will show you a free and open source alternative which is good enough for education purposes and for small codes. Suppose you want to debug the famous cpi.c code which computes an approximation to pi by a parallel numerical integration. This demo uses the free Data Display Debugger (ddd) which is based on gdb but has a nice GUI. Step 1, compile: mpicc -g -o cpi ./cpi.c Step 2, run (without a debugger): mpirun -np 2 ./cpi Step 3, run with the free Data Display Debugger (ddd) : mpirun -np 2 ddd ./cpi The last command will open two MPI processes (instances) of ddd each running cpi, see screen capture: Finally, I enclose here a short video which I hope is convincing: hi-res version:  https://youtu.be/N_J4NKJkuMs Happy (Free) MPI debugging!
Read more

[mpich-discuss] Errors while testing MPICH2 on Vista

-
[mpich-discuss] Errors while testing MPICH2 on Vista Guy Tel-Zur gtelzur at gmail.com Sat Oct 24 14:39:26 CDT 2009 Previous message: [mpich-discuss] MPI-3 Fortran feedback Next message: [mpich-discuss] MPI function to test a machine alive or not. Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] I am trying to test a fresh installation of MPICH2 (version 1.2) on Vista32 and get the following errors: c:\temp>"c:\Program Files\MPICH2\bin\mpiexec.exe" -n 4 cpi.exe Please specify an authentication passphrase for smpd: passphrase, e.g. behappy*> Error while connecting to host, No connection could be made because the target machine actively refused it. (10061) Connect on sock (host=user-PC, port=8676) failed, exhaused all end points Unable to connect to 'user-PC:8676', sock error: Error = -1 I will appreciate any comment. --Guy -------------- next...
Read more