Posts

Showing posts from 2007

A comment on Rebecca's post

A useful teaching example for Parallel Processing courses is given in Rebecca's Blog: "Example Performance Evaluation"
I added the following comment:

"I liked the example and I think this kind of demonstrations really help explaining were bottlenecks may occur in the parallel code. I would like to add two comments:
1. Solving the matrix and obtaining 0 for the startup time, after already neglecting the bandwidth, means that the problem is totally Embarrassingly Parallel and this is indeed the case for MC calculations. It would be interesting to repeat the exercise for a communication intensive application, e.g. Laplace equation.
2. I think it worth mentioning Collective Communication commands, i.e. Bcast and Reduce in this context. In particular Reduce+Summation can be more efficient than the loop over the Recv because it can do summation out of order and this may hide some of the holding/communication time. Another alternative is to use MPI ANY_SOURCE in the Recv comma…

Cloud Computing

Image
I created a new group in Facebook about Cloud Computing.
You are all welcome to join!
Click here to visit the group web page:
http://www.facebook.com/group.php?gid=8450870046

TechnologyTrends: The next big thing must be Schnoodle

Google - The famous search engine and Internet related technologies
Doodle - Event scheduling
Moodle - A free open source course management system (CMS)
Schnoodle - A Schnoodle is a Poodle hybrid that is a cross-breed of a Poodle and a Schnauzer

Do Google results make sense?

Image
I did a Google digging research and became puzzled:
I searched for a few Buzz Words and then repeated the search with a combination of them.
The number of entries returned is not in accordance with the known laws of arithmetics!

"Grid Computing" 171,000 entries
where "XXX YYY" means with the exact phrase XXX YYY!
virtualization 1,530,000 entries
"Grid Computing" Virtualization 1,300,000 entries
where space between the terms means with all of the words.
virtualization -"grid computing" 2,580,000 entries
where "-" means exclude the term from the search
-virtualization "grid computing" 4,510,000 entries

In order to verify this mystery I repeated the test with two other terms:

Israel 26,600,000 entries
Jerusalem 4,190,000 entries
Jerusalem Israel 1,120,000 entries
jerusalem -israel 2,150,000 entries
-jerusalem israel 45,300,000 entries

To my understanding the situation can be demonstrated as in the following plot:




Does this mean we should make…

"Grid Confinement" - A Physicist perspective

Image
The following post should not be taken too seriously!

If GridComputing is so good but what we see practically is a complete chaos in the Data Center then there must exist an Anti-Grid that cancels all its benefits.
If one tries to make an order and to split the Data Center into two separate sites then immediately Grid - Anti-Grid pairs are produced out of the vacuum and chaos is being restored.

Social Networks and Grid Computing - Some similarities

Image
There are some similarities between Social Networks, e.g. Facebook and Grid Computing.
In the table below I mention a few.
.

Mounting a memory stick under Linux

A short reminder for myself and for all those who are lost when no auto mount works:

1. Verify that the device is recognized in 'dmesg', write down the device name. e.g. /dev/sdb
2. mkdir /disk_on_key
3. mount -t vfat /dev/sdb /disk_on_key
4. at the end un-mount by: umount /disk_on_key

Good luck!

GSS authentication failure - A question to the gt-users mailing list

I posted today the following question:

I installed Globus version gt4.0.4-x86_64_rhas_4-installer on an Opteron node running CentOS 4.4.
output of uname -a is:
2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 26 14:14:47 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

In the globus-gatekeeper.log there is the following error message:

TIME: Thu Oct 25 18:16:50 2007
PID: 14589 -- Notice: 6: globus-gatekeeper pid=14589 starting at Thu Oct 25 18:16:50 2007

TIME: Thu Oct 25 18:16:50 2007
PID: 14589 -- Notice: 6: Got connection xxx.xxx.xxx.xxx at Thu Oct 25 18:16:50 2007

GSS authentication failure
GSS Major Status: General failure
GSS Minor Status Error Chain:
globus_gsi_gssapi: Error during delegation: Delegation protocol violation
Failure: GSS failed Major:000d0000 Minor:00000002 Token:00000000

TIME: Thu Oct 25 18:16:50 2007
PID: 14589 -- Failure: GSS failed Major:000d0000 Minor:00000002 Token:00000000

I would appreciate any help resolving this problem

Regards,

Guy Tel-Zur

-------------------------------------------------…

Software Vulnerability: A new group in Facebook

I created a new group in Facebook to discuss and share knowledge in Software Vulnerability.
I keep there references to relevant links and documents.
You are welcome to join the group: Facebook | Software Vulnerability

My page in Facebook

(Note: In order to access the group you need first to login to your Facebook account)


--Guy

The 9th edition of PhysicaPlus is on the air

Image
The Israel Physical Society (IPS) on-line magazine, PhysicaPlus, is published a few times a year both in English and in Hebrew and is an excellent source for semi-popular Physics articles.
You are invited to visit PhysicaPlus

IGT Annual Event & Exhibition

Image
Dec. 3rd, 2007, The Next Generation Data Center

Click on the image for further information:

Que es mas macho, BlueGene o EGEE? (*)

Image
I just read in Supercomputingonline about EGEE III, the 3rd phase of the gigantic Grid Computing project:

"EGEE-III will last for 24 months, with a total manpower bid of almost 10,000 person months and an EC budget of Euro 36 million. As with EGEE-II, partners will provide extra effort to the project beyond that funded by the EC, bringing the total project budget up to Euro 70 million, and also contributing a further estimated Euro 50 million worth of computing resources."

and I ask myself what is more justified, a single extended 3PFLOPs BlueGene/P with 884,736-processors or thousands of various kinds of old 32 bit boxes distributed all over the world running Scientific Linux 3.0.X and gLite?

For years we were told that no single data center site will be able to cope with a few PB of data per year that will be produced by the LHC; Nice Powerpoint presentations showed a 20km tower of CDROMs, higher than the Mt. Blanc, indicating that only distributed Grid Computing environment …

The End of Grid Computing?

Image
In the year 2003 the MIT Technology review ranked "Grid Computing" among the 10 Emerging Technologies That Will Change the World [1].
We are now four years later and something is not going well with "Grid Computing".
An indication that there is a problem can easily be seen by looking at the "Google Trends" plot for the term "Grid Computing":


(click on the image to get the current trend).
This finding can be compared with another buzz word, "Virtualization", which is older than "Grid Computing" and yet is gaining more and more momentum:


There is however one exception. The Academic Grid is still having lot's of glory thanks to the huge heavily funded European (EGEE) and other US projects. When LHC data will start to be taken at CERN it will reach it's top importance. But, it seems that for other scientific projects Grid Computing is not going to be such a success. It will remain as "Nice to have" but will never rep…

Live Traffic Map

GridKa School, Karlsruhe

Image
I participated in the Grid Computing school, GridKa2007, that took place in Karlsruhe, Germany.

If something can be simple it must be complicated in Gentoo

Eclipse version 3.3 does not work out of the box on Sabayon Linux (Gentoo). It likes Java 1.6.
Upon installing this JDK from SUN there was a strange error message that "run-java-tool is not available". Then, I noticed that /usr/bin/java was pointing to /usr/bin/run-java-tool. I know that it was invented with some good reasons but in order to start playing quickly with Eclipse I just removed this link and put instead:
"ln -s /usr/lib/jvm/sun-jdk-1.6/bin/java /usr/bin/java", then Eclipse was happy and me too!!!

Enjoy Eclipse

Grid-HPC WG Meeting

June 28th, 14:00-16:00 - Grid-HPC Work Group meeting Location: IGT Offices, Maskit 4, 5th Floor, HertzliyaAgenda:14:00-14:15: OPENING - Avner & Guy14:15-14:50: DEBUGGING AND OPTIMIZING APPLICATIONS FOR MULTICORE MPP ARCHITECTURES

Speaker:
Jacques Philouze, Vice President Sales & Marketing, Allinea

Abstract:
As two, four and potentially eight-core processors become the norm, the defacto HPC architecture is tending towards large clusters of modest 8-16 core shared-memory servers, potentially with co-processing devices (eg. GPGPUs, FPGAs, Clearspeed). Programming these machines optimally presents a number of challenges, and applications that use a mixed programming models are now becoming commonplace.
In this presentation we will discuss the challenges facing today's HPC application developers, and the need for simple tools that can address mixed programming models. We will present new multicore features of Allinea's Distributed Debugging Tool (DDT) and Optimisation and Profi…

How to convert data cd/dvd back into an iso file?

Very simple:
type "df" to find the dvd file system name, then:

dd if=/dev/dvd_fs of=mydvd.iso


That's it!

How To Install VMware Server On A Fedora 7 Desktop | HowtoForge - Linux Howtos and Tutorials

VMware on Fedora-7 (F7)

I found the following reference useful when I installed VMware:
How To Install VMware Server On A Fedora 7 Desktop | HowtoForge - Linux Howtos and Tutorials
However, there still was a problem: After defining a new virtual machine I got this error message:

"Unable to change virtual machine power state. The "/usr/lib/vmware/bin/vmware-vmx" process did not start properly...."

Any ideas what to do next????

================================
Later addition:
I think this is because the latest VMware is not ready yet for the advanced F7 and its latest kernel.
What it did was to fall back to CentOS5.

Diskless tips - Part 2

1) In order to enable services like NFS mount and ypbind on the diskless clients edit the rc.local file in the image tree on the server like this:
[root@grid02]# more /diskless/centos44_diskless_ver_03/root/etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

#--------added manually by Guy, 30/5/2007:
mount -t nfs 192.168.1.2:/home /home
ypdomainname WRITE_HERE_THE_DOMAIN_NAME
ypbind
ypset 192.168.1.2

2) In order to document the installation procedure, here is the files & directory structure on the server which is relevant for the pxe-boot & tftp stage:
[root@grid02 ~]# ls /tftpboot/
linux-install

[root@grid02 ~]# ls /tftpboot/linux-install/
centos44_diskless_ver_01 centos44_diskless_ver_03 pxelinux.0
centos44_diskless_ver_02 msgs pxelinux.cfg

[root@grid02 ~]# ls /tftpboot/linux-install/cent…

My first test with the IL-BGU EGEE Site

[telzur@cs-grid4 tests]$ voms-proxy-init --voms dteam
Enter GRID pass phrase:
Your identity: /C=IL/O=IUCC/OU=BGU/CN=Guy Tel-Zur
Cannot find file or dir: /home/telzur/.glite/vomses
Creating temporary proxy ........................................................ Done
Contacting lcg-voms.cern.ch:15004 [/C=CH/O=CERN/OU=GRID/CN=host/lcg-voms.cern.ch] "dteam" Done
Creating proxy .......................................................... Done
Your proxy is valid until Wed May 23 05:27:43 2007
[telzur@cs-grid4 tests]$ voms-proxy-info
subject : /C=IL/O=IUCC/OU=BGU/CN=Guy Tel-Zur/CN=proxy
issuer : /C=IL/O=IUCC/OU=BGU/CN=Guy Tel-Zur
identity : /C=IL/O=IUCC/OU=BGU/CN=Guy Tel-Zur
type : proxy
strength : 512 bits
path : /tmp/x509up_u33335
timeleft : 11:59:55
[telzur@cs-grid4 tests]$ glite-job-submit hello2.jdl

Selected Virtual Organisation name (from proxy certificate extension): dteam
Connecting to host g01.phy.bg.ac.yu, port 7772
Logging to host g01.phy.bg.ac.yu, port 9002


*************…

Kernel Virtual Machine is working on my laptop

Image
I followed the instruction from: http://kvm.qumranet.com/kvmwiki/Debian on my DELL420 laptop running Ubuntu 7.04.

Here is a screen shot showing Windows-XP and CentOS5 running as guests Operating Systems (click on the image to enlarge):

Dapper Drake on a Dell Latitude D420

I used this great link for installing my wireless card on my Dell 420 using Ubuntu 7.04

Dapper Drake on a Dell Latitude D420

Excellent job.

Thanks

IL-BGU home page

Image
Today I updated my IL-BGU portal.
visit: http://www.bgu.ac.il/~tel-zur/grid.html
or click on the map to enter:

Diskless client installation

In the RHEL documentation (Chapter 4. "Diskless Environments", item #4) there is an in-accuracy in the rsync syntax usage.
Instead of:
rsync -a -e ssh golden_client:/ /diskless/whatever/root/
Use the following syntax which does not produce error messages:
rsync -v -a -e ssh --exclude='/sys/*' --exclude='/proc/*' golden_client:/ /diskless/whatever/root/

Other comments (that I find critial in my experience):

1) Make sure the following RPM is installed in the client: busybox-anaconda
(In my CentOS4.4 case it is: busybox-anaconda-1.00.rc1-5.x86_64.rpm)
2) Make sure there are no active NFS mounted partitions on the client before executing 'rsync'.
3) Make sure that the following 2 directories exist:
/tftpboot/linux-install/msgs/
/tftpboot/linux-install/pxelinux.cfg

and that in the tftp definition file under /etc/xinted.d/tftp the following line is set correctly:
server_args = -s /tftpboot/linux-install

4) If pxelinux.0 does not exist under /tftp/linux-instal…

Howto add Java Plugin to Firefox on Linux

Although this is probably documented in numerous sites, here is my summary:

1) Install the JRE package from SUN (download and then 'chmod u+x' to the *.bin file).
2) cd to /usr/lib/firefox-1.5.0.5/plugins
3) Type:
[root@guydell plugins]# ln -s /usr/java/jre1.5.0_11/plugin/i386/ns7/libjavaplugin_oji.so
[root@guydell plugins]# ls -l
total 36
lrwxrwxrwx 1 root root 58 Mar 9 17:16 libjavaplugin_oji.so -> /usr/java/jre1.5.0_11/plugin/i386/ns7/libjavaplugin_oji.so
-rwxr-xr-x 1 root root 14288 Jul 29 2006 libnullplugin.so
-rwxr-xr-x 1 root root 7564 Jul 29 2006 libunixprintplugin.so

Invitation to the next IGT HPC Work group meeting

Image
Monday, March 26th, 2007
IGT Offices, Maskit 4, 5th Floor, Hertzliya
Agenda:
14:00-14:15: OPENING - Avner & Guy
14:15-14:50: PPF, JAVA, OPEN-SOURCE AND GRID: BEYOND THE TRADITIONAL GRID
Speaker: Laurent Cohen, ILOG, Inc. and JPPF founder
Abstract:

Traditional Grid architectures rely on a concept of job submission that comes with a set of constraints regarding the nature of the compute nodes, the ease of deployment of the jobs, and the effort required to use the Grid, which could otherwise be utilized to work on the problems to solve. JPPF offers an alternative, enabling a true heterogeneous nature of the Grid components, an ease of use that permits engineers, developers and scientists to focus on their domain rather than on the grid infrastructure, while retaining the benefits of the Grid technology to solve heavy and complex computational problems.We will present how the design and architectural choices in JPPF, in terms of programming language…

Summary of the IGT HPC Work group first year

Image
The following two slides give a perspective of the IGT HPC Work group activity during its first year.
There were 4 meetings with an average number of about 20 attendees.

If your company/research group would like to give a presentation in this forum please contact me.

The 2nd Israeli Association of Grid Technologies Annual Event

Image
Check this link.

OpenVPN my first trial

Image
This note describes establishing VPN connection between my laptop and my remote Linux server.
I was inspired by the first example in "Casting Your Net with OpenVPN" by Paul Duncan article that was published in the Linux Magazine.

First I created configuration files at both ends, see screen dump for the client configure file:

Then I started OpenVPN on both computers:

and similarly at the remote server:
After establishing the connection, I could connect via my private network:
I did SSH from my client (10.55.55.2) to my server (10.55.55.1):