1. Introduction à EDGeS

Dans le contexte du Traitement Distribué des Données, EDGeS est un projet permettant la communication entre la Grille de Service EGEE, qui utilise l’intergiciel gLite, et les Grilles de PC utilisant les intergiciels BOINC ou XtremWeb-HEP.EDGeS est un projet européen de Traitement Distribué des Données.

Financé par EC FP7 Research Infrastructures sous le numéro 211727, EDGeS a démarré le 01 Janvier 2008 et s’est achevé avec succès le 31 Mars 2010.

Le but était de permettre la communication entre la Grille de Service EGEE, qui utilise l’intergiciel gLite, et les Grilles de PC utilisant les intergiciels Boinc ou XtremWeb-HEP.

EDGeS a atteint ces objectifs et a même réussi à la communication entre la Grille de Service EELA, qui utilise l’intergiciel gLite, et la Grille de PC brésilienne utilisant l’intergiciel OurGrid.

EDGeS est maintenant relayé par les projets EDGI et DEGISCO.

Mise à jour de la passerelle XWHEP-EGEE

In the opposite of what announced before, each XWHEP deployment must have its own XWHEP->EGEE bridge due to performance problems. It appeared that it is too heavy to manage several XWHEP server on a single XWHEP->EGEE bridge. We then had to correct this bridge and implement a new version «V2» as member of activity «JRA1». As V2 was implemented and debugged, we deploy that new version in the context of this SA1 activity.

In V1, any job submitted with an X509 proxy had to be computed by an EGEE resource, and an EGEE resource only: an XWHEP resource had no chance to compute such a job. In V2, any job submitted with an X509 can be computed by any resource, including XWHEP (non secured resources) as well as EGEE ones (secured resources). The XWHEP logging and bookkeeping protocols and services ensure enough security to track any malicious behaviour. The end user X509 proxy security and integrity are still ensured because resources connected to XWHEP server never download X509 proxies. Their are exclusively and only downloaded and used by the bridge itself. This clearly means, in both versions, that the bridge must be run in a secured resource.

Figure «Fig 1 : XWHEP->EGEE bridge V1 Vs V2» shows the performances differences between V1 and V2. The blue line represent the amount of jobs submitted with an X509 certificate proxy -hence able to be computed by any EGEE resources. The green line represents the total amount of completed jobs, independently of their submission type (with or without X509 proxy), hence computed by XWHEP and EGEE resources. The red line represents job failures (data unavailable etc).
This figure also shows two different bar types: a black and a blue one. The black one represents the amount of connected EGEE resources.This resources are allocated when the XWHEP->EGEE bridge has found XWHEP jobs submitted with X509 proxy. When such a job is found, the bridge download the end user proxy and uses it to an XWHEP pilot job to EGEE (a pilot job being an XWHEP worker that will connect to the XWHEP server and request an XWHEP job to compute).
The blue bar represents the amount of job effectively computed by EGEE resources.
We can see two different periods in Figure 1: before october 2009 and since november 2009. In the first period, we were using the XWHEP->EGEE bridge version V1; since the second period, we use the bridge version V2. In first period we can see that the bridge unnecessarily overbooked EGEE resources which were really used in a 10% ratio only, which were just very bad. This was due to the fact that the bridge was designed to try to manage jobs from many XWHEP deployment connecting to all XWHEP server. In the second period, using the bridge V2, the average effective usage of connected EGEE resources is 75%, which is very good. this is due to the fact that the bridge does not connect to XWHEP servers, but directly to the database used by this server; of course the bridge does not modify the database itself : it only connects to the database in read only mode to ensure that it does not perturb the server. Connecting to the database ensures a better performance rates but does not allow the management of more than one XWHEP server per bridge since database accesses are usually not allowed outside an administrative domain.





Fig 1 : XWHEP->EGEE bridge V1 Vs V2

XWHEP pour EDGeS

Les partenaires du projet EDGeS ont décidé d’utiliser XWHEP 5.6.0.

Nous présentons ici les versions d’XWHEP testées et utilisées dans le cadre du projet EDGeS.

  • 18 août 2009 : XWHEP pour EDGeS 1.1.0
    • cette version est un tag EDGeS pour XWHEP 5.7.4 où :
      1. le nom du répertoire contenu dans le TAR est fixé à XWHEP-EDGeS
      2. la gestion des applications linkée dynamiquement est inactive
      3. on peut choisir le compte à utiliser pour tourner le middleware
        (il suffit de positionner la variable “system.user” dans le fichier build.conf)

    • 25 mai 2009 : XWHEP pour EDGeS 1.0.1
      • cette version est un tag EDGeS pour XWHEP 5.7.1 où :
        1. le nom du répertoire contenu dans le TAR est fixé à XWHEP-EDGeS
        2. la gestion des applications linkée dynamiquement est inactive
        3. on peut choisir le compte à utiliser pour tourner le middleware
          (il suffit de positionner la variable “system.user” dans le fichier build.conf)

      • 31 mars 2009 : Les sources XWHEP-EDGeS 1.0.0 ; Le binaire de XWHEP-EDGeS 1.0.0

      • 31 mars 2009 : XWHEP-EDGeS 0.0.0

La passerelle EDGeS de XWHEP à EGEE est en production

Nous avons le plaisir d’annoncer la mise en production du bridge XW->EGEE.

Le bridge est un service installé au LAL sur l’UI grid11 et connecte trois plates-formes XWHEP.
Les pages suivantes montrent l’activité de notre bridge:

Les “anciennes” actualités.

  • May 10th, 2005
    • Platform upgraded to 1.7.0.
  • Feb 3rd, 2006 : A major security hole found in Linux
    This does not directly concern XtremWeb, but may occur on Linux machines, including Linux workers:

    It seems that Linux kernel, up to 2.4.23, does not include OOM (Out Of Memory) process. This may leads to serious damages, including data loss, if any process (submitted with XtremWeb or not) requests far more than available memory.
    Linux 2.6.x does not seem to suffer such lacks.

    This is still under expertise…

  • Decembre 15th, 2005
    • Platform upgraded to 1.5.0.
  • Septembre 15th, 2005
    • EGEE/XtremWeb : a new gliding
  • August 23th, 2005
    • Platform upgraded to 1.5.0-beta.
  • April 20th, 2005
    • Platform upgraded to 1.4.0.
  • January 20th, 2005
    • Platform upgraded to 1.3.11.
  • January 14th, 2005
    • Platform upgraded to 1.3.10.
  • August 6th, 2004
    • Platform upgraded to 1.3.5.
  • July 1st, 2004
    • Platform upgraded to 1.2.0 ;
    • Documentation available : please click here
    • Downloads available : please click here.
  • Jun 25th, 2004
    • More than 7,000 jobs executed : please click here.
  • Jan 23rd, 2004
    The full production process is now installed :

    • the platform processes Aires simulations on demand, using the XtremWeb client;
    • simulation results are automatically stored on HPSS storage at CC IN2P3.
  • Oct 31th, 2003
      XtremWeb v1r2-rc5 installed at LAL