About CARMEN Services

What are CARMEN Services?

CARMEN Services allow users to take their own analysis code and deploy it on the CARMEN platform. Once deployed, sharing settings can be applied to the service so that it can be seen and run by the owner, colleagues, groups, or all CARMEN users.

A CARMEN service is simply some analysis code, written in either Matlab, Python, R, Java, C/C++, etc as a command-line application. This command line application is then wrapped in a dynamically loadable Java class (in a JAR file), and associated with a metadata document (XML file). The metadata documents provides information to the user and the system, such as what the service is called, what the parameters are called and their type, which OS platform to run the service on and which software environment to use.

The service runs in the CARMEN execution environment which is a heterogeneous network in our private cloud. CARMEN Services support multiple operating system (OS) and language platforms. The target platform/software is specified when the service is built, and automatically used at service runtime. A simple interface is provided on the CARMEN portal to allow users to run the service and see it’s progress and results. As the runtime details are hidden, the user can concentrate on what the service does, and need not care what OS the service runs on, and in what language.

A search facility allows users to find services in the service repository. Once a user finds a service of interest, it can be bookmarked for future use.

The following subsections are provided for those who may be interested in the technical details of CARMEN Services. It is in no way needed to be able to create and run services on our platform.

The CARMEN Execution Environment

The CARMEN execution environment runs on our private cloud of  HP Proliant BL465c G7 plus a few physical Xeon-based servers. Onto these servers we create our execution node instances which are 8-core 16GB Virtual Machines (VM) based on one of three operating systems; CentOS Linux 5, Scientific Linux 4, and Windows Server. On each execution node we run an execution manager which is a piece of software (java servlet) that communicates with the main CARMEN platform and co-ordinates the execution process.

What Happens When a Service Runs

To run a service, the user hits the “Run” button for that particular service and sets the input parameters. The user then needs to specify where in their CARMEN user space they wish the results to be placed. They can then start the execution process and sit back until it completes. The system then takes a note of all the settings selected by the user and performs the following tasks:

  • The CARMEN system looks at the service metadata to determine which platform (OS) to run the service on, and decides upon a suitable execution node.
  • The system contacts the execution node manager servlet and informs it about the following details: service ID, input parameter file IDs, input parameter values,  and software environment setup details.
  • The service manager then checks to see if it already has the required service on it’s local disks. If not, it downloads the service implementation file (jar file) from the data repository. This is then unjarred and the service wrapping class is dynamically loaded into the service manager client interface.
  • The service manager downloads any input files from the service repository to it’s local disks.
  • The software environment is set up on the execution node.
  • The service is called with the input parameters, and the underlying analysis code is monitored for completion.
  • Once the service completes, the required output files are placed into the user’s data space in the CARMEN data repository.
  • The user is informed of the completion status.