Copyright 2013 The Netherlands eScience Center
Xenon is a middleware abstraction library. It provides a simple programming interface to various pieces of software that can be used to access distributed compute and storage resources.
Xenon is developed by the Netherlands eScience Center as a support library for our projects. Several projects develop end-user applications that require access to distributed compute and storage resources. Xenon provides a simple API to those resources, allowing those applications to be developed more rapidly. The experience gained during end-user application development is used to improve the Xenon API and implementation.
Follow the instructions from jitpack.io to include Xenon as a dependency for Gradle, Maven, SBT, or Leiningen projects, e.g. Gradle:
allprojects {
repositories {
...
maven { url 'https://jitpack.io' }
}
}and
dependencies {
compile 'com.github.NLeSC:Xenon:2.0.0-rc2'
}
Or for a Maven project,
<repositories>
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
</repositories>
and
<dependency>
<groupId>com.github.NLeSC</groupId>
<artifactId>Xenon</artifactId>
<version>2.0.0-rc2</version>
</dependency>
Here are some examples of basic operations you can perform with Xenon:
import nl.esciencecenter.xenon.XenonException;
import nl.esciencecenter.xenon.credentials.PasswordCredential;
import nl.esciencecenter.xenon.filesystems.CopyMode;
import nl.esciencecenter.xenon.filesystems.CopyStatus;
import nl.esciencecenter.xenon.filesystems.FileSystem;
import nl.esciencecenter.xenon.filesystems.Path;
public class CopyFileLocalToSftpAbsolutePaths {
public static void main(String[] args) throws Exception {
// Use the file system adaptors to create file system representations; the remote file system
// requires credentials, so we need to create those too.
//
// Assume the remote system is actually just a Docker container (e.g.
// https://hub.docker.com/r/nlesc/xenon-ssh/), accessible via
// port 10022 on localhost
String location = "localhost:10022";
String username = "xenon";
char[] password = "javagat".toCharArray();
PasswordCredential credential = new PasswordCredential(username, password);
FileSystem localFileSystem = FileSystem.create("file");
FileSystem remoteFileSystem = FileSystem.create("sftp", location, credential);
// create Paths for the source and destination files, using absolute paths
Path sourceFile = new Path("/etc/passwd");
Path destFile = new Path("/tmp/password");
// create the destination file only if the destination path doesn't exist yet
CopyMode mode = CopyMode.CREATE;
boolean recursive = false;
// perform the copy and wait 1000 ms for the successful or otherwise
// completion of the operation
String copyId = localFileSystem.copy(sourceFile, remoteFileSystem, destFile, mode, recursive);
long timeoutMilliSecs = 1000;
CopyStatus copyStatus = localFileSystem.waitUntilDone(copyId, timeoutMilliSecs);
// throw any exceptions
XenonException copyException = copyStatus.getException();
if (copyException != null) {
throw copyException;
}
}
}The following code performs a wordcount of a file residing on a remote machine:
import nl.esciencecenter.xenon.credentials.PasswordCredential;
import nl.esciencecenter.xenon.schedulers.JobDescription;
import nl.esciencecenter.xenon.schedulers.JobStatus;
import nl.esciencecenter.xenon.schedulers.Scheduler;
public class SlurmSubmitWordCountJob {
public static void main(String[] args) throws Exception {
// Assume the remote system is actually just a Docker container (e.g.
// https://hub.docker.com/r/nlesc/xenon-slurm/), accessible to user 'xenon' via
// port 10022 on localhost, using password 'javagat'
String location = "localhost:10022";
String username = "xenon";
char[] password = "javagat".toCharArray();
PasswordCredential credential = new PasswordCredential(username, password);
// create the SLURM scheduler representation
Scheduler scheduler = Scheduler.create("slurm", location, credential);
JobDescription description = new JobDescription();
description.setExecutable("/usr/bin/wc");
description.setArguments("-l", "/etc/passwd");
description.setStdout("/tmp/wc.stdout.txt");
// submit the job
String jobId = scheduler.submitBatchJob(description);
long WAIT_INDEFINITELY = 0;
JobStatus jobStatus = scheduler.waitUntilDone(jobId, WAIT_INDEFINITELY);
// print any exceptions
Exception jobException = jobStatus.getException();
if (jobException != null) {
throw jobException;
}
}
}The output of the job will be written to /tmp/wc.stdout.txt file in the nlesc/xenon-slurm Docker container.
Xenon currently supports the following file access mechanisms:
file(local file manipulation)ftpsftpwebdavs3
Xenon currently supports the following job submission mechanisms:
localsshgridengineslurmtorque
Planned extensions include:
- Swift
- HDFS (almost done)
- YARN
- GridFTP
- glite
- Azure-Batch
- Amazon-Batch
Xenon's JavaDoc is available online at https://jitpack.io/com/github/NLeSC/Xenon/master-SNAPSHOT/javadoc/index.html.
Xenon is copyrighted by the Netherlands eScience Center and releases under the Apache License, Version 2.0.
See http://www.esciencecenter.nl for more information on the Netherlands eScience Center.
See the "LICENSE" and "NOTICE" files for more information.
This product includes the SLF4J library, which is Copyright (c) 2004-2013 QOS.ch See "notices/LICENSE.slf4j.txt" for the licence information of the SLF4J library.
This product includes the JSch library, which is Copyright (c) 2002-2012 Atsuhiko Yamanaka, JCraft,Inc. See "notices/LICENSE.jsch.txt" for the licence information of the JSch library.
This product includes the Logback library, which is Copyright (c) 1999-2012, QOS.ch. See "notices/LICENSE.logback.txt" for the licence information of the Logback library.
This product includes the JaCoCo library, which is Copyright (c) 2009, 2013 Mountainminds GmbH & Co. KG and Contributors. See "notices/LICENSE.jacoco.txt" for the licence information of the JaCoCo library.
This project includes the JUnit library. See "notices/LICENSE.junit.txt" for the licence information of the JUnit library.
This project includes the Mockito library, which is Copyright (c) 2007 Mockito contributors. See "notices/LICENSE.mockito.txt" for the licence information of the Mockito library.
This project includes the Java CoG Kit, which is Copyright (c) 1999-2006 University of Chicago. See "notices/LICENSE.cog-jglobus.txt" for the licence information of the Java CoG Kit.
This project includes the Commons-logging library, which is Copyright (c) Apache Software Foundation. See "notices/LICENSE.commons-logging.txt" for the licence information of the Commons-logging library.
This project includes the log4j library, which is Copyright (c) Apache Software Foundation. See "notices/LICENSE.log4j.txt" for the licence information of the log4j library.
This project includes the Legion of the Bouncy Castle Java cryptography APIs, which are Copyright (c) 2000-2013 The Legion Of The Bouncy Castle. See "notices/LICENSE.bouncycastle.txt" for the licence information of this library.
This project includes the pureTLS library, which is Copyright (c) Claymore Systems, Inc. See "notices/LICENSE.puretls.txt" for the licence information of the pureTLS library.
This project includes libraries produced by the Cryptix Project. See "notices/LICENSE.cryptix.txt" for the licence information of these libraries.