[Clas12_software] A new queue is created for your jobs

[Vardan Gyurjyan]

Thank you Ying,

We run our single data-stream test using infiband. The test shows that ClaRA scales linearly over 10 nodes, and is able to keep up with 2KHz DAQ rate. However, we would like to scale more than 10 nodes to see when we will face a bottleneck. I would like to mention that this mode is for the online/real-time processing of the experimental data. 

The next step will be to start tests using multiple data-streams, i.e. separate IO for each node (off-line data processing/cooking mode). Currently I am writing so called staging service that is responsible for the data-file management (copying data-files to and from the local persistent storage). We will need, again, more than 10 farm nodes in the queue for the off-line data processing mode test.

Sincerely,
Vardan


----- Original Message -----
From: "Ying Chen" <ychen at jlab.org>
To: "Vardan Gyurjyan" <gurjyan at jlab.org>
Cc: scicomp at jlab.org
Sent: Monday, June 10, 2013 12:21:08 PM
Subject: A new queue is created for your jobs

Vardan,

We have finished the work to set up a separated queue for jobs that use a whole node. So far a few 16 core nodes (6-10 farm12xxx) are configured for this queue. There is no change in user's script to use this new queue, the queue routing is made in server level (all jobs ask 16 cores will be route to this new queue by Auger). The only change in your jsub is to ask 16 cpu (not 17) so it use the whole node. We will configure the max jobs in these nodes to 16 (it is 20 now).

In the future, we can easily add/remove nodes, increase the priority of this new queue when need. Let me know if you experience any problem.


Ying