The shared memory parallel processor
euler.cs.ubc.ca is available
for the use of faculty, staff, postdocs and students in the Scientific Computing Lab (SCL). It has a
lot of main memory (128 GB) but a relatively small number of processors (4 dual core), so it is best suited for jobs with large memory requirements but little or no parallelism. For jobs more suitable to highly parallel cluster environments, consider using the WestGrid machines (which also has very powerful batching systems), or other clusters in the CS Department.
Rather than institute a heavy-duty batching system, we would prefer to allow everybody interactive access to
euler whenever the mood to code strikes.
In order for this protocol (or lack thereof) to work, everybody has to behave nicely. Before starting a large job, a long job or lots of jobs, use the command
top to examine the current machine load. If the CPUs, memory or swap are in heavy use, maybe your job can wait a little bit.
Mailing list for users
However, there will be times when (near) exclusive use of
euler is necessary. In order to minimize conflicts and extraneous coordinating email, we have set up a mailing list
scl-compute so that users of
euler can inform each other of such situations without bothering the rest of the lab.
If you are using
euler, please subscribe to the mailing list by sending
an email message containing the text "
subscribe scl-compute" to
email@example.com. Other mailing list commands (eg
info, etc.) are standard to the majordomo system.
If you will need exclusive or near exclusive access to
euler for some period of time, please send an email specifying the time period to
firstname.lastname@example.org -- preferably several days in advance -- to warn other users. The same holds true if you plan to run a job or jobs that will take a long period of time (eg several days or more) and occupy a significant amount of memory (eg 32 GB or more) even if you do not need exclusive access.
If you find that somebody is interfering with exclusive access despite
a notice posted to
scl-compute (probably by looking at
top), please email that user directly (with a copy to Ian Mitchell) and request that the job(s) be stopped.
Users from outside of the Scientific Computing Lab
Although priority is given to members of SCL, everybody in the CS department
has access to
euler for compute purposes. If you think that you might want to use
euler in the future, please join the mailing list as described above so that you can find a suitable quiet period in which to run your jobs.
If you have a large job to run, please follow the protocol for the mailing list.
|Hardware Architecture||Sun x4600|
|Processors||4 x AMD 2.8GHz Dual Core|
|Main memory||128GB RAM (that's right -- GB)|
|Local disk||4 x 146GB HD|
|Operating System||OpenSuSE 10.3 (64 bit Linux)|
NUMA memory architecture
Note that this machine is a Non-Uniform Memory Access (NUMA) architecture.
In other words, it is shared memory -- every processor can access all of the
memory -- but the time it takes to access the memory varies significantly.
Each dual core processor has 32 GB directly attached, which will be
significantly faster to access for that processor than the remaining 96 GB
attached to the other processors. However, all main memory will be much
quicker than either network or disk access.
Local disk space
The local disks are primarily intended for high bandwidth, low latency swap
and temporary space. There are three directories (
/var/tmp2) with no quota restrictions for temporary files. There is about
245 GB disk space shared by swap (eg virtual memory) and
/tmp. Note that
/tmp is erased after a reboot. There are also about 137 GB in each of
/var/tmp2, which is space that will not be erased after a
Share the space & clean up when you are done
Keep in mind that main memory and these three directories on the local hard
drives are shared resources and are intended to be used for active
computations only. When they fill up, bad things might happen (and a single
full memory dump will almost fill them up). If you want to save files over
the longer term, use your home directories or the
/scl network drive space
-- slightly slower to access than the drives that are local to the machine,
but those network drives are backed up and we can get more space if we need
There should be the usual array of software available on the machine,
including the gnu compilers,
Matlab. Please let Ian Mitchell know if you
identify anything else that you want but is not installed so that I can pass
it along to the tech staff.
givens.cs.ubc.ca has been purchased as a "visualization workstation." The machine is sitting in the lab, so anybody is welcome to use it either in person or by remote login.
|Hardware Architecture||Standard Intel Workstation|
|Processors||Core 2 Duo 3GHz|
|Main memory||8GB RAM|
We added a mid-range programmable NVIDIA Quadro FX 3700 video card to
givens. This card has 512MB internal RAM and 112 parallel processing cores, and can be used for either high end visualization or some types of parallel scientific computing. The card supports NVIDIA's CUDA, which is a general purpose parallel computing architecture used by many of NVIDIA's graphics cards. You can download libraries for FFT and BLAS, program it in C, and even do some Matlab, in some cases demonstrating significant speedups.
Feel free to try some GPU scientific computing on
givens. If you run into any difficulties with the current configuration (eg: the GPU does not work, drivers are missing, you need a different OS, the monitor or physical location is poor, etc.) please let Ian Mitchell know -- we have started with a department standard installation, but would be happy to modify it if that would make the machine more useful.
If you are planning to use
givens for any significant work, please read and follow the protocols listed above for
euler. In particular, please subscribe to the
scl-compute mailing list so that you will be kept up-to-date on any system issues.