[Orca-users] Re: Orca in too many servers

[Sean O'Neill]

At 04:37 PM 8/27/2002 -0300, Edgard Dapás wrote:

>I'm new with orca
>I have 14 servers with orca, I'm creating the HTML and the GIF files  in
>every server.
>  Is there anyway to create all the html pages and gif in one server?

Let the babble begin ... sorry big response but it should give you a good 
idea how it can be done.

This is the way most folks do it who have lots of machines.  A great 
utility to use for this purpose is rsync in combination with SSH using 
pub/priv key authentication.  Assuming you do it this way, the easiest 
method is to create your ssh keys without passwords so your crontab 
controlled rsync process doesn't get hit with a password prompt when trying 
to rsync the data over from all the machines.

The way I have it setup is the ORCA data collector runs as the login 
"perfboy" on all my machines.  The "perfboy" login is a member of the "sys" 
group so it has read-only priviledges for /dev/kmem and /dev/mem.  Assuming 
you do it this way, don't run the collectors as root.  This opens up 
problems with security when collecting the data using rsync over 
ssh.  Actually, there is no reason to run the ORCA collectors as root for 
any reason even if you don't do it this way that I can think of.

My "graphing" machine is setup so that the "perfboy" login on this machine 
can ssh into all my other machines using ssh pub/priv keys without a 
password.  I have a crontab job that rsync's the data from the other 
machines over every 10 minutes.  My ORCA "graphing" process runs as root on 
the graphing machine and it writes the result images and HTML docs into my 
Apache docroot area.

Collection script looks like this:

#!/bin/ksh
#
PATH=/usr/bin:/usr/sbin:/sbin:/usr/local/bin; export PATH

LOGIN=perfboy
SRC_LIST=/local/home/${LOGIN}/scripts/sources.txt

for LINE in $(cat ${SRC_LIST})
do
    # Just for reference, assuming a MINIMAL data transfer of only deltas 
on the
    # current day's data file, the ssh command below takes about 2-3 
seconds per
    # server.  This means, conservatively, 20 servers / minute, 200 servers /
    # 10 minutes. Important to note this because the crontab running this
    # script runs every ten minutes.
    #
    # Uses ${parameter%%pattern} Remove large right pattern ksh parameter
    # expansion
    SRC=${LINE%%:*}
    # Uses ${parameter##pattern} Remove large left pattern ksh parameter
    # expansion
    HOSTNAME=${LINE##*:}
    #
    # Note the "/" trailing slash in SRC_DIR IS VERY IMPORTANT !!!! RTFM
    #
    SRC_DIR=${LOGIN}@${SRC}:/local/home/${LOGIN}/orca/var/orca/orcallator/${HOSTNAME}/
    DST_DIR=/local/home/${LOGIN}/orca/var/orca/orcallator/${HOSTNAME}

    rsync -a -q --delete -essh ${SRC_DIR} ${DST_DIR}

done

The sources.txt file simply looks like this:

10.107.145.22:certification-app01
10.107.145.24:certification-app02
<SNIP>

I have another "maintenance" crontab job on each machine that expires the 
ORCA text data when the data is 15+ days old.

#
# Delete orcallator directory data files that are 15 days or older.
#
0 1 * * * [ -d /local/home/perfboy/orca/var/orca/orcallator/`hostname | 
nawk '{split($0,A,".");print A[1]}'` ] && find 
/local/home/perfboy/orca/var/orca/orcallator/`hostname | nawk 
'{split($0,A,".");print A[1]}'` -name "orcallator-*" -mtime +15 -exec rm {} \;

On the graphing machine, I backup all the data for all machines every to a 
"backup" directory that is currently maintained indefinitely.  This is so I 
can do analysis for stuff in the past and also so I can recreate my graphs 
should some bonehead [like me :)] nuke my RRD files.

Backup script looks like this:

#!/bin/ksh
#
# First include means to include all directories
# Second include means to include all files matching pattern orcallator*gz
# The exclude means drops EVERYTHING else.
# So this basically means the orcallator.pid file and the uncompressed data
# files are ignored.  This could cause a problem though.  I have seen some
# cases where previous day data files are not compressed for some unknown
# reasons e.g. bad stop/start sequence, process kill and restarted in unknown
# state, etc. Need to think about some way to overcome this little problem.
#
/usr/local/bin/rsync -a -q --include "*/" --include "orcallator*gz" 
--exclude "*" /local/home/perfboy/orca/var/orca/orcallator/ 
/local/home/perfboy/databackup

Hope this BIG response help you out.


--
........................................................
......... ..- -. .. -..- .-. ..- .-.. . ... ............
.-- .. -. -... .-.. --- .-- ... -.. .-. --- --- .-.. ...

Sean O'Neill