[Orca-dev] Re: orcora - an oracle collection agent for orca
D.C.Lawie
duncanl at demon.net
Fri Sep 17 08:53:56 PDT 1999
This is getting a bit too messy for me to try to intersperse responses again.
I don't think orcora would (or should) go into orcallator.se. I think I misunderstood you earlier. orcora is
written in perl, using DBD and DBI::Oracle. I think I've managed to hit the attach button this time.
There are also two further attachments - the short one is a diff of my orcallator.se with the version supplied
when I downloaded orca (1.7). The second is a full diff with the current version 1.12 sent today. I am only
slightly hopeful that all these will get forwarded to the mailing list properly.
Cheers,
Duncan.
Blair Zajac wrote:
> Duncan,
>
> I'm cc'ing a copy of these email to the orca-developers email list, since
> it seems appropriate.
>
> "D.C.Lawie" wrote:
> >
> > Blair Zajac wrote:
> >
> > > Duncan,
> > >
> > > Your work looks great. It would be interesting to integrate this
> > > with orcallator.se and could be included with the Orca package.
> > > It would be great for trend and problem analysis.
> >
> > I feel that the major way in which "third parties" such as myself can help on
> > orca at the moment is by providing additional collection agents. As it stands,
> > orcora ought to run on most u*ix platforms, making orca useful outside the
> > Solaris community. Hopefully, this would encourage others to write further
> > collection agents also.
>
> Yes, I agree here. I think one possible collection tool for cross-OS use
> would be a C code that uses the code with top to generate orcallator.se
> style output that Orca would read.
>
> >
> > I don't see how orcora can integrate with SE - is it possible to kick off a
> > separate program from within SE? Are you interested in the changes I've made
> > to orcallator.se for filesystem/disk busy info?
>
> I think the orcora would be a totally separate process from orcallator.se
> unless it could be made into one more handle_ function. What kind of
> work would this take? Do you need some special hooks to get the data
> you are looking for?
>
> Please send me (or the list) the filesystem/disk busy changes you made. I
> am attaching the latest version of orcallator.se to this message.
>
> >
> > My file header looks like this:
> >
> > timestamp locltime started BPrsrlTt bchr dchr memsrt shrdpfree shrdprel wtwl
> > imml lcghr lcphr rdospc rdoalloc rbckcntsn recursv shrtblscn prsex chnd
> > freelist tsmax tsavg dbsize datasize TS.SYSTEM TS.....
> >
> > In fact, orcora is attached.
>
> It didn't come through to me.
>
> >
> > > It sounds like
> > > a totally separate tool would be appropriate for generating
> > > your pages, instead of integrating this into orca.pl itself.
> > > Currently orca.pl is going through a major restructing anyway
> > > and would probably be left separate.
> >
> > I agree that the "traffic light" side of what I have written should be kept
> > reasonably separate. I'm not sure how far I want to go with the issue of
> > alerts as many sites are likely to use gui tools that come with SE. It was
> > written because this site cannot use those X-tools, and may be useful for
> > other similar sites with an "email on RED/BLACK" kind of addition.
> > Alternatively, it may work better as a client of something like Jim Trocki's
> > mon system.
> >
> > > Thanks for the description page, the next version of Orca will
> > > have HREFs for the data targets that can point to a document.
> > > I'll probably take your document and help flesh it out some.
> >
> > I'm glad to be any help I can. Orca is leaps and bounds ahead of what I was
> > originally doing. Using your stuff has resulted in a much better monitoring
> > system here.
>
> Great to hear that Orca made a difference.
>
> Cheers,
> Blair
>
> >
> > Have a good weekend,
> > Duncan.
> >
> > >
> > > Blair
> > >
> > > Duncan Lawie wrote:
> > > >
> > > > Blair,
> > > >
> > > > I have a couple of developments for orca.
> > > >
> > > > 1. As orcollator.se produces a gggwAggg type field in each line, I have
> > > > written a "status map" perl script which converts this field (and the
> > > > time stamp) into a web page with appropriately coloured boxes. It is
> > > > pretty simple but may be of use where sites aren't using Cockroft's
> > > > monitoring tools directly. You can have a look at sample web pages at
> > > > http://www.hoopoes.com/sol.html and http://www.hoopoes.com/dbs.html .
> > > >
> > > > 2. As the second page indicates, I have also written an oracle
> > > > "collection agent" for orca, producing output similar to that from
> > > > orcallator. It is written in perl and uses perl DBI.
> > > >
> > > > If you are interested in either/both of these I can send the code and/or
> > > > more documentation. I would like to announce them on orca-discuss at
> > > > some point but am happy to take your guidance on integrating them better
> > > > with orca first ( or, alternatively, separating them more completely).
> > > >
> > > > I have also had a stab at providing some documentation on what is being
> > > > presented in the standard orca graphs - plus my odd extensions. It is
> > > > derived essentially from Cockroft and surfing around Sun's websites so
> > > > I'm not really prepared to unleash it on the public as yet. There is a
> > > > copy of this at http://www.hoopoes.com/orca_explain.html .
> > > >
> > > > Yours,
> > > > Duncan.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: orcora.pl
Type: application/x-perl
Size: 13459 bytes
Desc: not available
Url : /pipermail/orca-dev/attachments/19990917/2119b263/attachment.bin
-------------- next part --------------
37a38
> #define WATCH_MOUNTS
56a58,64
> #ifdef WATCH_MOUNTS
> #include <mnt_class.se>
> #include <statvfs.se>
> #include "fs_class.se"
> #endif
>
>
88a97,102
> #ifdef WATCH_MOUNTS
> mnttab_t mnt$m;
> mnttab_t tmp_mnt;
> fs_t vfsstat$f;
> fs_t tmp_fs;
> #endif
279a294
> fprintf(stderr, " -DWATCH_MOUNTS watch usage of mount points\n");
456a472,475
>
> // Take care of mount points
> #ifdef WATCH_MOUNTS
> handle_mounts();
457a477
> #endif
517a538
> put_output("scnrt", sprintf("%5.1f", pvm.scan));
595c616,620
< for (i=0; i < GLOBAL_disk[0].disk_count; i++) {
---
> for (i=0; i < GLOBAL_disk_count; i++) {
> put_output(sprintf("dsk.c%dt%dd%d",
> GLOBAL_disk[i].info.controller, GLOBAL_disk[i].info.target,
> GLOBAL_disk[i].info.device),
> sprintf("%6.2f", GLOBAL_disk[i].run_percent));
644a670,709
> #ifdef WATCH_MOUNTS
> handle_mounts()
> {
>
> /* traverse the mount table to find mounted ufs/vxfs file systems */
>
> for (mnt$m.number$ = 0; ; mnt$m.number$++)
> {
> tmp_mnt = mnt$m;
>
> if (tmp_mnt.number$ == -1) { break; }
>
> if ( ( tmp_mnt.mnt_fstype == "ufs" || tmp_mnt.mnt_fstype ==
"vxfs" )
> && (tmp_mnt.mnt_flags & MNT_RW) != 0
> )
> {
> vfsstat$f.mp$ = tmp_mnt.mnt_mountp;
> tmp_fs = vfsstat$f;
>
> if (vfsstat$f.status$ == -1)
> {
> printf("WARNING: statvfs returned error for %s, ",
tmp_mnt.mnt_mountp);
> printf("filesystem will be ignored\n");
>
> continue;
> }
> else
> {
> put_output( sprintf("mntu.%s", tmp_mnt.mnt_mountp),
> sprintf("%10ld", tmp_fs.f_used));
> put_output( sprintf("mnta.%s", tmp_mnt.mnt_mountp),
> sprintf("%10ld", tmp_fs.f_avail));
> }
> }
> }
> }
> #endif
>
>
>
-------------- next part --------------
0a1
>
7c8
< // Author: Blair Zajac <bzajac at geostaff.com>
---
> // Author: Blair Zajac <bzajac at geostaff.com>.
11c12,33
< // Version 1.7: Mar 25, 1999 Speed up by 20% and simplify count_proc.
---
> // Version 1.12: Sep 14, 1999 Add the page scan rate as scanrate in
> // measure_cpu.
> // Version 1.11: Aug 13, 1999 Add the number of CPUs as ncpus. Move
> // measure_disk and measure_ram sooner in the
> // list of subsystems to handle. Increase
the
> // number of characters for each network
> // interface from four to five. Add new disk
> // reads, writes, Kbytes read, and Kbytes
> // written per second. Add number of bytes
> // of free memory in bytes as freememK.
> // Version 1.10: Jul 28, 1999 Measure the process spawn rate if
WATCH_CPU
> // is defined and the user is root.
> // Version 1.9: Jun 2, 1999 If WATCH_YAHOO is defined, then process
the
> // access log as a Yahoo! style access log.
> // Restructure the code to handle different
> // web server access log formats.
> // Version 1.8: Jun 1, 1999 If the environmental variable WEB_SERVER
is
> // defined, use its value of the as the name
> // of the process to count for the number of
> // web servers on the system. If WEB_SERVER
> // is not defined, then count number of
httpd's.
> // Version 1.7: Mar 25, 1999 Simplify and speed up count_proc by 20%.
28,38c50,59
< #define WATCH_CPU
< #define WATCH_MUTEX
< #define WATCH_NET
< #define WATCH_TCP
< #define WATCH_NFS
< #define WATCH_DISK
< #define WATCH_DNLC
< #define WATCH_INODE
< #define WATCH_RAM
< #define WATCH_PAGES
< #define WATCH_MOUNTS
---
> #define WATCH_CPU 1
> #define WATCH_MUTEX 1
> #define WATCH_NET 1
> #define WATCH_TCP 1
> #define WATCH_NFS 1
> #define WATCH_DISK 1
> #define WATCH_DNLC 1
> #define WATCH_INODE 1
> #define WATCH_RAM 1
> #define WATCH_PAGES 1
58,61c79,85
< #ifdef WATCH_MOUNTS
< #include <mnt_class.se>
< #include <statvfs.se>
< #include "fs_class.se"
---
> #if WATCH_CPU || WATCH_WEB
> #include <proc.se>
>
> #ifdef WATCH_CPU
> // This is the maximum pid on Solaris hosts.
> #define DEFAULT_MAXPID 30000
> #include <fcntl.se>
64,66c88
<
< #ifdef WATCH_HTTPD
< #include <proc.se>
---
> #ifdef WATCH_WEB
67a90,153
>
> // Define this macro which returns the size index for a file of a
> // particular size. This saves the overhead of a function call.
> #define WWW_SIZE_INDEX(size, size_index) \
> if (size < 1024) { \
> size_index=0; /* under 1KB */ \
> } else { \
> if (size < 10240) { \
> size_index=1; /* under 10K */ \
> } else { \
> if (size < 102400) { \
> size_index=2; /* under 100KB */ \
> } else { \
> if (size < 1048576) { \
> size_index=3; /* under 1MB */ \
> } else { \
> size_index=4; /* over 1MB */ \
> } \
> } \
> } \
> } \
> dwnld_size[size_index]++;
>
> // Handle the reply code from the server.
> #define WWW_REPLY_CODE(word) \
> if (word != nil) { \
> if (word == "304") { \
> httpop_condgets++; \
> } \
> else { \
> first_byte = word; \
> if (first_byte[0] == '4' || first_byte[0] == '5') { \
> httpop_errors++; \
> } \
> } \
> }
>
> // Handle the method of the object served. This define only works with
> // non-proxy servers.
> #define WWW_METHOD1(word) \
> if (word != nil) { \
> switch (word) { \
> case "get": \
> case "GET": \
> httpop_gets++; \
> break; \
> case "post": \
> case "POST": \
> httpop_posts++; \
> break; \
> case "head": \
> case "HEAD": \
> ishead = 1; \
> httpop_condgets++; \
> break;
>
> #ifdef WATCH_SQUID
> #define WWW_METHOD2 \
> case "icp_query": \
> case "ICP_QUERY": \
> squid_icp_queries++; \
> break;
> #else
> #define WWW_METHOD2
69a156,164
> #define WWW_METHOD_END \
> default: \
> break; \
> } \
> }
> #define WWW_METHOD(word) WWW_METHOD1(word) WWW_METHOD2 WWW_METHOD_END
> #endif
> #endif
>
97,102d191
< #ifdef WATCH_MOUNTS
< mnttab_t mnt$m;
< mnttab_t tmp_mnt;
< fs_t vfsstat$f;
< fs_t tmp_fs;
< #endif
111c200
< string nodename; // Name of machine running the
script.
---
> string nodename; // Name of this machine.
117a207,224
> #ifdef WATCH_CPU
> int can_read_kernel = 0; // If the kernel can be
read.
> long kvm$mpid; // The last created PID.
>
> // These variables store the mpid before and after the standard interval.
> long mpid_previous;
> long mpid_current;
> ulonglong mpid_then;
> ulonglong mpid_now;
>
> // These variables store the mpid before and after 5 second intervals.
> long mpid5_previous;
> long mpid5_current;
> ulonglong mpid5_then;
> ulonglong mpid5_now;
> double mpid5_rate;
> #endif
>
119,126c226,235
< #ifdef WATCH_HTTPD
< ulong log_file;
< string log_name = getenv("WEB_LOG");
< string log_gateway = getenv("GATEWAY");
< uint log_gatelen;
< stat_t log_stat[1];
< ulong log_ino;
< long log_size;
---
> #ifdef WATCH_WEB
> string www_server_proc_name = getenv("WEB_SERVER");
> string search_url = getenv("SEARCHURL");
> string www_log_filename = getenv("WEB_LOG");
> string www_gateway = getenv("GATEWAY");
> ulong www_fd;
> uint www_gatelen;
> stat_t www_stat[1];
> ulong www_ino;
> long www_size;
128,130c237,239
< double log_interval; // Hi-res interval time.
< ulonglong log_then;
< ulonglong log_now;
---
> double www_interval; // Hi-res interval time.
> ulonglong www_then;
> ulonglong www_now;
132,134c241,243
< double log5_interval; // Actual hi-res 5 second interval.
< ulonglong log5_then;
< ulonglong log5_now;
---
> double www5_interval; // Actual hi-res 5 second
interval.
> ulonglong www5_then;
> ulonglong www5_now;
147d255
< string search_url;
150a259,264
> #if WATCH_PROXY || WATCH_SQUID || WATCH_YAHOO
> // If we're watching a Yahoo log, then take the transfer time to be the
> // processing time.
> double www_dwnld_time_sum; // transfer time
> double www_dwnld_time_by_size[5]; // mean transfer time by size bin
> #endif
152,153d265
< double prxy_squid_xfer_sum; // transfer time
< double prxy_squid_xfer_by_size[5]; // mean transfer time by size bin
276c388,390
< fprintf(stderr, " setenv WEB_LOG /ns-home/httpd-80/logs/access
- location of web server log\n");
---
> fprintf(stderr, " setenv WEB_SERVER netscape\n");
> fprintf(stderr, " setenv WEB_LOG /ns-home/httpd-80/logs/access
- location of web
> server log\n");
279c393,394
< fprintf(stderr, " setenv SEARCHURL srch.cgi - match for search
scripts, default is search.cgi\n");
---
> fprintf(stderr, " setenv SEARCHURL srch.cgi - match for search
scripts, default is
> search.cgi\n");
281c396
< fprintf(stderr, " -DWATCH_HTTPD watch httpd access log\n");
---
> fprintf(stderr, " -DWATCH_WEB watch web server access logs\n");
294d408
< fprintf(stderr, " -DWATCH_MOUNTS watch usage of mount points\n");
302c416
< // Run forever. If WATCH_HTTPD is defined, then have measure_httpd()
---
> // Run forever. If WATCH_WEB is defined, then have measure_web()
312c426
< while (now + interval*0.5 > sleep_till) {
---
> while (sleep_till < now + interval*0.5) {
316,317c430,431
< #ifdef WATCH_HTTPD
< measure_httpd(sleep_till);
---
> #ifdef WATCH_WEB
> measure_web(sleep_till);
319,322c433
< while (now < sleep_till) {
< sleep(sleep_till - now);
< now = time(0);
< }
---
> sleep_till_and_count_new_proceses(sleep_till);
332c443
< handle_os(now, tm_now);
---
> measure_os(now, tm_now);
334c445
< #ifdef WATCH_HTTPD
---
> #ifdef WATCH_WEB
349,350c460,462
< // Sleep to give the disks a chance to update.
< sleep(DISK_UPDATE_RATE);
---
> #ifdef WATCH_CPU
> int i;
> #endif
352,353c464,473
< // Get the clock tick rate.
< hz = sysconf(_SC_CLK_TCK);
---
> #ifdef WATCH_CPU
> // Initialize the process spawning rate measurement variables.
> // Determine if the kernel can be read to measure the last pid.
> i = open("/dev/kmem", O_RDONLY);
> if (i != -1) {
> close(i);
> can_read_kernel = 1;
> mpid_previous = kvm$mpid;
> mpid_then = gethrtime();
> mpid_current = mpid_previous;
355,356c475,480
< // Get the page size.
< page_size = sysconf(_SC_PAGESIZE);
---
> mpid5_then = mpid_then;
> mpid5_previous = mpid_previous;
> mpid5_current = mpid_previous;
> mpid5_rate = 0;
> }
> #endif
358,363c482,485
< // Calculate the system boot time.
< boot_time = time(0) - (kstat$misc.clk_intr / hz);
<
< #ifdef WATCH_HTTPD
< search_url = getenv("SEARCHURL");
< if (search_url == nil) {
---
> #ifdef WATCH_WEB
> // Initialize those variables that were not set with environmental
> // variables.
> if (search_url == nil || search_url == "") {
367,369c489,490
< if (log_gateway == nil) {
< log_gateway = "NoGatway";
< log_gatelen = 0;
---
> if (www_server_proc_name == nil || www_server_proc_name == "") {
> www_server_proc_name = "httpd";
370a492,496
>
> if (www_gateway == nil || www_gateway == "" ) {
> www_gateway = "NoGatway";
> www_gatelen = 0;
> }
372c498
< log_gatelen = strlen(log_gateway);
---
> www_gatelen = strlen(www_gateway);
375,379c501,508
< log_file = fopen(log_name, "r");
< if (log_file != 0) {
< stat(log_name, log_stat);
< log_ino = log_stat[0].st_ino;
< log_size = log_stat[0].st_size;
---
> // Initialize the web server watching variables. Move the file pointer
> // to the end of the web access log and note the current time.
> if (www_log_filename != nil) {
> www_fd = fopen(www_log_filename, "r");
> if (www_fd != 0) {
> stat(www_log_filename, www_stat);
> www_ino = www_stat[0].st_ino;
> www_size = www_stat[0].st_size;
381c510
< fseek(log_file, 0, 2);
---
> fseek(www_fd, 0, 2);
382a512
> }
384,385c514,515
< log_then = gethrtime();
< log5_then = log_then;
---
> www_then = gethrtime();
> www5_then = www_then;
387a518,529
> // Sleep to give the disks a chance to update.
> sleep(DISK_UPDATE_RATE);
>
> // Get the clock tick rate.
> hz = sysconf(_SC_CLK_TCK);
>
> // Get the page size.
> page_size = sysconf(_SC_PAGESIZE);
>
> // Calculate the system boot time.
> boot_time = time(0) - (kstat$misc.clk_intr / hz);
>
389c531
< measure_os();
---
> _measure_os();
393c535
< measure_os()
---
> _measure_os()
415c557
< handle_os(long now, tm_t tm_now)
---
> measure_os(long now, tm_t tm_now)
418c560
< measure_os();
---
> _measure_os();
421c563
< handle_misc(now, tm_now);
---
> measure_misc(now, tm_now);
425c567
< handle_cpu();
---
> measure_cpu();
430c572
< handle_mutex();
---
> measure_mutex();
432a575,584
> // Take care of the disks.
> #ifdef WATCH_DISK
> measure_disk();
> #endif
>
> // Take care of ram.
> #ifdef WATCH_RAM
> measure_ram();
> #endif
>
435c587
< handle_net();
---
> measure_net();
440c592
< handle_tcp();
---
> measure_tcp();
445c597
< handle_nfs();
---
> measure_nfs();
448,452d599
< // Take care of the disks.
< #ifdef WATCH_DISK
< handle_disk();
< #endif
<
455c602
< handle_dnlc();
---
> measure_dnlc();
460c607
< handle_inode();
---
> measure_inode();
463,467d609
< // Take care of ram.
< #ifdef WATCH_RAM
< handle_ram();
< #endif
<
470c612
< handle_pages();
---
> measure_pages();
472,475d613
<
< // Take care of mount points
< #ifdef WATCH_MOUNTS
< handle_mounts();
477d614
< #endif
492c629
< handle_misc(long now, tm_t tm_now)
---
> measure_misc(long now, tm_t tm_now)
516d652
< put_output(" uptime", sprintf("%8d", uptime));
517a654
> put_output(" uptime", sprintf("%8d", uptime));
519a657,659
> sleep_till_and_count_new_proceses(long sleep_till)
> {
> long now;
521c661,714
< handle_cpu()
---
> long sleep_till1;
> long mpid5_diff;
> double mpid5_interval;
> double rate;
> #endif
>
> now = time(0);
> while (now < sleep_till) {
> #ifdef WATCH_CPU
> if (can_read_kernel != 0) {
> // Sleep at least 5 seconds to make a measurement.
> sleep_till1 = now + 5;
> while (now < sleep_till1) {
> sleep(sleep_till1 - now);
> now = time(0);
> }
>
> // Measure the 5 second process creation rate.
> mpid5_current = kvm$mpid;
> mpid5_now = gethrtime();
> mpid5_interval = (mpid5_now - mpid5_then) * 0.000000001;
> mpid5_then = mpid5_now;
> if (mpid5_current >= mpid5_previous) {
> mpid5_diff = mpid5_current - mpid5_previous;
> }
> else {
> mpid5_diff = mpid5_current + DEFAULT_MAXPID - mpid5_previous;
> }
> rate = mpid5_diff/mpid5_interval;
> if (rate > mpid5_rate) {
> mpid5_rate = rate;
> }
> mpid5_previous = mpid5_current;
>
> // Now take these results to measure the long interval rate.
> // Because the mpid may flip over DEFAULT_MAXPID more than once
> // in the long interval time span, use the difference between
> // the previous and current mpid over a 5 second interval to
> // calculate the long interval difference.
> mpid_current += mpid5_diff;
> mpid_now = mpid5_now;
> }
> else {
> sleep(sleep_till - now);
> }
> #else
> sleep(sleep_till - now);
> #endif
> now = time(0);
> }
> }
>
> #ifdef WATCH_CPU
> measure_cpu()
523a717,718
> double mpid_interval;
> double mpid_rate;
531,533d725
< put_output(" 1load", sprintf("%6.2f",
tmp_kstat_misc.avenrun_1min/256.0));
< put_output(" 5load", sprintf("%6.2f",
tmp_kstat_misc.avenrun_5min/256.0));
< put_output("15load", sprintf("%6.2f",
tmp_kstat_misc.avenrun_15min/256.0));
538c730,742
< put_output("scnrt", sprintf("%5.1f", pvm.scan));
---
> put_output("scanrate", sprintf("%8.3f", pvm.scan));
>
> // Calculate the rate of new process spawning.
> if (can_read_kernel != 0) {
> mpid_interval = (mpid_now - mpid_then) * 0.000000001;
> mpid_rate = (mpid_current - mpid_previous) / mpid_interval;
> put_output("#proc/s", sprintf("%7.3f", mpid_rate));
> put_output("#proc/p5s", sprintf("%9.4f", mpid5_rate));
>
> // Reset counters.
> mpid_then = mpid_now;
> mpid_previous = mpid_current;
> mpid5_rate = 0;
539a744
> }
543c748
< handle_mutex()
---
> measure_mutex()
546a752
> put_output("ncpus", sprintf("%5d", tmp_mutex.ncpus));
551c757
< handle_net()
---
> measure_net()
556,573c762,783
< put_output(sprintf("%4sIpkt/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].ipackets));
< put_output(sprintf("%4sOpkt/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].opackets));
< put_output(sprintf("%4sInKB/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].ioctets/1024.0));
< put_output(sprintf("%4sOuKB/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].ooctets/1024.0));
< put_output(sprintf("%4sIErr/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].ierrors));
< put_output(sprintf("%4sOErr/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].oerrors));
< put_output(sprintf("%4sColl%%", tmp_nr.names[i]),
< sprintf("%9.3f", GLOBAL_net[i].collpercent));
< put_output(sprintf("%4sNoCP/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].nocanput));
< put_output(sprintf("%4sDefr/s", tmp_nr.names[i]),
< sprintf("%10.3f", GLOBAL_net[i].defer));
---
> // Skip unused interfaces.
> // if (GLOBAL_net[i].up == 0) {
> // continue;
> // }
> put_output(sprintf("%5sIpkt/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].ipackets));
> put_output(sprintf("%5sOpkt/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].opackets));
> put_output(sprintf("%5sInKB/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].ioctets/1024.0));
> put_output(sprintf("%5sOuKB/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].ooctets/1024.0));
> put_output(sprintf("%5sIErr/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].ierrors));
> put_output(sprintf("%5sOErr/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].oerrors));
> put_output(sprintf("%5sColl%%", tmp_nr.names[i]),
> sprintf("%10.3f", GLOBAL_net[i].collpercent));
> put_output(sprintf("%5sNoCP/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].nocanput));
> put_output(sprintf("%5sDefr/s", tmp_nr.names[i]),
> sprintf("%11.3f", GLOBAL_net[i].defer));
579c789
< handle_tcp()
---
> measure_tcp()
599c809
< handle_nfs()
---
> measure_nfs()
608c818
< handle_disk()
---
> measure_disk()
611a822,825
> double total_reads;
> double total_writes;
> double total_readk;
> double total_writek;
616,620c830,838
< for (i=0; i < GLOBAL_disk_count; i++) {
< put_output(sprintf("dsk.c%dt%dd%d",
< GLOBAL_disk[i].info.controller, GLOBAL_disk[i].info.target,
< GLOBAL_disk[i].info.device),
< sprintf("%6.2f", GLOBAL_disk[i].run_percent));
---
> total_reads = 0.0;
> total_writes = 0.0;
> total_readk = 0.0;
> total_writek = 0.0;
> for (i=0; i < GLOBAL_disk[0].disk_count; i++) {
> total_reads += GLOBAL_disk[i].reads;
> total_writes += GLOBAL_disk[i].writes;
> total_readk += GLOBAL_disk[i].kreads;
> total_writek += GLOBAL_disk[i].kwrites;
629a848,851
> put_output("disk_rd/s", sprintf("%9.1f", total_reads));
> put_output("disk_wr/s", sprintf("%9.1f", total_writes));
> put_output("disk_rK/s", sprintf("%9.1f", total_readk));
> put_output("disk_wK/s", sprintf("%9.1f", total_writek));
634c856
< handle_dnlc()
---
> measure_dnlc()
642c864
< handle_inode()
---
> measure_inode()
651c873
< handle_ram()
---
> measure_ram()
654a877
> put_output(" freememK", sprintf("%10d", GLOBAL_pvm[0].freemem));
660c883
< handle_pages()
---
> measure_pages()
670,711c893,894
< #ifdef WATCH_MOUNTS
< handle_mounts()
< {
<
< /* traverse the mount table to find mounted ufs/vxfs file systems */
<
< for (mnt$m.number$ = 0; ; mnt$m.number$++)
< {
< tmp_mnt = mnt$m;
<
< if (tmp_mnt.number$ == -1) { break; }
<
< if ( ( tmp_mnt.mnt_fstype == "ufs" || tmp_mnt.mnt_fstype ==
"vxfs" )
< && (tmp_mnt.mnt_flags & MNT_RW) != 0
< )
< {
< vfsstat$f.mp$ = tmp_mnt.mnt_mountp;
< tmp_fs = vfsstat$f;
<
< if (vfsstat$f.status$ == -1)
< {
< printf("WARNING: statvfs returned error for %s, ",
tmp_mnt.mnt_mountp);
< printf("filesystem will be ignored\n");
<
< continue;
< }
< else
< {
< put_output( sprintf("mntu.%s", tmp_mnt.mnt_mountp),
< sprintf("%10ld", tmp_fs.f_used));
< put_output( sprintf("mnta.%s", tmp_mnt.mnt_mountp),
< sprintf("%10ld", tmp_fs.f_avail));
< }
< }
< }
< }
< #endif
<
<
<
< #ifdef WATCH_HTTPD
< /* breakdown access log format */
---
> #ifdef WATCH_WEB
> // Breakdown access log format.
713,714d895
< int size_index;
< string word;
715a897
> int size_index;
717c899,902
< #if WATCH_PROXY || WATCH_SQUID
---
> string word;
> char first_byte[1];
>
> #if WATCH_PROXY || WATCH_SQUID || WATCH_YAHOO
722a908,912
> #ifdef WATCH_YAHOO
> string arg;
> ulong ptr;
> ulong tmp;
> ulong ulong_xf;
723a914,915
> #endif
>
724a917,924
> #ifdef WATCH_YAHOO
> // Make sure that the input line has at least 32 bytes of data plus a
new
> // line, for a total length of 33.
> if (strlen(buf) < 33) {
> return;
> }
> word = strtok(buf,"\05");
> #else
725a926
> #endif
728a930
>
730,731c932,933
< /* word contains unix time in seconds.milliseconds */
< word = strtok(nil, " "); /* elapsed request time in ms */
---
> // Word contains unix time in seconds.milliseconds.
> word = strtok(nil, " "); // Elapsed request time in
ms
733c935
< prxy_squid_xfer_sum += xf;
---
> www_dwnld_time_sum += xf;
737,748c939,943
< word = strtok(nil, " "); /* client IP address */
< logtag = strtok(nil, "/"); /* Log tag */
< word = strtok(nil, " "); /* reply code */
< if (word != nil) {
< if (strstr(word,"304") != nil) {
< httpop_condgets++;
< }
< if (strncmp(word, "4", 1) == 0 || strncmp(word, "5", 1) ==
0) {
< httpop_errors++;
< }
< }
< word = strtok(nil, " "); /* size sent to client */
---
> word = strtok(nil, " "); // Client IP address
> logtag = strtok(nil, "/"); // Log tag
> word = strtok(nil, " "); // Reply code
> WWW_REPLY_CODE(word)
> word = strtok(nil, " "); // Size sent to client
750,751c945,946
< request = strtok(nil, " "); /* Request method */
< word = strtok(nil, " "); /* URL */
---
> request = strtok(nil, " "); // Request method
> word = strtok(nil, " "); // URL
753c948
< if (strstr(word,"cgi-bin") != nil) {
---
> if (word =~ "cgi-bin") {
756c951
< if (strstr(word, search_url) != nil) {
---
> if (word =~ search_url) {
760,761c955,956
< word = strtok(nil, " "); /* optional user ident */
< word = strtok(nil, "/"); /* Hierarchy */
---
> word = strtok(nil, " "); // Optional user ident
> word = strtok(nil, "/"); // Hierarchy
763c958
< if (strstr(word, "DIRECT") == nil) {
---
> if (word =~ "DIRECT") {
767,768c962,963
< word = strtok(nil, " "); /* Hostname */
< word = strtok(nil, " "); /* content-type */
---
> word = strtok(nil, " "); // Hostname
> word = strtok(nil, " "); // Content-type
770,771c965,966
< /* process the collected data */
< if (strstr(logtag, "TCP") != nil) {
---
> // Process the collected data.
> if (logtag =~ "TCP") {
774c969
< if (strstr(logtag, "UDP") != nil) {
---
> if (logtag =~ "UDP") {
777c972
< if (strstr(logtag, "HIT") != nil) {
---
> if (logtag =~ "HIT") {
780c975
< if (strstr(logtag, "MISS") != nil) {
---
> if (logtag =~ "MISS") {
783,798c978,981
< switch(request) {
< case "GET":
< httpop_gets++;
< break;
< case "POST":
< httpop_posts++;
< break;
< case "HEAD":
< ishead = 1;
< httpop_condgets++;
< break;
< case "ICP_QUERY":
< squid_icp_queries++;
< default: break;
< }
< if (ishead == 0) { /* don't add size if its a HEAD */
---
> WWW_METHOD(request)
>
> // Do not add size if its a HEAD.
> if (ishead == 0) {
801,813c984,1010
< if (z < 1024) {
< size_index=0; /* under 1K */
< } else {
< if (z < 10240) {
< size_index=1; /* under 10K */
< } else {
< if (z < 102400) {
< size_index=2; /* under 100K */
< } else {
< if (z < 1048576) {
< size_index=3; /* < 1MB */
< } else {
< size_index=4; /* >= 1MB */
---
>
> WWW_SIZE_INDEX(z, size_index)
> www_dwnld_time_by_size[size_index] += xf;
>
> #elif WATCH_YAHOO
> // Yahoo log format. Fields in square brackets will only appear in the
> // log file if the data actually exists (ie. you will never see a null
> // Referrer field). Further, fields labelled here with "(CONFIG)" will
> // only appear if they are enabled via the YahooLogOptions configuration
> // directive.
> //
> // IP Address (8 hex digits)
> // Timestamp (time_t as 8 hex digits)
> // Processing Time (in microseconds, as 8 hex digits)
> // Bytes Sent (8 hex digits)
> // URL
> // [^Er referrer] (CONFIG)
> // [^Em method] (CONFIG)
> // [^Es status_code]
> // ^Ed signature
> // \n
>
> // Ignore the IP address and timestamp. Get the processing time, the
> // number of bytes sent and the URL. For each portion of the line,
split
> // it up into separate pieces.
> if (sscanf(word, "%8lx%8lx%8x%8x", &tmp, &tmp, &ulong_xf, &z) != 4) {
> return;
814a1012,1019
>
> xf = ulong_xf/1000000.0;
> WWW_SIZE_INDEX(z, size_index)
> www_dwnld_time_sum += xf;
> www_dwnld_time_by_size[size_index] += xf;
>
> if (word =~ "cgi-bin") {
> httpop_cgi_bins++;
815a1021,1022
> if (word =~ search_url) {
> httpop_searches++;
816a1024,1028
>
> for (;;) {
> word = strtok(nil, "\05");
> if (word == nil) {
> break;
818,829c1030,1037
< dwnld_size[size_index]++;
< prxy_squid_xfer_by_size[size_index] += xf;
< #else /* common or netscape proxy formats */
< strtok(nil, " "); /* - */
< strtok(nil, " "); /* - */
< strtok(nil, " ["); /* date */
< strtok(nil, " "); /* zone] */
< word = strtok(nil, " \""); /* GET or POST */
< switch (word) {
< case "get":
< case "GET":
< httpop_gets++;
---
> first_byte = word;
> ptr = &word + 1;
> arg = ((string) ptr);
> ptr = 0;
> switch (first_byte[0]) {
> case 'm':
> WWW_METHOD(arg)
> ptr = 1;
831,833c1039,1040
< case "post":
< case "POST":
< httpop_posts++;
---
> case 's':
> WWW_REPLY_CODE(arg)
835,839d1041
< case "head":
< case "HEAD":
< ishead = 1;
< httpop_condgets++;
< break;
843c1045,1064
< word = strtok(nil, " "); /* URL */
---
> }
>
> // If no method was seen, then assume it was a GET.
> if (ptr == 0) {
> httpop_gets++;
> }
>
> // Do not add size if its a HEAD.
> if (ishead == 0) {
> dwnld_totalz += z;
> }
>
> #else /* common or netscape proxy formats */
> strtok(nil, " "); // -
> strtok(nil, " "); // -
> strtok(nil, " ["); // date
> strtok(nil, " "); // zone]
> word = strtok(nil, " \""); // GET or POST
> WWW_METHOD(word)
> word = strtok(nil, " "); // URL
845c1066
< if (strstr(word,"cgi-bin") != nil) {
---
> if (word =~ "cgi-bin") {
848c1069
< if (strstr(word, search_url) != nil) {
---
> if (word =~ search_url) {
855,856c1076
< if (word != nil) {
< if (strstr(word, "HTTP") != nil || strstr(word, "http") !=
nil) {
---
> if (word != nil && (word =~ "HTTP" || word =~ "http")) {
859c1079,1080
< }
---
> WWW_REPLY_CODE(word)
> word = strtok(nil, " "); // Bytes transferred.
861,871d1081
< if (strstr(word,"304") != nil) {
< httpop_condgets++;
< }
< else {
< if (strncmp(word, "4", 1) == 0 || strncmp(word, "5",
1) == 0) {
< httpop_errors++;
< }
< }
< }
< word = strtok(nil, " "); // bytes transferred
< if (word != nil) {
873c1083
< if (ishead == 0) { // don't add size if its a HEAD
---
> if (ishead == 0) { // Do not add size if its a HEAD.
876,888c1086
< if ((z % 1024) == z) {
< size_index=0; /* under 1K */
< } else {
< if ((z % 10240) == z) {
< size_index=1; /* under 10K */
< } else {
< if ((z % 102400) == z) {
< size_index=2; /* under 100K */
< } else {
< if ((z % 1048576) == z) {
< size_index=3; /* < 1MB */
< } else {
< size_index=4; /* >= 1MB */
---
> WWW_SIZE_INDEX(z, size_index)
890,894d1087
< }
< }
< }
< dwnld_size[size_index]++;
< }
896,906c1089,1101
< word = strtok(nil, " "); /* status from server */
< word = strtok(nil, " "); /* length from server */
< word = strtok(nil, " "); /* length from client POST */
< word = strtok(nil, " "); /* length POSTed to remote */
< word = strtok(nil, " "); /* client header req */
< word = strtok(nil, " "); /* proxy header resp */
< word = strtok(nil, " "); /* proxy header req */
< word = strtok(nil, " "); /* server header resp */
< word = strtok(nil, " "); /* transfer total secs */
< word = strtok(nil, " "); /* route */
< if (word != nil) { /* - DIRECT PROXY(host.domain:port) SOCKS */
---
> word = strtok(nil, " "); // status from server
> word = strtok(nil, " "); // length from server
> word = strtok(nil, " "); // length from client POST
> word = strtok(nil, " "); // length POSTed to remote
> word = strtok(nil, " "); // client header req
> word = strtok(nil, " "); // proxy header resp
> word = strtok(nil, " "); // proxy header req
> word = strtok(nil, " "); // server header resp
> word = strtok(nil, " "); // transfer total secs
> word = strtok(nil, " "); // route
>
> // - DIRECT PROXY(host.domain:port) SOCKS
> if (word != nil) {
912,918c1107,1113
< word = strtok(nil, " "); /* client finish status */
< word = strtok(nil, " "); /* server finish status */
< word = strtok(nil, " "); /* cache finish status */
< /* - ERROR HOST-NOT-AVAILABLE = error or incomplete op
< WRITTEN REFRESHED CL-MISMATCH(content length mismatch) =
cache_writes
< NO-CHECK UP-TO-DATE = cache_hits
< DO-NOT-CACHE NON-CACHEABLE = uncacheable */
---
> word = strtok(nil, " "); // client finish status
> word = strtok(nil, " "); // server finish status
> word = strtok(nil, " "); // cache finish status
> // ERROR HOST-NOT-AVAILABLE = error or incomplete op
> // WRITTEN REFRESHED CL-MISMATCH(content length mismatch) = cache_writes
> // NO-CHECK UP-TO-DATE = cache_hits
> // DO-NOT-CACHE NON-CACHEABLE = uncacheable
923c1118,1119
< case "CL-MISMATCH": prxy_cache_writes++;
---
> case "CL-MISMATCH":
> prxy_cache_writes++;
926c1122,1123
< case "UP-TO-DATE": prxy_squid_cache_hits++;
---
> case "UP-TO-DATE":
> prxy_squid_cache_hits++;
929c1126,1127
< case "NON-CACHEABLE": prxy_uncacheable++;
---
> case "NON-CACHEABLE":
> prxy_uncacheable++;
931c1129,1130
< default: break;
---
> default:
> break;
934c1133
< word = strtok(nil, " ["); /* [transfer total time x.xxx */
---
> word = strtok(nil, " ["); // [transfer total time x.xxx
937,938c1136,1137
< prxy_squid_xfer_by_size[size_index] += xf;
< prxy_squid_xfer_sum += xf;
---
> www_dwnld_time_sum += xf;
> www_dwnld_time_by_size[size_index] += xf;
944c1143
< measure_httpd(long sleep_till)
---
> measure_web(long sleep_till)
963,964c1162,1163
< #if WATCH_PROXY || WATCH_SQUID
< prxy_squid_xfer_by_size[i] = 0.0;
---
> #if WATCH_PROXY || WATCH_SQUID || WATCH_YAHOO
> www_dwnld_time_by_size[i] = 0.0;
968a1168,1170
> #if WATCH_PROXY || WATCH_SQUID || WATCH_YAHOO
> www_dwnld_time_sum = 0.0;
> #endif
970d1171
< prxy_squid_xfer_sum = 0.0;
984,985c1185,1190
< if (log_name != nil) {
< while (time(0) < sleep_till) {
---
> if (www_log_filename != nil) {
> now = time(0);
> while (now < sleep_till) {
> #ifdef WATCH_CPU
> sleep_till_and_count_new_proceses(now + 5);
> #else
987,988c1192,1195
< if (log_file != 0) {
< while (fgets(buf, BUFSIZ, log_file) != nil) {
---
> #endif
> now = time(0);
> if (www_fd != 0) {
> while (fgets(buf, BUFSIZ, www_fd) != nil) {
990,991c1197,1198
< if (log_gatelen > 0) {
< if (strncmp(buf, log_gateway, log_gatelen) == 0) {
---
> if (www_gatelen > 0) {
> if (strncmp(buf, www_gateway, www_gatelen) == 0) {
1000,1003c1207,1210
< stat(log_name, log_stat);
< if (log_ino != log_stat[0].st_ino || log_size > log_stat[0].st_size)
{
< if (log_file != 0) {
< fclose(log_file); /* close the old log */
---
> stat(www_log_filename, www_stat);
> if (www_ino != www_stat[0].st_ino || www_size > www_stat[0].st_size)
{
> if (www_fd != 0) {
> fclose(www_fd); /* close the old log */
1006,1009c1213,1216
< log_file = fopen(log_name, "r");
< if (log_file != 0) {
< log_ino = log_stat[0].st_ino;
< while(fgets(buf, BUFSIZ, log_file) != nil) {
---
> www_fd = fopen(www_log_filename, "r");
> if (www_fd != 0) {
> www_ino = www_stat[0].st_ino;
> while(fgets(buf, BUFSIZ, www_fd) != nil) {
1011,1012c1218,1219
< if (log_gatelen > 0) {
< if (strncmp(buf, log_gateway, log_gatelen) == 0) {
---
> if (www_gatelen > 0) {
> if (strncmp(buf, www_gateway, www_gatelen) == 0) {
1021,1024c1228,1231
< log5_now = gethrtime();
< log5_interval = (log5_now - log5_then) * 0.000000001;
< log5_then = log5_now;
< dtmp = (httpops - lastops)/log5_interval;
---
> www5_now = gethrtime();
> www5_interval = (www5_now - www5_then) * 0.000000001;
> www5_then = www5_now;
> dtmp = (httpops - lastops)/www5_interval;
1031c1238
< log_size = log_stat[0].st_size;
---
> www_size = www_stat[0].st_size;
1035,1038c1242,1243
< now = time(0);
< while (now < sleep_till) {
< sleep(sleep_till - now);
< now = time(0);
---
> sleep_till_and_count_new_proceses(sleep_till);
> www5_now = gethrtime();
1040d1244
< }
1042,1044c1246,1248
< log_now = gethrtime();
< log_interval = (log_now - log_then) * 0.000000001;
< log_then = log_now;
---
> www_now = www5_now;
> www_interval = (www_now - www_then) * 0.000000001;
> www_then = www_now;
1054c1258
< #if WATCH_PROXY || WATCH_SQUID
---
> #if WATCH_PROXY || WATCH_SQUID || WATCH_YAHOO
1057c1261
< prxy_squid_xfer_by_size[i] = 0.0;
---
> www_dwnld_time_by_size[i] = 0.0;
1060c1264
< prxy_squid_xfer_by_size[i] =
prxy_squid_xfer_by_size[i]/dwnld_size[i];
---
> www_dwnld_time_by_size[i] = www_dwnld_time_by_size[i]/dwnld_size[i];
1082,1083c1286,1287
< put_output("#httpds", sprintf("%7d", count_proc("httpd")));
< put_output("httpop/s", sprintf("%8.2f", httpops/log_interval));
---
> put_output("#httpds", sprintf("%7d",
count_proc(www_server_proc_name)));
> put_output("httpop/s", sprintf("%8.2f", httpops/www_interval));
1085,1089c1289,1293
< put_output("cndget/s", sprintf("%8.2f", httpop_condgets/log_interval));
< put_output("search/s", sprintf("%8.3f", httpop_searches/log_interval));
< put_output(" cgi/s", sprintf("%8.3f", httpop_cgi_bins/log_interval));
< put_output(" htErr/s", sprintf("%8.3f", httpop_errors/log_interval));
< put_output(" httpb/s", sprintf("%8.0f", dwnld_totalz/log_interval));
---
> put_output("cndget/s", sprintf("%8.2f",
httpop_condgets/www_interval));
> put_output("search/s", sprintf("%8.3f",
httpop_searches/www_interval));
> put_output(" cgi/s", sprintf("%8.3f",
httpop_cgi_bins/www_interval));
> put_output(" htErr/s", sprintf("%8.3f", httpop_errors/www_interval));
> put_output(" httpb/s", sprintf("%8.0f", dwnld_totalz/www_interval));
1095c1299
< put_output(log_gateway, sprintf("%8.2f", gateops/log_interval));
---
> put_output(www_gateway, sprintf("%8.2f", gateops/www_interval));
1107,1112c1311,1323
< put_output(" xfr_t", sprintf("%8.2f", 0.01 * dtmp *
prxy_squid_xfer_sum));
< put_output(" xfr1_t", sprintf("%8.2f", prxy_squid_xfer_by_size[0]));
< put_output(" xfr10_t", sprintf("%8.2f", prxy_squid_xfer_by_size[1]));
< put_output("xfr100_t", sprintf("%8.2f", prxy_squid_xfer_by_size[2]));
< put_output(" xfr1M_t", sprintf("%8.2f", prxy_squid_xfer_by_size[3]));
< put_output("xfro1M_t", sprintf("%8.2f", prxy_squid_xfer_by_size[4]));
---
> put_output(" xfr_t", sprintf("%8.2f", 0.01 * dtmp *
www_dwnld_time_sum));
> put_output(" xfr1_t", sprintf("%8.2f", www_dwnld_time_by_size[0]));
> put_output(" xfr10_t", sprintf("%8.2f", www_dwnld_time_by_size[1]));
> put_output("xfr100_t", sprintf("%8.2f", www_dwnld_time_by_size[2]));
> put_output(" xfr1M_t", sprintf("%8.2f", www_dwnld_time_by_size[3]));
> put_output("xfro1M_t", sprintf("%8.2f", www_dwnld_time_by_size[4]));
> #elif WATCH_YAHOO
> put_output(" wprc_t", sprintf("%9.5f", 0.01 * dtmp *
www_dwnld_time_sum));
> put_output(" wprc1_t", sprintf("%9.5f", www_dwnld_time_by_size[0]));
> put_output(" wprc10_t", sprintf("%9.5f", www_dwnld_time_by_size[1]));
> put_output("wprc100_t", sprintf("%9.5f", www_dwnld_time_by_size[2]));
> put_output(" wprc1M_t", sprintf("%9.5f", www_dwnld_time_by_size[3]));
> put_output("wprco1M_t", sprintf("%9.5f", www_dwnld_time_by_size[4]));
More information about the Orca-dev
mailing list