GEM Documentation Navigation Page. Documentation is also available at The GEM Home Page. Documentation and Code are both Copyright © 1995-2006 by Sql Technologies. All Rights are Reserved.

The Complete GEM Documentation	Slide Show: What Is GEM	Download GEM
An Introduction To GEM	How To Install	Getting Started Guide
Batch Jobs Overview	GEM Screenshots	Server Backup and Maintenance
Utilities Guide	The GEM Console	New System Stored Procedures
FAQ and Change Log	Common Perl Libraries	Pricing & Right To Use

The GEM CONSOLE is a set web pages that allow you to see a consolidated set of system reports. These reports range from fairly static configuration reports to very dynamic reports (system monitoring, batch job state, performance reports). The console can be viewed with a standard web browser. A web server is not needed.

The GEM Console is assembled and managed by the GEM batch scripts. This is a batch reporting system - designed intentionally that way to minimize production system impact. This means that some of the console reports will be assembled in the middle of the night and some will be reassembled every few minutes using the data collected by the monitoring system. The console is designed with several goals in mind.

1. Completely document your environment
2. Provide configuration auditing & security reports
3. Historical reporting
4. Assist Disaster Recovery
5. System sanity check for misconfigurations, space problems etc...
6. Advanced Reporting on the state of your backups
7. Provide extensive morning review reports
8. Monitor GEM and specifically monitor GEM Batch Jobs
9. Provide trivial customization to YOUR specific needs.

The GEM Console also had a design goal. It was designed over time, as a freely available open architecture system. It was specifically designed to be easily extendible into my consulting clients site specific needs. The customization features of the console are simple, elegant, and do pretty much whatever you might need to monitor or report on specific things with your systems. It will work very well with your existing monitoring, auditing, and reporting.

This document explains the Console with the aim of permiting you to install, use, and maintain it.

Documented separately are the GEM batch jobs. These batch jobs are integral to the workings of the Console. To synopsize, the configuration utility creates a set of batch jobs. Jobs for your unix systems exist in the unix_batch_scripts directory and win32 jobs exist in the win32_batch_scripts directory. You are responsible for scheduling these jobs (although we do provide help on this). If you have a hybrid win32/unix environment, each batch job should be scheduled in one and only one spot. The batch jobs perform system monitoring and collect information into the data subdirectory of the GEM installation. There are a few batch jobs that can be considered "Console" batch jobs. These jobs either put together and manage the console or create specific console reports. One final job takes all the "pre-prepared" console parts and assembles them into a final set of web pages. The GEM configuration utility will build an initial set of Console pages, but a working console requires you to schedule your batch jobs. While installing, you will end up with a minimal console, but that console will be fleshed out with more information as the batch jobs you sheduled run. The console is built in the data/CONSOLE_REPORTS directory of the GEM root. The console can optionally be published to another system or directory. This feature allows you and your co-administrators to view the console over the company intranet. Be aware - the console publishes detailed configuration reports about your systems that you may not want revealed for security reasons. It is normal to publish the console on a shared drive (not a web server) that can be controlled with standard file access permissions. The batch jobs also produce output. Job standard output and standard error is stored in data/GEM_BATCHJOB_LOGS and and data/batchjob_errors. As mentioned earlier, data/CONSOLE_REPORTS is where the console is assembled. The data/BACKUP_LOGS directory is read for output of your backup scripts. The data/html_output directory is a place you can put your own reports for inclusion in the Console.

This package needs to be installed through the master configuration utility of GEM. It is not supported as a standalone utility.

The following table summarizes the locations mentioned in the introduction.

DIRECTORY	USAGE
ADMIN_SCRIPTS/console	Perl Code Location
data/CONSOLE_REPORTS	Final Web Browsable Output
data/GEM_BATCHJOB_LOGS	Source Of Batch Job Log Files
data/batchjob_errors	Source Of Batch Job Error Logs
data/html_output	Where you put user assembled log files/reports
data/system_information_data	Spot GEM places collected server logs and configuration data
unix_batch_scripts/batch	The Unix Batch Scripts That Create The Console
win32_batch_scripts/batch	The Win32 Batch Scripts That Create The Console

The above table reveals a lot about the order of activities. Scheduled batch scritps in the unix_batch_scripts and win32_batch_scripts call perl programs that reside in ADMIN_SCRIPTS/console. These programs perform a variety of tasks but many of them collect data into data/system_information_data. All Batch jobs create log files in data. A few of the batch jobs collect data and create the actual console.

The console also gives you additional protection by crosschecking your backup files to ensure that your backups were really created. Backups are perhaps the most critical database administration task. The critical nature of the success of your backup strategy has not been considered by the major vendors - it is left as an exercise for the DBA to validate their backups. GEM changes this. It checks that your backups actually happened and that everything looks reasonable. Of course, we can not KNOW what you intend in terms of backups. Should the database "test" on your production server be backed up? Do you have log shipping? The console creates reports describing your backup state and uses heuristic rules to figure out if everything is ok. A human must read these reports to ascertain that everything is working and to investigate any failures. This will prevent you from running with inadequate backups or inadequate transaction log loads (in hot backup systems). Surprisingly, while you may think that your backups are reliable, stuff will happen... This system protects you.

The GEM console specific batch jobs are as follows.

BATCH JOB NAME	DESCRIPTION
ConsoleArchiver.ksh	Saves a copy of the console. Recommended that this is run once per month to give you a historical configuration audit for your servers.
ConsoleBuildAndFtp.ksh	Assembles Console And Copies/Ftp's Final Output Directory To Final Destination Calls ADMIN_SCRIPTS/console/console_build_and_ftp.pl
ConsoleUnixWeekly.ksh ConsoleWin32Weekly.ksh	Performs Task That Should Be Done 1ce Per Week Calls ADMIN_SCRIPTS/console/console_weekly.pl
ConsoleUnixNightly.ksh ConsoleWin32Nightly.ksh	Performs Task That Should Be Done 1ce Per Night Calls ADMIN_SCRIPTS/console/console_nightly.pl
ConsoleUnixHourly.ksh ConsoleWin32Hourly.ksh	Performs Task That Should Be Done Hourly Calls ADMIN_SCRIPTS/console/console_hourly.pl

Gem allows you to create your own reports that will be directly incorporated into the console. These reports are defined in the file conf/console.dat. The following is the online help for this file. This file is used to easily incorporate custom business logic into your console.

   # THIS FILE CONTROLS THE REPORTS FOUND IN THE CUSTOM SITE REPORTS
   #
   # NOTE: anything found in the log directory AUTOMATICALLY gets reported
   #  under the log files section of the web page.
   #
   # MAIN PURPOSE: Custom Reports section of the server documenter
   #
   # You can create custom reports here and set up a list of messages to
   # ignore.  Reports can be queries run on one server, commands that
   # return html, and external commands that return txt.
   #
   # Sometimes you will explicity wish to ignore some messages from the
   # reports (like user with no login messages from databases that
   # have been copied from an other server).  Do that here as well.
   #
   # The format for this file is (one row per report)
   #   REPORT_NAME SERVER Database QUERY
   #   REPORT_NAME html <command to run returning html results>
   #   REPORT_NAME cmd <command to run returning ascii/txt results>
   #   REPORT_NAME sql server login pass <sql command>
   #   REPORT_NAME file host <filename>
   #    IGNORE server database <message ids (space separated)>
   #
   # Queries can be run on individual servers with the sql and null keywords.
   #
   # Use the sql keyword to run on non ASE servers (like HISTORICAL and BACKUP
   # servers) that are not in the password file for whatever reason.
   #
   # For file reports - host may be the word null in which case the file is local.
   #
   # External commands can be called using the cmd (ascii output) and html
   # (formatted html output) keywords.

   # The IGNORE line ignores particular messages on the Server Analysis For
   # Problems report by server/db.  The message ids should be numbers (eg. 31008)
   # from that report.
   #
   # ALL underscores in REPORT_NAME will be replaced with spaces when printing
   #
   # sql is a special query for HIST/BACKUP/REPORT servers that are not
   # in the password file for whatever reason
   #
   # Database name can be the word 'null' if that is needed (no quotes)
   #
   # ALL underscores in REPORT_NAME will be replaced with spaces on output
   #
   # STORED PROCEDURES
   # ------
   # When you run a stored procedure that returns some stuff it is embedded
   # into the html report as is
   #
   # example to run 2 queries on BPSAPROD database bpsadata:
   #
   # Bpsa_Status BPSAPROD bpsadata exec bpsadata..show_bpsa_load_status
   # Bpsa_Transaction_Row_Count BPSAPROD bpsadata exec bpsadata..show_trans_row_counts
   #
   # RUN SYSTEM COMMAND (PERL MUST BE IN PATH or PREPENDED)
   # ------
   # If you are running a perl command and you are on NT, you will need to
   # have C:/program_files/perl (or whatever your perl is) prepended
   # to your program path so it can interpret it ok.  ie.
   #    Rsh_Program_Type cmd C:/program_files/perl /optapp2/syb_admin/x.pl -cx
   #
   # this code checks for core dumps on 3 systems
   #
   # examples:
   # Core_Dump_Checker_On_Prod html /optapp2/syb_admin/development/ADMIN_SCRIPTS/console/core_dump_checker.pl  2>/dev/null
   # Core_Dump_Checker_On_Sybprod html rsh -l sybase sybprod /opt2/sybdump/MONITORING_SCRIPTS/core_dump_checker.pl -o.  2>/dev/null
   #
   # run file command on rsh and ftp on all hosts in password file so that i can
   # see if they have been appropriately modified for my current site (we use
   # shell wrappers on development/qa so these systems cant go to our production.
   #
   # Rsh_Program_Type cmd /optapp2/syb_admin/development/ADMIN_SCRIPTS/rdist/run_cmd_on_all_hosts.pl -c"file /usr/bin/rsh"
   # Ftp_Program_Type cmd /optapp2/syb_admin/development/ADMIN_SCRIPTS/rdist/run_cmd_on_all_hosts.pl -c"file /usr/bin/ftp"
   #
   # we also have a program that identifies recently modified files and their
   # differences within a set of servers
   #
   # Recently_Changed_Files html /scripts/chksum_compare.pl  -hx  -pappprod
   # Oracle_Backups cmd rsh -l oracle sybint1 'ls -l /optapp1/oracle/tools/*'

GEM automatically includes external reports based on information in conf/custom_reports.dat. This configuration file contains information regarding This file is self documenting. The header for console_reports.dat currently reads.

   #
   # CONSOLE CONFIGURATION FILE
   #
   # This file lists console reports and sources in a format that allows customization and stuff.  Basically, reports
   # are either assembled by the console manager (document_all.pl) or by external programs.  The page header, however,
   # needs to be assembled by document_all.pl.  To make this work,   reports that are assembled by batch scritps are
   # placed in the html_output directory.  They are then read by the console manager, converted from .txt or .html format,
   # incorporated into the javascript navigation bar, and moved into the CONSOLE_REPORTS directory.  This file guides
   # that process.  The format is a standard GEM csv file.
   #
   # FILE,   MENU,   TITLE,   IS_OPTIONAL
   #
   # note.  this means that you should not put commas in your report titles...
   #
   # The following are legal MENU items
   #   BYAMREVIEW
   #   SECURITY
   #   BYGEMLOG
   #   BYCUSTOM
   #   BACKUP
   #   BYHOST
   #   INDEXPAGE

Html File	Menu Item	Report Name	Batch Name
Win32_Hosts_Report.html	BYHOST	Windows Servers Hosts Files	PROGNAME
MssqlScheduledJobRpt.html	BYAMREVIEW	Sql Server Jobs With Errors	PROGNAME
Win32_Scheduler.html	BYAMREVIEW	Windows Task Scheduler	PROGNAME
mssql_jobstatus_rpt.html	BYAMREVIEW	Sql Server Job Scheduler	PROGNAME
PcDiskspace.txt	BYAMREVIEW	Windows Disk Space Report	PROGNAME
SybRepMonReport.html	BYAMREVIEW	Sybase Replication Latency	PROGNAME
GemRpt_SybRepSvrLogRpt.html	BYAMREVIEW	Sybase Replication Log Report	PROGNAME
GemRpt_SybMonSvrLogRpt.html	BYAMREVIEW	Sybase Mon Server Log Report	PROGNAME
GemRpt_SybHistSvrLogRpt.html	BYAMREVIEW	Sybase Hist Server Log Report	PROGNAME
GemRpt_SybErrLogRpt.html	BYAMREVIEW	Sybase Error Log - Filtered	PROGNAME
GemRpt_SybErrLogAllRpt.html	BYAMREVIEW	Sybase Error Log - UnFiltered	PROGNAME
GemRpt_SybErrLogAlarm.html	BYAMREVIEW	Sybase Error Log - Recent	PROGNAME
OptdiagAnalyze.html	BYAMREVIEW	Sybase Fragmentation Analysis	PROGNAME
GemRpt_SybaseRepServer.html	BYAMREVIEW	GEM Sybase Replication Logs	PROGNAME
GemRpt_SybaseBackupServer.html	BYAMREVIEW	GEM Sybase Backup Logs	PROGNAME
GemRpt_SybaseASE.html	BYAMREVIEW	GEM Sybase ASE Error Logs	PROGNAME
GemRpt_ProductionOvernight.html	BYAMREVIEW	GEM Production Errors	PROGNAME
GemRpt_ProductionWarnings.html	BYAMREVIEW	GEM Production Warnings	PROGNAME
GemRpt_PCEventlog.html	BYAMREVIEW	GEM Windows Event Log Report	PROGNAME
GemRpt_PCDiskspace.html	BYAMREVIEW	GEM Windows Disk Space Report	PROGNAME
GemRpt_UnixWarnings.html	BYAMREVIEW	GEM Unix Warnings Report	PROGNAME
GemRpt_UnixErrors.html	BYAMREVIEW	GEM Unix Errors Report	PROGNAME
GetBackupStateSybase.html	BACKUP	Get Backup Info - Sybase	PROGNAME
GetBackupStateMssql.txt	BACKUP	Get Backup Info - MS SQL	PROGNAME
BackupErrorReport.html	BACKUP	Backup CrossCheck - Possible Errors	PROGNAME
BackupReport.html	BACKUP	Backup CrossCheck - All Systems	PROGNAME
SybaseBackupFileReport.html	BACKUP	Sybase Backup Configuration	PROGNAME
GemRpt_BackupReport.html	BACKUP	GEM Backup Manager Summary	PROGNAME
GemRpt_Backups.html	BACKUP	GEM Backup Manager Job State	PROGNAME
SybRepMonAgent.html	BYGEMLOG	Sybase Replication Latency Monitor	PROGNAME
GemRpt_SybRepSvrLogFetch.html	BYGEMLOG	Sybase Replication Log Fetch	PROGNAME
GemRpt_SybRepSvrConfig.html	BYGEMLOG	Sybase Replication Log Configuration	PROGNAME
GemRpt_SybRepSvrSaveAlarms.html	BYGEMLOG	Sybase Replication Log Save	PROGNAME
GemRpt_SybMonSvrLogFetch.html	BYGEMLOG	Sybase Mon Server Log Fetch	PROGNAME
GemRpt_SybMonSvrConfig.html	BYGEMLOG	Sybase Mon Server Log Configuration	PROGNAME
GemRpt_SybHistSvrLogFetch.html	BYGEMLOG	Sybase Hist Server Log Fetch	PROGNAME
GemRpt_SybHistSvrConfig.html	BYGEMLOG	Sybase Hist Server Log Configuration	PROGNAME
GemRpt_SybErrLogFetch.html	BYGEMLOG	Sybase Error Log Fetch	PROGNAME
GemRpt_SybErrLogConfig.html	BYGEMLOG	Sybase Error Log Configuration	PROGNAME
GemRpt_BlackoutReport.html	BYGEMLOG	GEM Ignore / Blackout Report	PROGNAME
GemRpt_ProductionList.html	BYGEMLOG	GEM Production Servers Report	PROGNAME
GemRpt_AgentReport.html	BYGEMLOG	GEM Agents Run At Report	PROGNAME
GemRpt_WinSecurityRpt.html	SECURITY	GEM Windows Security Report	PROGNAME

The console will generate reports on its own when it runs. The console can run (by default) in Hourly, Nightly, and Weekly flavors, with SYBASE and SQL Server options.

Page	Report Name	File Name	Assembled?	Notes On Creation
MAIN PAGE	Server Names & Ports	interfaces.html	Y	Internally assembled by document_all.pl if DO_SYBASE=1
MAIN PAGE	Interface File Conflicts	interfaces_conflicts.html	Y1	console/interfaces_file_rpt.pl
MAIN PAGE	Database Use Report	dbuserpt.html	ALL	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Server Overview	server_summary.html	ALL	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Server Version Report	server_version.html	ALL	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Server Sysconfig Values	server_configures.html	2	BROKEN! console/sysconfigures_rpt.pl
MAIN PAGE	Space By Database (Sybase)	space_by_db_syb.html	DO_SYBASE	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Space By Database (SQL Server)	space_by_db_sql.html	DO_SQLSVR	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Server by database (Sybase)	server_by_db_syb.html	All	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Server by database (SQL Server)	server_by_db_sql.html	All	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Devices By Servers	server_devices.html	2	console/devices_html_rpt.pl
MAIN PAGE	Space Used By Database (Sybase)	space_used_syb.html	All	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Space Used By Database (SQL Server)	space_used_sql.html	All	Assembled by document_all.pl From Stored XML Data
MAIN PAGE	Windows Version/Restart Report	win32_getversion.html	W5	Assembled by bin/win32_getversion.pl --DAYS=7
MAIN PAGE	SQL Server Data Paths	sql_path_report.html	S5	bin/mssql_datapath_report.pl --HTML
MAIN PAGE	Space Used History	space_report.html	9	bin/space_report.pl
ByServer	[Click ]	by_server.html	Always	Assembled by document_all.pl - This page is a container
ByServer	Overview	byserver/$srvr.html	ALL	Assembled by document_all.pl From Stored XML Data
ByServer	Config	$srvr.html	Server Type	config_report.pl
ByServer	Login Report	$svr_logins.html	6 + Server Type	custom_rpt.pl -Ssvr -h -R6
ByServer	Users	$svr_users.html	6 + Server Type	custom_rpt.pl -Ssvr -h -R6
ByServer	Largest 20 Tables	$svr_largetables.html	22	biggest_table.pl -h -S$srvr -U$login -P$password -n20 -s
ByServer	Debug	$svr_debug.html	4	debug_one_server -h -Ssvr
ByServer	$srvr_dbschema.txt	$svr_dbschema.txt	(NEVER?)	dbschema.pl -Ssvr -km -Dmaster
By Host	Bad Symbolic Links	bad_symbolic_links.html	20 and !nt	core_dump_checker.pl
ByHost	Database Disk Usage	disk_report.html	6	Assembled by document_all.pl From Stored XML Data
By Host	Host Network Setup	ip_info.txt	U13	run_cmd_on_all_hosts.pl -c"/usr/sbin/ifconfig -a" -C"/sbin/ifconfig -a" -n
By Host	Host Operating System	sys_info.html	U11	run_cmd_on_all_hosts.pl -c"uname -a" -f1,3,4,5,6,7,8 -o
By Host	Host Uptime	rup.html	12	/usr/bin/rup -t
By Host	Hosts File	host_info.html	U13	bin/build_hosts.pl
By Host	Windows IP Configuration Report	win32_ipconfig.htm	W11	bin/win32_ipconfig_report.pl
By Host	Windows Hardware Report	win32_hardware_report.htm	W11	bin/win32_system_report.pl -SYSRPT
By Host	Windows Server Hosts File	Win32_Hosts_Report.html	N.A.	EXTERNAL REPORT (See ABOVE)
By Host	Windows Software Config Report	win32_software_report.htm	W11	bin/win32_system_report.pl -SWRPT
By Host	Core Dumps Report	you_have_mail.txt	U15	run_cmd_on_all_hosts.pl -cls -l /var/mail -n
By Host	Large File Report	large_files.txt	15	core_dump_checker.pl
By Host	Users With Crontabs	core_dump_checker.html	15	core_dump_checker.pl
By Host	Users With Mail	you_have_crontab.txt	U15	./rdist/run_cmd_on_all_hosts.pl -c'ls -l /var/spool/cron/crontabs' -n
Security	Access By Logins - SQL Server	sql_suid_by_server.html	W5	crosstab.pl -C0 -R2 -O3...
Security	Access By Logins - Sybase	syb_suid_by_server.html	Y3	crosstab.pl -C0 -R2 -O3...
Security	GEM Windows Security Report	GemRpt_WinSecurityRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Security	Object Creation Time - SQL Server	sql_crdate.html	S5	crdate_report.pl -PRODONLY --HTML --TYPE=sqlsvr
Security	Object Creation Time - Sybase	syb_crdate.html	Y5	crdate_report.pl -PRODONLY --HTML --TYPE=sybase
Security	Role Report - Sql Server	sql_role.txt	S5	passwd_rpt.pl -h -Tsqlsvr
Security	Role Report - Sybase	syb_role.txt	Y5	passwd_rpt.pl -h -Tsybase
Morning Review	Dbcc Results Summary	check_dbcc_output.html	16	check_dbcc_output.pl -D14
Morning Review	GEM Production Errors	GemRpt_ProductionOvernight.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Production Warnings	GemRpt_ProductionWarnings.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Sybase ASE Error Logs	GemRpt_SybaseASE.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Sybase Backup Logs	GemRpt_SybaseBackupServer.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Sybase Replication Logs	GemRpt_SybaseRepServer.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Unix Errors Report	GemRpt_UnixErrors.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Unix Warnings Report	GemRpt_UnixWarnings.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Server Analysis for Problems	error.html	4	Internally Generated From Stored Data
Morning Review	Sql Server Job Scheduler	mssql_jobstatus_rpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sql Server Jobs With Errors	MssqlScheduledJobRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	SqlServer Db Configuration Fixer	fix_db_sqlsvr.html	S4	fix_db.pl -x -Tsqlsvr
Morning Review	Sybase Db Configuration Fixer	fix_db_sybase.html	Y4	fix_db.pl -x -Tsybase
Morning Review	Sybase Error Log Filtered	GemRpt_SybErrLogRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Error Log - Recent	GemRpt_SybErrLogAlarm.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Error Log - UnFiltered	GemRpt_SybErrLogAllRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Hist Server Log	GemRpt_SybHistSvrLogRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Mon Server Log Report	GemRpt_SybMonSvrLogRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Replication Latency	SybRepMonReport.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Replication Log Report	GemRpt_SybRepSvrLogRpt.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Sybase Space Errors Report	space_errors.html	9	space_errors.pl -H
Morning Review	Sybase Thresholds Crossed Report	Sybase_Threshold_Report.html	Y9	threshold_manager.pl --DEBUG --HTML --ACTION=REPORT
Morning Review	Unix Disk Space	unix_df_info.html	U14	unix_space_monitor_via_rsh.pl -A
Morning Review	Windows Disk Space Report	PcDiskspace.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	Windows Task Scheduler	Win32_Scheduler.html	N.A.	EXTERNAL REPORT (See ABOVE)
Morning Review	GEM Windows Disk Space Report
Morning Review	GEM Windows Event Log Report
Morning Review	Sybase Fragmentation Analysis
GEM Logs	BATCH FILE EXECUTION LOGS	by_logs.html		Internally Assembled
GEM Logs	GEM Agents Run At Report	GemRpt_AgentReport.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	GEM File Sources Report	gem_file_manager.html	21	gem_file_manger.pl -mh
GEM Logs	GEM Ignore / Blackout Report	GemRpt_BlackoutReport.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	GEM Production Servers Report	GemRpt_ProductionList.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Error Log Configuration	GemRpt_SybErrLogConfig.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Error Log Fetch	GemRpt_SybErrLogFetch.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Hist Server Log Configuration	GemRpt_SybHistSvrConfig.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Hist Server Log Fetch	GemRpt_SybHistSvrLogFetch.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Mon Server Log Configuration	GemRpt_SybMonSvrConfig.html
GEM Logs	Sybase Mon Server Log Fetch	GemRpt_SybMonSvrLogFetch.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Replication Latency Monitor	SybRepMonAgent.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Replication Log Configuration	GemRpt_SybRepSvrConfig.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Replication Log Fetch	GemRpt_SybRepSvrLogFetch.html	N.A.	EXTERNAL REPORT (See ABOVE)
GEM Logs	Sybase Replication Log Save	GemRpt_SybRepSvrSaveAlarms.html	N.A.	EXTERNAL REPORT (See ABOVE)
Custom Reports	ALL	$custom_report_name{$count}.*	17	$custom_report_query{$count}
Backup Logs	Backup Batch Logs: Errors	by_backuperrs.html		Internally Assembled
Backup Logs	Backup Batch Logs: Logs	by_backuplogs.html		Internally Assembled
Backup Logs	Backup CrossCheck - All Systems	BackupReport.html	N.A.	EXTERNAL REPORT (See ABOVE)
Backup Logs	Backup CrossCheck - Possible Errors	BackupErrorReport.html	N.A.	EXTERNAL REPORT (See ABOVE)
Backup Logs	GEM Backup Manager Job State	GemRpt_Backups.html	N.A.	EXTERNAL REPORT (See ABOVE)
Backup Logs	GEM Backup Manager Summary	GemRpt_BackupReport.html	N.A.	EXTERNAL REPORT (See ABOVE)
Backup Logs	Get Backup Info - MS SQL	GetBackupStateMssql.html	N.A.	EXTERNAL REPORT (See ABOVE)
Backup Logs	Get Backup Info - Sybase	GetBackupStateSybase.html	N.A.	EXTERNAL REPORT (See ABOVE)
Backup Logs	Sybase Backup Configuration	SybaseBackupFileReport.html	N.A.	EXTERNAL REPORT (See ABOVE)

The system uses a variety of configuration information including the conf/gem.xml library which is managed by the GemData.pm perl module. This file stores, in xml, static configuration data about your environment - data often collected using the internal RosettaStone.pm module. This seems a superior approach to creating flat files for everything or to the approach of discovering all configuration informaton in real time. The main program that loads this database is called discover.pl. The actual file gem.xml is stored in the conf subdirectory of your appliation.

The tool kind of gets everything i could think of. It collects:

1. Sybase Error Logs
2. Sybase Backup Server Logs
3. Sybase Run Files
4. Sybase Configuration Information From the server

You will need to schedule your batch jobs. Luckily, the GEM configuration utility will create the crontab file that you will need to schedule. Here are some samples of a crontab file.

10 1 * * 0       /apps/sybmon/dev/unix_batch_scripts/batch/console_weekly.ksh
37 5 * * 1-6      /apps/sybmon/dev/unix_batch_scripts/batch/console_nightly.ksh
24,32,48,4 8-16 * * 1-5  /apps/sybmon/dev/unix_batch_scripts/batch/console_hourly.ksh
41 7-19 * * 1-6   /apps/sybmon/dev/unix_batch_scripts/batch/space_monitor.ksh
33 * * * *        /apps/sybmon/dev/unix_batch_scripts/batch/get_unix_diskspace.ksh

Gem contains numerous pre built reports that are derived from the monitoring system data. These reports are named with the string “GEM” at the beginning of them. For example, a report named $REPORTNAME is created by the GemRpt_${REPORTNAME}.ksh script (or GemRpt_${REPORTNAME}.bat on Win32) and appears in the Console as “GEM $REPORTNAME Report”.

Under the covers, the batch file calls the perl script RunMimiReport.pl – which in turn calls mimi.pl the CGI version of the monitoring interface. In other words – the prebuilt reports are a static (point in time) snapshot of the dynamic alarm system monitor. The exact syntax under the covers would be :

RunMimiReport.pl –REPORT=$REPORTNAME --OUTFILE=$GemRpt_INSTALL_DIR/data/html_output/$REPORTNAME.html --BATCHID=$REPORTNAME

The above command generates a report based on dynamic data stored in the alarming database. RunMimiReport.pl calls mimi.pl with a set of arguments associated to $REPORTNAME that are fetched from the ReportArgs table. You can, therefore create your own reports! RunMimiReport.pl has many associated features (like mailing the output to someone!) allowing you to say – mail an hourly snapshot of just the systems zen_app_1, zen_app_3, and zen_manager to bob, the zen guy every hour! This report can include EVERY thing monitored on any set of systems, servers, processes, or whatever… Send the pre built Unix_Errors and Unix_Warnings reports by mail to your harried unix guy. Send the Win32_Errors report to your frantic windows guy.

So… Backtracking… Let us assume you want a report… First you would create it in the monitoring page, which allows you to set all kinds of filters - then you note the URL – which (due to CGI conventions) contains the report arguments. The next thing you do is enter them into the ReportArgs table using a syntax like:

   delete ReportArgs where reportname = 'GemRpt_Backups'
   insert ReportArgs values  ('GemRpt_Backups','filter_severity','All')
   insert ReportArgs values   ('GemRpt_Backups','filter_container','All')
   insert ReportArgs values   ('GemRpt_Backups','filter_time','1 Day')
   insert ReportArgs values   ('GemRpt_Backups','ShowProgram','NO')
   insert ReportArgs values   ('GemRpt_Backups','ReportTitle','All Backup    Messages')
   insert ReportArgs values   ('GemRpt_Backups','progname','BACKUPS')

Then create/run your batchfile. I recommend using the –DEBUG argument to RunMimiReport.pl and saving the results into a .html file that you can view in brilliant color and magnificence in your web browser. When I run the above report with –DEBUG I see within the output the fact that the essential query was:

   select {stuff}
   from Heartbeat
   where monitor_program
   in ('backup.pl', 'BackupFull', 'BackupTran', 'LoadAllTranlogs','FixLogship')
   and ( reviewed_time is null or reviewed_until and internal_state!='D'
   order by t.system,subsystem}

To debug a report (say the example GemRpt_Backups report which comes prepackaged with the system) you can try something like:

   /usr/local/bin/perl /apps/sybmon/dev/ADMIN_SCRIPTS/monitoring/RunMimiReport.pl --BATCHID=GemRpt_Backups --REPORT=GemRpt_Backups --OUTFILE=/apps/sybmon/dev/data/html_output/GemRpt_Backups.html --DEBUG --MAILTO=ebarlow@home.com

Which will send you a mail report with the output of the report! The --DEBUG optional arg will give additional diagnostics.

The following report is generated by RunMimiReport.pl --GENREPORT and lists the arguments that are passed for each of the pre built reports.

Reportname	ProdChkBox	ReportTitle	ShowProgram	admin	filter_container	filter_severity	filter_time	progname	radio_screen	subsystem
Backups	NO	Backup Error Report	NO		BACKUP	Warnings	2 Hours
ExchangeFiles	YES	Exchange Files	NO		ExchangeFiles	All	2 Hours
ExchangeFiles2	YES	Exchange Files	NO		ExchangeFiles	Errors	2 Hours
FeedDf	NO	Market Data Disk Space	NO		FEED	Warnings	5 Days
Feeds	NO	Feed Error Report	NO		Feeds	Warnings	2 Hours
GemRpt_AgentReport						All			Agents
GemRpt_BackupReport						All			Backups
GemRpt_BlackoutReport		GEM Blackout Report		Blackout Report
GemRpt_Ignorelist				Ignore List
GemRpt_ProductionErrors	YES	Production Overnight Report	NO		All	Errors	Since 4PM
GemRpt_ProductionList				Production
GemRpt_ProductionWarnings	YES	Production Warnings	NO		All	Warnings	Since 4PM
GemRpt_Sql_Server_Errors	NO	Sql Server Errors	NO		SQL_SERVER	Errors	2 Hours
GemRpt_Sybase_ASE		Sybase ASE Error Logs	NO		All	Warnings	1 Day	Sybase_ASE_Log
GemRpt_Sybase_BackupServer		Sybase Backup Logs	NO		All	Warnings	1 Day	Sybase_Backup_Log
GemRpt_Sybase_Errorlog		Sybase Error Logs	NO		All	Warnings	1 Day	SybaseErrorlog
GemRpt_Sybase_Errors	NO	Sybase Errors	NO		SYBASE	Errors	2 Hours
GemRpt_Sybase_RepServer		Sybase Replication Logs	NO		All	Warnings	1 Day	Sybase_Replication_Log
GemRpt_Unix_2Hour	NO	GemRpt_Unix_2Hour	NO		UNIX	Warnings	2 Hours
GemRpt_Unix_Errors	NO	Unix Error Report	NO		All	Errors	1 Day	UnixLogmon	Events
GemRpt_Unix_Warnings		Unix Warning Report	NO		All	Warnings	1 Day	UnixLogmon	Events
GemRpt_Win32_Diskspace		Windows Disk Space	NO		All	Warnings	1 Day	PcDiskspace
GemRpt_Win32_Errors	NO	Win32 Errors	NO		WIN32	Errors	2 Hours
GemRpt_Win32_Eventlog		Windows Event Logs	NO		All	Warnings	1 Day	PcEventlog
GemRpt_Win32_SecurityRpt		Windows Security Events	NO		All	Errors	1 Day	PcEventlog	Events	Security
Imagine	NO	Imagine Error Report	NO		IMAGINE SYSTEM	Warnings	2 Hours
ProdOvernight	YES	ProdOvernight	NO		All	Errors	Since 4PM
ProdWarnings	YES	Production Warnings	NO		All	Warnings	2 Hours
Unix2Hour	NO	Unix2Hour	NO		Unix	Warnings	2 Hours

A container is a set of servers or programs that can be used to filter stuff for particular users. This system is not totally flexible - you can choose a container to include a set of other containers, systems, or monitoring programs - but you can not restrict it further. For most users this is acceptable.

Users will probably be interested either in a set of servers (APP1, APP2, DB1) or a set of monitor programs (ie all the unix monitoring programs for your unix admin).

As a note: we should probably should rename ContainerMap to ContainerProgram and ContainerOverride to ContainerSystem

Container Definition via mimi.pl

   Program -> Container Mapping (Program=Sybase_ASE_Log) Container=DELETEME
      Note: Must add container by hand or it wont show
      Insert ContainerMap Values ('Sybase_ASE_Log','DELETEME')
   System -> Container Mapping (CONTAINER=DELETEME, SYSTEM=mlpsyb)
      if exists ( select * from Container where name='mlpsyb' )
            Insert ContainerOverride Values ('DELETEME','c','mlpsyb',null)
      else    Insert ContainerOverride Values ('DELETEME','s','mlpsyb',null)

TABLE: ContainerMap

            fields monitor_program,container

TABLE: Container_full

            fields name,element

TABLE: ContainerOverride

            fields name, type, element, owner

TABLE: Container

            fields name (container), type (s=system, c=container, p=program), element, owner (unused)

PROCEDURE: build_container

   delete Container

   -- First map containers from ContainerMap
   INSERT Container
      select cm.container,'s',e.system,null
      from ContainerMap cm, Event e
      where e.monitor_program = cm.monitor_program
   union
      select cm.container,'s',e.system,null
      from ContainerMap cm, Heartbeat e
      where e.monitor_program = cm.monitor_program
      and e.batchjob is null

   delete Container
   FROM Container c1,  ContainerOverride c2
   where c1.name=c2.name
   and   c1.type=c2.type
   and   c1.element=c2.element

   INSERT Container
   select * from ContainerOverride

   delete Container_full

   INSERT Container_full
      select 'All',e.system from Event e
      where monitor_program!='Monitor'
   union
      select 'All',e.system from Heartbeat e
      where monitor_program!='Monitor'
      and e.batchjob is null

   INSERT  Container_full
   select  distinct name, element
   from    Container
   where     type = 's'
 union
   select  c1.name, c2.element
   from    Container c1, Container c2
   where   c1.type = 'c'
   and     c2.type = 's'
   and     c1.element = c2.name

   select  c1.name, c3.element
   from    Container c1, Container c2, Container c3
   where   c1.type = 'c'
   and   c2.type = 'c'
   and   c3.type = 's'
   and   c1.element = c2.name
   and   c2.element = c3.name

   select  c1.name, c4.element
   from    Container c1, Container c2, Container c3, Container c4
   where   c1.type = 'c'
   and   c2.type = 'c'
   and   c3.type = 'c'
   and   c4.type = 's'
   and   c1.element = c2.name
   and   c2.element = c3.name
   and   c3.element = c4.name

   INSERT    Container_full
   select    distinct 'All',e.system
   from    IgnoreList e
   where    e.system not in ( select element from Container_full )

   INSERT ProductionServers
   select distinct element,0
   from   Container_full
   where  element not in ( select system from ProductionServers )

REPORTS: Containers

   select    name,
         Type=case when type='c' then 'Container'   when type='s' then 'Server'   else 'unknown'      end,
         element
   from Container order by name,type"

REPORTS: System->Container

      select    name,   element from ContainerOverride order by name,type

REPORTS: Program->Container

      select    monitor_program, container from ContainerMap order by monitor_progra

USAGE

OneQueryPerServerReport.pl

Generic procedure runner. Will run a single query on each of your servers, formatting output correctly.

REQUIRED ARGUMENTS

        -QUERY=QUERY
        -PROGRAMNAME=PROGRAMNAME        (default to BATCHID)
        -TITLE=TITLE

OPTIONAL ARGUMENTS

        -HTML
        -NOLOCK                 (ignore lock file to prevent repeat runs)
        -BATCHID=batchid
        -NOHEADER               (ignore header rows from results)
        -PREREQ                 (prerequisite query - only continue if this returns something)
        -NOPRINTMSGS    (ignore messages from the server)
        -DEBUG                  (debug)
        -NOLOCK
        -SYSTEMS=systems
        -SERVER=Server
        -PROGRAMNAME
        -TITLE
        -PRINTSERVERLABEL       (pretty print a nice label for each server)
        -TYPE=(DB Type - sqlsvr / sybase / both)

SYNTAX

perl console_build_and_ftp.pl $SYBASE

SCHEDULING

This program should be scheduled frequenly. It rebuilds the console from saved pages and can ftp it to remote sites.

SYNTAX

perl console_hourly.pl $SYBASE

SCHEDULING

This program should be scheduled about once an hour. It does needed maintenance on the web pages.

SYNTAX

perl console_nightly.pl $SYBASE

SCHEDULING

This program should be scheduled about once a night. It does needed maintenance on the web pages for static stuff. Specifically it does the following

  1.  Log itself with the alarming system.
  2.  Run document_all.pl -I10,15,18

     Rebuild all your web pages - heck its the middle of the night.  For a variety of reasons, the
     following 3 reports are NOT run (thats what the -I means = IGNORE)

     Report #10 - schema report
     Report #15 - File system checks
     Report #18- hourly server logs - format a recent error logs report

  3.  Run ftp_to_website.pl to copy the resulting web pages if you have decided that you want
      them to be copied to another web site.

SYNTAX

perl console_weekly.pl $SYBASE

SCHEDULING

This program should be scheduled once a week. It does a survey on the whole world and rebuilds the whole web site. This is basically discover.pl plus all the functionality of the console_nightly.pl.

SYNTAX

perl console_hourly.pl $SYBASE

SCHEDULING

This program should be scheduled about once an hour. It does needed maintenance on the web pages.

SYNTAX

perl console_nightly.pl $SYBASE

SCHEDULING

This program should be scheduled about once a night. It does needed maintenance on the web pages for static stuff. Specifically it does the following

  1.  Log itself with the alarming system.
  2.  Run document_all.pl -I10,15,18

     Rebuild all your web pages - heck its the middle of the night.  For a variety of reasons, the
     following 3 reports are NOT run (thats what the -I means = IGNORE)

     Report #10 - schema report
     Report #15 - File system checks
     Report #18- hourly server logs - format a recent error logs report

  3.  Run ftp_to_website.pl to copy the resulting web pages if you have decided that you want
      them to be copied to another web site.

SYNTAX

perl console_weekly.pl $SYBASE

SCHEDULING

This program should be scheduled once a week. It does a survey on the whole world and rebuilds the whole web site. This is basically discover.pl plus all the functionality of the console_nightly.pl.

USAGE

core_dump_checker.pl -o outdir

DESCRIPTION

Survey your servers and store the results in gem.xml.

USAGE

  discover.pl -Ssys1,sys2,sys3 -Tunix,oracle,sqlsvr -d

where sys1..3 are systems and -T represents the 5 types currently supported (unix,oracle,sqlsvr,win32servers,sybase).

-O to print diagnostic Output dump

DESCRIPTION

The GEM COnsole is a web system with html documentation on all your servers. The input (server names, passwords, report definitions) comes from your configuration files.

document_all.pl is the main script called by the GEM batch jobs in order to create your console. The scheduled batch scripts are simply drivers around this script. document_all.pl is responsible for creating html output reports.

This program integrates with the Alarming library to save alarms as approriate.

USAGE

document_all.pl -OUTDIR=outdir -REPORT=1,2,3... -IGNOREREPORT=1,2,3

        -OUTDIR dir       : output directory
        -QUICK                          : dont do the glue stuff
        -IGNOREREPORT=list: list of reports to ignore (comma separated)
        -REPORT=list      : list of reports to run (comma separated)
        -PRINTTIMING      : print timing information for reports
        -NOREPORTS        : no rebuild of pages
        -ALARM            : use mlp alarms
        -DO_SQLSVR        : run sql servers
        -DO_SYBASE        : run sybase servers
        -DO_ORACLE                      : run oracle servers
        -BATCHID          : label of which batch this was run by
        -CONSOLE_REBUILD  : quick run - just reformat main web site

Report numbers are identified as follows

        1)  Interfaces file report
        4)  Create Debug/error report (by reading _debug.html files)
        5)  Weekly Security Reports
        6)  Disk usage report
        7)  Get backup server logs
        8)  Sybase server error logs
        9)  Space report
        10) Schema report
        11) Host os info (some reports require rsh setup)
        12) Host uptime info (requires rsh setup for rup command)
        13) ifconfig -a (requires rsh setup for rup command)
        14) Unix disk space (requires rsh for df -k)
        15) File checks (cores, crontabs, failed links, mail -> unix only)
        16) Dbcc & Backup reports
        17) User defined reports (see console.dat)
        18) Unused
        19) Unused
        20) Core Dump Checker (local, long)
        21) Gem Reports
        22) By Server Pages
        23) Weekly Updates (need this on windows or unix)
        24) Nighly Updates (need this on windows or unix)
        25) Database Debugger - Calls CheckServerDaily.pl
        26) Standard Internal Reports

The program rebuilds your main web page automatically based on whatever output files exist in the output directory, so you do not need to run all reports every night.

This program should be totally config file driven.

FILES

Console_Info.dat = identify console report layout

   these reports are found in $gem_root_dir."/data/html_output"
   when found they are reformatted and moved
   file format is comma separated ($FILENAME,$MENU,$TITLE,$IS_OPTIONAL)
   These reports are placed in $MENU/$TITLE with link referring to $FILENAME

CONSOLE.dat

   These are custom reports processed by run_custom_defined and read_custom_defined_reports

OUTPUT DIRECTORY

The output directory is set via the driver scripts but is normaly data/CONSOLE_REPORTS. This directory contains a variety of subdirectories as needed.

RELATED BATCH JOBS

ConsoleArchiver.ksh

   /usr/local/bin/perl /apps/sybmon/dev/ADMIN_SCRIPTS/bin/ConsoleArchiver.pl --BATCHID=ConsoleArchiver $*

ConsoleBuildAndFtp.ksh

   /usr/local/bin/perl  /apps/sybmon/dev/ADMIN_SCRIPTS/console/console_build_and_ftp.pl $*

ConsoleUnixHourly.ksh

   /usr/local/bin/perl  /apps/sybmon/dev/ADMIN_SCRIPTS/console/console_hourly.pl $*

ConsoleUnixNightly.ksh

   /usr/local/bin/perl  /apps/sybmon/dev/ADMIN_SCRIPTS/console/console_nightly.pl $*

ConsoleUnixWeekly.ksh

   /usr/local/bin/perl  /apps/sybmon/dev/ADMIN_SCRIPTS/console/console_weekly.pl $*

ConsoleWin32Hourly.ksh

   C:/perl/bin/perl.exe  //samba666/sybmon/dev/ADMIN_SCRIPTS/console/console_win32_hourly.pl $*

ConsoleWin32Nightly.ksh

   C:/perl/bin/perl.exe  //samba666/sybmon/dev/ADMIN_SCRIPTS/console/console_win32_nightly.pl $*

ConsoleWin32Weekly.ksh

   C:/perl/bin/perl.exe  //samba666/sybmon/dev/ADMIN_SCRIPTS/console/console_win32_weekly.pl $*

UNDERLYING FUNCTIONALITY OF THESE BATCH JOBS

The following convenience batche jobs are provided. They map 1:1 to batch jobs set up by the scheduler. The win32 jobs should be run on windows, the others only on unix. console_build_and_ftp.pl should be run either on windows or unix (but not both).

console_build_and_ftp.pl => document_all.pl --BATCHID=ConsoleBuildAndFtp --PRINTTIMING -CONSOLE_REBUILD console_hourly.pl => document_all.pl --BATCHID=ConsoleUnixHourly -PRINTTIMING -REPORT=7,8,9 -ALARM -NATIVE console_nightly.pl => document_all.pl -BATCHID=ConsoleUnixNightly -IGNOREREPORT=10,15,18,20,22 -PRINTTIMING -NATIVE -ALARM console_weekly.pl => discover.pl -Tunix,oracle,sybase

                           => space_monitor.pl --BATCHID=WeeklyTablespaceWrite --REWRITE --NOGEMXMLWRITE --DOHELPTABLE
                           => document_all.pl -BATCHID=ConsoleUnixWeekly -IGNOREREPORT=100 -PRINTTIMING -ALARM -NATIVE

                           => space_monitor.pl --BATCHID=WeeklyTablespaceWrite --REWRITE--NOGEMXMLWRITE
                           => document_all.pl --BATCHID=ConsoleWin32Weekly -IGNOREREPORT=100 -PRINTTIMING -ALARM  -NATIVE

NATIVE SETTINGS

The GEM Console is designed to identify your environment and set the standard variables as appropriate when --NATIVE is passed in. Specifically --NATIVE will set -DO_SQLSVR, -DO_SYBASE, -DO_ORACLE and --PRIMARY.

   if( $CONFIG{INSTALLTYPE} == WINDOWS and is_nt() ) {
         $DO_SQLSVR=1 if $CONFIG{USE_SQLSERVER} eq "Y";
         $DO_SYBASE=1 if $CONFIG{USE_SYBASE_ASE} eq "Y";
         $DO_ORACLE=1 if $CONFIG{USE_ORACLE} eq "Y";
   } elsif( $CONFIG{INSTALLTYPE} == UNIX & ! is_nt() ) {
         $DO_SYBASE=1 if $CONFIG{USE_SYBASE_ASE} eq "Y";
         $DO_ORACLE=1 if $CONFIG{USE_ORACLE} eq "Y";
   } elsif( $CONFIG{INSTALLTYPE} == SAMBA and is_nt() ) {
         $DO_SQLSVR=1 if $CONFIG{USE_SQLSERVER} eq "Y";
         $DO_SYBASE=0;
         $DO_ORACLE=1 if $CONFIG{USE_ORACLE} eq "Y";
   } elsif( $CONFIG{INSTALLTYPE} == SAMBA and ! is_nt() ) {
         $DO_SYBASE=1 if $CONFIG{USE_SYBASE_ASE} eq "Y";
   }

USAGE

findid.pl -i sourcedir -A -N

   -d debug mode
   -N dont use html

SYNOPSIS

This program copys the files to your web site - as specified in the GEM configuration utility. Sometimes, the local directory is not where you wish to publish your console. This program does the copy, or ftp if needed, to your actual web site. This modified version of ftp_to_website.dist is customized during the configuration process to include your server information.

DESCRIPTION

Optionally Fetch files or Print File Information From gem.xml.

USAGE

gem_file_manager.pl -SERVERNAME=sys1,sys2,sys3 -DEBUG -DOREPORT -DOANALYSIS -DOF ETCH -OUTFILE=file -FILETYPE=FILE_TYPE

This is the main program Retrieve, Parse, and Handle Files From Your Systems

Required: -FILETYPE=FILE TYPE

      BACKUP_SERVER or ASE_SERVER or HISTORICAL_SERVER or MONITOR_SERVER
         or REP_SERVER or Envronment File, Errorlog, Interfaces File, Run File

Required: One of these 3 operations

   -DOREPORT      print a report of files to fetch
   -DOANALYSIS    analyze the files you have collected
   -DOFETCH       fetch files

Optional:

    -HTML               html output
    -SAVE_TO_ALARMDB    save to alarm database
    -OUTFILE=outputfile
    -BATCHID=BATCHID    key for the alarms system (internal)
    -PATFILE=paternfile patern files allows restart for logging
    -SERVERNAME=Svrs    to work on (default is all of appropriate type)
    -TODAYYESTERDAYONLY filter for today/yesterday errors (10000 messages)
    -NOERRORFILTER      dont filter non errors out (show all)
    -PURGESIZE=<size>   purge source files larger than <size> bytes (after fetch

    The Default Filter will filter to a maximum 1000 messages per file

USAGE

interfaces_file_rpt.pl [--PORTLIST] [--DEBUG] [--HTML]

USAGE

space_errors.pl -i sourcedir -A -N

   -d debug mode
   -N dont use html

USED BY

SybSpaceMonitor SpaceMonitorSqlsvr

DESCRIPTION

This utility monitors database space

It also monitors space history into ascii files. This utility will save table space information the first time it is run every day. These files are named helptable.SERVER.DB.DATE.TIME. It will also save database level space information so that can be tracked. Data is saved in the gem/data directory in ascii file format.

You may need to run two batches of this job - one for sqlserver (only works from windows) and one for sybase (works both windows and sql).

USAGE

space_monitor.pl -outdir=outdir

MONITOR SPACE USAGE ON YOUR SYBASE/SQL SERVER DATABASES

-BATCHID=batchid -DEBUG (debug) -SERVER=Server -DATABASE=Database -TYPE=(DB Type - sqlsvr / sybase / both) -DOHELPTABLE ( table space ) -REWRITE (Rewrite) -ALARM (Alarm appropriately) -NOGEMXMLWRITE (no gem.xml write)

With --DOHELPTABLE saves sp_helptable into output directory in files named

   helptable.SERVER.DB.DATE.TIME

This directory output is useful for the program plot_tablespace.pl This program works with space_report.pl

OVERRIDES

The program uses threshold_overrides.dat for override information.

USAGE

space_report.pl -i outdir -d

NOTES

looks through the system_information_data tree and gets files starting with "dbspace_". These files contain space used information - one line per day. This data is parsed and put into a reasonable report.

The files are created by space_monitor.pl

DESCRIPTION

Monitor Unix Disk Space on all servers tagged COMM_METHOD=RSH or SSH and then store results using the alarming subsystem. Should be run frequently. When run from GEM console, the report created is put onto web pages.

ARGUMENTS

Use -A to save heartbeats.

SYNTAX

unix_space_monitor_via_rsh.pl -o outdir -A

                        -A use mlp alarms
                        -t test mode - additional prints
                        -S hostname (comma separated)

Monitor Unix Disk Space on all servers tagged with COMM_METHOD=SSH or RSH and STORE results using the alarming subsystem. Should be run frequently. When run from GEM console, the report created is put onto web pages.