GEM Getting Started With GEM

This guide will give information on how to get started with the Generic Enterprise Manger (GEM). The scope of this project is large, but if you follow this guide, we can make this tool a success for you. This document describes the structure of the code line - necessary information to understand the application.

GEM provides a framework to manage multiple database systems. The system is built upon a set of perl libraries which provide the glue to bind the system together. Since its all open source perl, expansion and customization are easy. This guide endeavors to guide you through the process of installing and customizing GEM.

One thing to remember. GEM is designed for simplicity. The goal is to put code, data, and libraries in locations where a reasonable administrator would expect them to be found. After you extract GEM, please poke around and familiarize yourself with the product. Look for utilities in the ADMIN_SCRIPTS subdirectory and configuration files in the conf subdirectory. Knowledge of the layout is essential to maintaining and operating GEM as your production server managment interface.

This document provides an overview of GEM layout. To truly understand the layout, you will need to have an existing copy of the code. To get this, please read the Installation Guide.

	GEM Base Directory	This is where you extracted the software.
	gem.pl	This is the GEM enterprise manager - which rolls up all the plugins. It is not currently functional.
	configure.pl	This is the GEM setup utility
	troubleshoot.pl	This is the diagnostic utility. It is used to diagnose configuration and setup problems.
	monitor.pl	The systems monitor. It allows drill down server diagnostics (works on system tables)
	eventviewer.pl	The system event viewer

The main command line programs of the GEM are stored in ADMIN_SCRIPTS and broken down by package

	GEM Base Directory	This is where you extracted the software.
	ADMIN_SCRIPTS	Command line programs reside here.
	bin	Miscellaneous utilities
	dbi_backup_scripts	Sybase and Sql Server Backup Scripts
	MDA_procs	Sybase MDA Table monitoring procedures
	monitoring	Programs designed for monitoring
	oracle	Oracle Scripts
	procs	Sybase and Sql Server Extended Stored Procedure Library
	console	Scripts that manage the GEM Console

The data directory is where information is stored about your systems. This information is broken down by package.

	data	Sub directory for generated data - this directory can get big
	BACKUP_LOGS	This directory contains the output from your maintenance plan scripts broken down by server. Within each server specific subdirectory exists a directory for audits, dbcc, errors, and sessionlogs.
	batchjob_errors	Standard Error from the maintenance scripts
	GEM_BATCHJOB_LOGS	Standard Output from the maintenance scripts
	CONSOLE_REPORTS	This is the console. You point your browser here to see it.
	depends	Files required for the dependency analyzer
	cronlogs	Logs of your scheduled jobs
	html_output	Output of any html reports you want auto included in documentation
	lockfiles	Lock Files for programs that can be overscheduled
	system_information_data	Monitoring data and configuration information by server

The conf directory contains system configuration files. The following are the main configuration files

Files are shiped in the distribution with .sample extensions - these are the templates with standard comments. No files are shipped with .dat, .xml, or .cfg extensions. This permits you to perform an upgrade by extracting the new tarball over the existing code line with no problems.

GEM Scheduled Tasks

One of the core features of GEM is its monitoring and maintenance scripts. These scripts perform the systems monitoring and alarming, perform the system administration operations you define in your maintenance plans, and provide useful routine maintenance activities. The scripts are customized to your environment by the GEM configuration manager. For version 1, some of these scripts may involve hand definition of a configuration file.

The configuration manager will create scheduled for some of these jobs on your Windows scheduler and will create a Unix crontab file for jobs you indicate you wish to run on Unix. These tasks fundamentally entail a call to one of the command line scripts located in the ADMIN_SCRIPTS directory, but the GEM hides this from you. GEM creates a set of .bat and .ksh driver scripts that have obvious names to be the scripts that you actually run or schedule.

These scripts are stored in the win32_batch_scripts and Unix_batch_scritpts directories. Within those directories are 4 subdirectories as follows:

	GEM Base
	unix_batch_scripts	This is where scripts that run under Unix exist. These directories are populated with customized scripts.
	interactive	Output from these commands can bee seen. These are the scripts that you use
	batch	These scripts are for use by your scheduler. They redirect output to specified log and error files. If you run them you will get no output.
	plan_interactive	Scripts for the backup / maintenance plan system. These scripts are interactive as per above
	plan_batch	Scripts for the backup / maintenance plan system These scripts are for use by the scheduler as per above

The above tree is duplicated in the win32_batch_scripts subdirectory.

Simply put, each script is created as a .ksh on Unix and as both a .bat and .ksh script on Windows. Copies of the script exist in an interactive and batch directory (or a plan_interactive/plan_batch directory). The batch scripts redirect output to locations in the data tree (so you cant see it) but are identical to the interactive scripts. When you run a scheduled task, use the batch version, and when you run interactively, use the interactive version (you want to see output if you are running interactively). The plan_* directories store pregenerated maintenance plan scritps. The plan directories were separated from the regular directories because the tree becomes hard to navigate when you get large numbers of servers (and a large number of maintenance plans).

Backup and Maintenance Plans

GEM offers a complete maintenance plan based system maintenace facility. Plans are set up using the GEM Installer and stored in the configuration file configure.cfg. All normal features related to systems maintenance are covered including Backups, DBCC, Update Statistics, and Log Shipping. Additionally, backup compression for SQL Server, notification, and extensive reporting are provided. The command line interface provides a level of flexibility not available with native SQL Server facilities.

GEM Enterprise Manager

The GEM Enterprise Manager is the graphical user interface for GEM. It is a modular system that uses plugins for all functionality. Plugins use single console for all your systems. It uses perl DBI - a database independant api - and provides you with a seamless api. The trick here is the RosettaStone.pm perl module, which basically maps database independant requests (get me locks/logins/space) to database specific ddl. This gives you consistency. The GEM. is designed to use plugins, which are basically small perl modules that conform to an api. Most of your services are provided by the other GEM libraries, so these modules basically define what user interface elements they want (menus, how they want their explorer to look, right click functionality) and then manage a Tk frame however they want. The goal here is to have third parties develop plugins that add functionality. And the nice thing here, is that this can be business specific functionality (ie. a trade or order volume report). The GEM plugins are managed from a central store on the internet which is available to you to distribute your plugins in the same manner that CPAN does for perl modules.

Documentation and Diagnostics

All system documentation uses the standard perldoc format. Perldoc is a utility provided with perl to extract documentation from programs. To get full documentation on a program, simply run perldoc program. The majority of the GEM documentation is composed from the embedded perldoc. You can see a short version of the help by simply running perl program.pl -?, which will provide useful syntax help.

Older database programs use the standard syntax of -Uuser -Ppassword -Sserver -Ddatabase for arguments. Newer programs use --USER=username --PASSWORD=password --SERVER=servername --DATABASE=database convention.

If you have problems with a programs, try running with -d or --DEBUG flags set. These will run the program in verbose/diagnostic mode and the program will print useful diagnostic information. This is the standard way of debugging problems.

This output is documentation for the SQL Technologies Getting Started With GEM.

The Complete GEM Documentation	Slide Show: What Is GEM	Download GEM

An Introduction To GEM	How To Install	Getting Started Guide
Batch Jobs Overview	GEM Screenshots	Server Backup and Maintenance
Utilities Guide	The GEM Console	New System Stored Procedures
FAQ and Change Log	Common Perl Libraries	Pricing & Right To Use

FILE NAME	FILE DESCRIPTION
gem.xml	This file is managed by the survey scripts and used by the enterprise manager. It contains discovered configuration information.
configure.cfg	this file is the main configuration file for the backup scripts
documenter.dat	this file determines extra reports and programs to include in the GEM Console
oracle_passwords.dat	Oracle Password File
pc_password.dat	Password file for Windows servers;
pc_service.dat	File listing services on Windows that must be up
port_monitor.dat	File listing systems/ports to monitor
sqlsvr_password.dat	Sql Server Password file
sybase_passwords.dat	Sybase Password file
threshold_overrides.dat
Unix_passwords.dat	Unix password file
Win32_Cleanup.dat	Directories under Windows for routine purge/cleanup

Getting Started With GEM

Getting Started Guide

for the GEM

Directory Structure

GEM User Interfaces

The ADMIN SCRIPTS Directory

The Data Directory

The conf Directory

GEM Scheduled Tasks

Backup and Maintenance Plans

GEM Enterprise Manager

Documentation and Diagnostics