A Catalog of DBA Coverage Necessities
by Steve Woody, CommitDBA Director
CHALLENGESThe enterprise database is the most important, sensitive, and financially valuable intangible asset for data driven organizations.
Today’s DBAs are faced with significant challenges with evolving enterprise database technology stacks and complexities.
Holistic Database Management is required to ensure that there are no coverage gaps which could destabilize the database environment.
SOLUTIONThe Database Essentials present a Catalog of High Level Database Coverages which DBA tasks are defined within to administer databases efficiently. By establishing the Top 10 Database Essentials, there is now a holistic approach and framework for optimal database management coverage without any gaps.
TOP10Database Essentials include catalog topics of database Management, Security, Recovery, High Availability, Performance & Monitoring, Architecture, Scalability & Capacity, Development, Remediation, and Maintenance.
MANAGEMENTHaving the appropriate oversight and governance of database tasks and activities requires strong management policies and procedures within the database domain. This essential coverage requires communication, reporting, best practices, quality assurance, change management and ongoing service improvements.
SECURITY SAFEGUARDSThe database is the last line of defense regarding security and data breach. This essential coverage requires the monitoring and management of your database security appropriately from every type of threat.
RECOVERYIf you are not convinced that the recovery processes of your enterprise databases are rock solid, then your com-pany is at high risk for ongoing business continuity.
This essential coverage requires the monitoring and management of your database backup and recovery processes.
Appropriate “checks & balances” need to be included with the overall plan. Some of these include testing, monitoring, change control, scalability, premium backup software and tools and audits, redundancy strategies, offsite storage and others.
AVAILABILITYDatabase High Availability is critical for production operations in today’s e-business. This essential coverage requires best HA solution options based on database processing requirements.
PERFORMANCEOptimal database performance is mandatory for application processing.
This essential coverage addresses the best practices for sustainable performance while utilizing best-in-class monitoring tools for proactive incident management and historical analysis.
ARCHITECTUREDatabase architecture relates to internal and external components.
This essential coverage addresses DBA support requirements to optimize these architectural components.
Properly architected databases help improve other essentials such as backup & recovery, high availability, database performance, scalability and development.
On the other hand, poor architecture can lead to significant risks.
Virtualization is now playing a key role in database architecture decisions and implementation.
SCALABILITYMany organizations do not realize that with intensive data growth, scalability & capacity are critical for database sustainment.
With this essential coverage the DBA manages scalability requirements via monitoring, usage patterns and historical analysis. Based on the analysis both hardware and software scalable options can be determined.
DEVELOPMENTApplication and evelopment DBA support is required at many sites.
With this essential coverage, the DBA manages the various database internal components. These can include stored procedures, functions, triggers and others along with re-porting requirements from the database.
REMEDIATIONDatabase problems and incidents do occur no matter how well the database is managed.
With this essential, the DBA is able to be proactive 24x7 utilizing best-in-class monitoring tools to assist with root cause analysis and is able to remediate problematic conditions quickly.
Appropriate DBA response standards and processes should be in place for the Primary or Secondary DBA to respond and remediate within the timeline dictated by the severity of the database incident, problem or issue.
MAINTENANCEDatabase maintenance may not be the most interesting aspect of DBA services but is very important.
With this essential coverage maintenance functions such as patching, updates, scans, file cleanup and others help keep the database in excellent condition and compliance.
DEPLOYMENTDeploying new database versions can range from the simple to the complex.
The DBA must be prepared for all types of deployments.
CONCLUSIONDue to the DBA role expanding with emerging technologies it is critical to have a framework of support categories which the Top 10 Database Essentials provide.
BUSINESS VALUEOutsourcing to a Database Managed Service Provider (DMSP) provides business value with complete coverage for the Top 10 Database Essentials.
Economies of scale and efficiencies has improved performance with Cost savings from utilizing CommitDBA services.
How do data-driven companies know that their enterprise databases are managed holistically utilizing industry best practices that cover all administrative tasks? There are many aspects to that question which include such requirements as - utilizing expert DBAs or DBA services, database management policies and standards, change control procedures, proper oversight and embedded project management. In addition, and very importantly, a catalog of database topology coverages is required so that the DBA understands completely what database components are to be serviced.
Have you heard marketing spin from database vendors to the effect of - “our enterprise databases, middleware and add-on options are so advanced that DBA services are not needed or minimally needed”? Nothing could be further from the truth! Basically, this is a marketing ploy to influence the uneducated buyer. While new features and enhancements provide solutions to improve database integration, availability, recovery, performance and other attributes, database management is as complex and challenging as ever due to the evolution of database and application stacks. That’s why it is most important to utilize best practices and why CommitDBA has created a Catalog of the Top 10 Database Essentials by which DBA best practices are administered.
The enterprise database is the most important, sensitive and financially valuable intangible asset for any organization. Today’s technology advancements provide tiered enterprise applications that initiate transactions to databases through mobile and web based applications. Local and custom applications are still embedded into many organizations as well. These applications enable organizations to run their business requirements, such as Business Processing, Marketing, Business Intelligence, Information Management, Inventory, Sales, CRM, Social and Data Mining, along with other customized business needs.
All critical and key data is stored in the enterprise database, whether it be in the form of a Transactional database, Mirrored/Replicated database, Data Warehouse/Data Mart/BI database, Big Data, hybrid database or various other database types. Data types processed by applications can include Corporate data, Customer data, Financial data, Sup-ply Chain data, Big data, Archival data and on and on. The emerging Cloud model includes Cloud based hosting of databases along with SaaS (Software as a Service) applications. While this model has matured from the previous hosting center model, it still requires holistic database management and monitoring in the same way that on premise data-bases require these services.
No matter what the mixture of applications, data types, database struc-tures or other variances, if an organization incorporates the “Top 10 Database Essential Areas of Coverage” their enterprise databases will be accessible, protected, sustainable, optimized and reliable. By breaking down DBA tasks into categorized domains, database services can be or-ganized holistically minimizing the potential for any service gaps to a minimum. This also provides an organization with a blueprint of what DBA services are to be expected for operations to run efficiently. The DBA now has the proper guidelines and a unified approach for holistic database management. While some categories have services and tasks that might overlap, each essential category contains its own topologies and potentially hundreds of DBA support elements, depending on the type of database.
So what are the Top 10 Database Essentials that every organization should have database services provide?
1. Database Management Execution
Enterprise databases are the most important intangible asset of any organization and without appropriate database management and data-base essentials coverage, the danger of uninterrupted business continuity is a constant threat. Database management execution reflects the professionalism and ownership of the DBA or DBA Services firm to manage, implement and execute necessary database support requirements successfully. Without proper database management responsibilities and oversight, there is high risk that the remaining Database Essentials will not be implemented or administered properly.
Database Management Execution includes: (1) Providing oversight and guidance regarding the Top 10 Database Essentials; (2) Proper communicating with other IT units and business management regarding database administration functions; (3) Providing reports on the activities, risks and improvements needed; (4) Embedding best practice and quality assurance controls within DBA services; (5) Working with Change Management processes for efficient implementation of database changes; (6) Managing incident control and remediation; and (7) Providing continual service improvement so that the DBA services are optimized to the fullest. While there are additional database management roles and responsibilities those listed above are key.
2. Database Security Safeguards
With ever increasing data breaches and data security incidents, it has never been more important to ensure that all database protection points are covered and preventive controls are in place. The database is the “last line of defense” once other IT infrastructure perimeter components have been compromised. Security safeguards and compliance for databases range from the simple to complex. These safeguards can include managing user account credentials, privileges and roles, user label identity, password authentication and policies, auditing, data masking, transparent data encryption, data privacy, data labels, ongoing security patches, SQL injection safeguards, database firewalls, secure backups and many other advanced security features. Appropriate access control policies should be in place so that unauthorized individuals do not have highly privileged administrative accounts to the database. In addition, obtaining regulatory compliance certifications and audits for such pro-grams as SOX, HIPAA, PCI, SSAE 16 and others further reinforces security measures for all aspects of a company.
Sensitive data residing in the database is often exported to external dump files and the data is offloaded to secure data storage centers, such as Iron Mountain, for backup archival purposes. Engaging encryption options with exports including password encryption in order to access the export reduces risk for data breaches. In addition, if your databases are backed up with a full service Cloud backup and recovery provider, adequate endpoint protection is essential. This could require such security provisions such as full data transfer encryption (e.g. AES/3DES encryption with FIPS 140-2); decryption keys, data destruction compliance; the auditing of encrypted devices, backup and deletion events and other security protection points.
Database services should include a thorough assessment and audit to look for database security vulnerabilities to prevent breaches and the loss of data as an ongoing preventative control. Database security audits may find orphan/defunct accounts that should have been removed along with inappropriate or inaccurate privileges assigned. Collaborating with other IT infrastructure staff to assess protection points outside the database is essential as well. Finally, automating your processes for preventive controls expedites risk management and remediation. Implementing a risk management approach for database security enables each organization with a proactive model for critical database security safeguards.
3. Database Recovery Reliability
Solid backup execution and recoverability is a key component of IT con-tinuity operations and disaster recovery strategies. It is one of the criti-cal business continuity factors for every organization that values its cor-porate data. So, what can you say about Database Recovery Safeguards, other than it is one of the top two critical Database Essential functions for all DBAs? The other being Security Safeguards. Database backup/recovery reliability is essential and should demand the full attention of DBA services! Solid database backup strategies and processes should be foolproof and worry free from whatever points of failure that might occur. There have been plenty of cases where data recovery was slow or impossible or due to a variety of reasons. Backup and recovery failure could be caused by DBA apathy, sloppiness, poor strategy or lack of strategy, lack of plan-ning, lack of testing, lack of execution, lack of competence and various other reasons that could have prevented failure points. The top “checks and balances” that database services must provide solu-tions to ensure Recovery Reliability include; (1) Redundant Backup Strategies, (2) Reliable backup processes and procedures, (3) Backup & Recovery Testing, (4) Backup Monitoring, (5) Change Control, (6) Scala-bility and Capacity, (7) Best-in-Class Backup Tools/Software, (8) Back-up Audits and (9) secure offsite storage with a firm like Iron Mountain. New technology solutions include Cloud based backup services where an organization can outsource backups to Cloud based service centers. The databases are auto replicated to the Cloud centers which offloads most of the DBA’s on premise backup responsibilities. However, you will need to vet out each Cloud service provider for issues such as secu-rity, performance levels and cost. For a more thorough understanding of Recovery Safeguards relating to databases, please read the blog “Good Backups Gone Bad – The Down-side of Assuming your Backups are Good.”
4. Database High Availability Maximized
The Database High Availability Maximized Essential for enterprise data-bases is a top priority for DBAs in today’s web and e-commerce environment. With rapid and explosive data growth, ongoing Big Data requirements, e-commerce/e-business and web service advancements, emerging mobile apps and enterprise application software (EAS) the norm, 24x7 high availability (HA) mandates are common. Database services and DBAs must possess the expertise to implement best-possible HA availability options and redundancy solutions along with the skills and methodology to support such solutions. Each database environment is different and various availability options can be expensive both from the hardware and software perspective. However, an experienced DBA can provide the right solutions for optimal balance that will ensure that database availability is maximized to meet business requirements.
Today’s primary options and solutions for maximizing database high availability include: (1) database clustering providing immediate fail-over, (2) hot-active standby databases, (3) utilizing databases in Virtual Machines (VMs) and (4) databases utilizing Cloud services. While there are additional high availability options, the above four guarantee the highest reliability and performance.
5. Database Performance & Monitoring Optimized
The Database Performance and Monitoring Essential is challenging for any DBA and can be a “can of worms” due to the potential of “hundreds of issues” that can affect poor database performance. Performance is-sues can occur at both the macro and micro level with severity levels from low to extreme. The DBA may need collaboration with other IT and Developer teams to vet out other causes outside the database.
Some performance related topics the DBA may run into include: application design and coding, database procedures, database physical architecture/configuration, database design, database lock and latch contention, database server and hardware/CPU scalability, memory/buffer cache challenges, running on legacy versions, database parameters, database internal objects such as indexes, optimizer challenges, resource governance, data replication, heavy user loads and on and on. In addition, the DBA should be proactive with implementing new applicable tuning features based on major database version upgrades. New version features can provide significant performance improvements and possibly resolve existing performance issues completely.
Poor performing databases at worst can cause financial business loss. For example, if mobile and web app customers cannot place timely orders or normal business processes are delayed, then the customers may not return. Database services should include database performance experts to properly manage the numerous database options, configurations and decisions that are critical for sustained and optimal performance. In addition, a best-in-class proactive database monitoring tool, such as Dell Foglight®, is absolutely necessary for the DBA to respond immediately to performance related issues. Implementing a proactive monitoring solution utilizing a real-time monitoring tool that is able to diagnose current performance conditions along with the severity level should be one of the highest priorities for the DBA. Monitoring can also help with the testing of performance related modifications. It is many times difficult to simulate production load tests after making adjustments depending on the level of performance adjustments made. Therefore, it is challenging at times to see what improvements might be made until modifications are implemented. Taking a baseline of production performance analytics before adjustments are made, and doing historical analysis (via monitoring tools) for a period of time after the adjustments have been implemented, can provide empirical evidence of possible improvements.
6. Database Architecture Optimized
Today’s databases are comprised with many types of physical and structural components. Database architecture itself is a subsystem of organizational enterprise architecture and technology strategies. It fits into the organization’s infrastructure topology. The DBA must manage data-base architectural and design components in collaboration with other IT initiatives as well.
Proper database architecture affects database performance, backup and recovery, redundancy, high availability, data growth and other areas. Internal architecture components include storage for transaction data, index data, data dictionary, temporary data, rollback data, archive data, BLOB data and various other data types. External database architecture includes database files containing the internal data, virtual provisioning, servers, disk groups and tiered storage, SANs, backup hardware and other hardware components. The physical design and architecture of the database determines how successful the database is aligned with the necessities for maximum performance and sustainment. Also, the physical hardware is driven by business requirements for redundancy, high availability, capacity, performance, storage and additional purposes.
Within the last 15 years, database virtualization has become a best practice architectural component for DBAs and database services. Virtualizing databases provides clear advantages that include data protection, consolidation, workload management, high availability, backup and recovery, disaster recovery and other advantages. Additional virtualization business benefits include cost efficiencies, operational efficiencies and risk mitigation. However, virtualization for database purposes has its own unique challenges and optimized requirements such as adequate CPU, memory, storage, database utilization and updated hard-ware that must be vetted out and architected properly.
As you can see, database architecture is critical and should not be taken lightly by the DBA since poorly constructed architecture can lead to significant risks and problems. Databases have been known to have outages due to poor architecture and configuration.
7. Database Scalability Maximized
With today’s unprecedented data growth, increased database traffic, user base expansion and application advances there is a constant strain on database infrastructure capacity. In addition, new releases of data-base systems may also require upgraded hardware and additional capacity in order to take full advantage of new features and enhancements. DBAs and database services must monitor these capacity limits and be able to recommend and implement scalable solutions. Without proactive monitoring and planning, significant risks such as database performance problems, data storage constraints and even outages can easily occur due to scalability limits. Hardware infrastructure such as physical database servers, memory and storage require appropriate capacity alignment whether they are standalone, virtual or pools of clustered servers.
Being proactive with data growth and processing requirements involves always evaluating capacity levels and reviewing data growth trends via monitoring and historical analysis. This can be done through best-in-class tools such as Foglight®, which collects historical data trends, tracks incremental data, and provides estimates for capacity and scalability purposes. With analyzed data growth estimates and usage pat-terns, the DBA can establish the need for scalable hardware or virtual machines with cost efficient solutions. In addition, the DBA can work with other IT strategic staff to determine data retention and data archival policies based on the business requirements. DBA services can also assist application scalability and responsiveness by working with software development teams to reduce bottlenecks, data contention and retrieval.
Scalability is not limited to hardware infrastructure, however. Today, all enterprise RDBMS software solutions provide feature rich capacity improvements. Some features may include infrastructure upgrades while others do not. Some features reside outside of the core RDBMS package and are packaged as database options while others are not. In addition, middleware features can provide scalability solutions. For instance, middleware with embedded data grid software for accelerated access to frequently used data is an example of improving capacity and through-put.
8. Database Development Execution
Some organizations have designed and built internal applications which normally require back-end development DBA support for such things as stored procedures, functions, packages, triggers, design, object modifications, privilege modifications, application problem detection (related to database), ETL and other internal database needs. Another important support activity for the DBA in this area relates to the building and assisting of database reporting needs.
For those organizations that have internally built applications, the need for database development requirements is very high due to the constant upgrades and enhancements of these applications, along with fixing problematic issues that were not detected through testing and change control. Other organizations may only have commercial off-the-shelf (COTS) applications; however, the DBA may still need to try and isolate application database components that could be causing issues such as offline objects or performance problems with a stored procedure not running efficiently.
While the level of DBA development requirements may vary based on specific application and database environments, it is an important skill to have within the Database Essentials Catalog of Services.
9. Database Remediation Assurance
Database problems, incidents and issues can cause business operation disruption that can range from minor pain and interruption to extreme pain and interruption and various levels of discomfort in between. If you cannot trust your DBAs or DBA Services to remediate database is-sues in a timely manner, then database sustainment remains at high risk. Lack of remediation assurance could be due to incompetence, poor processes or procedures, a small pool of DBA resource availability, no training, poor communication, poor ticket tracking, non-enforceable SLA or any number of other reasons.
The Database Remediation Essential requires the DBA or database services to always be proactive and able to remediate database problems quickly based on the business hours which could be 24x7x365. DBA investigation and diagnosis should quickly detect root cause database is-sues. Database issues such as data corruption, down databases and poor performance require the highest prioritization. These incidents are considered either at a high or extreme severity level and the DBA must provide response and remediation efforts immediately.
A compelling and practical way proactive problem management is achieved is through the utilization of best-in-class tools such as Foglight® which monitor databases 24x7 real-time with little overhead. Database monitoring implemented with fine-tuned alerts, severity level error detection and a centralized dashboard will provide essential information quickly to the DBA. Based on problem severity level the DBA can proceed on remediation to meet Service Level Agreement (SLA) requirements of the company or client.
Superior DBA services must also have process controls and standards in place so that escalation occurs between a primary and secondary DBA in case backup is required to quickly mitigate and remediate the problematic conditions. Larger firms or DBA Service firms may have a Service Desk unit with procedures that provide work-flow based on incident levels along with ticket systems to log the incidents.
10. Database Maintenance & Deployment Execution
Enterprise databases will always require ongoing maintenance and up-dates along with rolling release upgrade and deployment requirements. Although database maintenance and deployment may not be as interesting or exciting as other DBA catalog coverages, it is nevertheless very important to the ongoing sustainment of production environments.
Patching is the most critical database maintenance function for DBAs. Keeping up with the latest database patch releases prevents all types of potential setbacks which could range from security problems to performance issues. However, the DBA will need to coordinate with application teams for testing before putting patch updates into production environments.
Other maintenance functions could include ongoing internal and external database cleanup, data migrations, ongoing data dictionary updates including statistics updates, nightly process maintenance, large file truncations, updating configuration settings, scans for data corruption and much more. Having credible and reliable maintenance plans will continue to be a key essential to the success of database continuity and optimal performance.
Deploying new database versions can range from the simple to the com-plex. Major version upgrades can require a massive collaboration effort between various IT and Application teams. Exhaustive planning, testing and quality assurance processes are normally a part of major database upgrade and deployment efforts. The DBA must be totally prepared for all types of database upgrades and deployments to ensure smooth and risk averse execution.
The expansion of DBA responsibilities and coverages continues to grow rapidly with enterprise database new features and options, overlapping middleware, unrelenting data growth, greater high availability require-ments, emerging web and mobile application requirements along with ever increasing complexities. Without a holistic database services ap-proach, which provides the framework and catalog of DBA coverage es-sentials, there is high risk and potential for DBA service gaps that could cause significant operational issues for data-driven organizations.
The Top 10 Database Essentials described in this white paper provides the full catalog and portfolio of high level areas of coverage required by DBAs to perform the full spectrum of DBA services. By categorizing these coverages, the DBA understands all of the functions necessary for complete and holistic management of the enterprise database environment.
About the Author
Steven Woody is a managing partner with Ross Group Inc and President of the CommitDBA® division of Ross Group, a specialized database management service provider (DMSP) with headquarters in Dayton, Ohio. Mr. Woody has both technology and business degrees and has been working in the database industry since the late 1980s.
CommitDBA Business Value
CommitDBA, a division of Ross Group Inc, has been providing remote database support to companies across the U.S. and Canada for over 15 years. Our remote database management services - DBAManage - is a cost effective solution for managing your enterprise databases while implementing best practices based on the “Top 10 Database Essentials” as discussed in this paper.
CommitDBA has partnered with Dell Software® to bring best-in-class database monitoring tool Foglight® to proactively monitor your data-bases 24x7x365. With Foglight® we are able to continually scan the database environment in real-time and be alerted quickly of any anomaly that may be degrading the databases.
For more information on how CommitDBA Services can help your organization, please call 800.871.0942 or email email@example.com.Download Now