|By Michael Kopp||
|October 27, 2012 10:00 AM EDT||
Traditional Enterprise Database vendors often bring up the lack of professional monitoring and management tool support for NoSQL solutions. Their argument is that enterprise applications require sophisticated tuning and monitoring of the database in order to ensure a performant and smooth operation. NoSQL Vendors, while arguing that this lack is not enough to favor RDBMS over their respective solutions, do agree. Several vendors try to differentiate themselves by providing enterprise level monitoring and management software, for example, Cassandra, MongoDB, HBase or others. Both are of course correct that monitoring and management of especially the performance aspect is important, but at the same time they are making the same mistake that RDBMS vendors have made for the last decade: they ignore the application.
Application Performance Management for Databases
What matters in the end is not the database performance itself, but the performance of the application that uses the database. We have explained the different problem patterns numerous times in this blog (...) so I won't go into them. All of them however have one thing in common: the application logic drives how the database is used and there is only so much that you can tune on the database to cover for mistakes on the application side. So we need to monitor and optimize that usage pattern itself. Application Logic again is driven by input data or in most cases end user interaction, thus we need to understand how end user behavior and end user actions drive the database usage. On the other hand we need to understand the impact of the database on these actions. What's important to understand is that the database can work and perform to the highest standards and still be the main bottleneck as far as the application is concerned - if it is wrongly used or has a bad access pattern. RDBMS and NoSQL Databases have that in common. Therefore the fundamental way I, as a performance engineer, do application performance analysis and management does not change:
First we need to understand if a particular business transaction slows down, has a general performance problem and if this has an impact on the end user:
Do any of the business transactions violate a baseline or have negative end user experience
Next we would isolate the high level cause of the slow down or performance issue. There are many ways of doing this, but it's always some kind of fault domain isolation:
Transaction Flow shows that we spend ~15 % waiting for the Database
This Transaction Flow shows that the Business Backend is calling a Cassandra Database Cluster
This tells us if we spend time waiting on the database. We see that there is not much difference between a regular database and something like an Apache Cassandra!
What's important here is that if the database shows up as the main contributor, this does not mean that the database itself is at fault, it might just be the applications usage of it. Thus I would need to check the usage and access pattern next:
This shows the select statements executed within a particular transaction type
This shows all Cassandra database statements against all participating Cassandra servers executed in a particular transaction
Here I might see that the reason for bad performance is that we execute too many statements per transaction, or that we read too much data. If that is the case I would need to check the application logic itself and potentially need a developer to fix it. The developer would of course want to understand where, why and in which particular transaction the statements are executed.
This shows a single Transaction (PurePath) and the Cassandra Statements executed within it
If however a particular statement is slow we, might very well have a database issue and I would talk to the DBA. The only difference in case of a NoSQL solution in this process is that you often have a database cluster, so I would want to understand if the problem is isolated to a particular node or not. And the DBA will want to understand if my access pattern leads to a good distribution across that cluster or if I am hammering away at a subset of them.
This hotspot view shows that Cassandra Server Node3 consumes much more Wait and I/O time than the others
The nice thing is that all in all my analysis does not differ between JDBC, ADO, Cassandra or one of the many NoSQL solutions.
APM Solution Support
There is of course a caveat here; it requires some level of support of the APM solution of choice. Sometimes it might be enough to see API calls on the NoSQL client in my response time breakdown. More often than not of course a little bit more context is desired, like which ColumnFamily is accessed and how many rows are read or which Database Node in the Cluster is serving a read. For this and the afore mentioned reasons I argue that APM solution support of your chosen Database or NoSQL solution is as important as the monitoring of the database itself.
I have spent considerable time tuning SQL statements and indexes, but in the end the best optimizations have always been those on the application and how the application uses the database. SQL Tuning almost always adds complexity and often is a workaround over bad application or data structure design. In the NoSQL world "SQL statement" tuning for the most part is a task of the past, but Data Structure Design has retained its importance! At the same time logic that traditionally resided in the database is now in the application layer, making application design even more important than before. So while some things have shifted, from an Application Performance Engineering Perspective I have to say: nothing really changed, it's still about the application. Now more than ever!
SYS-CON Events announced today that ActiveState, the leading independent Cloud Foundry and Docker-based PaaS provider, has been named “Silver Sponsor” of SYS-CON's DevOps Summit New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. ActiveState believes that enterprises gain a competitive advantage when they are able to quickly create, deploy and efficiently manage software solutions that immediately create business value, but they face many challenges that prevent them from doing so. The Company is uniquely positioned to help address these challenges thro...
Apr. 27, 2015 06:15 AM EDT Reads: 3,182
BroadSoft on Tuesday announced that it is a recipient of the 2014 Frost & Sullivan Market Leadership Award in the Hosted/Cloud Internet Protocol (IP) Telephony market for Latin America. According to Frost & Sullivan market research, the Latin America (LATAM) hosted/cloud Internet Protocol (IP) telephony market, including integrated unified communications and collaboration (UC&C) applications, is currently experiencing a rapid growth trajectory and is expected to exhibit a tenfold rise in annual revenues in the 2013-2020 period. With more than 600 cloud deployments internationally, BroadSoft w...
Apr. 27, 2015 06:00 AM EDT Reads: 2,405
SYS-CON Events announced today that Vitria Technology, Inc. will exhibit at SYS-CON’s @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Vitria will showcase the company’s new IoT Analytics Platform through live demonstrations at booth #330. Vitria’s IoT Analytics Platform, fully integrated and powered by an operational intelligence engine, enables customers to rapidly build and operationalize advanced analytics to deliver timely business outcomes for use cases across the industrial, enterprise, and consumer segments.
Apr. 27, 2015 06:00 AM EDT Reads: 2,660
SYS-CON Events announced today that Alert Logic, the leading provider of Security-as-a-Service solutions for the cloud, has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo® and DevOps Summit 2015 New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo® and DevOps Summit 2015 Silicon Valley, which will take place November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
Apr. 27, 2015 05:45 AM EDT Reads: 2,070
SYS-CON Events announced today that Akana, formerly SOA Software, has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo® New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. Akana’s comprehensive suite of API Management, API Security, Integrated SOA Governance, and Cloud Integration solutions helps businesses accelerate digital transformation by securely extending their reach across multiple channels – mobile, cloud and Internet of Things. Akana enables enterprises to share data as APIs, connect and integrate applications, drive part...
Apr. 27, 2015 05:00 AM EDT Reads: 2,050
SYS-CON Events announced today that CommVault has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. A singular vision – a belief in a better way to address current and future data management needs – guides CommVault in the development of Singular Information Management® solutions for high-performance data protection, universal availability and sim...
Apr. 27, 2015 04:45 AM EDT Reads: 2,068
SYS-CON Events announced today that SafeLogic has been named “Bag Sponsor” of SYS-CON's 16th International Cloud Expo® New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. SafeLogic provides security products for applications in mobile and server/appliance environments. SafeLogic’s flagship product CryptoComply is a FIPS 140-2 validated cryptographic engine designed to secure data on servers, workstations, appliances, mobile devices, and in the Cloud.
Apr. 27, 2015 04:30 AM EDT Reads: 2,125
SYS-CON Events announced today that StorPool Storage will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. StorPool is distributed storage software that allows service providers, enterprises and other cloud builders to run data storage on standard x86 servers, instead of using expensive and inefficient storage arrays (SAN).
Apr. 27, 2015 04:00 AM EDT Reads: 2,249
GENBAND has announced that SageNet is leveraging the Nuvia platform to deliver Unified Communications as a Service (UCaaS) to its large base of retail and enterprise customers. Nuvia’s cloud-based solution provides SageNet’s customers with a full suite of business communications and collaboration tools. Two large national SageNet retail customers have recently signed up to deploy the Nuvia platform and the company will continue to sell the service to new and existing customers. Nuvia’s capabilities include HD voice, video, multimedia messaging, mobility, conferencing, Web collaboration, deskt...
Apr. 27, 2015 03:45 AM EDT Reads: 1,923
Sonus Networks introduced the Sonus WebRTC Services Solution, a virtualized Web Real-Time Communications (WebRTC) offer, purpose-built for the Cloud. The WebRTC Services Solution provides signaling from WebRTC-to-WebRTC applications and interworking from WebRTC-to-Session Initiation Protocol (SIP), delivering advanced real-time communications capabilities on mobile applications and on websites, which are accessible via a browser.
Apr. 27, 2015 02:30 AM EDT Reads: 2,004
SYS-CON Events announced today that Site24x7, the cloud infrastructure monitoring service, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Site24x7 is a cloud infrastructure monitoring service that helps monitor the uptime and performance of websites, online applications, servers, mobile websites and custom APIs. The monitoring is done from 50+ locations across the world and from various wireless carriers, thus providing a global perspective of the end-user experience. Site24x7 supports monitoring H...
Apr. 27, 2015 02:30 AM EDT Reads: 2,091
SYS-CON Events announced today that Intelligent Systems Services will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Established in 1994, Intelligent Systems Services Inc. is located near Washington, DC, with representatives and partners nationwide. ISS’s well-established track record is based on the continuous pursuit of excellence in designing, implementing and supporting nationwide clients’ mission-critical systems. ISS has completed many successful projects in Healthcare, Commercial, Manufacturing, ...
Apr. 27, 2015 02:15 AM EDT Reads: 2,819
SYS-CON Events announced today that B2Cloud, a provider of enterprise resource planning software, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. B2cloud develops the software you need. They have the ideal tools to help you work with your clients. B2Cloud’s main solutions include AGIS – ERP, CLOHC, AGIS – Invoice, and IZUM
Apr. 27, 2015 02:00 AM EDT Reads: 3,587
SYS-CON Events announced today that Tufin, the market-leading provider of Security Policy Orchestration Solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. As the market leader of Security Policy Orchestration, Tufin automates and accelerates network configuration changes while maintaining security and compliance. Tufin's award-winning Orchestration Suite™ gives IT organizations the power and agility to enforce security policy across complex, multi-vendor enterprise networks. With more than 1...
Apr. 27, 2015 01:45 AM EDT Reads: 3,625
SYS-CON Events announced today that Cloudian, Inc., the leading provider of hybrid cloud storage solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Cloudian, Inc., is a Foster City, California - based software company specializing in cloud storage software. The main product is Cloudian, an Amazon S3-compliant cloud object storage platform, the bedrock of cloud computing systems, that enables cloud service providers and enterprises to build reliable, affordable and scalable cloud storage solu...
Apr. 27, 2015 01:00 AM EDT Reads: 2,756
SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the industry’s first all flash version of HyperConverged Appliances that include both compute and storag...
Apr. 27, 2015 12:45 AM EDT Reads: 4,665
Temasys has announced senior management additions to its team. Joining are David Holloway as Vice President of Commercial and Nadine Yap as Vice President of Product. Over the past 12 months Temasys has doubled in size as it adds new customers and expands the development of its Skylink platform. Skylink leads the charge to move WebRTC, traditionally seen as a desktop, browser based technology, to become a ubiquitous web communications technology on web and mobile, as well as Internet of Things compatible devices.
Apr. 27, 2015 12:45 AM EDT Reads: 2,334
SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada. Our partner network encompasses some 300 of the world's leading systems integrators and security s...
Apr. 27, 2015 12:00 AM EDT Reads: 5,277
“With easy-to-use SDKs for Atmel’s platforms, IoT developers can now reap the benefits of realtime communication, and bypass the security pitfalls and configuration complexities that put IoT deployments at risk,” said Todd Greene, founder & CEO of PubNub. PubNub will team with Atmel at CES 2015 to launch full SDK support for Atmel’s MCU, MPU, and Wireless SoC platforms. Atmel developers now have access to PubNub’s secure Publish/Subscribe messaging with guaranteed ¼ second latencies across PubNub’s 14 global points-of-presence. PubNub delivers secure communication through firewalls, proxy ser...
Apr. 27, 2015 12:00 AM EDT Reads: 3,883
SYS-CON Events announced today that On the Avenue Marketing Group, a sales and marketing firm that utilizes events to market and sell products to consumers, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. On the Avenue Marketing Group (OTA) is a sales and marketing firm that utilizes events to market and sell products to consumers. On behalf of our clients, we attend thousands of fairs, festivals, expos, concerts, conferences, and sporting events annually, helping them reach millions of individuals ...
Apr. 26, 2015 07:30 PM EDT Reads: 3,507