Welcome!

Industrial IoT Authors: William Schmarzo, Elizabeth White, Stackify Blog, Yeshim Deniz, SmartBear Blog

Blog Feed Post

VMware or Microsoft?–Agentless Backup for Virtual Environments

Today’s post comes to us courtesy of an old friend and coworker, who is still a friend but who now works for Veeam.  I’m talking about none-other than Chris Henley

Thank you, Chris, for this excellent write-up on backing up virtual environments!

---

One of the most important things to remember when talking about backup best practices in virtual environments is that virtual environments are not physical environments. I know that sounds really silly but that it is really quite important because physical environments have a different architecture than virtual environments. When we talk about running one operating system and one underlying hardware set, it’s important to understand that one hardware, one disk, one operating system, relationship demands a specific architectural design for the software that would be used to backup that physical architecture. In that physical environment the software designers used an architecture that focused heavily on the use of agents to provide the interactions between backup software and the physical hardware they were trying to back up. This agent based approach was incredibly successful for a very long time. Decades! The agent based approach is still successful in physical environments today, and probably represents the best possible backup solution for the physical environment. The problem is virtual environments are not physical environments, and the world of IT is headed for the virtual environment. Virtual environments differ from physical environments because the hypervisor, whether that’s VMware or Hyper-V, provides a layer of abstraction between the underlying hardware and overlying operating systems that will actually run above the hypervisor in the virtual architecture. The important consequence that goes right along with this architectural change in the virtual world means that if you try and use the agent based approach of the past in conjunction with a virtual environment it just doesn’t work. Now the reason it doesn’t work is not that you couldn’t force the old agent based model into the virtual environment where you added an agent to every virtual machine and then monitored, managed, administrated, and maintained those agents. The challenge here is that the virtual environment would demand a dramatic additional measure of work in order to get the backup operations to work properly, and frankly it is not necessary. VMware and Microsoft, the two major players in the hypervisors space, with ESX and Microsoft Hyper-V respectively, have each made a recommendation that we do not use agents in the virtual machines! Instead the recommendation is that you use an open set of APIs and connect to those APIs using standards that will allow us to interact with a virtual machine. This technique allows the software to interact with the underlying host for that Hyper-V or ESX VM. The host provides the tracking mechanism for us to do data protection or data protection mechanisms. Agentless data protection is a big deal.

The Agentless Backup Approach.

When we think about Hyper-V we want to make certain that we take an agentless approach to backup, replication, restoration, monitoring, and management so that we maximize the capabilities that have been built into the hypervisor by Microsoft as well as minimizing the impact in resources that data protection will have on the actual virtual machines themselves. The Microsoft VSS process allows for the imaging of virtual machines in their entirety along with the associated binary, configuration, xml, snapshot, settings, and any other associated virtual machine files which would allow you to make a very complete copy of a virtual machine and its data for backup or other data protection uses. The cool thing is that this is all without the use of any installed agent inside the virtual machine. Of course all of this relies on the fact that you are using the standards based approach, where you have built a set of tools that work directly in conjunction with VSS, and with the way that Hyper-V is built.

When we think about Agentless Systems we don’t necessarily mean that we will not use any agents anywhere in the architecture. Instead what we’re talking about is the fact that the agents will not be installed in the virtual machines. In most cases the actual software that is going to provide data protection to a virtual environment running Hyper-V will have some kind of interactive component that is actually installed or configured on the Hyper-V host. These “agents” and I use the term loosely run in conjunction with the windows operating system that is actually supporting that Hyper-V host. Generally these “agents” come in the form of drivers and or services. They are really not agents in the traditional sense. The key here is that when we make the installation of components that those installed components are not going to the virtual machines, meaning there is no additional overhead to the running virtual machine, or to its application based workflow, or services, and you are not providing any additional requirement for the usage of administrative time and resources necessary to update and manage those agents.

The VSS process

Microsoft has this really cool process called the volume shadow copy service and it is the base for agentless backup of VM’s in Hyper-V. The Volume Shadow Copy service is not new, in fact, it has been around since 2003. Microsoft introduced the volume shadow copy service with Windows Server 2003 and initially it was designed to provide just what its title suggests, shadow copies or previous version copies of existing documents inside the Windows Server operating system. Today we rely on that same functionality and in fact the same VSS.exe service that was used for volume shadow copies to make image copies of virtual machines in Hyper-V. It’s important that you have a brief understanding of the volume shadow copy service so let’s talk about it now.

The volume shadow copy service is made up of three essential components first the Vss.exe service, second the VSS Requestor, and finally the VSS writer.

The VSS.exe service is responsible for taking requests from a VSS Requestor and fulfilling those requests. In this case the requests will be associated with virtual machines and image copies of those VM’s. VSS is installed with each version of Windows Server.

The VSS Requestor will formulate requests to the VSS service for a specific image to be created of a specific virtual machine. The VSS Requestor is not written by Microsoft; instead it’s a piece of software that is written by a third party in order to formulate a request that would then be passed to the VSS Service. You can make your own VSS Requestor with a little help from Microsoft who provides code samples and guidance for those interested in writing a VSS Requestor.

The VSS writer is responsible for taking the image copy of the data that is requested. The VSS writer does the actual writing of that data to disk. Depending on exactly what is requested there are a number of different VSS writers that might be used. For example if you wanted to make an image of a virtual machine running on Hyper-V the volume shadow copy service would use the Hyper-V VSS writer in order to write the image of the virtual machine that was requested by the requester.

image

For more information on the VSS process please see the following link to Technet.Microsoft.com. http://technet.microsoft.com/en-us/library/cc785914(v=WS.10).aspx

Fast recovery

Agentless backup is cool, VSS process is cool, and new ways to implement the 3-2-1 rule are cool, none of this really makes any difference if we can’t get that data back quickly. The defining point in any disaster recovery plan is the ability to recover the data. When we think about recovering data, not only is it important that we understand where the data is located, it’s also important that we know and can clearly work with the format in which the data is stored, and be able to extend the new capabilities to enable advanced data recovery options at a moment’s notice. Virtual machines are built to run application workloads and those application workloads support lots of individual users. A virtual machine running Microsoft Exchange is providing e-mail services to the users in an organization. Those users do not want downtime of the virtual machine that supports their email. In the event of data loss (small or large scale) as administrators we need to find a way to recover e-mail items direct from the backup into the running virtual machine that is supporting the Microsoft Exchange email application. The data protection market has changed dramatically over the past two years with companies focusing more and more on application specific tools and less and less on the legacy methods of data restoration.

With innovative tools like Veeam’s Explorer for Exchange an organization might receive a request from a user who needs to recover an erroneously deleted email message with an associated attachment. The tool allows for the mounting of the Exchange.edb database from within the backup file. Once mounted the helpdesk professional can then search for the desired email, or simply select the user’s mailbox and browse to the email. At this point the email can be restored to the running Exchange VM, emailed directly back to the user, saved as an .msg file, or a .pst file. All of this is done in seconds while the user is on the phone, and while the Exchange server is still running and providing the desired services to the rest of the network.

This new paradigm of agentless data protection at the application level is changing the way we think about data protection and disaster recovery in virtual environments. Best of all its free!

Get the Veeam Backup Free Edition tools at http://www.veeam.com

Read the original blog entry...

More Stories By Kevin Remde

Kevin is an engaging and highly sought-after speaker and webcaster who has landed several times on Microsoft's top 10 webcast list, and has delivered many top-scoring TechNet events and webcasts. In his past outside of Microsoft, Kevin has held positions such as software engineer, information systems professional, and information systems manager. He loves sharing helpful new solutions and technologies with his IT professional peers.

A prolific blogger, Kevin shares his thoughts, ideas and tips on his “Full of I.T.” blog (http://aka.ms/FullOfIT). He also contributes to and moderates the TechNet Forum IT Manager discussion (http://aka.ms/ITManager), and presents live TechNet Events throughout the central U.S. (http://www.technetevents.com). When he's not busy learning or blogging about new technologies, Kevin enjoys digital photography and videography, and sings in a band. (Q: Midlife crisis? A: More cowbell!) He continues to challenge his TechNet Event audiences to sing Karaoke with him.

IoT & Smart Cities Stories
Bill Schmarzo, Tech Chair of "Big Data | Analytics" of upcoming CloudEXPO | DXWorldEXPO New York (November 12-13, 2018, New York City) today announced the outline and schedule of the track. "The track has been designed in experience/degree order," said Schmarzo. "So, that folks who attend the entire track can leave the conference with some of the skills necessary to get their work done when they get back to their offices. It actually ties back to some work that I'm doing at the University of San...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in development and launches of disruptive technologies to create new market opportunities as well as enhance enterprise product portfolios with emerging technologies. His most recent venture was Octoblu, a cross-protocol Internet of Things (IoT) mesh network platform, acquired by Citrix. Prior to co-founding Octoblu, Chris was founder of Nodester, an open-source Node.JS PaaS which was acquired by AppFog and ...
The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...