Welcome!

Industrial IoT Authors: Yeshim Deniz, Liz McMillan, William Schmarzo, Allwyn Sequeira, Pat Romanski

Related Topics: Machine Learning , Java IoT, Industrial IoT, Microservices Expo, PowerBuilder, Containers Expo Blog

Machine Learning : Blog Post

Where Is My Disk Space?

An HTML 5 File System Visualizer

by Nick Mueller, Zetta.net

Hello new users! The file system visualizer can be found at wheresmydiskspace.com - continue reading to learn more about the development of the tool and the visualization options.

Before buying more storage space it's a good idea to make sure your existing space isn't filled with redundant or old data - or hundreds of downloaded cat videos.

Disk capacity is increasing and while prices continue to drop, those savings are offset by demands for new capacity to store more and larger files. Not only does this mean more primary disk space, but 2x that amount for backups.

Zetta co-founder Lou Montulli may have the answer to this problem. Recently Lou combined his experience with browsers and storage in creating an open-source tool - a File System Visualizer (www.wheresmydiskspace.com) - for analyzing storage usage.

Lou was a founding engineer at Netscape in 1994 when he helped create the first commercial web browser Netscape Navigator. Over the years he's been responsible for the development of many browser related innovations, and co-founded Zetta.net in 2008 - where he continues to serve as VP of Engineering and Chief Scientist.

"The tool was conceived as a method for visualizing multiple aspects of any large file set: an existing file system, a backup or an archive," he says. "This can be a great tool to use if you find yourself running low on disk space and need to find files to delete to free up space."

The tool makes heavy use of the Data-Driven Documents JavaScript library together with jQuery, Dojo, PrettyPhoto, JavaScript and Scalable Vector Graphics. Sponsored by Zetta, all the source code for the File System Visualizer is available and a BSD license allows anyone to use it, commercially and non-commercially, free of charge.

"Part of the challenge and opportunity of this tool was writing it in JavaScript and using HTML as the user interface," Lou says. "I was part of the team who wrote the very first web browsers, so I was personally motivated to design a tool that takes advantage of some of the great new technologies coming out of HTML5, Mozilla.org and the broader web community."

Getting Started
The File System Visualizer is free to use and doesn't require installing any software. You just need a web browser that supports SVG and has a fast JavaScript engine. Go to www.wheresmydiskspace.com. The home page has a few video demonstration of the product that you can view before running the software.

Or you can:

  1. Click the link at the top of the page to take you directly to the visualizer.
  2. There you have three options: you can look at some sample data sets, use a Java applet to collect the data from your local machine and create a manifest file detailing what is in the file system, or you can load a manifest file created in a previous scan.
  3. If you choose to do a new scan, and there are a large number of folders, the software will prompt you to save the manifest to your disk rather than keeping it in the browser.

Test Setup
We recently had the File System Visualizer tested on a Windows 7 desktop with a third generation Intel Core i7 processor and 16 GB RAM. The scan took approximately 5 minutes. When completed, a message came up that there were 52,993 folders.

The software can analyze a local disk, or an administrator can run it remotely on any mountable drive. At this point it runs on Windows (32-bit and 64-bit) and OSX.

Visualizing Your Data
After running the scan, the software then presents seven different views of the data. The views are illustrated at the top of the page and you can click on any of the images to access that view of the data.

Summary Page - This showed that the test computer had 353.1 GB of data in 52,993 folders containing 364,931 items, with an average file size of 967.7 KB.

Visual Tree - This gives a hierarchical tree visualization of the data. On the left is a pull-down box where you can select to view the data by size, by type or by date. There is also a slider where you can select the tree display depth from one to seven levels.

Screenshot of the Tree View

Viewing by size shows a hierarchical view of the file system and the amount of data in each folder with up to seven levels of depth. To look at just the contents of a single folder, rather than the entire file system at once, just click on the dot next to that folder.

Viewing by type at the first level divided the data into known types and uncategorized. Going to the second depth level divided the uncategorized by their file extension and the categorized into groups such as disk images, games, database, software development, fonts, plugins, office types, settings, executables, media, backup and system. For most of those categories, going to the next level would give the file extensions, but some categories (media, office types and encodings) would further subdivide before getting to their final level.

Viewing by date, the first level divides the data into "1 year and older" and "within 1 year" and shows the GB of data in each category. Taking it to the second level splits the "within 1 year" branch into five levels and the "1 year and older" into each of the years for which you have data. There is no third level available.

Hierarchical List - This view presents the data in list rather than tree format. To get to deeper levels, click the + sign next to any of the categories. In addition to the file names, there are columns for Size in Directory, Total Size and % with children. When you click on the headers for the columns, up and down arrows appear, making it look like the data is sortable by those columns, but it isn't.

Flattened List - This is a sortable, non-hierarchical list of the folders. When viewing by Size, in addition to File Name, there are seven other sortable columns of data in each folder, including Size and Number of Items. The Type and Date views are similarly sortable. In none of these views can you look at a subtree, only at the entire file system. To view a subtree, go to one of the other views and narrow it down to the subtree and view type you want, and then click on the Flattened List visualization.

Your hard drive in "sun burst" view.

Sunburst - A type of pie chart, with rings showing each of the levels of depth. The chart can display each slice as an even size, or can adjust the sizes by the file count or amount of data in the slice. Clicking on any of the slices will move that folder or data point into the center circle, with the rings showing the subfolders or subcategories of that particular subdirectory.

Tree Map - A box type view of the data. As with the Sunburst, the boxes can be sized equally, or sized by data size or number of files. Clicking on any of the boxes will show the details within that subdirectory or data type.

Bubble Chart - This gives two layout options for showing the data: Bubble Chart or Circle Pack. The Bubble Chart shows bubbles for all the items in that category sized by the amount of data in that folder or file type. The Circle Pack presents a hierarchical view of the bubbles. In either view, clicking on a bubble or circle will give the bubbles showing the subcategories of that item.

Conclusion
The File System Visualizer is a quick and easy way to gain understanding of what's on your file system. It's intuitive to use and within minutes, you can start locating what is taking up disk space. Then you can delete or archive anything that is no longer needed, or establish policies to prevent wasted space. Then, if additional storage space is still needed, you can give management a clear visual presentation of how storage is being used in your environment. You can start visualizing your hard drive right now.

Nick is Zetta's Corporate Reporter, and has been writing and telling stories about technology with blogs, social media, and content marketing since the days when the BBS reigned.

More Stories By Derek Kol

Derek Kol is a technology specialist focused on SMB and enterprise IT innovations.

@ThingsExpo Stories
As businesses adopt functionalities in cloud computing, it’s imperative that IT operations consistently ensure cloud systems work correctly – all of the time, and to their best capabilities. In his session at @BigDataExpo, Bernd Harzog, CEO and founder of OpsDataStore, will present an industry answer to the common question, “Are you running IT operations as efficiently and as cost effectively as you need to?” He will expound on the industry issues he frequently came up against as an analyst, and...
Microservices are a very exciting architectural approach that many organizations are looking to as a way to accelerate innovation. Microservices promise to allow teams to move away from monolithic "ball of mud" systems, but the reality is that, in the vast majority of organizations, different projects and technologies will continue to be developed at different speeds. How to handle the dependencies between these disparate systems with different iteration cycles? Consider the "canoncial problem" ...
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor - all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
My team embarked on building a data lake for our sales and marketing data to better understand customer journeys. This required building a hybrid data pipeline to connect our cloud CRM with the new Hadoop Data Lake. One challenge is that IT was not in a position to provide support until we proved value and marketing did not have the experience, so we embarked on the journey ourselves within the product marketing team for our line of business within Progress. In his session at @BigDataExpo, Sum...
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex softw...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, will discuss the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information,
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Apache Hadoop is emerging as a distributed platform for handling large and fast incoming streams of data. Predictive maintenance, supply chain optimization, and Internet-of-Things analysis are examples where Hadoop provides the scalable storage, processing, and analytics platform to gain meaningful insights from granular data that is typically only valuable from a large-scale, aggregate view. One architecture useful for capturing and analyzing streaming data is the Lambda Architecture, represent...
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
With billions of sensors deployed worldwide, the amount of machine-generated data will soon exceed what our networks can handle. But consumers and businesses will expect seamless experiences and real-time responsiveness. What does this mean for IoT devices and the infrastructure that supports them? More of the data will need to be handled at - or closer to - the devices themselves.
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), will provide an overview of various initiatives to certifiy the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldw...
SYS-CON Events announced today that Loom Systems will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2015, Loom Systems delivers an advanced AI solution to predict and prevent problems in the digital business. Loom stands alone in the industry as an AI analysis platform requiring no prior math knowledge from operators, leveraging the existing staff to succeed in the digital era. With offices in S...
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 20th Cloud Expo, which will take place on June 6-8, 2017 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 add...
SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on ...
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...