Commercial Tool to get Web Response Time and availability

Discussions

Performance and scalability: Commercial Tool to get Web Response Time and availability

  1. Hi All,
       We are looking for a commercial monitoring tool that let us to get the web app. response time, availability, http code, etc ... our web app are integrated with Microsoft security (NTLM...) so that tool must support microsoft security .

    by.
  2. Over here we just did a quick scan of the market for the same purpose.
    We are basically a MS shop as far as out Inter/intranet apps is concerned, we also run an IBM mainframe as well.
    Here's list of vendors we identified:

    Vendor: Compuware
    Tools
    Vantage Suite (including Application Vantage, Application Expert, ClientVantage, and ServerVantage) provides diagnostic data to determine the cause of performance problems before users or critical business processes are affected. These tools track response times, resource usage, application faults, and availability using either passive or active monitoring of applications, servers, and databases. A powerful scripting language enables the identification of any type of business application: web, client/server, or mainframe. Features include web-based reporting and console tools, automatic problem notification, and automated corrective action without operator intervention. Problem areas can be diagnosed using the Thread Analysis and Bounce Diagram, which identifies the timing and sequence of each transaction’s component traversing the network, as well as relationships between threads.
     
    Strobe (and iStrobe) measures and reports the activity and performance of OS/390-based online and batch applications, providing details about where and how time is being spent during an application’s execution.
    Application Level Monitoring
    This vendor claims end to end monitoring and explicitly mentions business applications, most notably singling out .NET applications.
    Technology Compatibility
    The following technologies are explicitly mentioned as being supported: .NET, DCOM, MS SQL Server, DB2, Win NT, Win 2000, OS/390, PeopleSoft, MS Exchange, IIS, and Active Directory.
     
    The Company
    This company has been in business since 1973, and has offices around the world to serve approximately 25,000 of the world’s largest and most respected businesses. Gartner rates this company “Caution” overall, which means it faces challenges in one or more areas. However, Gartner rates it “Promising” for Product/Service and “Positive” for both Technology and Support. The “Caution” rating stems from the company’s dependence on mainframe revenue.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Hewlett-Packard
    Tools
    OpenView can, among other things, provide insight into the structure, availability, and performance of applications through Smart Plug-ins, which have application-specific intelligence regarding expected behavior and structure, and make appropriate notifications when application processes, log files, and other mechanisms suggest a problem with performance or availability. It offers application management for nearly every major business application on the market, and can be easily extended to any off-the-shelf or in-house application. It allows operators to configure custom actions on alarms such as paging or sending electronic mail, and can filter by severity, type, or system node, and can provide interval-driven alarm status updates. Data is also stored, and may be analyzed for past trends or used to forecast future behaviour.
     
    Application Level Monitoring
    This product will monitor some third party commercial applications out-of-the-box, and claims to be highly configurable to monitor virtually any application.
     
    Technology Compatibility
    OpenView can run on several platforms, including MS Windows. Out-of-the-box plug-ins are available for .NET, Active Directory, Exchange Server, SQL Server, and PeopleSoft.
     
    The Company
    Hewlett-Packard was founded in 1939 and has grown to be one of the world’s largest suppliers of technology products and solutions. Gartner rates this company “Caution” overall, which means it faces challenges in one or more areas, though it is such a large and diverse company that this overall rating may not be meaningful. Gartner rates HP “Positive” for the OpenView software and for Customer Service/Support.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: InfoVista
    Tools
    InfoVista Server (in conjunction with other products in the Vista suite) monitors performance and availability of servers and services via both polling and importing log files. It claims sophisticated real-time reporting to quickly pinpoint problems, with dynamic modeling and trending to predict service degradations. It includes an API to facilitate custom integration so it can collect data from any source in a multi-vendor, multi-technology environment. It has the scalability to poll data from a virtually unlimited number of devices. Complex monitoring algorithms are computed, and notifications are distributed in real time. Historic data is stored with customizable aging and summary intervals for flexible long and short term trend analysis.
     
    Application Level Monitoring
    This product appears to have a very open architecture, but few specifics are given as to what it actually monitors. The focus is on its flexibility and adaptability, as is shown by this quote, “InfoVista can be adapted dynamically to collect and analyze the performance of any IT infrastructure environment, as well as any service or application running on that infrastructure.” It looks like this tool can monitor pretty much anything, but what it can do out-of-the-box is unclear.
     
    Technology Compatibility
    InfoVista claims that it “polls performance metrics from virtually any source”, but no details are given.
     
    The Company
    Founded in 1995, this vendor claims that eighty percent of the world’s largest service providers as ranked by Fortune, as well as leading Global 2000 enterprises, have selected InfoVista.
     
    This vendor offers 24x7 customer support, as well as training and consulting.
     
    Vendor: Argent Software
    Tools
    Argent Guardian provides universal device coverage on wide area and local area networks using SNMP, while monitoring specific Windows events and statistics using APIs, with or without installing server agents. It proactively manages the enterprise, identifying and correcting problems before users or administrators even know they exist, and providing alerts through alphanumeric paging, email, or SMS. The monitoring console can be viewed in a Windows GUI or through a browser, and can be configured for viewing alerts from the entire enterprise, or from any configurable subset.
     
    Argent Defender offers stress testing, capacity planning and performance monitoring for Intranet and web applications. It can identify reliability and load-bearing issues prior to full deployment. It monitors production applications and generates alerts when performance slows below pre-defined thresholds.
     
    Argent Exchange Monitor is a robust, scalable solution that provides every conceivable level of monitoring, alerting, and detailed reporting for complex Exchange 2000 (or 5.5) environments.
     
    Argent Console consolidates all alerts from Argent tools as well as other monitoring products. All alerts and alarms from in-house products and any number of third party packages can be sent directly to the Console, and consolidated using Argent's powerful ActiveX and XML interfaces.
     
    Application Level Monitoring
    Guardian appears to be primarily network and system monitoring. While Defender will monitor Intranet and web applications, it is all done from the user perspective by sending sample test transactions with no back-end monitoring, and its emphasis is clearly on pre-release testing. The Console looks promising, as it appears to be configurable to monitor virtually any application at any level, though it’s not clear how much useful monitoring could be done without coding specific alerts into WCB’s inhouse applications.
     
    Technology Compatibility
    The following technologies are explicitly mentioned as being supported: Windows 2000, MS Exchange, ActiveX, Visual Basic, and XML.
     
    The Company
    Argent is a privately held company founded in 1991, with 75 employees in 11 offices worldwide. They have more than 1000 customers.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Visual Networks
    Tools
    Visual eWatcher monitors and tests web applications to ensure that transactions are processed and information is delivered. It can test against predetermined thresholds to ensure acceptable application performance, and in the case of a problem, can generate a variety of alerts via email, network, or pager, or can send executable commands for automated response or resolution. It can also test a web site’s performance at different dial-up/modem speeds.
     
    Application Level Monitoring
    This tool is specifically for monitoring web based applications. It appears to do so strictly from the end-user’s perspective, with no separate monitoring of back end processes.
     
    Technology Compatibility
    Since this tool doesn’t need to integrate with site specific technology beyond a standard browser interface, this is not an issue. No technology details are given regarding the automated alerts via network, or the sending of executable commands for automated problem resolution.
     
    The Company
    Visual Networks was founded in 1993, and received ISO 9001:1994 certification in 1998 and again in 2001.
     
    This vendor offers support and training.
     
    Vendor: NetScout
    Tools
    nGenius Probes are non-intrusive, high-performance, real-time, application-aware monitoring devices that collect key performance metrics such as traffic and application utilization, conversations, error conditions, resource utilization, response time, and data captures
    nGenius Performance Monitor combines real-time and historical information on a single platform for monitoring, troubleshooting, capacity planning, and reporting on application traffic flowing across the network. It uses information from the Probes to assess website and networked application performance.
     
    Application Level Monitoring
    NetScout claims the nGenius Probes can monitor everything from the physical layer up to the application layer, including 250+ protocols out-of-the-box and 100 user configurable custom applications. While this seems to be primarily a network level monitoring tool, this configurability and application-awareness could be used for application level monitoring.
     
    Technology Compatibility
    The Performance Monitor runs on Windows 2000 Server and Advanced Server SP2. The probes plug directly into an ethernet network port.
     
    The Company
    NetScout was founded in 1984, received ISO 9001 certification in 1998, and today more than 3,500 enterprises world wide use NetScout’s monitoring products.
     
    This vendor offers support and training.
     
    Vendor: Candle Corp.
    Tools
    OMEGAMON suite – XE (mainframe), DE (dashboard), II (performance monitors), et al – lets you do real time and historical monitoring and performance analysis. You can define and report on all the resources that make up an application or line of business. The dashboard helps you tie together reports, graphs, and other data into a single, customizable workspace. You can define policies to have the system resolve business problems automatically, and/or generate alerts.
     
    Application Level Monitoring
    Candle states that it can monitor business applications, including in-house applications, using the Universal Agent. It appears to be highly customizable, but no details are given as to specifically what aspects of an application can be monitored.
     
    Technology Compatibility
    Various parts of this software claim compatibility with/support for: OS/390, 3270, Win NT, DB2, SQL Server, web services, SOAP, and Tivoli. It also has a customizable “Universal Agent” for monitoring a platform, database, or application that is not explicitly supported by one of the standard OMEGAMON products. There is also a gateway product that allows integrating information from disparate sources.
     
    This suite of tools appears to have originated with mainframe monitoring, but many additional tools seem to have made it usable on a broad spectrum of platforms.
     
    The Company
    Candle claims to have been chosen by 83% of the Fortune 100, and 73% of the Global 100. They claim that OMEGAMON is being used to monitor more than 70% of all mainframes in the world. Gartner rates Candle Corporation as ‘Promising’ overall, which means this company shows potential, but has not fully matured, yet they are rated ‘Positive’ for Product/Service. While Gartner states that Candle is still facing challenges, these ratings are substantially higher than the company received in February 2002.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Legato Co.
    Tools
    AppPanel provides performance management that helps to rapidly pinpoint bottlenecks and stress points, send alerts, and store performance data for analysis and future resource planning. Legato claims to provide performance monitoring of Windows servers, workstations, networks, and applications, to ensure that systems perform well end-to-end. It can provide simplified single-server restart of stopped windows services without manual intervention.
     
    It claims to use industry-standard interfaces, so benefits are seen immediately when applications publish more data. It “works with” the Microsoft Management Console, with some indication that this means it comes as a snap-in that extends the console, though it may provide a new interface for it. AppPanel also has a Web Console for data analysis, and a MicroPanel for wireless devices, but the capabilities of these features are vague.
     
    Application Level Monitoring
    AppPanel appears to focus mostly on System level monitoring, but it does claim business application monitoring. It also implies that there is an interface that applications can use to monitor and report on particular custom metrics.
     
    Technology Compatibility
    AppPanel works on Windows NT, 2000, 2003, and XP. It also has built in modules for Exchange, SQL Server, IIS, and print services.
     
    The Company
    Legato claims a world-wide presence and a long history of adopting and driving industry standards, but few details are given, and there is no indication of how mature their application monitoring tool is.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Identify Software
    Tools
    AppSight Black Box is a lightweight software module that operates like a Black Box flight recorder on a plane. Deployed on servers and clients, the Black Box software records applications at the user, system and code levels, thus providing visibility into application execution while eliminating the need to recreate problems.
     
    AppSight Console allows all problem resolution teams to view and drill down through Black Box recordings. AppSight fully supports distributed applications, allowing the user to follow transactions across servers and clients and drill-down into any component. It also enables developers to replay Black Box recordings at the code-level, providing a complete trace of function calls and variable values to pinpoint coding errors in the application.
     
    Application Level Monitoring
    The AppSight Solution claims to help pinpoint all common application problems (performance, functional, configuration, and user errors). As a record of everything that lead up to a problem this is likely true, but no mention is made of real time alerts, so this appears to be simply a recording tool useful during after-the-fact resolution, rather than a preventative or active monitoring tool. Because recording everything would require a huge repository, it is configurable with user-defined recording profiles, which means it only helps with those areas being continually recorded.
     
    Technology Compatibility
    The following technologies are explictly stated as being supported: Windows, .NET, and IIS.
     
    The Company
    Identify Software was established in 1996 and claims that hundreds of enterprises and software vendors are currently using AppSight.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: NetIQ
    Note that some NetIQ monitoring is already in use at WCB. It is included here to provide a comparison with other vendors and tools that were researched.
     
    Tools
    AppManager provides a solution for managing, diagnosing, and analyzing the performance and availability of distributed Windows-based systems, applications, and server infrastructures. Its central console allows management of virtually every component of a distributed, heterogeneous environment, from hardware to server applications, to end-user response time.
    WebTrends Suite delivers data and statistics that can be used to interpret user experience, understand self-service needs, and analyze performance.
    AppAnalyzer for Exchange delivers Exchange analytics across the enterprise, such as traffic analysis, delivery times, storage and usage reports, and content notification essential for optimizing performance and controlling a secure environment.
     
    Application Level Monitoring
    This vendor’s tools appear to be primarily focussed on system level monitoring. Application level monitoring appears to be limited to third party server applications, such as MS Exchange. There was no mention of monitoring in-house applications, beyond general monitoring of the servers they run on. The monitoring provided by WebTrends does appear to monitor how a web application is used, but the focus is on the end user’s experience, rather than the operation of the application.
     
    Technology Compatibility
    AppManager is certified by MicroSoft to be compliant with Windows Server 2003. It can also monitor MS Exchange, SQL Server, Windows XP, MQ Series, BizTalk, IIS, MSMQ, SMS, MTS, and Tivoli.
     
    The Company
    NetIQ was founded in 1995, and now has more than 60,000 customers world wide, including over 4,000 of the world’s leading enterprises. They also have partnerships with other industry leaders, notably including MicroSoft and IBM.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Precise
    Tools
    Precise/Insight measures response time and segments it by tier/component.
    Precise/Indepth samples non-intrusively at the database level, identifying users/programs using the most resources, and making recommendations for improvements.
    Precise/Inform communicates key performance indicators to the right people at the right time.
    Precise i3 combines the above three tools to provide integrated monitoring of certain environments and applications. Business transactions are monitored end-to-end, alerting support groups of breaches in service quality, identifying the root cause, and generating expert corrective advice. Monitored components include web server, application server, database server, and storage.
    Precise/Interpoint monitors SQL from the user through the application to the database.
     
    Application Level Monitoring
    Aside from application response time, most of the tools available from Precise appear to be for system monitoring. Application monitoring appears to be primarily for third party products like PeopleSoft. Web-based applications are covered, but the details only refer to response time and system level monitoring. Precise/Interpoint for COM+ appears to be a SQL monitoring product, centering on the database, but with the ability to monitor which COM+ component is responsible for each SQL query.
     
    Technology Compatibility
    They seem to be focussed on Oracle; while they do have Precise/Indepth for DB2 and SQL Server, their newest developments (e.g. Precise/Savvy) seem to be made for Oracle first, with others coming along later.
    They have a version of Precise i3 for PeopleSoft. Precise/Interpoint is available for COM+.
     
    The Company
    More than 6,000 leading enterprises worldwide, including 80% of the Fortune 100, are using Precise’s solutions. Gartner rates Precise as “Promising” overall, which means this company shows potential, but has not fully matured, yet it’s rated “Positive” for Products and “Strong Positive” for Support.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Computer Associates, Int.
    Tools
    Unicenter Management solutions go beyond monitoring application availability and performance, to manage all aspects of the application (though little detail is given as to what ‘all aspects’ includes). Application servers are monitored for service availability and performance, and cross-domain analysis is possible, claiming advanced root cause determination for rapid problem resolution.
     
    However, this ability appears to be limited to commercially available third party products, such as web servers, application servers, database servers, ERP, and groupware.
     
    Application Level Monitoring
    It appears that this vendor’s suite of tools is geared primarily for System level monitoring, with some network monitoring as well. The only application level monitoring appears to be for commercially available applications, such as Peoplesoft.
     
    Technology Compatibility
    Unicenter Management solutions claim to work with IIS, MQ Series, DB2, and PeopleSoft.
     
    The Company
    Computer Associates has been in business for over 25 years and has a global presence. They claim to be the first and only global enterprise software company to get ISO 9002 quality certification. While Gartner rates Computer Associates “Promising” for Product/Service, they rate the company as a whole as “Caution”, which means they are facing challenges in one or more areas, and potential customers are advised to become aware of those challenges before making decisions.
     
    This vendor offers support and training.
     
    Vendor: Mercury Interactive
    Tools
    SiteScope uses an agentless architecture to monitor the status and performance of web, network, and application infrastructures.
    SiteSeer is a hosted service based on the monitoring in SiteScope that provides self-service monitoring from outside the firewall.
    Topaz is a broad suite of modular, scalable tools for monitoring and managing system availability, performance, and service levels.
    ProTune assists in the tuning and configuration of applications, infrastructure, and security devices by leveraging Mercury Interactive’s performance tuning expertise.
     
    These tools gather information from a wide variety of performance monitors, with the ability to send alerts or execute recovery scripts when metrics pass acceptable threshold levels. There are a lot of references to monitoring from outside the firewall, and the emphasis seems to be on the availability and performance of internet-based applications, but there are also components that act within the firewall.
     
    It appears that most of the data gathered is from existing third party monitors. The Mercury Interactive tools, and Topaz in particular, provide a central tool for gathering and organizing this data, and for centralizing and controlling problem diagnosis and corrective action.
     
    Beyond gathering performance data, there are tools to simulate user actions, both inside and outside the firewall, to proactively identify problems before they are experienced by real users.
     
    Application Level Monitoring
    These products appear to be primarily for system level monitoring, but some features, such as those that simulate user actions, might be considered application level monitoring.
     
    Technology Compatibility
    Mercury Interactive claims integration with the following technologies in use at the WCB: PeopleSoft, .NET and wireless, legacy applications and client/server, Visual Studio, XML and COM+. It further claims to be able to monitor performance regardless of hardware or operating system.
     
    The Company
    Gartner rates this company a ‘Strong Positive’ overall, and notably in the Product/Service and Support Services categories. Mercury Interactive has been around for 13 years, but it appears as though most of its historical focus has been on testing tools, with application performance monitoring being a relatively new addition.
     
    This vendor offers support, training, and consulting services.
     
    Vendor: Micromuse
    Tools
    Netcool suite monitors network and system devices as well as service and application availability, including eCommerce, eMail, wireless, infrastructure, and bandwidth usage.
     
    Netcool monitors the network to locate and isolate problems as they begin to develop, then looks into devices such as a servers, switches, or routers to diagnose the problem and recommend solutions. Netcool also allows you to determine in advance the impact of faults in the IT environment.
     
    Data is also collected and stored, and may be formatted into reports showing the behaviour of devices, systems, and services over time.
     
    Application Level Monitoring
    While it claims to monitor applications, very little information is given. The heavy emphasis is on network devices.
     
    Technology Compatibility
    This system claims compatibility with many low level network products, but more of interest to application monitoring are: MS IIS, MS SMS, FTP, HTTP, HTTPS, TCP/IP.
     
    The Company
    Micromuse began shipping NetCool in 1996. It claims to be the market leader in “service and business assurance software”.
     
    This vendor offers support and training.
     
    Vendor: Equant
    Tool
    Enterprise Server Monitoring provides a remote monitoring service for the detection of problems on a server. It provides automatic paging, email, and fax notification, and maintains statistics for summary and detail reports. This monitoring is done remotely by the vendor, rather than an actual tool that WCB could use in-house. They provide a website which WCB can access to monitor any problems their service reports.
     
    No details are given about the nature of monitoring, or the types of problems that can be detected. It appears to be simply a problem detection service, and states that problem resolution is turned over to their customer (WCB). They do claim 24x7 customer service.
     
    Application Level Monitoring
    This appears to be primarily a system and/or network level monitoring service, but no details are given. It may include some rudimentary application monitoring, such as availability.
     
    Technology Compatibility
    No details are given, though they state that it “operates with server hardware from many different vendors”.
     
    The Company
    Equant was formed as the result of a merger in 2000/2001. It appears to be a world leader in providing network and telecom services. They are not a software vendor.
     
    This vendor offers support and consulting services.
     
    Vendor: VIEO
    Tool
    Adaptive Application Infrastructure Management (AAIM) combines application-aware networking hardware with intelligent management software.
     
    The core of AAIM is a hardware appliance that plugs into a network port, acting as a Layer 2 switch to monitor all network traffic from the inside, where it can take automated actions to improve application response times, avoid bottlenecks, and prioritize traffic. It claims deep visibility into all application infrastructure components, including hosts, operating systems, web servers, application servers, and database servers.
     
    AAIM includes three analysis engines that listen to traffic and auto-discover computers on the network and the software running on them in order to dynamically build performance models and determine changes for optimization. It can automatically adjust resource allocations, or provide recommendations that are accepted or rejected by authorized administrators.
     
    Application Level Monitoring
    This appliance appears to be focussed on network and system level monitoring. However, it may be configurable to monitor application components as well.
     
    Technology Compatibility
    AAIM plugs into a TCP/IP network, but beyond that the first release does not appear to target WCB’s technology platforms. There are plans for .Net tools and PeopleSoft compatibility in the future.
     
    The Company
    VIEO is a new company, founded in 2000, and referred to as a start-up in the April 21, 2003 issue of Network World. References are positive, but the company and its product are as yet unproven. The first product release is due mid 2003.
     
    This vendor offers support and consulting services.

    Roland Stens
    http://www.performancetester.com
  3. Apart from the tools appointed by Roland, Quest also has some availability monitoring tools.

    Besides, we have a very positive experience with OmegaMon, at least in OS/390 enviroments, although they also provide tools for open systems.


    Jose R. Huerga
    http://www.terra.es/personal/jrhuerga
  4. The AppSight Solution claims to help pinpoint all common application problems (performance, functional, configuration, and user errors). As a record of everything that lead up to a problem this is likely true, but no mention is made of real time alerts, so this appears to be simply a recording tool useful during after-the-fact resolution, rather than a preventative or active monitoring tool. Because recording everything would require a huge repository, it is configurable with user-defined recording profiles, which means it only helps with those areas being continually recorded.
    AppSight does include a very sophisticated trigger/alert mechanism, and thus can be used either after-the-fact or in a preventive manner. Moreover, AppSight not only alerts you to the fact that a problem occurred, but it also has the ability to automatically change its recording configuration to capture problems at a deep level, so that 1) you know that there is a problem; and 2) you already have all the data (e.g. user experience, database calls, WS calls, function calls, variable values, and more) in order to determine what's causing the problem. The performance overhead is very low, so this is something that may run on an ongoing basis.

    p.s. the system also supports J2EE apps, not only .Net and native Windows

    Ran Gishri.
  5. I just wanted to add that there is a tool called "Transaction Analyzer" from HP that integrates nicely with OpenView to provide a decent systems and application management framework.

    Here is the link for it
    Transaction Analyzer
  6. Cristian, what was your decision, I am looking at a few of the same tools.