This is a guest post from Guy Leech.

Citrix Virtual Apps and Desktops is a powerful, secure, and scalable solution for solving the challenges involved in deploying mission critical applications to users. Citrix admins in a typical enterprise use it to deploy a wide range of apps and desktops.

A great deal of effort has been put into Citrix products to remove the complexity of application deployment and operation. But when problems do occur, it’s essential to be able to identify and drill down to the root cause. As the number of applications and desktops deployed grows, the ability to find the real culprit behind issues becomes increasingly difficult.

In this post I’m going to introduce just some of the fruits of my labours in writing script-based actions (SBAs) for ControlUp to give their real-time console even more troubleshooting capabilities. I hope the three scripts introduced here will help you save time and pinpoint issues faster.

Whilst these scripts have been developed to slot seamlessly into the ControlUp product, they can also be used standalone by Citrix admins. The ControlUp team is committed to contributing to the Citrix community and makes all of the scripts created available for download in their ControlUp script library.

Display Network Connections by Process

This SBA is a fast netstat-based script, which enables users to dive deeper than the process level into the network connections established by each process. It takes output from the netstat command, processes it, and displays it in a far more easily digested format.

Where an IP address cannot be resolved to a name by DNS, it will optionally use the RIPE API to get registrant details for the IP address to help track down what the remote system may be. RIPE is the Regional Internet Registry (RIR) for EMEA and provides a public RESTful API, which can be used for various purposes including requesting details of the registrant for a given IP address.

Where the IP address is not assigned to an entity in the EMEA region, other RIRs can be queried, within the same single query, to further try and ascertain the registrant. The script uses PowerShell’s Invoke-WebRequest cmdlet, which converts the results of the web request to the RIPE API into a structure. It’s easily processed in PowerShell, which, in this case, is in JSON format. You can find details on this API here.

You can run the script for a single process, all the processes in a specific user’s session, or the computer as a whole. Download the script here.

Analyze Process Network Bandwidth Consumption

By means of a short packet-capturing sample, which doesn’t use any third-party software, this script analyzes network traffic in real time and displays which sources or destinations account for the highest bandwidth consumption.

The network packet capture uses Event Tracing for Windows (ETW), which is an efficient mechanism for obtaining low-level trace information from the Windows kernel. Windows operating systems have many built-in providers of ETW information, and this script uses the Microsoft-Windows-Kernel-Network one. Run ” logman query providers” to see those available on any particular system (on my Windows 10 laptop there are over 1,100). PowerShell version 5 includes native cmdlets for controlling ETW traces but for older versions and operating systems, traces can be controlled via the logman or netsh built-in commands.

The output from ETW traces are etl files, which can be loaded into the Windows event viewer and also by PowerShell using the Get-WinEvent cmdlet. This allows the raw ETW trace to be manipulated to present the required information. However, if the trace output is viewed in event viewer, it is not particularly easy to visualise the data. Also, there will typically be thousands of events, even for short tracing periods:

Because we require the amount of data sent or received per event, which is contained in the “size” event data field above, we have to analyze every event with id 11 (for received TCP data) or id 10 (for sent TCP data). The “daddr” and “saddr” fields are the destination and source IPv4 addresses, respectively, but in decimal format rather than the more common (and easier remembered) dotted notation. Fortunately, all this data is also available to Get-WinEvent in PowerShell so it can be processed and presented per source or destination address.

Again, the script can be run for a single process, all the processes in a specific user’s session, or the whole computer and can query the RIPE database for non-DNS resolvable addresses. Download the script here.

Analyze Process Disk Access

This script leverages the Sysinternals Process Monitor (procmon) utility to show the file system locations that are being accessed by specified processes, which can assist with troubleshooting anomalous I/O issues.

Procmon is an valuable tool for many troubleshooting scenarios but it can present a daunting experience for inexperienced users, particularly when there is a large amount of trace data produced. This SBA not only automates the capture of procmon data but also its analysis so users do not have to use procmon; in fact, they don’t even see it when this SBA is running.

If procmon is not already available on a system, the path to which can be specified as an argument to the SBA, it will be downloaded from the internet via the live.sysinternals.com site (although its integrity, via digital certificates, will be checked before it is allowed to run). Procmon is then run for the duration specified via an argument to the SBA using the /runtime switch. In order to only capture data for the process under investigation, a filter file (.pmc) is dynamically constructedm which reduces the amount of data collected and thus the time required to process it and disk space needed.

The data produced by procmon at this point is in a binary pml file that needs to be converted to csv format data that PowerShell can consume, which is achieved by running procmon for a second time with /OpenLog and /SaveAs arguments. It’s then a matter of using the Import-CSV PowerShell cmdlet to read in the converted trace file, process it, and produce a summary of the most frequent operations and file system objects involved as shown below:

Of course, all files created by the SBA are deleted, even if the SBA is terminated prematurely by the user, so as not to impact on free disk space on the computer where the SBA is run. Similarly, all instances of procmon are terminated. Download the script here.

I am working closely with the ControlUp team to continue and build out useful scripts to help you more effectively troubleshoot and fix your Citrix Virtual Apps and Desktop deployments.  You can put in a request for specific scripts or provide feedback on existing scripts here. And you can find these scripts in the ControlUp script library.

Citrix Tech Bytes – Created by Citrix Experts, made for Citrix Technologists! Learn from passionate Citrix Experts and gain technical insights into the latest Citrix Technologies.

Click here for more Tech Bytes and subscribe.

Want specific Tech Bytes? Let us know! tech-content-feedback@citrix.com.

Guy Leech wrote his first (Basic) program in 1980, was a Unix developer after graduation from Manchester University, and then became a consultant, initially with Citrix WinFrame, in 1995 and later into Terminal Server/Services and more recently virtualisation. He has also had various stints in Technical Pre-Sales, Support and R&D. He works as an independent consultant, scripter, and trainer; lives in West Yorkshire, England; has a wife, three children, and three dogs; and is a keen competitive runner when not injured.