Home » pvs 7.1

Tag: pvs 7.1

Citrix PVS vdiskdif.vhdx cache file filling up and servers crashing after reboot.

Situation:

We had a number of production VMs running a Citrix desktop workload on Citrix Xenapp 6.5 via Citrix PVS 7.1 (SP3) with cache to RAM with overflow to hard drive, set at 2048Mb. VMware Hypervisor 5.5 and Appv 5 SP2 with HRP4 running in full infrastructure mode (no Microsoft SCCM) and 30GB of packages running in Shared content store mode from a network file share \\server\appvshare$

Servers have 12GB Ram, with 4 vCPU’s and Windows 2008 R2 SP1

C: Drive is 40GB (with 35GB utilised)

D: Drive is 20 GB (typically 12GB utilised at any one time) with a 6.1GB Page file redirected here, an the vdiskdif.vhdx at 4mb

We have a daily reboot for the entire Citrix desktop estate estate at 3am.

Appv GPO Settings are to use Shared content mode, use the local path of c:\programdata\appv (not app-v), autoload = ALL, user and computer refresh on logon = true

APPV_Streaming

Problem/Symptoms:

After an IE11 upgrade on the Citrix vDisk, every 2-3 days some of the citrix servers would reboot (as per policy) and quickly fill up the D drive, finally crashing the server and rendering it unusable. Hoorah for PVS cache to disk.

As the machine had crashed there was little way to troubleshoot it after the fact, so we needed to capture it during.

Steps to resolve:

Increase C: Drive to 60GB using the process listed in the first section of this article – NO FIX

Increase D: Drive to 40GB to resolve. The server stopped crashing as the server didnt run out of D drive space, we then noticed that the vdiskdif.vhds was sitting at 13.6GB and growing (vs unaffected servers sitting at 4Mb) When checking the system processes, nothing was obviously running or processing. This was a nice workaround but not a final solution or fix.

We rebooted and remotely monitored the system with PSTools (sysinternals) executable “pslist -s \\servername”. During the reboot the appvclient.exe was siting 25% CPU, whilst the GPO to do a full client sync was ‘called’ and run (powershell: get-appvpublishingserver | sync-appvpublishingserver)

Our crashing was finally narrowed to the AppV client that was sporadically somehow crashing and filling the D drive. When the AppV client was stopped or the server removed from any ‘global’ publishing – the server never crashed. As soon as the production APPV packages were republished to the device, and after every 2 or 3 reboots the appvclient.exe would run, start caching files, and start filling up the D drive as it was ‘copying / caching’ the packages.

This felt very much like the AppV client was ignoring the shared content mode, and trying to cache everything locally.

Final Resolution:

The ‘gut feeling’ on this problem was that the AppV client wasnt getting the correct settings from GPO in time. The local AppV Client settings (registry) were reviewed and it seemed there were some improvements to be made.

OLD SETTING

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\AppV\Client\Streaming]

“AutoLoad”=dword:00000001

“SharedContentStoreMode”=dword:00000000

Turns out the Appv Client settings were set to disable Shared content mode, and to autoload all apps which we are assuming its doing sometimes before it gets its required settings from GPO.

 

NEW SETTING

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\AppV\Client\Streaming]

“AutoLoad”=dword:00000000

“SharedContentStoreMode”=dword:00000001

After creating a new vDisk version with the ‘New Setting’ registry keys listed above tattooed in the vDisk image  the servers haven’t crashed since.

Also we were using a GPO powershell script that tells the client to go and ‘autoload’ all the apps. So the autoload registry option was just confusing things, hence we disabled it.

Citrix PVS 7.1 and AppV 5 applications crashing, launching once

This issue absolutely did my head in. I had to publish it for you all to save you the time if you ever come across it as I was close to QUITTING because I couldn’t get this to work.

Scenario:

  1. Windows 2008 R2, Citrix XenApp 6.5 and Microsoft AppV 5 SP2.
  2. Applications published globally to the XA Servers
  3. All Servers streamed from Citrix PVS 7.1
  4. User profiles managed with Citrix User Profile Manager

This article for the Citrix UPM was already followed and the AppV Exclusions were definitely set

https://support.citrix.com/proddocs/topic/user-profile-manager-5-x/upm-using-with-app-v.html

Some App-V packaged applications would launch once, then never again when the Package Installation Root was pointing to the Read Only PVS Image (%programdata%AppV) and ONLY when the server was in Read only mode. If I was editing the image in PVS maintenance mode to update the vDisk – the issue never presented itself.

Ideally I wanted the App-V cache to point to the ‘out of the box’ %programdata%AppV – and in conjunction with PVS you can guarantee its fresh everytime

A temporary work around was to move the Package Installation Root to the D: (Read Write Disk) < but we soon started getting errors with this as no amount of scripting and startup scripts would cleanly remove this directory 100% all the time. I.E:

Get-AppvClientPackage -All | Remove-AppVClientPackage

remove-item -Recurse -Force d:app-v

get-appvpublishingserver | sync-appvpublishingserver -Global

This script would only remove the currently published applications, too bad for any legacy packages that were no longer published to the servers,

AppV_D_Notrefreshing

and for whatever reason the SYSTEM account was unable to delete the d:app-v folder (access denied even though it was owner and had full rights) sporadically the folder wouldn’t update at all and no applications would be refreshed until a user logged into the server. (Poinless on a citrix server where the applications have to exist if delivered as published apps’  Citrix could not find / see the executable and the published app would not run.

Eventually App-V Clients services started failing unable to start.

appv_d_Error7024

The Microsoft App-V Client service terminated with service-specific error The system cannot find the path specified.. EventID 7024 (Update: After further troubleshooting it turn out this was because some of the legacy packages were referenced in the Windows registry)

There had to be a better solution.

Solution:

Change the PVS 7.1 Disk Cache mode to  “RAM with Overflow to Disk”, Return the Package Install Root to %programdata%AppV and generalise the AppV 5 Client before shutting down the image after each PVS update.

 

vdisk_properties_pvs7.1_appv5_Fix

 

AppV 5 Generalise:

c:admintoolspstoolspsexec.exe -s powershell.exe “Get-AppvClientPackage -All | Remove-AppVClientPackage”
regedit.exe /s “C:adminToolsAppV_Generalise.reg”

AppvGeneralise.reg Contents:

Windows Registry Editor Version 5.00

[-HKEY_LOCAL_MACHINESOFTWAREMicrosoftAppVClientIntegrationPackages]

[-HKEY_LOCAL_MACHINESOFTWAREMicrosoftAppVClientPackageGroups]

[-HKEY_LOCAL_MACHINESOFTWAREMicrosoftAppVClientPackages]

[-HKEY_LOCAL_MACHINESOFTWAREMicrosoftAppVClientStreamingPackages]

Articles / References:

http://discussions.citrix.com/topic/348198-app-v-5-client-citrix-provisioning/ – Explains the situation clearly, and finally provided me the solution to fix it once and for all. Much thanks to Carl Fallis!