With the current global situation, pressure on remote working environments has increased substantially. Cloud bursting is an option, but reportedly even Azure is cracking at the seams, and it is not likely to be a lot better at Amazon or Google. You may just have to work with what you have right now. This post contains some basic tips for reducing the load on your environment.
A lot of these measures will affect users. Let them know what is happening and why. Explain to the @home workers how they can access certain resources without going through the Citrix environment, and help them if required. If you think you can shut down certain services in your environment, talk to your users and make sure that this is the case. Inform the servicedesk what you are going to do, and be prepared for a fast rollback if you have gone too far.
A first step could be to simply ASK your users to take it easy. Not to leave dozens of programs open in the background while they work in one program 99% of the time. Simple things like that.
Note: check your stats. If you have zero memory issues but a stressed out CPU, asking users to open and close programs a lot may make your problems worse. As for some of the other suggestions in this post, everybody’s environment is different, see what could work well for your environment.
As always, all tips in this blog should be tested in your environment before deploying, but right now the odds are that you do these tests just a little bit faster than usual. There are always unforeseen results, be ready to Ctrl-Z.
While streaming YouTube or radio at work is often allowed, it is not always business critical. Also, a lot of users working at home who may innocently start a stream in their Citrix session will be better served by just going to the streaming site from their local machine. Yes, you may have content redirection configured, but that is no guarantee it will work all the time. Sites can be placed on a blacklist for streaming if there are issues. And with resource contention on the Citrix servers the odds of something going wrong are higher than usual.
See if you can block streaming. Don’t forget popular news websites. Everybody is checking the news right now, again they can also check that using their local resources. And a lot of news that has video feeds also has a written article with the essentials as well.
Are there videos which are business critical on, say, YouTube? If you can identify these it is often possible to allow only these specific URLs through your firewall.
Coordinating a firewall change can take some time. If you want a quick fix, take a look at the Redirect URLs to 127.0.0.1 in HOSTS file script I wrote for ControlUp (available in the console and here: ControlUp Script Library). This script can add entries to the local HOSTS file that resolve to 127.0.0.1, so a user can only browse to the specified website if they know the actual IP address.
Teams, Skype, all very useful at a time like this. But is it necessary to run that webvideo stream as well? Very rarely, and it eats resources. Disable video in those programs. Or even better, ask your @home users if they can run these programs locally.
You can disable IP Video for Teams in the Office 365 Admin Center, in the Teams Meeting Policies: https://admin.teams.microsoft.com/policies/meetings
These days a browser is often used for accessing web based applications. It is not strictly necessary to have that application open together with 50 other tabs of nonsense, but that’s what’s going to happen. Talk to your users and make them aware that right now they need to be a bit more conservative with their tab use.
For Chrome, try adding the –process-per-site switch a for Chrome (so change the shortcut for blah\blah\Chrome.exe and make it blah\blah\Chrome.exe –process-per-site). Make this the default shortcut but keep the original shortcut tucked away somewhere. This way if the ‘–process-per-site’ switch causes any issues you can guide a user to the ‘regular’ Chrome shortcut straight away.
Quick test on my home computer with about 10 tabs open:
‘Regular Chrome’: 45 processes, 1800 MB memory used
‘Process per site Chrome’ :36 processes, 1550 MB memory used
It’s only a 250 MB difference, but hey, it’s free real estate.
And get that Ad blocker in there!
(you can use the HOSTS file script to do some quick & dirty ad blocking as well)
I browse all over the place, this is what my PiHole statistics look like after about 15 minutes of browsing:
That’s a lot of ads you can block…
5. Turn down VisualFX
Turning down Windows visual effects has been a standard practice for a while, but when resources are plenty these days they are often not tuned. Sliding menus, fade in and out and Aero peek are all very pretty but totally unnecessary. Not only do all those small effects start adding up in CPU use if there are many users, it will also use network bandwidth. Personally, even on my local machine I prefer the snappier response with these turned off.
You can use settings in the Citrix and Terminal Server GPOs to tune some of these settings. You can also place registry settings directly in the registry to achieve this. However, be aware that injecting the settings in the user registry during a session does not work, the settings are not applied and are discarded when the user logs off. The settings have to be there before the user shell initializes during logon. If you use the Default profile as a base for your user profiles (hybrid style) you can change the settings in the .Default user registry. Be careful here though, the last thing you want to do is mess up your Default user profile.
These are the settings I have used for years for basic tuning of the visual effects, use with care, ymmv:
For max tuning set FontSmoothing to ‘0’, but the ‘2’ setting does keep things more legible.
I have these settings on my (Windows 10) machine, and my Visual Effects tab looks as follows. These registry settings do not control every setting on the Visual Effects tab, there is a bit more going on under the hood but they are a pretty good start:
Incidentally, the MinAnimate value corresponding to the Display – Show animations in Windows switch also controls visual effects in Office. On my computer with an admittedly old GPU, GPU use while clicking through the menus in Word was halved after switching this off. It’s only a few %GPU (if your servers have a dedicated GPU, otherwise it’s CPU and probably more impact) but again, a few % multiplied by dozens of users can make a difference.
Sure. Cool. But do you really need this?
And this registry key maps to Display – Show Transparency in Windows on Windows 10:
It all adds up.
6. Session Idle times
You can tighten up the Disconnect and Logoff settings in your session settings. Usually it is great if a user can walk away from their session and it is available straight away when they return hours later because it has not even Disconnected. But right now you may need every bit of memory you can get so consider logging off Disconnected sessions sooner.
Use common sense, if you start logging off users after every 5 min of Idle time but they just log in again a minute later all you have done is cause a hefty extra CPU load.
7. Monitor your monitoring software.
It is not unusual to see more than one monitoring product on a machine. One product is monitoring hardware health, another performance, another network etc. Unfortunately all too often I see several products monitoring the same thing because different departments prefer their own products. But right now you should be focusing on the basics. Take a look at what you are monitoring and why, choose wisely.
Follow Ton de Vreede on Twitter at @WillCode4Cheese