Since version 10.1 NetScaler (Gateway) uses a new mechanism for deploying and maintaining custom design themes for login pages. This works quite a bit better than the old way with manual boot scripts to copy customized files. But it still is prone to issues especially after updating the NetScaler firmware.
Before updating firmware, make sure to set the theme back to „Default“. This will not impact your users, because you do this on the SECONDARY node in your HA pair, which is not taking connections at that time. After the update, set the theme to „Custom“ again and re-implement customization, force HA failover and repeat on the other node.
What might happen, if you change this procedure? A couple of chances to mess things up:
- No login possible through or malfunction of admin UI. The Configuration Utility is part of the ns_gui folder being deployed through the custom theme mechanism. If theme stays at „Custom“, the updated files will not be part of the customized ns_gui folder and thus the admin UI will be old and possibly incompatible with the new firmware.
- No login possible through or malfunction of NetScaler Gateway and AAA-TM. If the files for login, tmlogin et al. are not updated, they might be missing changes needed for correct function with new firmware. Furthermore, the client components (Gateway Plugin, EPA Plugin) will not be updated and especially establishing an SSL-VPN will fail subsequently;
- Update-downgrade-loop of Gateway Plugin: Even if you noticed that the client components need an update and you manually uploaded the new AGEE_setup.exe to your NetScaler Gateway (after you have manually updated the admin_ui folder in your ns_gui_custom folder to be able to login again…) and your clients have successfully updated, they will still fail to establish SSL-VPN connections. First they were told they need to update Gateway Plugin to a new version, which they thought they downloaded from the NetScaler Gateway, but they got the same old version. Now they have the new version installed, but upon connection they will be told they need to downgrade to an older version. After which they would be told to update again. This is due to another file not being updated, which compares the client’s version to its own information – which still has the old firmware version.
There might be even more issues, but at least these have been seen in the wild already. So make sure to simply go back to default and redo the customization. A script for creating the archive out of the newly customized files might be helpful. Yes, the archive. Make sure to recreate it after every customization, because it will be extracted and its contents will be used upon every NetScaler boot.
Update: Thanks to Stuart Carroll (@stuart_carroll) for the comment on using Rewrite feature to modify default themes to reduce risks even more. Whenever possible (complexity of customization is limited, of course), this is the best way to go. See our (German, sorry) post on using Rewrite for customizing Clientless Access view to get an idea on that.