What are critical security releases and why do I need them?
A critical security release or patch is an update to Civi in response to a security flaw that has been found.
These could range from something that could allow a hacker to read files that do not contain any sensitive information all the way up to allowing them to access your user data, change the root user, or disable your site.
We must apply these patches/ releases as you are in contract for software maintenance with Circle Interactive, and also as we are your data processor, we have a legal obligation to keep your site safe at all times.
Further, if you are on a shared server, failure to do this may also put our other customers at risk.
You can find release notes for each Civi release here.
Can I opt out of critical security releases?
The short answer is no, but note that we will look to minimise disruption by only insisting on applying critical security updates and major version updates.
You are in contract for software maintenance with Circle Interactive, and also as we are your data processor, we have a legal obligation to keep your site safe at all times.
If there is a major version update, the need to move you on to this is still driven by security in that you need to be up to date in readiness for a critical security update.
An ESR release may give you a little more longevity, but security updates will still need to be applied and these could still affect your site’s operation.
What notice will I have for critical security releases?
Due to the nature of these events, there can never be a set in stone schedule or process, but typically, as official Civi partners, Circle tends to have wind of a forthcoming update about a month in advance.
From this point, as more information becomes available, we will decide on our approach and then communicate this to our customers, which will usually be around 2 weeks ahead of the release.
We will not have any details of how critical the security release will be, which elements of Civi it will affect and so forth until the release day when it becomes public.
It’s also possible a vulnerability could be found that needs to be addressed more immediately and therefore has to be addressed as a patch or a version outside of such a process with little or no notice.
How quickly will critical security releases be applied?
We aim to apply critical security updates for all sites as quickly as is feasible after they are released. We will usually apply these to shared servers and uncustomised sites first, and then to customised sites on their own machines.
Some releases will relate to more serious security issues than others which will affect not only how quickly we apply them, but also how much time we have to test and how we communicate with you throughout the process.
Once the release is public, the vulnerability is known to hackers, so although we would avoid this wherever possible, there could be a scenario where a patch or upgrade is applied universally after basic testing and our clients are only informed afterwards.
How much downtime will there be?
We will look to minimise downtime while the upgrade is being applied, but the length of time in each instance may vary. As a rule of thumb, this can be as low as 15 mins per site, but there are a lot of variables, some of which won’t even be apparent until we start the process.
We will upgrade sites early in the morning (c. 7am UK time) in order to minimise your downtime and this also means you have a full working day in support hours so any issues raised on tickets can be responded to.
Can I choose when my Civi is upgraded?
Circle needs to apply critical security updates to all the sites we manage in a very short window of time, so it’s simply not practical to offer much flexibility in terms of times and dates, especially for customers on shared servers.
If you have a strong and exceptional business case to, for example, avoid one particular date, please contact us and we will see if this is possible to incorporate.
What testing do Circle do ahead of applying these?
Circle needs to balance the need to identify issues with the release and how it works with real life sites against their responsibility to keep your site up to date and secure. To this end, we will do as much testing as time allows between being able to get a copy of the version series, the security release itself and the actual security release or patch becoming available.
Based on an example where we know a security release will come on a given date and the advice from the Civi team is that we update all of our client’s sites to the version they specify, a typical plan would be;
Circle review the elements in Civi that have changed since the last version and prepare some test sites with copies of real life sites that use those features.
Circle test the release candidate* on these, paying particular attention to functional areas we know have been changed. We will feed any bugs back to the Civi team so these can hopefully be fixed before the public release.
For customised sites, we will run a ‘diff’ (current version vs new version) against each individual site and then take appropriate action to make sure any relevant differences are addressed. This will involve such things as porting customisations, and updating modules/ extensions.
Where customised sites require more attention and/or the customer will be involved in testing, we will refresh their staging site with a copy of live and then update.
*A release candidate is a test version of a forthcoming public release.
What if modules/ extensions don’t work with the new Civi version?
If you use modules/ extensions that are affected by an upgrade and there is a suitable update available, we will usually apply this FOC.
Where we are aware that an upgrade will render a module unusable, we will approach you with alternatives and cost estimates for these.
If a module breaks when an upgrade is applied, Circle will raise an issue with the maintainer. Although we have no control over those who contribute modules, we can usually take an educated guess where the maintainer is likely to supply an amended version themselves within a reasonable timeframe.
Where Circle believes the maintainer will provide a solution, we would usually recommend you wait for this.
Failing this, Circle will provide an estimate to either produce a patch for the module, or to create a new solution (whichever is most feasible).
Please try to manage your sites so modules no longer in use are disabled so that Circle doesn't spend time trying to solve upgrade issues relating to things you don’t actually use.
Can I test my own site?
Whenever an upgrade is applied to your live site, we strongly suggest you test as much as you can as quickly as possible and raise any issues as support tickets.
Although we will unit test the elements that make up your site, we do not know all your user workflows, may not be familiar with any contributed modules you may use, or the full detail of your customisations.
If you have a site with complex customisations that is not on a shared server, we can update your test site with the upgrade on request in order for you to test ahead of the live release and include this in your cost estimate.
Note that the window for doing this, especially if the critical security release turns out to be a real threat to your site and data, is narrow and we may end up in a position where we need to progress to a direct release to live anyway.
Why hasn’t my test site been updated?
The way we test and roll out these upgrades means it’s usually not necessary to upgrade organisations' test sites and by not doing so, we also retain a useful record of how things were working pre upgrade.
Where a site is heavily customised and we establish that the version changes do affect these customisations, we will update the test site first, and likewise where there is a complex, customised site on its own server where the owner would like to test the upgrade themselves.
Your test site will be updated to match the Civi version on live the next time we need to use it to deliver a piece of work or investigate a support issue that can’t be progressed on the live site.
What is covered by my software maintenance fee?
Security and major version upgrades are covered by your maintenance fee for the uncustomised elements of your Civi.
If you have customisations, including customised modules, these require porting and additional testing which is chargeable.
Why do I have to pay to fix things that have broken due to an upgrade?
As an open source and community maintained piece of software, Circle cannot be responsible for bugs in any Civi version or any contributed modules or extensions.
We will test as extensively as we can as described elsewhere in this FAQ, but ultimately you are as responsible as we are for finding any issues which we can then report back on your behalf and work to fix as per our support contract with you.
If there are any issues due to oversights on our part, e.g. Documented customisations we have not ported, you will not be charged for any remedial work.
Why is my site upgraded twice with each security release?
Usually when a security release will come as a civi version (e.g. 5.19.4), official advice is to get all sites onto a version in the same series as soon as it becomes available (e.g. 5.19.0)
The reason for this is the pre- security release version will contain all the changes that will be in the security release itself, just missing those that pertain specifically to the security aspect.
By getting all sites onto the right series, we have time to test and iron out any bugs so that once the actual critical security release is available, all we need to do is move everyone onto it without delay.
So, going back to the examples above, your site may be moved onto 5.19.0, then up to 5.19.4 within a week or so afterwards.
Why can’t you just roll back the Civi version if things aren’t working properly?
Rolling back an upgrade is, at best, a complex process and often actually impossible as the upgrade changes the database schema, so the only option would be to roll back to a backup which would lose everything done in the interim period.
If you have a strong business case to insist on a restore from backup, the time taken will be chargeable as will fixing any further issues that have arisen as a result of this.
What are ESR Civi versions?
CiviCRM Extended Security Release (ESR) provides a longer release of CiviCRM, allowing for fewer upgrades but fewer new features.
This could be attractive to Civi users with a lot of customisations as it could mean fewer upgrades, as some critical security releases can be applied as patches rather than version upgrades.
This is a paid service and Circle clients can access a reduced rate for these.
More information is on; https://civicrm.org/esr
