Upgrade problems with the new Fixed IP quota
In the last few weeks a new quota has been added to Nova covering Fixed IPs. This was done in response to LaunchPad bug 1125468, which was disclosed as CVE 2013-1838.
To be honest I think there are some things the vulnerability management team learned the hard way with this disclosure. For example, we didn't realize that we needed to update python-novaclient to allow users to set the quota, or that adding a quota would require changes in Horizon. Both of these errors have been corrected.
More importanly, the default value of the new quota was set to 10. I made this decision based on the default value of the instances quota coupled with a desire to protect deployments from denial of service. However, this decision combined with a failure to explicitly call out the new quota in the release notes for the Folsom stable release have resulted in some deployers experiencing upgrade problems. This was drawn to our attention by LaunchPad bug 1161190.
We have therefore moved to set the default quota for fixed IPs to unlimited. If you want to protect yourself from a potential DoS, then you should seriously consider changing this default value in your deployment. This can be done with the quota_fixed_ips flag. The code reviews implementing this change are either merged, or under review depending on the release. At the time of writing this Havana and Grizzly have a fix merged, with Folsom and Essex still under review.
I think this experience also reinforces the importance of testing all upgrades in a lab environment before doing them in production.
Sorry for any inconvenience caused.
Tags for this post: openstack nova quota fixed_ip vmt cve denial_of_service
Related posts: Further adventures with base images in OpenStack; Havana Nova PTL elections; Openstack compute node cleanup
Comment