| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-22 | |||
| 15:23:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Require Migration object arg to migrate_instance_finish method https://review.opendev.org/690106 | |
| 15:24:49 | umbSublime | I have a specific workload where I need to enable invTSC, for this to work with migrations I also need to set the tsc frequency. How can I configure the tsc frequency in nova.conf on my compute hosts (is it possible with cpu_model_extra_flags)? in qemu it looks like this: "<clock><timer name='tsc' frequency='3000000000'/></clock>" | |
| 15:25:25 | sean-k-mooney | umbSublime: im pretty sure you cant | |
| 15:26:02 | sean-k-mooney | you could enabel a specific cpu flag but i dont think you can spcify the frequency | |
| 15:29:06 | umbSublime | Would the solution in this case be to create a custom CPU model and use that in nova.conf ? | |
| 15:29:16 | umbSublime | or is there an alternative workaround I didn't think about | |
| 15:29:54 | sean-k-mooney | well this is really a feature request so i would start with the uscase | |
| 15:30:33 | sean-k-mooney | you are trying to fake a specifc cpu frequece and provide a "stable" tsc to the guest correct? | |
| 15:30:39 | umbSublime | I'm only the operator for this cloud. But the use case is a software that requires that flag. Only issue is with only the flag enbaled and the frequency unset, migrations are not possible | |
| 15:30:43 | umbSublime | due to the nature of the flag | |
| 15:30:47 | sean-k-mooney | so that if you migrate betwwen hosts with differen cpu fequecne it does not chagne | |
| 15:31:10 | umbSublime | exactly there are some extensive discussions about that and how it was implemented in qemu/libvirt | |
| 15:31:51 | umbSublime | when you set a specific frequency, there is a calculation made so that migration is possible and the value of the register is mirrored over | |
| 15:32:44 | sean-k-mooney | ya so nova dose not currently suport hat | |
| 15:32:48 | sean-k-mooney | *that | |
| 15:32:51 | umbSublime | All I know is the software they run on the guest requires that invTSC is presented to the guest. From my operation pov I also need the VM to be migratable | |
| 15:33:00 | sean-k-mooney | we could ad support if we had a defied usecase | |
| 15:33:28 | sean-k-mooney | do they need invTSC set or do the want the TSC to be marked as reliable in the vm | |
| 15:33:31 | umbSublime | It's a game engine software so using other counters isn't really possible | |
| 15:33:41 | umbSublime | they need invTSC to be set | |
| 15:34:06 | sean-k-mooney | ok so setting tsc=reliable on the kernel commandline wont be viable | |
| 15:35:01 | sean-k-mooney | im basicaly wonder if https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/kernel-parameters.txt#L4832-L4838 | |
| 15:35:06 | umbSublime | Would the custom cpu_model be a working temporary solution ? | |
| 15:35:07 | sean-k-mooney | was another alternitive | |
| 15:35:14 | umbSublime | Ohh I didn't mention it's a windows guest :/ | |
| 15:35:20 | sean-k-mooney | oh :) | |
| 15:35:31 | sean-k-mooney | so that would be a know to linux kernel args then | |
| 15:35:48 | sean-k-mooney | in which case we shoudl be settin ghte hyperv features | |
| 15:37:14 | sean-k-mooney | if you modified the libvirt xml defition and created a new cpu model it might work | |
| 15:37:20 | sean-k-mooney | but i doubt it | |
| 15:37:45 | umbSublime | If I virsh edit a running vm and add the frequency, then reboot the VM it is migrated just fine by openstack | |
| 15:38:16 | sean-k-mooney | umbSublime: if you virsh edit it then it will be lost if nova regenerate the xml | |
| 15:38:24 | sean-k-mooney | e.g. via hard reboot or any move opertaiton | |
| 15:38:26 | umbSublime | Yes of course ~! | |
| 15:38:46 | sean-k-mooney | e.g. i would not expect it to survie a migraton | |
| 15:38:51 | sean-k-mooney | oh | |
| 15:38:54 | sean-k-mooney | you mean | |
| 15:38:55 | umbSublime | It was just to confirm that migration would work if forcing the freq. The next step was to configure in nova :) | |
| 15:39:10 | sean-k-mooney | before you live migrate you remove the invTSC flag | |
| 15:39:26 | umbSublime | nope both the invTSC and freq migrated with the VM | |
| 15:39:39 | umbSublime | I was actually surprised by that | |
| 15:40:02 | sean-k-mooney | well we dont update all the xml on live migration | |
| 15:40:05 | sean-k-mooney | we just do a diff | |
| 15:40:12 | sean-k-mooney | and modify what needs to be chagned | |
| 15:40:23 | umbSublime | Ok that explains it ! | |
| 15:40:26 | sean-k-mooney | but a hard reboot of the vm would loose any manual change you make | |
| 15:40:32 | umbSublime | yes absolutely | |
| 15:40:48 | sean-k-mooney | dumb question but do you neeed live migration. or would cold migration be enough | |
| 15:40:53 | umbSublime | this was just for testing purposes | |
| 15:41:23 | umbSublime | live-migration would really be the ideal goal (we need sometimes to update compute hosts without impacting guests) | |
| 15:42:06 | umbSublime | If this really isn't possible at the moment, I'm sure that aspect could be discussed internally | |
| 15:42:08 | sean-k-mooney | i think unfotuetly the best path forwad would be to add this as a new feature | |
| 15:42:15 | sean-k-mooney | what release of openstack are you running | |
| 15:43:13 | umbSublime | Currently that stack is on Queens | |
| 15:44:24 | sean-k-mooney | this is the only reference to invtsc in the nova code base | |
| 15:44:26 | sean-k-mooney | https://github.com/openstack/nova/blob/1ec87c227c4a506ddc3355108a95390cdaa57d62/nova/conf/libvirt.py#L608-L613 | |
| 15:45:32 | umbSublime | hehe yes, that's the only one I found also. Most of the work to make exposing that flag to the guest possible was made in qemu/libvirt projects. There many discussions in the MLs about this | |
| 15:46:21 | sean-k-mooney | umbSublime: can i suggest you file a bluepirnt for this. or maybe a bug fix. | |
| 15:47:25 | sean-k-mooney | i would like to call it a bug so it could be backproted but its really a feautre since invtsc frequesce configuration was never previously supproted so it shoudl be a specless blueprint not a bug | |
| 15:49:45 | umbSublime | sean-k-mooney: I'll add it to my todo list :) (I've never done any of those, but would gladly do it) | |
| 15:50:20 | umbSublime | Thanks a lot helping me out :) really appreciated | |
| 15:51:05 | sean-k-mooney | no worries. sorry i could not point you at an exisitng knob to tweak | |
| 15:51:30 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Stop converting Migration objects to dicts for migrate_instance_start https://review.opendev.org/690115 | |
| 15:59:49 | mriedem | dansmith: this is the requirements bump you'll need for your osc change https://review.opendev.org/#/c/690097/ | |
| 15:59:57 | mriedem | but the latest novaclient is breaking horizon... | |
| 15:59:59 | dansmith | mriedem: woot, I was just looking for it | |
| 16:00:05 | dansmith | guh | |
| 16:02:32 | openstack | Launchpad bug 1849351 in OpenStack Dashboard (Horizon) "horizon doesn't work with python-novaclient 16.0.0" [Undecided,New] | |
| 16:02:32 | mriedem | https://bugs.launchpad.net/horizon/+bug/1849351 | |
| 16:04:36 | cdent | I think I know that bug. That's the "horizon needs more maintainers" bug. | |
| 16:04:37 | dansmith | mriedem: do you know what the deal is? brin's list of changes does't include anything that pops out at me | |
| 16:04:43 | mriedem | https://docs.openstack.org/releasenotes/python-novaclient/unreleased.html#upgrade-notes | |
| 16:04:56 | mriedem | we removed the list_extensions stuff | |
| 16:05:13 | mriedem | which horizon uses here https://github.com/openstack/horizon/blob/master/openstack_dashboard/api/nova.py#L1018 | |
| 16:05:30 | mriedem | i'm guessing to enable/disable features based on extensions but since i think liberty everything is enabled and non-configurable | |
| 16:05:34 | mriedem | unless you have out of tree hacks | |
| 16:05:42 | mriedem | anyway, i'm cloning horizon to hack on it | |
| 16:05:43 | dansmith | oh this is more of a bump for them just the latest novaclient release? | |
| 16:06:13 | mriedem | horizon just has to remove usage of deprecated novaclient code that is now gone | |
| 16:07:35 | dansmith | novaclient hash 364cad41912e2c0f99a30f78b2835f3480a18d6e is what I was looking for | |
| 16:07:40 | dansmith | I guess that's why we went from 15 to 16 | |
| 16:07:44 | mriedem | efried: ooo hot damn this is working now https://review.opendev.org/#/c/683988/ | |
| 16:07:56 | mriedem | dansmith: that and the cells v1 things that were removed also yeah | |
| 16:08:03 | dansmith | ack | |
| 16:08:13 | efried | \o/ | |
| 16:13:07 | mriedem | since stephen is MIA until after the summit, i wonder if we should move py27 jobs to experimental so we can run them on-demand but not automatically on every change? | |
| 16:13:27 | mriedem | that would at least be py27 and functional | |
| 16:13:41 | mriedem | maybe that's not worth the trouble right now | |
| 16:14:07 | cdent | why bother? | |
| 16:14:07 | dansmith | shouldn't we change functional-py3whatever to functional and functional to functional-py27 ? | |
| 16:14:19 | cdent | that was in response to mriedem not dansmith | |
| 16:14:21 | dansmith | just to make it default for people running local? | |
| 16:14:27 | dansmith | cdent: I know :) | |
| 16:14:49 | mriedem | dansmith: stephen's patch might do that...i'd have to check. but at some point i'm guessing we'll have functional-py37 too | |
| 16:14:55 | mriedem | i think that's already in experimental | |
| 16:15:11 | mriedem | yup https://github.com/openstack/nova/blob/master/.zuul.yaml#L97 | |
| 16:15:23 | dansmith | mriedem: I've been typing py27 for unit for ten years almost, it's annoying to have the target name change every time we bump py3 versions | |
| 16:15:36 | dansmith | so I'd love to have a -efunctional and -eunit so I don't have to track | |
| 16:15:54 | sean-k-mooney | dansmith: we did that in os-vif | |
| 16:15:59 | mriedem | i've been typing -e functional-py36 for years now b/c that's the one i always broke :) | |