| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-09 | |||
| 14:07:23 | gibi | when I set the allocation ratio to lower to induce overallocation | |
| 14:15:27 | opendevreview | Balazs Gibizer proposed openstack/placement master: DNM: extra logs to troubleshoot overcapacity https://review.opendev.org/c/openstack/placement/+/808083 | |
| 15:33:10 | sean-k-mooney | ah nice | |
| 15:47:17 | sean-k-mooney | although if the compute agent is down you might not see that | |
| 17:03:11 | gibi | elodilles_pto: there is probably a stable only bug here https://bugs.launchpad.net/nova/+bug/1941819 but as it is probably only affect stein and older which are in EM I don't think I will spend time fixing it. Maybe the bug author try it. | |
| 18:40:37 | legochen | hey nova experts, one question - can someone point me the best practice about configuring nova-scheduler filter to distribute VMs to user-specified cabinets? | |
| 18:41:31 | dansmith | user-specified, meaning "at boot a user specifies where this should go" ? | |
| 18:41:35 | dansmith | or did you mean user-specific? | |
| 18:42:05 | legochen | user-specified, meaning "at boot a user specifies where this should go" ? <= yes | |
| 18:42:24 | dansmith | in general this is not a thing nova allows or intends to allow, with one exception: AZs | |
| 18:42:42 | dansmith | users can choose AZs, so if you want them to be able to choose, make AZs for them to specify | |
| 18:43:00 | dansmith | you can do an AZ per site, or per aisle, or per rack or something | |
| 18:43:58 | legochen | For example, I have multiple cabients in data center, users want to distribute their VMs to different cabinets equally in order to avoid SPOF of tor switch or power stuff. | |
| 18:44:15 | dansmith | that's what AZs are for | |
| 18:44:32 | legochen | hmm, per cabinet per AZ seems not that reasonable to me :( | |
| 18:49:23 | legochen | I was thinking to configure per aggregate per cabinet. And set property - cabient=A for aggregate A, cabinet=B for aggregate B…. then users can specify —hint cabinet=A while creating a VM. | |
| 18:49:50 | dansmith | that's what AZs are for | |
| 19:18:09 | opendevreview | xiaoxin yang proposed openstack/nova master: Secure boot requires SMM feature enabled https://review.opendev.org/c/openstack/nova/+/808126 | |
| #openstack-nova - 2021-09-10 | |||
| 02:12:18 | opendevreview | xiaoxin yang proposed openstack/nova master: Secure boot requires SMM feature enabled https://review.opendev.org/c/openstack/nova/+/808126 | |
| 02:41:15 | opendevreview | Jorhson Deng proposed openstack/nova master: remove some redundant parameters in migrate_server https://review.opendev.org/c/openstack/nova/+/808143 | |
| 06:08:09 | opendevreview | Jorhson Deng proposed openstack/nova master: remove some redundant parameters in migrate_server https://review.opendev.org/c/openstack/nova/+/808143 | |
| 07:00:49 | admin1 | seeking info on what happens during a migration .. say i have h1 ( hypervisor1 ) and h2 ( hypervisor2 ) .. when I want to migrate vm1 on h1 -> h2, what makes the ssh conection .. is it h1 nova -> h2 nova .. or h1 libvirt -> h2 libvirt or h1 root -> h2 root .. | |
| 07:24:31 | gibi | bauzas: o/ friendly reminder that we need to land melwitt's placemnt fix in Xena RC1 https://review.opendev.org/c/openstack/placement/+/807014 | |
| 07:25:08 | gibi | melwitt, dansmith: we are out of EU cores for now, so could you please look at a small doc patch before RC1 https://review.opendev.org/c/openstack/nova/+/807564 ? | |
| 07:25:38 | gibi | melwitt, dansmith: also the reno prelude is ready https://review.opendev.org/c/openstack/nova/+/807786 | |
| 07:56:14 | bauzas | gibi: yeah I know and I started to review it | |
| 07:56:26 | bauzas | but y'know how it's a bit difficult | |
| 07:56:42 | bauzas | + loving escalations during the last 2 days :( | |
| 07:57:17 | gibi | bauzas: thanks. feel free to ask questions about the patch I should be able to answer them | |
| 07:57:21 | gibi | ahh escalations. :/ | |
| 07:59:11 | bauzas | don't know what happended | |
| 07:59:31 | bauzas | maybe because of the moon or because of puppets | |
| 07:59:43 | bauzas | but looks like we have a lot of customers asking us to help them | |
| 08:00:01 | gibi | maybe they are back from summer vacation :) | |
| 08:01:16 | bauzas | they're not French :p | |
| 08:08:41 | gibi | then they are just nice and waited for the French to return from their summer vacation :P | |
| 08:28:07 | admin1 | one more question .. if cold migration works but live does not -- ceph backend .. where do i need to start looking .. the nova logs ( debug true ) does not provide any actionable insight .. | |
| 09:30:42 | gibi | bauzas: do you agree with my triage here https://bugs.launchpad.net/nova/+bug/1939856 that it is probably a new feature not a bugfix? | |
| 09:34:44 | bauzas | gibi: fwiw, I +wd https://review.opendev.org/c/openstack/nova/+/792357 | |
| 09:35:04 | gibi | bauzas: thanks | |
| 09:35:08 | bauzas | so we can start to use the new label | |
| 09:35:28 | bauzas | but then I'll provide a new change explaining what I'd like to have to helping contributors | |
| 09:35:40 | gibi | OK, I will check that | |
| 09:36:46 | bauzas | gibi: about the bug you asked me | |
| 09:37:06 | bauzas | it's honestly a bug but Wishlist looks good to me | |
| 09:37:40 | bauzas | as you said, given the instance was offloaded from the compute, it's no longer possible to ask nova to wait for deleting it | |
| 09:42:50 | gibi | I would be affraid to change the delete behavior of offloaded instances as the current behavior was there forever and it is not really leaking any resources (I checked ports and volumes are cleaned up) | |
| 09:43:01 | gibi | so probalby people out there are adapted to this | |
| 09:51:19 | bauzas | gibi: we can fix this by documenting :) | |
| 09:51:57 | gibi | I can accept a documentation patch for it sure | |
| 10:13:05 | kevko | type for 'vhost', expected: boolean | |
| 10:13:05 | kevko | Hi folks, in victoria I am hitting this -> /var/log/kolla/nova/nova-compute.log:2021-09-10 09:49:09.815 8 ERROR nova.virt.libvirt.driver [req-8082a663-ca30-4845-9e93-2429c500ba76 31031550a7c94928abbd9dfa8634fba1 7b2ac086026741aa82c6c9c96ded42ee - default default] [instance: cf3ae6f9-a971-4123-ae73-9b597364985a] attaching network adapter failed.: libvirt.libvirtError: internal error: unable to execute QEMU command 'netdev_add': Invalid parameter | |
| 10:13:18 | kevko | is it bug ? | |
| 10:25:05 | sean-k-mooney | kevko: can you past the libvirt xml somewhere so we can see what was generated | |
| 10:25:19 | sean-k-mooney | also what libvirt verions are you useing | |
| 10:28:54 | kevko | sean-k-mooney: https://paste.opendev.org/show/809223/ | |
| 10:28:57 | kevko | debian version | |
| 10:28:59 | kevko | buster one | |
| 10:32:07 | sean-k-mooney | that meets our min libvirt if only barely https://github.com/openstack/nova/blob/stable/victoria/nova/virt/libvirt/driver.py#L225 | |
| 10:32:40 | sean-k-mooney | can you provide teh xml that was generated so we can see where the type element was added | |
| 10:32:53 | sean-k-mooney | i assume this was an interface attach? | |
| 10:33:09 | kevko | sean-k-mooney: well, i'm running tempests against stack .. if i turn on debug .. will i see xml ? | |
| 10:33:16 | kevko | sean-k-mooney: yeah, interface attach | |
| 10:33:33 | opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [PoC][yoga] Off-path Networking Backends Support https://review.opendev.org/c/openstack/nova/+/808199 | |
| 10:33:41 | sean-k-mooney | yes if you trun on debug you see a about a 8 line segment of xml printed for attach | |
| 10:34:10 | sean-k-mooney | we print the full xml when startign the vm and just the bit we are appending for attach | |
| 10:34:55 | kevko | ok, give me 5 minutes cca | |
| 10:37:46 | kevko | sean-k-mooney: https://paste.opendev.org/show/809224/ | |
| 10:39:15 | kevko | sean-k-mooney: bigger log -> https://paste.opendev.org/show/809225/ | |
| 10:39:48 | sean-k-mooney | that is odd that looks normal to me | |
| 10:40:05 | sean-k-mooney | oh what is your qemu version | |
| 10:40:26 | sean-k-mooney | is it below 4.0.0 | |
| 10:40:46 | kevko | https://paste.opendev.org/show/809226/ | |
| 10:40:54 | kevko | nope, it's 5.2 | |
| 10:41:14 | sean-k-mooney | ok i was wondering if it was older then libvirt suported | |
| 10:41:40 | sean-k-mooney | thye xml we are generating looks corect to me | |
| 10:41:58 | sean-k-mooney | so i suspect the issue is in the qemu args that libvirt is creeating | |
| 10:42:12 | sean-k-mooney | have you looked in the qemu instance long | |
| 10:42:15 | sean-k-mooney | *log | |
| 10:42:26 | sean-k-mooney | in /var/log/libvirt/qemu/instace* | |
| 10:42:35 | sean-k-mooney | is there any lower level error there | |
| 10:42:56 | kevko | let me check | |
| 10:45:57 | kevko | sean-k-mooney: nothing https://paste.opendev.org/show/809227/ | |
| 10:46:44 | sean-k-mooney | odd i dont even see thte attach | |
| 10:46:57 | sean-k-mooney | although it might not log that if it does not work | |
| 10:49:00 | sean-k-mooney | kevko: other then comparing the full xml to the fragment to see if there is any delta (comparing to the exising interface) | |
| 10:49:14 | sean-k-mooney | im not sure we can be much more help this looks like a libvirt bug | |
| 10:49:40 | kevko | i'm trying to replace victoria libvrit container for wallaby | |
| 10:49:52 | kevko | let me check what will happen :P | |
| 10:50:44 | sean-k-mooney | it likely should work nova support a fairly wide range | |
| 10:50:53 | sean-k-mooney | and libvirt has a pretty stable api | |
| 10:55:14 | kevko | sean-k-mooney: success | |
| 10:55:18 | kevko | sean-k-mooney: it's working | |
| 10:55:24 | kevko | nice | |
| 10:55:36 | gibi | hm, this sounds like https://bugzilla.redhat.com/show_bug.cgi?id=1835312 | |
| 10:55:36 | kevko | so minimal version 5.0.0 is buggy | |
| 10:55:45 | gibi | that points to https://bugzilla.redhat.com/show_bug.cgi?id=1832710 | |
| 10:55:54 | gibi | which has a lot of libvirt commits to fix | |
| 10:56:29 | gibi | so yeah, possible libvirt - qemu incompatibility | |
| 10:57:06 | kevko | aaaaa | |