| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-06-04 | |||
| 17:59:46 | sean-k-mooney | which woudl depend on the rpc call | |
| 17:59:54 | sean-k-mooney | for spwan i guess put the vm to error state? | |
| 18:00:11 | sean-k-mooney | not really sure about what we woudl do for anything else | |
| 18:01:10 | melwitt | ah so this is more complicated than I thought | |
| 18:01:24 | melwitt | I thought all we'd have to do is set the flag and then oslo.messaging would reconnect us and retry or something like that | |
| 18:01:37 | sean-k-mooney | the issue is that the queue that the compute agent created nolonger exsits but the compute agent does not know that. so unless we had the compute agent call itself on that queue in a heart beat we would not know we had to recreate the queue | |
| 18:02:03 | sean-k-mooney | melwitt: im not sure if we can create the queue or if it has to be the compute agent | |
| 18:02:17 | sean-k-mooney | the compute agent normally creates the queue when it connect to rabbitmq | |
| 18:02:40 | sean-k-mooney | which is why restarting the compuate agent fixes the problem | |
| 18:03:23 | sean-k-mooney | so at a minium we can log that the agent is unreachable. we could even mark the agent as down but that leaves the question of who will make it up again once its restarted | |
| 18:03:31 | melwitt | yeah. well, then I'd wonder if we could put try-except in the nova/rpc layer that will create the queue if we get MessageUndeliverable? that way whoever gets the "no queue" state will recreate it? | |
| 18:04:27 | sean-k-mooney | if we had a periodic task in teh compute agent that sent to its own queue then that would work | |
| 18:04:38 | sean-k-mooney | it might work if the conductor or api created it too | |
| 18:05:00 | sean-k-mooney | i just dont know if the compute agent will start litening to the queue when its created by someone else | |
| 18:05:41 | melwitt | yeah I guess I'm thinking it wouldn't matter who creates the queue. like if nova-conductor gets MessageUndeliverable trying to talk to compute, assume the queue is gone and recreate it, and resend | |
| 18:05:47 | melwitt | oh, right | |
| 18:05:50 | melwitt | hm | |
| 18:06:22 | sean-k-mooney | that is why i was suggesting havign the compute agent send to its own queue | |
| 18:06:30 | sean-k-mooney | jsut a simple ping/heatbeat | |
| 18:06:40 | melwitt | periodic does sound like it would work but that doesn't help someone trying to boot an instance and it fails due to this | |
| 18:07:01 | melwitt | but of course better than what we have today | |
| 18:07:02 | sean-k-mooney | we could retry after a short interval? | |
| 18:07:13 | sean-k-mooney | hoping the agent fixes its self? | |
| 18:08:08 | sean-k-mooney | maybe if we have time tommorow and ben or some of the oslo folks are aroudn we could ask them | |
| 18:08:36 | sean-k-mooney | our i guess i could try it locally | |
| 18:08:44 | melwitt | yeah... I don't love that idea but yeah, would take some thought on some options | |
| 18:08:51 | sean-k-mooney | i could use the rabbitmq gui to delete the queue manually | |
| 18:09:49 | sean-k-mooney | i dont think the manamge ui is isntall by devstack by default but i dont think its that hard to enable | |
| 18:10:15 | sean-k-mooney | if i could repoduce it manually then we could test teh recreate behavior | |
| 18:10:36 | sean-k-mooney | e.g. if i make the condutor recreate it and i still can boot a vm i know it need to be the agent | |
| 18:10:41 | melwitt | right | |
| 18:11:53 | melwitt | yeah doing that would demystify a lot. but yeah gonna be a pain/not ideal if nova-compute would need to re-init to pick up the new queue | |
| 18:14:11 | sean-k-mooney | im going to go figure out what im doing for dinner and then i might give it a try although i kind of want to look into it someother time | |
| 18:14:28 | sean-k-mooney | can you bug me about this if you remember tommorow or next week | |
| 18:14:54 | bnemec | If you have messaging stuff to talk about I would suggest pinging kgiusti to make sure he's available. | |
| 18:14:56 | melwitt | sure. I'll try to write something on the launchpad bug so I don't forget about this a sixth time or whatever we're at now | |
| 18:15:22 | melwitt | in addition | |
| 18:15:45 | sean-k-mooney | bnemec: well we are just trying to figure out how to use the mandataory flag that ye enabled via the transport options correctly | |
| 18:16:16 | sean-k-mooney | but my rabbit mq knoladge is really not good enough to have an intuition about this | |
| 18:16:20 | melwitt | use it + handle the exception that will raise as a result of setting it | |
| 18:16:38 | melwitt | *set it | |
| 18:17:14 | sean-k-mooney | melwitt: hehe yep 6th might be a bit much but at least thrid or forth :) | |
| 18:17:24 | melwitt | :) | |
| 18:19:31 | sean-k-mooney | we finally merge the patch t silance the amqp heart beat error by the way | |
| 18:19:48 | melwitt | #success | |
| 18:19:53 | openstackstatus | melwitt: Added success to Success page (https://wiki.openstack.org/wiki/Successes) | |
| 18:20:04 | melwitt | lol oops | |
| 18:20:36 | sean-k-mooney | hehe | |
| 18:20:59 | sean-k-mooney | https://review.opendev.org/#/c/724188/5 | |
| 18:21:41 | sean-k-mooney | ok time for food o/ | |
| 18:21:57 | melwitt | o/ | |
| 18:22:44 | melwitt | I forgot about the hashtag success being a thing 😬 | |
| 18:22:44 | melwitt | I forgot about the hashtag success being a thing 😬 | |
| 18:56:41 | dansmith | melwitt: from a quick scan, nobody has ever made that mistake in the history of the hashtag | |
| 18:57:09 | dansmith | except for the second most recent :P | |
| 18:57:38 | dansmith | (three days go) | |
| 19:30:08 | melwitt | dansmith: I noticed that. go me | |
| 19:30:57 | melwitt | #special | |
| 21:23:52 | openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Routed network scheduling spec https://review.opendev.org/733703 | |
| #openstack-nova - 2020-06-05 | |||
| 02:29:27 | xiaolin | kashyap: gibi: thanks for the help. It works! The error "unexpected host-model" disappear, but I got a new error "Instance failed to spawn: UEFINotSupported: UEFI is not supported". In order to successfully run the VM on my host with MIPS architecture, I made a patch for nova/virt/libvirt/driver.py. Need I report the bug? | |
| 07:27:06 | bauzas | good morning Nova | |
| 07:34:11 | alex_xu | bauzas: good morning | |
| 07:35:18 | bauzas | alex_xu: hope you're all good after 2 days of very late meetings | |
| 07:37:57 | alex_xu | bauzas: thanks, totally better than long flight :) | |
| 07:50:17 | aarents | Hi nova | |
| 07:51:25 | aarents | bauzas: can we flag this abondonned: https://blueprints.launchpad.net/nova/+spec/include-swap-memory i'm not sure I can do it myself | |
| 07:51:42 | bauzas | aarents: ack, looking | |
| 07:52:47 | bauzas | aarents: I can't either | |
| 07:52:53 | bauzas | it's a blueprint | |
| 07:52:57 | bauzas | just leave a comment | |
| 07:53:09 | aarents | k | |
| 08:35:06 | kashyap | xiaolin: Hi | |
| 08:35:59 | kashyap | xiaolin: I don't think anyone ever tested Nova with MIPS arch, nor is it officially supported. I'm surprised you've got this far. For the UEFI bits - need to dig more | |
| 08:36:12 | kashyap | xiaolin: Does your patch to libvirt/driver.py fix it? Are you able to boot an instance? | |
| 08:41:36 | xiaolin | kashyap: Yes, it fixed, I can boot an instance. | |
| 08:49:09 | gibi | xiaolin: please file bug an publish the patch connected to that bug if possible | |
| 08:49:35 | xiaolin | Many people around me use hosts with MIPS architecture. I hope the official can add support for MIPS and We can maintain it. Is there a plan? | |
| 08:51:06 | kashyap | xiaolin: Yeah, please do file a bug | |
| 08:53:04 | CeeMac | melwitt: So i did some experimenting with the nova os-simple-tenant-usage api and this pulls back pretty much the exact same usage information as horizon project usage | |
| 08:53:39 | CeeMac | so, sadly, it looks like regardless of if an instance is shelved or not, it still consumes resources :( | |
| 08:53:57 | kashyap | xiaolin: Hmm, for MIPS, the issue is maintainers that will stick around and address the issues :-) | |
| 08:54:12 | kashyap | xiaolin: It's easy to "add initial support" (and let it rot) :-( | |
| 08:54:39 | kashyap | xiaolin: But go ahead and file a bug as gibi suggests, and attach your patch as a comment | |
| 08:57:42 | CeeMac | melwitt: I also don't see any image/snapshot for the shelved instance, should that be visible? | |
| 08:58:02 | CeeMac | s/shelved/shelved_offloaded | |
| 09:00:32 | xiaolin | gibi: kashyap: Thanks for the help. We will stick, as our company produce machines with MIPS architecture, and we believe our company will not go bankrupt :-) | |
| 09:03:15 | aarents | gibi Thanks for this bug confirmation https://bugs.launchpad.net/nova/+bug/1881881 | |
| 09:03:15 | openstack | Launchpad bug 1881881 in OpenStack Compute (nova) "nova api return HTTP 500 error when interface attach failed due to lack of instance PCI slots" [Undecided,New] | |
| 09:03:52 | aarents | gibi So according the thread you're reffering, we can extend the number of pluggable interface by adding pci-bridge, is nova able to do it now? or should we implement something like hw:pci-bridge=2 in order to ask libvirt to add 2 pci-brigde when defining instance. | |
| 09:05:14 | kashyap | xiaolin: :-) Would also be nice to start a discussion on 'openstack-discuss' list -- do you mind starting a thread? | |
| 09:05:38 | kashyap | xiaolin: Put a subject like "[nova][libvirt] Support for MIPS architecture?" or something like that | |
| 09:13:42 | xiaolin | kashyap: It would be nice, but I don`t know how to start a discussion temporarily | |
| 09:13:57 | kashyap | xiaolin: I'm on a call; but it's a mailing list | |
| 09:14:11 | kashyap | I'll link you shortly, unless someone beats me to it | |
| 09:25:52 | frickler | xiaolin: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss , just send a mail with the subject above. you may want to subscribe before to avoid your mail being held in moderation | |
| 09:26:21 | gibi | aarents: I don't think nova can add pci-bridges today. If the limit can be avoided by the q35 machine type then I would take that direction instead | |
| 09:28:32 | bauzas | I do agree with gibi | |
| 09:28:39 | bauzas | the 'pc' machine type is old | |
| 09:28:47 | bauzas | very old, even | |
| 09:28:56 | bauzas | and we shouldn't modify nova because of it | |