| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-05 | |||
| 20:15:40 | cfriesen_ | efried: esberglu: it appears that the return code is being mapped to the process exit value. it must be doing something clever. In any case, I opened https://bugs.launchpad.net/nova/+bug/1695965 because the return code for "nova-manage db online_data_migrations" isn't very helpful. | |
| 20:15:40 | openstack | Launchpad bug 1695965 in OpenStack Compute (nova) ""nova-manage db online_data_migrations" exit code is strange" [Undecided,New] | |
| 20:16:37 | efried | cfriesen_ Cool, esberglu just made a change on a similar theme: https://review.openstack.org/#/c/467218/ | |
| 20:16:56 | efried | cfriesen_ See bug https://bugs.launchpad.net/nova/+bug/1692982 | |
| 20:16:56 | openstack | Launchpad bug 1692982 in OpenStack Compute (nova) "Need --strict option for nova-manage cell_v2 discover_hosts" [Wishlist,Fix released] - Assigned to Eric Berglund (esberglu) | |
| 20:18:09 | cfriesen_ | efried: right. In the case of online_data_migrations we really want to distinguish between three separate scenarios, which doesn't map nicely to binary exit values | |
| 20:18:22 | efried | cfriesen_ In your case, it's possible the answer might be, "code to the API, not the CLI" ;-) | |
| 20:18:31 | efried | But I wouldn't be the one to advocate that. | |
| 20:19:32 | cfriesen_ | efried: heh...one nasty case is that the newton version of this command returns this "2 rows matched query pcidevice_online_data_migration, 0 migrated" even when there is no work left to do. | |
| 20:19:44 | cfriesen_ | outputs that, rather | |
| 20:20:01 | cfriesen_ | because the query is too generic. | |
| 20:21:37 | efried | cfriesen_ FWIW, dansmith was the one in our corner on that --strict thing. He had very useful feedback when the idea came up. | |
| 20:24:37 | cfriesen_ | dansmith: when you get a moment could you look at bug 1692982 ? | |
| 20:24:37 | openstack | bug 1692982 in OpenStack Compute (nova) "Need --strict option for nova-manage cell_v2 discover_hosts" [Wishlist,Fix released] https://launchpad.net/bugs/1692982 - Assigned to Eric Berglund (esberglu) | |
| 20:25:07 | cfriesen_ | dansmith: whoops, I meant bug 1695965 | |
| 20:25:07 | openstack | bug 1695965 in OpenStack Compute (nova) ""nova-manage db online_data_migrations" exit code is strange" [Undecided,New] https://launchpad.net/bugs/1695965 | |
| 20:25:47 | esberglu | efried: cfriesen_: Sorry was off for a few there. I think I dropped for some of the convo. You still need anything from me? | |
| 20:26:16 | cfriesen_ | esberglu: just wondering if you knew offhand how the return value of "nova-manage" gets converted to process exit value | |
| 20:26:17 | dansmith | cfriesen_: we had migrations in the past that raised an exception when something wasn't new enough, but you needed to proceed when in order to get to the newer stuff, hence the zero return in that case | |
| 20:26:37 | dansmith | thats why it was like that | |
| 20:26:43 | dansmith | but I don't disagree with nailing it down as you describe | |
| 20:27:22 | cfriesen_ | dansmith: also, in Newton you can't even tell from the output logs whether or not you need to call it again, because of the PCI issue I showed above | |
| 20:27:44 | cfriesen_ | dansmith: the "count_all" is counting cases where there is no migration to be done | |
| 20:28:11 | esberglu | efried: Not sure. I just took if granted based on other commands | |
| 20:47:41 | cfriesen_ | dansmith: related issue: https://bugs.launchpad.net/nova/+bug/1695991 | |
| 20:47:41 | openstack | Launchpad bug 1695991 in OpenStack Compute (nova) ""nova-manage db online_data_migrations" doesn't report matched/migrated properly " [Undecided,New] | |
| 20:54:04 | mriedem | dansmith: i think we've touched on this before, but since services are restricted to cell dbs, we don't have a global view of what a minimum compute version is in a deployment once we have multpile cells | |
| 20:54:06 | mriedem | thinking of https://github.com/openstack/nova/blob/78c69f61ab186c3d3eb0fe5cc5114cc71f0fbdad/nova/objects/service.py#L425 | |
| 20:54:36 | dansmith | mriedem: I have a helper | |
| 20:54:40 | mriedem | if we have an instance we can find the cell it's in and get the minimum compute for that cell, or get the service version for the compute the instance is on | |
| 20:55:59 | cfriesen_ | how is Service._check_minimum_version() going to work with cells then? check just the current cell? | |
| 20:56:16 | mriedem | well we could certainly iterate cells, | |
| 20:56:22 | mriedem | find the min for $service in all cells, | |
| 20:56:27 | mriedem | and return the lowest | |
| 20:56:42 | dansmith | mriedem: I'm sure I already wrote that helper.. looking | |
| 20:57:06 | mriedem | i randomly think about this because we don't really test for it in our grenade tests | |
| 20:57:12 | mriedem | since grenade is so limited in scope to smoke tests | |
| 20:57:59 | mriedem | e.g. https://github.com/openstack/nova/blob/78c69f61ab186c3d3eb0fe5cc5114cc71f0fbdad/nova/api/openstack/compute/servers.py#L496-L499 | |
| 20:58:06 | mriedem | ^ won't be accurate with multiple cells | |
| 20:58:24 | mriedem | i'll open a bug to track this | |
| 20:58:37 | dansmith | and unless we had multiple real cells we wouldn't hit it anyway | |
| 20:58:53 | mriedem | we had == in ci? | |
| 20:59:27 | dansmith | yeah | |
| 20:59:33 | mriedem | i thought we were going to have the/a multinode job have 2 cells with 1 compute each? | |
| 20:59:44 | dansmith | yeah but not one downlevel | |
| 20:59:49 | dansmith | not yet anyway | |
| 20:59:51 | mriedem | sure, but we could with grenade | |
| 20:59:52 | mriedem | eventually | |
| 20:59:55 | dansmith | eventually | |
| 21:00:07 | mriedem | but then we're stuck back with the grenade not running interesting tests issue | |
| 21:00:18 | dansmith | that helper doesn't appear to be in tree, but I know I wrote it.. might've been for a thing that we ended up not doing and so never committed it | |
| 21:00:44 | dansmith | don't we run smoke first and then full tempest post-upgrade? | |
| 21:00:49 | mriedem | nope | |
| 21:00:50 | mriedem | smoke both ways | |
| 21:00:55 | sdague | no, it's smoke on both sides | |
| 21:01:11 | mriedem | smoke'em if you got'em | |
| 21:01:54 | dansmith | okay | |
| 21:02:03 | sdague | honestly, for the multi node, multi cell version testing, we should probably purpose build something for that. Because it feels like we want to be pushing very specific edges on test cases | |
| 21:02:45 | dansmith | you mean a specific list of tests? | |
| 21:02:50 | mriedem | dansmith: https://bugs.launchpad.net/nova/+bug/1695993 | |
| 21:02:50 | openstack | Launchpad bug 1695993 in OpenStack Compute (nova) "supports_device_tagging check in /servers API won't work with multiple cells" [High,Triaged] | |
| 21:03:00 | mriedem | dansmith: i think he means not grenade | |
| 21:03:13 | sdague | yeh, maybe not grenade either | |
| 21:03:15 | mriedem | sdague: you're talking about the ansible idea from the PTG? | |
| 21:03:21 | dansmith | hmm | |
| 21:03:26 | sdague | mriedem: I'm not honestly sure | |
| 21:03:34 | mriedem | i remember you saying something like that once | |
| 21:03:40 | mriedem | when i was monkeying with d-g and cells v2 setup | |
| 21:04:01 | sdague | it just feels like we might be better off building something that builds out the mixed cell version environment from scratch | |
| 21:04:17 | sdague | instead of glomming too much onto the existing stuff | |
| 21:04:35 | mriedem | might have also been when talking about what rolling upgrades look like for cells v2 | |
| 21:04:36 | dansmith | I'm not sure why | |
| 21:04:38 | mriedem | with doing api at the end | |
| 21:04:47 | mriedem | since devstack/grenade starts/upgrades API first? | |
| 21:04:55 | mriedem | it had something to do with the upgrade order for api | |
| 21:05:03 | mriedem | we have notes from the BCN summit | |
| 21:05:57 | mriedem | anyway, i'm not proposing we tackle testing mixed-level cells in ci right now | |
| 21:06:37 | dansmith | mriedem: I will keep looking for my multicell service lookup patch in a couple dead end branches, but if I don't find it I'll just write it.. it's not hard | |
| 21:06:56 | mriedem | sure - you want to take that bug? | |
| 21:07:02 | mriedem | or just going to provide the helper? | |
| 21:07:31 | dansmith | mriedem: lemme write the helper and I'll survey the other uses... might be easy for me to just fix them uses too | |
| 21:07:50 | mriedem | dansmith: here it is https://review.openstack.org/#/c/442787/1/nova/objects/service.py | |
| 21:08:05 | dansmith | haha | |
| 21:08:15 | dansmith | no wonder I couldn't find it. that patch is dead to me | |
| 21:08:19 | mriedem | i know | |
| 21:08:24 | dansmith | lol | |
| 21:08:42 | mriedem | i was looking at it on friday considering taking it over | |
| 21:09:04 | mriedem | anyway, we should split that piece out into a separate change, and then the bug fix stuff on top probably | |
| 21:09:07 | dansmith | I've been working on sql injection attacks at gerrit so I can reassign it to you | |
| 21:09:08 | dansmith | yeah | |
| 21:09:10 | dansmith | I shall | |
| 21:28:49 | edleafe | jaypipes: dansmith: cdent: Started a thread on the ML: http://lists.openstack.org/pipermail/openstack-dev/2017-June/117913.html | |
| 21:29:02 | jaypipes | edleafe: thx ed | |
| 21:35:24 | openstackgerrit | Steve Noyes proposed openstack/nova master: WIP: update live migration to use v3 cinder api https://review.openstack.org/463987 | |
| 21:39:35 | openstackgerrit | Steve Noyes proposed openstack/nova master: objects: Add old_attachment_id to BlockDeviceMapping https://review.openstack.org/471142 | |
| 21:40:24 | mriedem | stvnoyes: that's not going to work | |
| 21:40:32 | stvnoyes | no? | |
| 21:40:42 | mriedem | primarily because you haven't added a column to the block_device_mappings table | |
| 21:41:28 | mriedem | however, we should talk to johnthetubaguy about this since i'd really hope we could avoid something like this | |
| 21:41:43 | stvnoyes | I was following this pattern that did somethng about the same - https://review.openstack.org/#/c/437665/ | |