| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-05 | |||
| 19:10:33 | tssurya | dansmith: I will try to go through all the nova list args and see | |
| 19:10:50 | dansmith | tssurya: I think you probably will just have to consider the appropriate behavior for each | |
| 19:10:52 | mgagne | johnthetubaguy, gmann: I greatly appreciate the work you have done with policy deprecation warnings. it's of great help when upgrading ++ | |
| 19:11:02 | dansmith | for most, shell instances or omission is the best idea I think, | |
| 19:11:05 | mriedem | i assume that filtering on az would be the same as status - we can't know, so ignore those results in a down cell | |
| 19:11:07 | dansmith | 500 for marker | |
| 19:11:15 | tssurya | dansmith: ack, | |
| 19:11:31 | dansmith | mriedem: oh right, duh, I forgot we couldn't know the instances in there | |
| 19:11:41 | dansmith | mriedem: we know hosts and instances in hosts, so we could work it out I guess | |
| 19:12:04 | mriedem | i don't think we know instances in hosts from the api | |
| 19:12:10 | dansmith | er, no | |
| 19:12:13 | dansmith | right | |
| 19:12:46 | dansmith | I was thinking instance->cell->host->az, but that's not fine-grained enough | |
| 19:12:52 | dansmith | obvi | |
| 19:12:55 | tssurya | mriedem: right so we print one shell for the down cell and print the results from the rest | |
| 19:13:06 | tssurya | this way users can know that the result is not fully right | |
| 19:13:17 | dansmith | hmm? | |
| 19:13:23 | dansmith | one shell per down cell? | |
| 19:13:39 | dansmith | I think you just have to leave out all shells for instances you can't tell are in an az | |
| 19:13:51 | tssurya | dansmith: oh okay | |
| 19:13:56 | tssurya | that wfm as well | |
| 19:14:05 | mriedem | would be helpful if we weren't talking about CLI UX wrt REST API semantics | |
| 19:15:20 | mriedem | would we ignore down cells if we're paging? let's say i've got 3 cells, A, B and C. my marker is in A, B is down, but C is up, so i get some instances from A, skip B and get the rest of the page from C with a new marker in C? | |
| 19:15:47 | dansmith | if you're doing no sort or filter, you could show B instances too | |
| 19:15:52 | dansmith | if you are, then you have to skip | |
| 19:17:18 | mriedem | ok posted some edge case stuff in the spec | |
| 19:17:46 | tssurya | mriedem: thanks, will have a look | |
| 19:17:49 | mriedem | also, i'm annoyed that i feel the need to qualify "edge case" lately as "not computers at the edge of a network, but rare scenarios" | |
| 19:18:04 | dansmith | heh | |
| 19:18:04 | tssurya | so we are going for a new API response ? | |
| 19:18:14 | melwitt | MFIN EDGE | |
| 19:18:16 | mriedem | tssurya: doesn't sound like it | |
| 19:18:49 | tssurya | and since we were on the topic of markers, I have this bug https://bugs.launchpad.net/nova/+bug/1773945 | |
| 19:18:50 | openstack | Launchpad bug 1773945 in OpenStack Compute (nova) "nova client servers.list crashes with bad marker" [Undecided,New] - Assigned to Surya Seetharaman (tssurya) | |
| 19:18:55 | tssurya | couldn't figure out what the issue is | |
| 19:19:10 | tssurya | if any of you have ideas | |
| 19:19:54 | mriedem | i saw something really weird the other day with markers | |
| 19:19:57 | mriedem | with osc | |
| 19:20:03 | mriedem | which uses nova python API bindings | |
| 19:20:26 | tssurya | oh | |
| 19:20:56 | mriedem | i was creating some instances when rabbitmq dropped | |
| 19:21:19 | mriedem | and i think it might have been what takashi said in the bug report, "if the record of the marker VM instance exists in the cell | |
| 19:21:19 | mriedem | but the 'cell_mapping' in the InstanceMapping of the marker VM instance is null (None), | |
| 19:21:19 | mriedem | the issue occurs." | |
| 19:21:54 | tssurya | but this would happen only during schedule time right ? | |
| 19:22:04 | tssurya | I thought takashi was right too at first | |
| 19:22:07 | mriedem | rabbit died during scheduling | |
| 19:22:26 | tssurya | ah :D | |
| 19:22:39 | mriedem | the weird thing is when i was debugging the CLI, it was passing a marker | |
| 19:22:46 | mriedem | and i don't know where that marker was coming from | |
| 19:22:58 | dansmith | tssurya: is that bug not just that the marker gets deleted during the paging? | |
| 19:23:21 | dansmith | oh, no, we do read_deleted=yes it | |
| 19:23:29 | tssurya | yea | |
| 19:24:04 | dansmith | so maybe we should check if the marker has a null cell mapping and log that case/ | |
| 19:24:10 | dansmith | either way, not much we can do there | |
| 19:24:29 | tssurya | dansmith: yep I agree, | |
| 19:24:53 | tssurya | I just never thought there would be case without any BuildRequest and NULL InstanceMapping | |
| 19:25:09 | dansmith | or, we just scan every cell! (no not really) | |
| 19:27:06 | tssurya | dansmith: I also wanted to ask another thing about have a new option --cell-name to nova service-list ; how you would feel about this, but we could also do this in tomorrow's cells meeting if you are busy now | |
| 19:27:29 | dansmith | tssurya: I don't want that, no | |
| 19:27:46 | dansmith | tssurya: put them in an aggregate and have a way to list via aggregate or something | |
| 19:28:01 | tssurya | right I thought so :), just tried my luck :D | |
| 19:28:04 | dansmith | or use nova-manage | |
| 19:28:33 | tssurya | yep, thanks! | |
| 19:29:47 | dansmith | gdi, why does zuul hate me? | |
| 19:30:09 | mriedem | tssurya: so this should only be called if the client passes in a marker https://github.com/openstack/nova/blob/f902e0d5d87fb05207e4a7aca73d185775d43df2/nova/compute/instance_list.py#L56-L73 | |
| 19:30:27 | mriedem | which when i ran openstack --debug service list, i saw it was passing a marker, but i wasn't | |
| 19:30:33 | tssurya | mriedem: yes | |
| 19:30:38 | dansmith | oh, it's python3 that hates me | |
| 19:30:38 | mriedem | so is there something in novaclient that always gets a marker? | |
| 19:30:47 | tssurya | oh no, we pass a marker | |
| 19:31:04 | mriedem | we? | |
| 19:31:20 | tssurya | I mean a marker is passed only if we give it via the CLI | |
| 19:31:50 | tssurya | what you are saying seems strange, but I haven't tested that | |
| 19:32:00 | tssurya | tested/observed | |
| 19:32:13 | melwitt | does getting MarkerNotFound mean that a marker had to have been passed? because the bug reporter was using servers.list() python API and got MarkerNotFound | |
| 19:32:46 | mriedem | right, and when i was doing openstack server list, i wasn't passing a marker, | |
| 19:32:48 | tssurya | melwitt: that's what I thought so too, | |
| 19:32:58 | mriedem | but i ran it with debug and saw it was doing GET /servers/ with a marker | |
| 19:33:00 | melwitt | that would mean that somewhere in servers.list() novaclient is automatically passing a marker. I guess it would if it were paging on its own | |
| 19:33:20 | mriedem | novaclient does do that | |
| 19:33:41 | mriedem | until it doesn't get any more results | |
| 19:33:44 | melwitt | yeah, that's what I'm referring to, the fact that it does that if the list is > api limit | |
| 19:33:54 | mriedem | i only had 2 instances | |
| 19:33:59 | melwitt | oh | |
| 19:34:08 | mriedem | i think it will happen at any point if you have one of these messed up mappings | |
| 19:34:31 | tssurya | mriedem: hmm yea, true at some point even we have NULL cell_mappings | |
| 19:34:51 | mriedem | https://github.com/openstack/python-novaclient/blob/10.2.0/novaclient/v2/servers.py#L863 | |
| 19:34:58 | melwitt | I missed why/how that is normal | |
| 19:35:28 | melwitt | if an instance is in the middle of being scheduled, it shouldn't have an instance_mapping yet | |
| 19:36:22 | tssurya | melwitt: it will have an instance_mapping but with a NULL cell_mapping | |
| 19:36:34 | mriedem | we create the empty instance mapping in the api | |
| 19:36:49 | melwitt | oh, I see. so any instance in the middle of scheduling would have the null marker | |
| 19:37:05 | melwitt | didn't realize that | |
| 19:37:10 | mriedem | https://github.com/openstack/nova/blob/f902e0d5d87fb05207e4a7aca73d185775d43df2/nova/compute/api.py#L913 | |
| 19:37:16 | mriedem | an instance during scheduling will have a build request | |
| 19:37:30 | dansmith | right, and we page those out of BRs not out of the cell anyway | |
| 19:37:32 | mriedem | once we pick a host, we update the instance mapping for the cell and delete the build request | |
| 19:37:50 | mriedem | https://github.com/openstack/nova/blob/f902e0d5d87fb05207e4a7aca73d185775d43df2/nova/conductor/manager.py#L1236-L1239 | |
| 19:38:08 | melwitt | ah, thanks. I can never remember this | |