| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-03 | |||
| 11:46:07 | sean-k-mooney | mdbooth: well if you were new to nova or did not think about it at the time then you can write races because of eventlets easier then if it was expcitly threaded | |
| 11:46:47 | mdbooth | Honestly, I never consider eventlets. I assume it's explicitly threaded. | |
| 11:47:11 | mdbooth | It's not, but that doesn't have any bearing on writing safe code, except when there's bugs in eventlet. | |
| 11:47:39 | sean-k-mooney | mdbooth: most new people i have talked to that work on openstack dont think about threads at all because the say oh its python and that has a gil so i dont need to care | |
| 11:48:13 | mdbooth | sean-k-mooney: That is just one of many issues with python in the real world :( | |
| 11:48:59 | mdbooth | Also, the gil doesn't prevent overlapping threads of execution, it just stops them running at the same time. The problems are the same. | |
| 11:49:15 | sean-k-mooney | mdbooth: yes and no. there perception is normally correct. its eventlets that violates the paradime | |
| 11:49:42 | mdbooth | As an old curmudgeon, I think python has been extremely detrimental to software engineering, particular in education | |
| 11:50:32 | mdbooth | No, it would not be correct. If you have a multi-threaded python program, even though the gil prevents multiple threads running concurrently, you still need locks. | |
| 11:50:43 | sean-k-mooney | mdbooth: i self taught myself c++ as my first langage then java so ya i like to understand exactly what is going on | |
| 11:51:46 | mdbooth | Basically: eventlets or python-multithreading: I don't care. It shouldn't change how your write code. | |
| 11:51:49 | sean-k-mooney | learning c++ fist made me a better engineer then learing python would have. | |
| 11:52:04 | mdbooth | They both use fake threading. | |
| 11:53:03 | openstackgerrit | Matthew Booth proposed openstack/nova master: Run evacuate tests with local/lvm and shared/rbd storage https://review.openstack.org/604400 | |
| 11:53:13 | mdbooth | I think ^^^ might fix that weird issue I was hitting | |
| 11:53:42 | mdbooth | There's a shortcut in service_is_up if the service is forced down, and we forced it down | |
| 11:54:29 | sean-k-mooney | mdbooth: yes i know if it was expcitly multi thraded it would be incorrect without the lock. anyway cool ill take a look at that too | |
| 11:55:10 | mdbooth | sean-k-mooney: Don't worry about ^^^ btw. Will just wait until zuul has voted. | |
| 11:55:22 | mdbooth | sean-k-mooney: I knew I was missing something simple there. | |
| 11:55:53 | sean-k-mooney | waith why was it forced down? | |
| 11:56:12 | sean-k-mooney | oh so you could evacuate | |
| 11:56:16 | mdbooth | The test_evacuate.sh script which ran before the second round... | |
| 11:56:18 | mdbooth | yeah | |
| 11:57:22 | sean-k-mooney | ha yet anthoer race this tiem between test :) | |
| 12:00:22 | nicolasbock | Morning | |
| 12:01:58 | nicolasbock | I have a run-away server, i.e. a VM that's running on a hypervisor different than what Nova thinks. So far I haven't quite been able to get the placement service to help me update Nova's view of reality... | |
| 12:02:22 | nicolasbock | I can see the server with 'openstack server show 2aa3a324-bf22-4e0c-912a-d7c52f59f1fd' | |
| 12:02:36 | nicolasbock | And it lists the wrong hypervisor | |
| 12:02:47 | mdbooth | sean-k-mooney: $ git grep test\.nested | wc -l | |
| 12:02:47 | mdbooth | 348 | |
| 12:02:54 | mdbooth | I'm not moving that ;) | |
| 12:03:13 | mdbooth | sean-k-mooney: Feel free to write a follow-up patch | |
| 12:03:22 | nicolasbock | I can also check that with 'openstack resource provider allocation show 2aa3a324-bf22-4e0c-912a-d7c52f59f1fd' | |
| 12:03:57 | sean-k-mooney | mdbooth: ok i would have just done nested=common.nested in test.py | |
| 12:04:34 | nicolasbock | The VM is really running on 6cbb84b0-02f4-4ee3-9df2-151475b1effe | |
| 12:04:47 | mdbooth | sean-k-mooney: Sure. I don't want to mess with test.nested here, though. | |
| 12:04:52 | nicolasbock | But `openstack resource provider allocation set --allocation rp=6cbb84b0-02f4-4ee3-9df2-151475b1effe 2aa3a324-bf22-4e0c-912a-d7c52f59f1fd` is not working | |
| 12:05:31 | sean-k-mooney | mdbooth: ok ill submist a follow up patch. eventully... i have added it to my whiteboard | |
| 12:05:34 | nicolasbock | I suppose I am missing a `resource-class-name` | |
| 12:05:50 | nicolasbock | But what do I put there? | |
| 12:08:32 | openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: Fix the help for the disk_weight_multiplier option https://review.openstack.org/607537 | |
| 12:11:55 | sean-k-mooney | nicolasbock: was the vm migrated | |
| 12:12:09 | nicolasbock | Yes sean-k-mooney | |
| 12:12:38 | sean-k-mooney | mdbooth: was lookin at an edgecase recently where if cleanup on the source fail we dont update the vm host | |
| 12:14:29 | sean-k-mooney | nicolasbock: e.g. when you finish migrating the instace if the post migrate job on the source node failts to say unplug a vif we fail before we update the instace db record to refect that the vm is running on the new node | |
| 12:14:49 | sean-k-mooney | mdbooth: did you ever proposea a patch for ^ | |
| 12:15:12 | mdbooth | sean-k-mooney: Probably. | |
| 12:17:27 | sean-k-mooney | nicolasbock: are the placement allocation currently associated with the vm correct for the host it is actully on | |
| 12:17:45 | sean-k-mooney | e.g. if you ignore the db host value and actully check the vms location | |
| 12:18:00 | mdbooth | sean-k-mooney: Trying to parse your comment here: https://review.openstack.org/#/c/605436/5/nova/compute/manager.py@1447 | |
| 12:18:18 | mdbooth | Are you saying we can release the lock there? | |
| 12:18:25 | mdbooth | Or yield the context manager there? | |
| 12:18:35 | mdbooth | Or something else, because neither of ^^^ would be correct. | |
| 12:18:48 | nicolasbock | The VM is running on `6cbb84b0-02f4-4ee3-9df2-151475b1effe` | |
| 12:19:03 | nicolasbock | But placement says that it's on `57b0e4d5-3a3e-4cf3-ba8c-b88c8ce4679b` | |
| 12:19:09 | sean-k-mooney | mdbooth: im saying if we dont have a lock when we invoke that line the db request could cause use to yeild causeing a race | |
| 12:19:32 | mdbooth | Ah, *eventlet* yield | |
| 12:19:37 | mdbooth | Ok. | |
| 12:19:54 | sean-k-mooney | e.g. this is the thing that definetly needs to be in the critcal section of the lock | |
| 12:20:01 | nicolasbock | I ran `openstack server show 2aa3a324-bf22-4e0c-912a-d7c52f59f1fd` | |
| 12:20:15 | nicolasbock | and `openstack resource provider allocation show 2aa3a324-bf22-4e0c-912a-d7c52f59f1fd` | |
| 12:21:06 | sean-k-mooney | mdbooth: ya re reading that was not clear | |
| 12:21:53 | sean-k-mooney | nicolasbock: is 57b0e4d5-3a3e-4cf3-ba8c-b88c8ce4679b the source or destination of the migration | |
| 12:22:23 | sean-k-mooney | nicolasbock: im assume the destination correct? | |
| 12:22:36 | sean-k-mooney | sorry source | |
| 12:23:07 | nicolasbock | I don't know what happened, but I would guess that it is the source | |
| 12:23:27 | nicolasbock | Sorry, I am not sure I completely grasp the terminology of source and destination | |
| 12:23:43 | sean-k-mooney | ya so the resouce being used on 6cbb84b0-02f4-4ee3-9df2-151475b1effe are likely still owned by the migration object in placement | |
| 12:24:44 | nicolasbock | What's the migration object? | |
| 12:26:21 | sean-k-mooney | when you do a migration we create a migration record that we use to calim resouces on the destination host then when the vm is move we use a special atomic oepration in the placemnt api to change the allocation consumer form the migration recordds uuid to the vms uuid | |
| 12:27:23 | nicolasbock | So you are saying that that atomic operation wasn't executed? | |
| 12:27:30 | sean-k-mooney | yes | |
| 12:27:35 | nicolasbock | Ok | |
| 12:27:55 | nicolasbock | Can I get it to execute? | |
| 12:28:01 | sean-k-mooney | so if you do nova server-migration-list 2aa3a324-bf22-4e0c-912a-d7c52f59f1fd does it have a migration object listed | |
| 12:28:53 | nicolasbock | No | |
| 12:30:10 | sean-k-mooney | oh hum strange. perhaps the migration has already been confirmed. | |
| 12:31:13 | sean-k-mooney | nicolasbock: efried might be able to help better the i if he is around | |
| 12:31:28 | nicolasbock | So I thought that `openstack resource provider allocation set` would allow me to update the DB | |
| 12:31:39 | nicolasbock | Thanks sean-k-mooney ! | |
| 12:32:06 | nicolasbock | But I am not using that command correctly since it complains about an incorrect 'allocation string format' | |
| 12:36:24 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/pike: libvirt: Use os.stat and os.path.getsize for RAW disk inspection https://review.openstack.org/607544 | |
| 12:37:16 | openstackgerrit | Matthew Booth proposed openstack/nova master: DNM: Run against mriedem's evacuate test https://review.openstack.org/604423 | |
| 12:44:15 | mdbooth | This is an interesting query: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22AssertionError%3A%20u'host3'%20%3D%3D%20u'host3'%5C%22 | |
| 12:44:49 | mdbooth | I wonder why that started spiking only a few days ago: code change, or infra change? | |
| 13:03:06 | stephenfin | lyarwood: RE: https://review.openstack.org/588570 I'd been putting it off but will do that now. Will keep you posted | |
| 13:04:26 | lyarwood | stephenfin: cheers | |
| 13:06:26 | s10 | How can I reopen bug https://bugs.launchpad.net/nova/+bug/1209101 ? It still exists. | |
| 13:06:26 | openstack | Launchpad bug 1209101 in OpenStack Compute (nova) "Non-public flavor cannot be used in created tenant" [High,Fix released] - Assigned to Sumanth Nagadavalli (sumanth-nagadavalli) | |
| 13:07:02 | sean-k-mooney | s10: just change status and leave a comment with details | |
| 13:08:02 | s10 | sean-k-mooney: I've left comment, but I can't change status, all of them are grey. | |
| 13:08:38 | sean-k-mooney | s10: that said i dont think its nessisarly the same but. that was fixed in 2013 | |
| 13:08:47 | sean-k-mooney | its more likely a regression | |
| 13:08:59 | sean-k-mooney | are you seeing this on master? | |
| 13:09:51 | s10 | sean-k-mooney: Yes, this regression have never been fixed. I will write a comment about how to reproduce it. | |
| 13:10:56 | sean-k-mooney | well it was fixed and then reverted so this likely needs to be treated as more then a bug fix but rather as a blueprint/spec | |
| 13:13:21 | sean-k-mooney | the original bug fix predates microversions so i think a mini spec + microversion but would be required to alter the api behavior | |
| 13:13:57 | sean-k-mooney | bauzas: is ^ correct | |
| 13:14:27 | bauzas | context ? | |
| 13:14:56 | sean-k-mooney | private flavor are automatically expsed to new tenants | |