| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-08-26 | |||
| 18:37:17 | stephenfin | sean-k-mooney: Okay. You've got reviews on https://review.opendev.org/#/c/745605/ too so if you get a chance to address that, I can re +2 tomorrow | |
| 18:38:01 | sean-k-mooney | ya ill take a look at it | |
| 19:04:10 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Add ability to download Glance images into the libvirt image cache via RBD https://review.opendev.org/574301 | |
| 19:22:16 | openstackgerrit | Merged openstack/nova master: Add type hints to 'nova.compute.manager' https://review.opendev.org/742863 | |
| 19:22:30 | openstackgerrit | Merged openstack/nova master: Avoid invalid file name, preventing git clone on win32 https://review.opendev.org/748250 | |
| 20:00:40 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | |
| 20:00:40 | openstackgerrit | sean mooney proposed openstack/nova master: pass vifs_already_plugged when reverting a migration https://review.opendev.org/748296 | |
| 20:00:57 | mnaser | does anyone know why regex based filtering is no longer working in the api with ussuri? | |
| 20:01:02 | mnaser | i cant track any commits that might have caused it | |
| 20:01:30 | mnaser | openstack --debug server list --all-projects --name 'foo' returns two servers which are 'foo1.bar' and 'foo2.bar' | |
| 20:01:47 | mnaser | but `openstack --debug server list --all-projects --name 'foo\d'` returns nothing | |
| 20:02:08 | sean-k-mooney | did you check the client | |
| 20:02:18 | sean-k-mooney | im not sure if this is api side | |
| 20:02:21 | sean-k-mooney | it might be | |
| 20:02:38 | sean-k-mooney | well i think it is but maybe there were some clinet changes that broke it | |
| 20:03:26 | mnaser | sean-k-mooney: its not, i looked into the api-ref | |
| 20:03:33 | mnaser | it says you can send actual regex | |
| 20:03:37 | sean-k-mooney | string | |
| 20:03:40 | sean-k-mooney | ||
| 20:03:42 | sean-k-mooney | Filters the response by a server name, as a string. You can use regular expressions in the query. For example, the ?name=bob regular expression returns both bob and bobb. If you must match on only bob, you can use a regular expression that matches the syntax of the underlying database server that is implemented for Compute, such as MySQL or PostgreSQL. | |
| 20:03:58 | mnaser | _regex_instance_filter() in https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py hasn't been touched for 5-8 years | |
| 20:04:01 | mnaser | according to blame | |
| 20:04:03 | sean-k-mooney | so if this behavior changed its likel a db change | |
| 20:04:16 | mnaser | i mean, it went from myslq to mysql | |
| 20:04:50 | mnaser | let me check if maybe its an option ;\ | |
| 20:06:00 | sean-k-mooney | this was the last change to that code https://github.com/openstack/nova/commit/117fad897d5310d66cc2e690f3cd32e72614d8fd | |
| 20:07:34 | mnaser | so it looks like it should be sending a REGEXP query | |
| 20:08:05 | mnaser | let me try it out against the actual db server | |
| 20:09:45 | mnaser | indeed, the regexp doesnt work | |
| 20:09:59 | mnaser | `SELECT * FROM instances WHERE display_name REGEXP 'foo\d';` returns an empty set | |
| 20:11:33 | sean-k-mooney | try 'foo\d.*' | |
| 20:11:46 | sean-k-mooney | foo\d would not match foo1.bar | |
| 20:14:15 | sean-k-mooney | https://regex101.com/r/uDuQBs/1/ | |
| 20:14:48 | mnaser | sean-k-mooney: ok so it turns out that mysql seemingly uses a differnt regex lib than mariadb | |
| 20:14:53 | mnaser | https://dev.mysql.com/doc/refman/5.7/en/regexp.html#regexp-operators | |
| 20:15:18 | sean-k-mooney | your current regex is not correct however | |
| 20:15:29 | sean-k-mooney | foo\d will not match foo1.bar | |
| 20:15:33 | sean-k-mooney | it will match foo1 | |
| 20:16:04 | sean-k-mooney | foo\d is the same as foo[0-9] | |
| 20:16:22 | mnaser | sean-k-mooney: right, i agree with you on that, but the regex impl using in mysql does not | |
| 20:16:32 | mnaser | SELECT display_name FROM instances WHERE display_name REGEXP 'foo[[:digit:]].'; works | |
| 20:17:24 | sean-k-mooney | it looks like they want you to double escape | |
| 20:17:32 | sean-k-mooney | so you could try foo\\d | |
| 20:18:49 | mnaser | sean-k-mooney: nope | |
| 20:19:02 | mnaser | see doc above, [[:digit:]] is what they use instead of \d according to that | |
| 20:19:08 | sean-k-mooney | ok will this is not part of the nova api defintion | |
| 20:19:09 | mnaser | i guess mariadb and mysql are different when it comes to this | |
| 20:19:19 | sean-k-mooney | maybe maybe not | |
| 20:19:27 | sean-k-mooney | \d works in pyton | |
| 20:19:35 | sean-k-mooney | but it might not work in sql | |
| 20:19:39 | mnaser | yeah well the fact we're relying on the db to do the filtering makes it very unpredictable | |
| 20:19:51 | mnaser | because the regex can be different dependign on the backend | |
| 20:21:08 | sean-k-mooney | ya well this is not a portable part of teh api | |
| 20:21:20 | sean-k-mooney | we dont actully give any guarenttes this will work | |
| 20:21:31 | sean-k-mooney | i know we looked at removing it at one point | |
| 20:22:07 | sean-k-mooney | it is called out in the api ref at least | |
| 20:22:08 | sean-k-mooney | If you must match on only bob, you can use a regular expression that matches the syntax of the underlying database server that is implemented for Compute, such as MySQL or PostgreSQL. | |
| 20:27:29 | sean-k-mooney | mnaser: for what its worth that regex support is from the nova v1 api | |
| 20:27:50 | sean-k-mooney | it predates v2 or microverions | |
| 20:28:03 | sean-k-mooney | i belive it even predates novas use of specs | |
| 20:28:19 | sean-k-mooney | so the fact it works at al is somewhat surprising | |
| 20:31:30 | melwitt | agreed, but apparently other people have known this works, someone fixed a thing related to it 3 years ago https://review.opendev.org/506760 | |
| 20:33:58 | sean-k-mooney | oh im sure people still use it | |
| 20:34:11 | sean-k-mooney | just its not portable and it proably not well tested | |
| 20:37:18 | melwitt | yeah | |
| 20:37:32 | sean-k-mooney | apparently sqlalcamey support regex filters | |
| 20:37:35 | sean-k-mooney | http://xion.io/post/code/sqlalchemy-regex-filters.html | |
| 20:37:46 | sean-k-mooney | nick_regexp = '^' + re.escape(nick) + r'\d+$' | |
| 20:37:49 | sean-k-mooney | return session.query(Person).filter(Person.nick.regexp(nick_regexp)).all() | |
| 20:38:10 | sean-k-mooney | melwitt: mnaser so if we wanted to make it portable in the futrue we could use those to do so | |
| 20:38:24 | sean-k-mooney | that would standardise on python regex syntax | |
| 20:38:29 | mnaser | oooh | |
| 20:38:34 | mnaser | yes i like that approach | |
| 20:38:46 | melwitt | project idea for mnaser xD | |
| 20:38:48 | sean-k-mooney | with a microversion since those are a thing now | |
| 20:39:08 | sean-k-mooney | so you can got old and busted or new an shiny | |
| 20:40:23 | mnaser | melwitt, sean-k-mooney: doesn't sound too harsh, probably once we're done upgrading everything to ussuri and this k8s operator for openstack project | |
| 20:40:52 | sean-k-mooney | are you collaberating with redhat on the k8s operator | |
| 20:41:17 | mnaser | i dont think redhat is building a k8s operator for openstack, we're building https://opendev.org/vexxhost/openstack-operator | |
| 20:41:30 | mnaser | all work is in gerrit / testing using tempest so nothing exotic in that sense | |
| 20:41:34 | sean-k-mooney | https://github.com/openstack-k8s-operators | |
| 20:41:47 | sean-k-mooney | mnaser: redhat is | |
| 20:41:50 | mnaser | lol, welp, TIL | |
| 20:42:00 | mnaser | this is very new | |
| 20:42:08 | sean-k-mooney | mnaser: and we will be supporting it in the next major verions | |
| 20:42:15 | mnaser | gosh, this is exactly what we're doing | |
| 20:42:16 | mnaser | lol | |
| 20:42:17 | sean-k-mooney | altough proably as tech preview | |
| 20:42:29 | mnaser | sean-k-mooney: who should i reach out to lol | |
| 20:43:15 | sean-k-mooney | that is a good question i can follow up internally tommorow and ask | |
| 20:44:18 | mnaser | sean-k-mooney: let me know because it pretty much already just works here... so we're really doing the same thing essentially :) | |
| 20:44:19 | sean-k-mooney | mdbooth and mschuppert are involed with the nova part | |
| 20:44:37 | mnaser | i.e. the keystone todo is already done in our case, heh | |
| 20:45:12 | sean-k-mooney | do you have a fully contiarerised nova contol plane | |
| 20:46:54 | sean-k-mooney | oh you are actully doing it in opendev | |
| 20:46:57 | mnaser | sean-k-mooney: yes :) | |
| 20:47:01 | sean-k-mooney | nice | |
| 20:47:13 | mnaser | and comptues too, the only thing that runs on the physical nodes is openvswitch and libvirt | |
| 20:47:26 | mnaser | the plan is to move openvswitch to containers too because we rely on the kernel data path so that should be ok | |
| 20:47:52 | mnaser | and libvirt, planning to move towards the split daemon model and run only the kvm virt driver on the host (and i think even then that has a way of being containerized but not being killed) | |