V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
UnitedStack
OpenStack
DevStack
OneStack
RDO
Fuel
StackOps
Install OpenStack on Ubuntu 12.04
OpenStack on GitHub
firejoke
V2EX  ›  OpenStack

openstack rocky 版本 多后端存储支持 保存镜像失败

  •  
  •   firejoke · 2018-12-24 16:23:29 +08:00 · 4633 次点击
    这是一个创建于 2159 天前的主题,其中的信息可能已经有所发展或是发生改变。

    官方文档Multi Store Support

    官方说在 rocky 版本是作为实验功能引入,

    参考后的配置(/etc/glance/glance-api.conf)

    [DEFAULT]
    ...
    enabled_backends = reliable:file, fast:rbd
    ...
    
    [glance_store]
    default_backend = reliable
    
    [reliable]
    filesystem_store_datadir = /opt/stack/data/glance/images/
    store_description = "Reliable filesystem backend"
    
    [fast]
    store_description = "Fast rbd backend"
    rbd_store_pool = images
    rbd_store_user = glance
    rbd_store_ceph_conf = /etc/ceph/ceph.conf
    rbd_store_chunk_size = 8
    rados_connect_timeout = 0
    
    [os_staging]
    filesystem_store_datadir = /opt/stack/data/glance/os_staging/
    store_description = "Filesystem store for staging purpose"
    

    源码内判断后端存储位置的代码(glance/glance/api/v2/image_data.py class ImageDataController)

    ...
    ...
        @utils.mutating
        def upload(self, req, image_id, data, size):
            backend = None
            if CONF.enabled_backends:
                backend = req.headers.get('x-image-meta-store',
                                          CONF.glance_store.default_backend)
    
                try:
                    LOG.debug(" = = = = = = backend: %s = = = = = = " % backend)
                    glance_store.get_store_from_store_identifier(backend)
                    LOG.debug("glance_store.backend %s" % glance_store.backend)
                except glance_store.UnknownScheme as exc:
                    raise webob.exc.HTTPBadRequest(explanation=exc.msg,
                                                   request=req,
                                                   content_type='text/plain')
    
            image_repo = self.gateway.get_repo(req.context)
            image = None
            refresher = None
            cxt = req.context
            try:
                self.policy.enforce(cxt, 'upload_image', {})
                image = image_repo.get(image_id)
                image.status = 'saving'
                try:
                    if CONF.data_api == 'glance.db.registry.api':
                        # create a trust if backend is registry
                        try:
                            # request user plugin for current token
                            user_plugin = req.environ.get('keystone.token_auth')
                            roles = []
                            # use roles from request environment because they
                            # are not transformed to lower-case unlike cxt.roles
                            for role_info in req.environ.get(
                                    'keystone.token_info')['token']['roles']:
                                roles.append(role_info['name'])
                            refresher = trust_auth.TokenRefresher(user_plugin,
                                                                  cxt.tenant,
                                                                  roles)
                        except Exception as e:
                            LOG.info(_LI("Unable to create trust: %s "
                                         "Use the existing user token."),
                                     encodeutils.exception_to_unicode(e))
    
                    image_repo.save(image, from_state='queued')
                    image.set_data(data, size, backend=backend)
    
                    try:
                        image_repo.save(image, from_state='saving')
                    except exception.NotAuthenticated:
                        if refresher is not None:
                            # request a new token to update an image in database
                            cxt.auth_token = refresher.refresh_token()
                            image_repo = self.gateway.get_repo(req.context)
                            image_repo.save(image, from_state='saving')
                        else:
                            raise
    
                    try:
                        # release resources required for re-auth
                        if refresher is not None:
                            refresher.release_resources()
                    except Exception as e:
                        LOG.info(_LI("Unable to delete trust %(trust)s: %(msg)s"),
                                 {"trust": refresher.trust_id,
                                  "msg": encodeutils.exception_to_unicode(e)})
    ...
    ...
    

    目前请求头并没有 x-image-meta-store 属性,直接按默认的 reliable 后端配置来

    相关日志:

    12 月 24 15:59:25 controller [email protected][6524]: DEBUG glance.api.v2.image_data [None req-081110c4-8d29-4567-a4fb-bbbc7313efbf alt_demo admin]  = = = = = = backend: reliable = = = = = =  {{(pid=6527) upload /opt/stack/glance/glance/api/v2/image_data.py:127}}
    12 月 24 15:59:25 controller [email protected][6524]: DEBUG glance.api.v2.image_data [None req-081110c4-8d29-4567-a4fb-bbbc7313efbf alt_demo admin] glance_store.backend <module 'glance_store.backend' from '/usr/lib/python2.7/site-packages/glance_store/backend.pyc'> {{(pid=6527) upload /opt/stack/glance/glance/api/v2/image_data.py:129}}
    ......
    12 月 24 15:59:26 controller [email protected][6524]: DEBUG glance_store._drivers.filesystem [None req-081110c4-8d29-4567-a4fb-bbbc7313efbf alt_demo admin] Wrote 0 bytes to /opt/stack/data/glance/images/9da337be-bb04-43af-ac1d-b9e5a15a0c70 with checksum d41d8cd98f00b204e9800998ecf8427e and multihash cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e {{(pid=6527) add /usr/lib/python2.7/site-packages/glance_store/_drivers/filesystem.py:728}}
    12 月 24 15:59:27 controller [email protected][6524]: DEBUG glance_store.multi_backend [None req-081110c4-8d29-4567-a4fb-bbbc7313efbf alt_demo admin] Skipping store.set_acls... not implemented {{(pid=6527) set_acls_for_multi_store /usr/lib/python2.7/site-packages/glance_store/multi_backend.py:439}}
    ......
    

    在 http 上传缓存目录能找到上传成功的文件,但在从缓存目录写入到 image 目录时,但写入数据就变成了 Wrote 0 bytes

    可能是最近比较浮躁

    一直跑流程也还是找不到为什么

    请求能有大大指点一二

    2 条回复    2018-12-25 19:28:46 +08:00
    firejoke
        1
    firejoke  
    OP
       2018-12-25 17:36:33 +08:00
    回滚后测试,已经确定不是代码层的问题
    firejoke
        2
    firejoke  
    OP
       2018-12-25 19:28:46 +08:00
    定位到了最终问题,是在
    def upload(self, req, image_id, data, size):
    backend = None
    if CONF.enabled_backends:
    下面打了个日志记录了整个 req,结果就导致写入数据 0
    具体原因不明
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   3729 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 24ms · UTC 10:32 · PVG 18:32 · LAX 02:32 · JFK 05:32
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.