慎用 AWS 中国 S3 Glacier

V2EX = way to explore

V2EX 是一个关于分享和探索的地方

For Existing Member Sign In

This topic created in 2941 days ago, the information mentioned may be changed or developed.

可能是个例，把遇到的情况发出来仅供参考

我厂在使用 AWS S3 Glacier 存储 Elasticsearch 快照，今天需要恢复大约 20 天的索引（按天建立的索引），总量约 1GB。

然而 Glacier 的解冻只能对每个对象进行解冻，不能使用通配符去解冻，当然可以使用 AWS cli 或 s3cmd 等工具来进行递归解冻。

问题就出在递归解冻上，每一个 Elasticsearch 索引快照会包含 100+ 的对象，所以每解冻一个快照索引都需要请求 100+ 次以上。

然而 AWS 中国 S3 Glacier 在请求 40+ 对象后就报错 503，尝试多次都是如此。

restore: 's3://BUCKET/index_20180401/1/__h'
restore: 's3://BUCKET/index_20180401/1/__i'
WARNING: Retrying failed request: /indices/index__20180401/1/__j?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 3 sec...
restore: 's3://BUCKET/index_20180401/1/__j'
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 3 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 6 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 9 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 12 sec...
WARNING: Retrying failed request: /indices/index__20180401/1/__k?restore (503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later)
WARNING: Waiting 15 sec...
ERROR: S3 error: 503 (GlacierExpeditedRetrievalNotAvailable): Glacier expedited retrievals are currently not available, please try again later

40+ 请求时间在 2 秒左右完成，大约 1 秒 20 个请求，如此大型的服务频率限制如此低？

目前还在联系 AWS 中国，询问具体原因，后续会附言更新

Supplement 1 · Apr 17, 2018

AWS 中国回复：

这个503报错属于Glacier服务返回的报错，在使用RestoreObject这个API解冻Glacier对象的时候，如果使用了“Expedited”类别则情况比较特殊，因为这个类别虽然是解冻时间最快但是对于API的Capacity有一定的限制(且默认情况下所有客户共享Cpacity池)，使用“Expedited”类别恢复数据建议每5分钟执行3个以下，否则会影响到Glacier服务器的Capacity性能。

所以是我用错了？加急取回限制真低..... 并没有在官网看见限制说明。

3 replies • 2018-04-17 18:58:22 +08:00