[英]How can I make integration tests with google cloud bigquery
We are in the processing of migrating from apache hbase to bigquery.我们正在从 apache hbase 迁移到 bigquery。
Currently we have end to end tests (using cucumbers) that work with a docker container running hbase.目前我们有端到端的测试(使用黄瓜)与运行 hbase 的 docker 容器一起工作。
There don't seem to be any bigquery docker containers or emulators ( https://cloud.google.com/sdk/gcloud/reference/beta/emulators/ )似乎没有任何 bigquery docker 容器或模拟器( https://cloud.google.com/sdk/gcloud/reference/beta/emulators/ )
How would we be able to create end to end tests for an application working with bigquery?我们如何才能为使用 bigquery 的应用程序创建端到端测试?
Currently there is not any kind of BigQuery local emulator or anything similar to that.目前没有任何类型的 BigQuery 本地模拟器或类似的东西。 As pointed out by the link you shared about available GCP emulators , there are some other products that have such a feature, but probably the reason why BigQuery does not have one is that its true potential is only seen when working in its real infrastructure, plus the fact that the costs of working with BigQuery can be relatively low depending on the usage you make of it, plus you have a Free Tier to start working with.
正如您分享的关于 可用 GCP 模拟器的链接所指出的,还有一些其他产品具有这样的功能,但 BigQuery 没有的原因可能是它的真正潜力只有在其真实基础设施中工作时才能看到,加上事实上,使用 BigQuery 的成本可能相对较低,具体取决于您对它的使用情况,而且您有一个免费套餐可以开始使用。
Let me summarize some info about BigQuery pricing that can be useful for you:让我总结一些可能对您有用的有关 BigQuery 定价的信息:
In any case, if you still think that working with BigQuery directly is not the best option for you, can always forward your requests to the Engineering team by creating a Feature Request in the Public Issue Tracker for BigQuery, although it will be in hands of the engineering team whether to decide if (and when) to implement such a feature, even more considering the complexity of BigQuery and that its performance is optimized for working in its current architecture.在任何情况下,如果您仍然认为直接使用 BigQuery 不是您的最佳选择,可以随时通过在 BigQuery 的公共问题跟踪器中创建功能请求将您的请求转发给工程团队,尽管它将由工程团队是否决定是否(以及何时)实现这样的功能,甚至更多地考虑 BigQuery 的复杂性以及它的性能针对在其当前架构中的工作进行了优化。
This is an old post but if you can use Python and you plan to test your SQL and assert your result based on input, I would suggest bq-test-kit .这是一篇旧文章,但如果您可以使用 Python 并且您计划测试您的 SQL 并根据输入断言您的结果,我建议使用 bq-test-kit 。 This framework allows you to interact with BigQuery in Python and make tests reliables.
该框架允许您在 Python 中与 BigQuery 交互并使测试可靠。
You have 3 ways to inject data into it:您可以通过 3 种方式向其中注入数据:
Hope that this helps.希望这会有所帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.