[英]Joining multiple tables multiple ways
I have 5 tables like below: Table A我有 5 个如下表:表 A
rank.![]() |
input.![]() |
---|---|
0. ![]() |
aa![]() |
1. ![]() |
bb ![]() |
2 ![]() |
cc![]() |
3 ![]() |
dd ![]() |
Table B表 B
rank.![]() |
input.![]() |
---|---|
0. ![]() |
aa![]() |
3 ![]() |
cc![]() |
4 ![]() |
dd ![]() |
5 ![]() |
ee ![]() |
Table C表C
rank.![]() |
input.![]() |
---|---|
0. ![]() |
aa![]() |
5 ![]() |
ee ![]() |
6 ![]() |
ff ![]() |
7 ![]() |
gg ![]() |
Table D表 D
rank.![]() |
input.![]() |
---|---|
0. ![]() |
aa![]() |
2 ![]() |
bb ![]() |
6 ![]() |
ff ![]() |
7 ![]() |
gg ![]() |
I need the output to be like below:我需要 output 如下所示:
Final table决赛桌
rank.![]() |
input.![]() |
---|---|
0. ![]() |
aa![]() |
2 ![]() |
bb ![]() |
3 ![]() |
cc![]() |
5 ![]() |
ee ![]() |
6 ![]() |
ff ![]() |
7 ![]() |
gg ![]() |
If i just cross join all the tables depending on the biggest table, i get the below output:如果我只是根据最大的表交叉连接所有表,我会得到以下 output:
rank.![]() |
input.![]() |
---|---|
0. ![]() |
aa![]() |
Is there a way to get the output i want without having to do multiple joins across AB,BC,CD,BD etc..有没有办法获得我想要的 output 而无需跨 AB、BC、CD、BD 等进行多次连接。
Please let me know.请告诉我。 I can either use SQL or Pyspark to do this.
我可以使用 SQL 或 Pyspark 来执行此操作。 Any suggestions would be appreciated.
任何建议,将不胜感激。
You can union all the tables, group by input and get the maximum of the rank:您可以合并所有表,按输入分组并获得排名的最大值:
select max(`rank`) as `rank`, input
from (
select * from tableA
union all
select * from tableB
union all
select * from tableC
union all
select * from tableD
) t
group by input
In Pyspark it would be在 Pyspark 它将是
from functools import reduce
df = reduce(lambda a, b: a.unionAll(b), [tableA,tableB,tableC,tableD])
result = df.groupBy('input').agg(F.max('rank').alias('rank'))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.