标签：hive join
SELECT ....
FROM A LEFT SEMI JOIN B
ON (A.col1 = B.col2)
WHERE ...
"
If A.col1 is of DOUBLE type, but B.col2 is of BIGINT, will print WARNING: Comparing a bigint and a double may result in a loss of precision. Why can‘t cast col2 to double automatically?
> If A.col1 is of DOUBLE type,
> but B.col2 is of BIGINT,
The automatic conversion is not acceptable according to the java language
spec (section 5.1.2)
https://docs.oracle.com/javase/specs/jls/se7/html/jls-5.html#jls-5.1.2
Also to be noted here is that in general, that even if you cast, you might
be casting the wrong way around.
Because joins on double columns will give incorrect (rather unintended,
but IEEE 754 correct) results when comparing byte serialized
representations - because of the nearly-equal property epsilon.
Easiest way to demonstrate this is to try the simplest off-by-epsilon case
(say, in python)
>>> import sys
>>> 0.1 + 0.2
0.30000000000000004
>>> 0.1 + 0.2 > 0.3
True
>>>
>>> ((0.1+0.2) - 0.3) < sys.float_info.epsilon
True
So if the RHS produced ETL values by sum() and the LHS was produced by
parsing log text, the JOIN will output zero rows.
If you want to do equijoins like that, the only valid case is to cast both
to fixed precision bigints (say, convert all dollars to cents, by *100),
not both to double.
Cheers,
Gopal
标签：hive join
原文地址：http://tangjj.blog.51cto.com/1848040/1642551
