Update on join fails to find variable "immediate built" in `i`

[iagogv3]

Let me show a minimal reprex as it is easier than describe and I am not sure why does it happen:

> library(data.table)
> irisdt<-as.data.table(iris)
> var="Sepal.Length"

Next works

> aux <- irisdt[, .(var = min(var)), env = list(var = var), by = Species]
> irisdt[, !c("Sepal.Length")][aux, on = "Species", SLM := var, env = list(var=paste0("i.", var))][]
     Sepal.Width Petal.Length Petal.Width   Species   SLM
           <num>        <num>       <num>    <fctr> <num>
  1:         3.5          1.4         0.2    setosa   4.3
  2:         3.0          1.4         0.2    setosa   4.3
  3:         3.2          1.3         0.2    setosa   4.3
  4:         3.1          1.5         0.2    setosa   4.3
  5:         3.6          1.4         0.2    setosa   4.3
 ---                                                     
146:         3.0          5.2         2.3 virginica   4.9
147:         2.5          5.0         1.9 virginica   4.9
148:         3.0          5.2         2.0 virginica   4.9
149:         3.4          5.4         2.3 virginica   4.9
150:         3.0          5.1         1.8 virginica   4.9

But if I do not use auxiliary variable aux and I include the summarised datatable in i, I get

> ## next, just to replicate my use case; irisdt2 is a data.table with a common variable to merge (Species)
>  irisdt2 <- irisdt[, !c("Sepal.Length")]
> irisdt2[irisdt[, .(var = min(var)), env = list(var = var), by = Species], on = "Species", SLM := var, env = list(var=paste0("i.", var))]
Error in .checkTypos(e, names_x) : 
  Object 'i.Sepal.Length' not found amongst [Sepal.Width, Petal.Length, Petal.Width, Species]

[iagogv3]

Thanks @trobx by your comment (where is it?). Actually, the issue seems to be using the same var name for two distinct strings ('Sepal.Length' and 'i.Sepal.Length') in the distinct environments env.

So, replacing the exterior var with var2, it works:

> irisdt2[irisdt[, .(var = min(var)), env = list(var = var), by = Species], on = "Species", SLM := var2, env = list(var2=paste0("i.", var))]

And only with this replacement. Replacing inside, I would try

> irisdt2[irisdt[, .(var2 = min(var2)), env = list(var2 = var), by = Species], on = "Species", SLM := var, env = list(var=paste0("i.", var))]

However, the var in env=list(var2=var) is the one defined in env = list(var=paste0("i.", var)), that is 'i.Sepal.Length', same issue as above.

[iagogv3]

Understanding this I would't say is a bug (and I will close the issue with this comment), but the error message could be more specific, detailing for example that those variable it mentions occur in the inside/right/i data.table

[trobx]

Not a bug but consistent env behaviour. When the auxiliary data.table call is nested in i, it is no longer evaluated in the global environment but in the environment of the outer call irisdt2[i=, j=, env=], where var has been substituted by the name i.Sepal.Length.

So what is intended (after the substitutions are resolved) is:

irisdt2[irisdt[, .(Sepal.Length=min(Sepal.Length)), by=Species], on="Species", SLM := i.Sepal.Length]

But what is produced is:

irisdt2[irisdt[, .(i.Sepal.Length=min(i.Sepal.Length)), by=Species], on="Species", SLM := i.Sepal.Length]

# Error in .checkTypos(e, names_x) : 
#   Object 'i.Sepal.Length' not found amongst [Sepal.Width, Petal.Length, Petal.Width, Species, SLM]

To ensure that the inner substitution only finds var defined in the global environment, use a different symbol (here var2) in the outer call:

irisdt2[irisdt[, .(var=min(var)), env=list(var=var), by=Species], on="Species", SLM := var2, env=list(var2=paste0("i.", var))][]

     Sepal.Width Petal.Length Petal.Width   Species   SLM
           <num>        <num>       <num>    <fctr> <num>
  1:         3.5          1.4         0.2    setosa   4.3
  2:         3.0          1.4         0.2    setosa   4.3
  3:         3.2          1.3         0.2    setosa   4.3
  4:         3.1          1.5         0.2    setosa   4.3
  5:         3.6          1.4         0.2    setosa   4.3
 ---                                                     
146:         3.0          5.2         2.3 virginica   4.9
147:         2.5          5.0         1.9 virginica   4.9
148:         3.0          5.2         2.0 virginica   4.9
149:         3.4          5.4         2.3 virginica   4.9
150:         3.0          5.1         1.8 virginica   4.9

It's clearer to avoid all reuse of var:

irisdt2[irisdt[, .(y=min(y)), env=list(y=var), by=Species], on="Species", SLM := x, env=list(x=paste0("i.", var))][]