Is there a way to run regular tasks async, like we can for HTTP request with ncurl_aio

[avhz]

Hi :)

I'm just wondering if there is an easy way to run a bunch or "regular" R tasks async using nanonest?

I can use ncurl_aio to send off a bunch of HTTP requests:

lapply(urls, \(url) nanonext::ncurl_aio(url))

I can then wait on the results later.

Is there a similar way to do this with nanonext but for regular R expressions ?

jobs <- lapply(x, \(i) nanonext::__???__(i))

finished <- nanonext::call_aio(jobs)
## or
finished <- nanonext::collect_aio(jobs)

Side question:

What is the difference between call_aio() and collect_aio() ?
Is it simply that collect_aio returns the x$data fields ?

[shikokuchuo]

What you're looking for is the mirai package (https://github.com/r-lib/mirai).
That runs things in other processes, rather than threads, but it is also the only way as R is not multi-threaded.

What is the difference between call_aio() and collect_aio() ?
Is it simply that collect_aio returns the x$data fields ?

Yes. collect_aio(x) is marginally more efficient than call_aio(x)$data.

[avhz]

I am currently using Mirai but was interested in this specifically to avoid separate processes, since NNG itself is multi-threaded.

I am currently having issues with passing the required objects to the separate processes using for a particular task, in particular for an R6 class that modifies itself in place.

[shikokuchuo]

Yes, but specifically it's not possible to call the R interpreter from another thread. It is also not safe to modify any R objects from another thread. nanonext is careful not to do any of these things. If you're having issues in mirai, please try and post a reprex in the mirai repo, or open a discussion there if you're not ready to post an issue.

[avhz]

I am definitely not very familiar with R internals and NNG, but couldn't a mutex be placed on the object?
There are quite a few examples of Python libraries that do this despite the GIL.

[shikokuchuo]

It's not just a synchronization problem unfortunately, it's a limitation of R.