How to read keyin while arrayfire windows is focused?

[3togo]

stdout.read_char() will only read when the terminal is focused. It can't read keyin when arrayfire windows is focused.

Since arrayfire window could be closed by pressing ESC, it should be able to receive keyin. However, I can't find any clue to do so.

Any help?

[3togo]

I guess there are several possibilities:

1. arrayfire-sdl2 interop => how to pass arrayfire array in rust to sdl2?
2. override Widget::pollEvents() in forge/src/backend/opengl/sdl/window.cpp?
3. use SDL_PollEvent(&evnt), get the arrayfire window id and then check whether it is in focus ==> how to get arrayfire window id?

[3togo]

How to call this function in Rust?

template<typename T>
static fg_image convert_and_copy_image(const af_array in) {
    const Array<T> _in = getArray<T>(in);
    dim4 inDims        = _in.dims();

    dim4 rdims = (inDims[2] > 1 ? dim4(2, 1, 0, 3) : dim4(1, 0, 2, 3));

    Array<T> imgData = reorder(_in, rdims);

    ForgeManager& fgMngr = forgeManager();

    // The inDims[2] * 100 is a hack to convert to fg_channel_format
    // TODO(pradeep): Write a proper conversion function
    fg_image ret_val = fgMngr.getImage(
        inDims[1], inDims[0], static_cast<fg_channel_format>(inDims[2] * 100),
        getGLType<T>());
    copy_image<T>(normalizePerType<T>(imgData), ret_val);

    return ret_val;
}

[9prady9]

the code will look something like this I create a class object called d. d will modify x,y, z after reading some keyboard input. image_out: Array<u8> will be regenerated by f(x,y,z)

   loop {
        let (x, y, z) = d.dump().into_iter().collect_tuple().unwrap();  // d modifies x, y, z
        let img_out = f(x, y, z);
        wnd.draw_image(&img_out, Some("img_out".to_string()));
        wnd.show();
        wnd.set_visibility(true);
        if wnd.is_closed() == true {
            break;
        } else {
            if let Ok(keyin) = Term::stdout().read_char() {
                if keyin == 'q' {
                    break;
                };
                d.parser(keyin);  // d read keyin
            }
        }
    }

I don't think I am able to understand your use case correctly. The example code you shared above seems like is right way to go about it if my understanding that d is some object with storage in CPU memory while f does some computation on GPU using d and returns af::Array that has GPU buffer.

If my above assertion is true, I would recommend moving the computation and the line creating d to GPU instead of trying to what you described earlier.

[9prady9]

How to call this function in Rust?

template<typename T>
static fg_image convert_and_copy_image(const af_array in) {
    const Array<T> _in = getArray<T>(in);
    dim4 inDims        = _in.dims();

    dim4 rdims = (inDims[2] > 1 ? dim4(2, 1, 0, 3) : dim4(1, 0, 2, 3));

    Array<T> imgData = reorder(_in, rdims);

    ForgeManager& fgMngr = forgeManager();

    // The inDims[2] * 100 is a hack to convert to fg_channel_format
    // TODO(pradeep): Write a proper conversion function
    fg_image ret_val = fgMngr.getImage(
        inDims[1], inDims[0], static_cast<fg_channel_format>(inDims[2] * 100),
        getGLType<T>());
    copy_image<T>(normalizePerType<T>(imgData), ret_val);

    return ret_val;
}

that is internal code to arrayfire, you have write and all upstream implementation on your own. I am not 100% sure but I feel like you have to fix the below lines because you seem to be trying to transfer your computation result from CPU to GPU. please correct me if I am wrong.

        let (x, y, z) = d.dump().into_iter().collect_tuple().unwrap();  // d modifies x, y, z
        let img_out = f(x, y, z);

[3togo]

f(a,b,c) = a*x + b*y + c*z where a, b and c are scalars; x, y and z are Arrayfire arrays
let img_out: Array<u8> = f(a, b, c);
the computation is still from GPU to GPU.

[9prady9]

I think I see what you want to achieve now, basically want to avoid switching to a console to trigger a re-compute (followed by a re-render) using a key stroke while the window is in focus.

Yes, forge and in turn arrayfire graphics weren't designed with this style of interaction in mind. To enable that, forge needs to provide event toolkit agnostic (glfw vs sdl) polling support.

[3togo]

As converting Array<u8> array to sdl2::surface is a GPU to CPU process, it is not desirable in terms of performance.
To ensure that it is a GPU to GPU conversion, Array<u8> needs to be converted to sdl2::texture directly. But, how?

fn gen_array() -> Array<u8> { 
    let (h, w) = (480, 640);
    let mut a = constant::<u8>(0, dim4!(h, w, 3));
    let b = constant::<u8>(97, dim4!(h, w, 1));
    eval!(a[1:1:0, 1:1:0, 0:0:1] = b);
    return a;on
}
    let a = gen_array();
    let [h, w, channels, ..] = a.dims().get().clone();
    let a = reorder_v2(&a, 2, 1, Some(vec![0]));
    let mut b = vec![u8::default(); a.elements()];
    a.host(&mut b);
  
    let d = Surface::from_data(
        &mut b,
        w as u32,
        h as u32,
        (w * channels) as u32,
        PixelFormatEnum::RGB24,
    ).unwrap();

[9prady9]

Are you familiar with CUDA-OpenGL interop ? if yes, then you have to use CUDA interop crate and SDL directly from rust to handle CUDA-OpenGL interop. I haven't tried this from rust (I have done this only from C++) so, I am not sure if there are caveats you need to be looking out for.

[3togo]

Thank you for your suggestion.