We are receiving - SystemCallError error="The interface is unknown. - OpenEventLog inside Windows Container

[api-py]

Hi,

Our block of configuration is triggering an error message that we cannot overcome. Can you please advise what is going wrong? We get the same with the version 2 of the plugin.

We are running fluentd on Windows as a container (AWS EKS Windows nodeS).

Regards,
Dejan

__

Error message when invoking the fluentd config for windows_eventlogor windows_eventlog2:

2020-11-30 08:03:36 +0000 [info]: starting fluentd-1.11.5 pid=6996 ruby="2.6.5"
2020-11-30 08:03:36 +0000 [info]: spawn command to main:  cmdline=["C:/ruby26/bin/ruby.exe", "-Eascii-8bit:ascii-8bit", "C:/ruby26/bin/fluentd", "-c", "C:\\fluent\\conf\\fluent.conf", "--under-supervisor"]
2020-11-30 08:03:46 +0000 [info]: adding match pattern="@FLUENT_LOG" type="null"
2020-11-30 08:03:47 +0000 [info]: adding filter pattern="@FLUENT_LOG" type="record_transformer"
2020-11-30 08:03:47 +0000 [info]: adding match pattern="@FLUENT_LOG" type="elasticsearch_dynamic"
2020-11-30 08:03:49 +0000 [info]: adding source type="windows_eventlog"
2020-11-30 08:03:49 +0000 [warn]: #0 in_windows_eventlog is deprecated. It will be removed in the future version.
2020-11-30 08:03:49 +0000 [info]: #0 starting fluentd worker pid=1632 ppid=6996 worker=0
2020-11-30 08:03:49 +0000 [error]: #0 unexpected error error_class=SystemCallError error="The interface is unknown. - OpenEventLog"
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/win32-eventlog-0.6.7/lib/win32/eventlog.rb:112:in `initialize'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluent-plugin-windows-eventlog-0.8.0/lib/fluent/plugin/in_windows_eventlog.rb:113:in `new'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluent-plugin-windows-eventlog-0.8.0/lib/fluent/plugin/in_windows_eventlog.rb:113:in `block in start'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluent-plugin-windows-eventlog-0.8.0/lib/fluent/plugin/in_windows_eventlog.rb:110:in `each'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluent-plugin-windows-eventlog-0.8.0/lib/fluent/plugin/in_windows_eventlog.rb:110:in `start'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:200:in `block in start'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:189:in `block (2 levels) in lifecycle'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:188:in `each'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:188:in `block in lifecycle'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:175:in `each'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:175:in `lifecycle'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/root_agent.rb:199:in `start'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/engine.rb:248:in `start'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/engine.rb:147:in `run'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/supervisor.rb:607:in `block in run_worker'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/supervisor.rb:845:in `main_process'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/supervisor.rb:598:in `run_worker'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/lib/fluent/command/fluentd.rb:361:in `<top (required)>'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/lib/ruby/gems/2.6.0/gems/fluentd-1.11.5-x64-mingw32/bin/fluentd:8:in `<top (required)>'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/bin/fluentd:23:in `load'
  2020-11-30 08:03:49 +0000 [error]: #0 C:/ruby26/bin/fluentd:23:in `<main>'
2020-11-30 08:03:49 +0000 [error]: #0 unexpected error error_class=SystemCallError error="The interface is unknown. - OpenEventLog"
  2020-11-30 08:03:49 +0000 [error]: #0 suppressed same stacktrace
2020-11-30 08:03:49 +0000 [info]: Worker 0 finished unexpectedly with status 1

Here is our fluentd config:

   <match @FLUENT_LOG>
      @type null
    </match>
    <source>
      @type windows_eventlog
      channels application,system
      read_interval 2
      tag winevt.raw
      <storage>
        @type local                   # @type local is the default.
        persistent true               # default is true. Set to false to use in-memory storage.
        path /var/log/fluentd-buffers # This is required when persistent is true.
                                      # Or, please consider using <system> section's `root_dir` parameter.
      </storage>
    </source>
    <filter @FLUENT_LOG>
      @type record_transformer
      @id filter_containers_stream_transformer
      <record>
        stream_name ${tag_parts[4]}
      </record>
    </filter>

[cosmo0920]

We are running fluentd on Windows as a container (AWS EKS Windows nodeS).

And I think that Windows container does not have EventLog interface...?

[cosmo0920]

We get the same with the version 2 of the plugin.

Hmm..., I see.
Inside Windows container, Windows EventLog does not work as expected.

Instead, k8s official document recommends to use LogMonitor to redirect Windows EventLog into stdout:
https://kubernetes.io/docs/setup/production-environment/windows/user-guide-windows-containers/#capturing-logs-from-workloads

Then, we might be able to handle Windows containers' logs with general kubernetes daemonset style log collector.
If LogMonitor also does not work on your Windows container on AWS EKS Windows nodeS, AWS EKS does not permit to call Windows EventLog related Windows systemcalls.

LogMonitor.exe can be downloaded here:
https://github.com/microsoft/windows-container-tools/releases/tag/v1.1

See also: https://docs.microsoft.com/en-us/virtualization/windowscontainers/troubleshooting#docker-container-logs