fix(debos/image): Add "quiet" to cmdline kernel options

[Saksham Singh (saks) (saks-singh)]

Add "quiet" functionality while bootup for Debian build, significantly improving the boot time userspace performance, giving significant improvement in kernel (8585ms to 996ms) and userspace (15.585s to 2.176s) for the Glymur device.

previous:

Startup finished in 8.585s (kernel) + 15.585s (userspace) = 24.170s 
graphical.target reached after 15.584s in userspace.

after:

Startup finished in 13.752s (firmware) + 181ms (loader) + 996ms (kernel) + 2.177s (userspace) = 17.108s
graphical.target reached after 2.176s in userspace.

[Loïc Minier (lool)]

Hi Saksham Singh (saks) (@saks-singh) are you measuring this with serial console attached? I'd expect that to slow down boot; do you see the same improvement without serial console attached?

[Loïc Minier (lool)]

For other maintainers: I'm not opposed to the change, but it shifts our images more towards production and less towards development, and might require re-enabling verbose boots for tests.

[Christopher Obbard (obbardc)]

FWIW i am not really a fan of adding quiet to the kernel command line for the default images, maybe this belongs under a $production flag (or similar) ?

[Saksham Singh (saks) (saks-singh)]

Hello Loïc Minier (@lool), we re-ran the boot time logs for the detailed analysis. The build kernel is the same as before, only difference being the Debian recipe highlighted in the PR, this time tested on a different Glymur CRD device.

Build & Setup	Firmware Avg.	Kernel Avg.	Userspace Avg.	Total Boot Time to graphical.target (loader time excluded)
Default without Serial Cable	12.259 s	8.102 s	16.026 s	36.387 s
Default with Serial Cable	13.248 s	8.137 s	16.061 s	37.446 s
Quiet without Serial Cable	11.775 s	1.383 s	2.189 s	15.347 s
Quiet with Serial Cable	13.227 s	1.180 s	2.125 s	16.532 s

As observable, Quiet mode gives significant improvement in boot time, and the serial cable only slows the firmware time, we believe it's a boost to the performance.

[Chitti Babu Theegala (ctheegal)]

Hi Saksham Singh (saks) (Saksham Singh (saks) (@saks-singh)) are you measuring this with serial console attached? I'd expect that to slow down boot; do you see the same improvement without serial console attached?

Loïc Minier (@lool) , Yes. We measured both configurations (with and w/o serial). The serial console impact is not much ~1s of boot time.

But, what really matters is "quiet" mode. This has improved the boot times:-
Kernel phase by 6.7s (8s -> 1.3s)
Multi-user.target by 14s ( 16s -> 2.1s )

[Chitti Babu Theegala (ctheegal)]

FWIW i am not really a fan of adding quiet to the kernel command line for the default images, maybe this belongs under a $production flag (or similar) ?

Christopher Obbard (@obbardc)

There was already a request for debug-build at the PR #568
That (once materializes) can cater to developers and most internal testers.

So, can we introduce 'quiet' cmdline this flavor build ?

[Robie Basak (basak-qcom)]

If the intention is that most developers will switch to using this build, then we will have the problem that those developers won't be able to give us effective bug reports when things go wrong. Given the types of kernel bugs and regressions we seem to deal with regularly, I'm not sure this is a good idea. Given that QLI is supposed to be a reference platform rather than directly a product intended for production use, wouldn't it make sense to provide a production flag as Chris suggests, but not enable it by default or on our CI images?

But it's not my decision.

[Robie Basak (basak-qcom)]

If the issue here is that there are cases where you want faster booting with CI images, don't want to regenerate images, and are happy to exclude yourself from filing kernel bug reports against those boots by throwing away the debug messages, then maybe we can find a way to enable that easily for you without changing the default. For example, could we arrange for a way to adjust your tooling or test harness that you are using to inject the quiet flag?

[Loïc Minier (lool)]

Hi, thanks for the analysis!

Same tradeoffs exist in meta-qcom; the default image is NOT to setting "quiet", instead there are two opt-in kas fragments on top:

• ci/performance.yml: adds quiet but also systemd.tty.term.console=linux
• ci/debug.yml: adds ftrace=... to cmdline, selects extra kernel configs (qcom_debug.config/debug.config) and turns other debugging functionality

This is the distro dimension of meta-qcom CI, so 5 build flavors:

        distro:
          - name: nodistro
            yamlfile: ""
          - name: qcom-distro
            yamlfile: ':ci/qcom-distro.yml'
          - name: qcom-distro-catchall
            yamlfile: ':ci/qcom-distro-catchall.yml'
          - name: debug
            yamlfile: ':ci/qcom-distro-catchall.yml:ci/debug.yml'
          - name: performance
            yamlfile: ':ci/qcom-distro-catchall.yml:ci/performance.yml'

That's multiplied by 3 kernels and 16 machines with a few options added/removed to the matrix by excludes/includes. Each build might output multiple images too, not sure.

So let's do something similar for qcom-deb-images:

• let's add "debug" and "performance" flags to our debos-recipes
• let's add some CI of the performance and debug images on at least one real board to make sure they work to some extent
• let's also check if systemd.tty.term.console is needed, see qualcomm-linux/meta-qcom@3abcf30 for the addition to meta-qcom

Saksham Singh (saks) (@saks-singh) Chitti Babu Theegala (@ctheegal), would you mind taking care of the performance part of the above?

(Cc: Christopher Obbard (@obbardc) for the debug part)

[Chitti Babu Theegala (ctheegal)]

For performance build, given out the requirements at #604

[Christopher Obbard (obbardc)]

This has been superseded by #607, I will leave this open for now though.