Use correlated subqueries on user_statistics table in the account_view for network stats

[sureshanaparti]

Description

This PR uses scalar correlated subqueries on user_statistics table in account_view for network stats (instead of lateral join, as in #12631 - this which was reverted here: #12965, due to incompatibility with MariaDB).
This works on both MySQL and MariaDB.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

• account_view updated with correlated subqueries.

SHOW CREATE VIEW account_view;

• account_netstats_view shouldn't exist, this should fail.

SHOW CREATE VIEW account_netstats_view;

• account_view queries execute in less time than before (query performance improved).

SELECT * FROM account_view WHERE id = 3;

• EXPLAIN shows ref access on user_statistics (not index scan)

EXPLAIN ANALYZE SELECT * FROM account_view WHERE id = 3 \G

How did you try to break this feature and the system with this change?

[codecov]

Codecov Report

❌ Patch coverage is 62.50000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.00%. Comparing base (5dcb8ab) to head (36321e0).
⚠️ Report is 104 commits behind head on 4.22.

Files with missing lines	Patch %	Lines
...ava/com/cloud/upgrade/dao/Upgrade42210to42220.java	57.14%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               4.22   #13899      +/-   ##
============================================
+ Coverage     17.71%   18.00%   +0.28%     
- Complexity    15858    16214     +356     
============================================
  Files          5926     5936      +10     
  Lines        533613   535714    +2101     
  Branches      65285    65596     +311     
============================================
+ Hits          94517    96430    +1913     
+ Misses       428415   428293     -122     
- Partials      10681    10991     +310     

Flag	Coverage Δ
uitests	4.02% <ø> (+0.33%)	⬆️
unittests	19.07% <62.50%> (+0.28%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sureshanaparti]

@blueorangutan package

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[DaanHoogland]

should the view create script also be removed from the code-base?

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19029

[kiranchavala]

@sureshanaparti getting the following exception

2026-09-25 07:06:22,400 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLSyntaxErrorException: Table 'cloud.account_netstats_view' doesn't exist
2026-09-25 07:06:22,402 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLSyntaxErrorException: Table 'cloud.account_netstats_view' doesn't exist
        at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:186)
        at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:88)
        at com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:258)
        at com.cloud.upgrade.DatabaseUpgradeChecker.executeViewScripts(DatabaseUpgradeChecker.java:449)
        at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:322)
        at com.cloud.upgrade.DatabaseUpgradeChecker.doUpgrades(DatabaseUpgradeChecker.java:531)
        at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:469)
        at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:62)
        at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:52)
        at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
        at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
        at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)

[sureshanaparti]

@blueorangutan package

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19321

[kiranchavala]

LGTM , Tested manually

The pr changes only database migration only. There are no Java behaviour changes apart from registering the new upgrade step and bumping the version to 4.22.2.0. The migration:

• Rebuilds account_view, replacing the join to account_netstats_view with two scalar subqueries on user_statistics (bytesReceived, bytesSent).
• Drops account_netstats_view.

[root@ref-trl-12431-k-Mol8-kiran-chavala-mgmt1 ~]# rpm -qa | grep cloudstack-management
cloudstack-management-4.22.2.0-shapeblue21838.noarch

SHOW CREATE VIEW cloud.account_view\G -- has "(select coalesce(sum(...)) from user_statistics ...)"; no account_netstats_view, no LATERAL

mysql> SHOW CREATE VIEW cloud.account_view\G;
*************************** 1. row ***************************
                View: account_view
         Create View: CREATE ALGORITHM=UNDEFINED DEFINER=`cloud`@`%` SQL SECURITY DEFINER VIEW `account_view` AS select `account`.`id` AS `id`,`account`.`uuid` AS `uuid`,`account`.`account_name` AS `account_name`,`account`.`type` AS `type`,`account`.`role_id` AS `role_id`,`account`.`state` AS `state`,`account`.`created` AS `created`,`account`.`removed` AS `removed`,`account`.`cleanup_needed` AS `cleanup_needed`,`account`.`network_domain` AS `network_domain`,`account`.`default` AS `default`,`account`.`api_key_access` AS `api_key_access`,`domain`.`id` AS `domain_id`,`domain`.`uuid` AS `domain_uuid`,`domain`.`name` AS `domain_name`,`domain`.`path` AS `domain_path`,`data_center`.`id` AS `data_center_id`,`data_center`.`uuid` AS `data_center_uuid`,`data_center`.`name` AS `data_center_name`,(select coalesce(sum((`us`.`net_bytes_received` + `us`.`current_bytes_received`)),0) from `user_statistics` `us` where (`us`.`account_id` = `account`.`id`)) AS `bytesReceived`,(select coalesce(sum((`us`.`net_bytes_sent` + `us`.`current_bytes_sent`)),0) from `user_statistics` `us` where (`us`.`account_id` = `account`.`id`)) AS `bytesSent`,`vmlimit`.`max` AS `vmLimit`,`vmcount`.`count` AS `vmTotal`,`runningvm`.`vmcount` AS `runningVms`,`stoppedvm`.`vmcount` AS `stoppedVms`,`iplimit`.`max` AS `ipLimit`,`ipcount`.`count` AS `ipTotal`,`free_ip_view`.`free_ip` AS `ipFree`,`volumelimit`.`max` AS `volumeLimit`,`volumecount`.`count` AS `volumeTotal`,`snapshotlimit`.`max` AS `snapshotLimit`,`snapshotcount`.`count` AS `snapshotTotal`,`templatelimit`.`max` AS `templateLimit`,`templatecount`.`count` AS `templateTotal`,`vpclimit`.`max` AS `vpcLimit`,`vpccount`.`count` AS `vpcTotal`,`projectlimit`.`max` AS `projectLimit`,`projectcount`.`count` AS `projectTotal`,`networklimit`.`max` AS `networkLimit`,`networkcount`.`count` AS `networkTotal`,`cpulimit`.`max` AS `cpuLimit`,`cpucount`.`count` AS `cpuTotal`,`memorylimit`.`max` AS `memoryLimit`,`memorycount`.`count` AS `memoryTotal`,`gpulimit`.`max` AS `gpuLimit`,`gpucount`.`count` AS `gpuTotal`,`primary_storage_limit`.`max` AS `primaryStorageLimit`,`primary_storage_count`.`count` AS `primaryStorageTotal`,`secondary_storage_limit`.`max` AS `secondaryStorageLimit`,`secondary_storage_count`.`count` AS `secondaryStorageTotal`,`backup_limit`.`max` AS `backupLimit`,`backup_count`.`count` AS `backupTotal`,`backup_storage_limit`.`max` AS `backupStorageLimit`,`backup_storage_count`.`count` AS `backupStorageTotal`,`bucket_limit`.`max` AS `bucketLimit`,`bucket_count`.`count` AS `bucketTotal`,`object_storage_limit`.`max` AS `objectStorageLimit`,`object_storage_count`.`count` AS `objectStorageTotal`,`async_job`.`id` AS `job_id`,`async_job`.`uuid` AS `job_uuid`,`async_job`.`job_status` AS `job_status`,`async_job`.`account_id` AS `job_account_id` from (`free_ip_view` join (((((((((((((((((((((((((((((((((((((((`account` join `domain` on((`account`.`domain_id` = `domain`.`id`))) left join `data_center` on((`account`.`default_zone_id` = `data_center`.`id`))) left join `resource_limit` `vmlimit` on(((`account`.`id` = `vmlimit`.`account_id`) and (`vmlimit`.`type` = 'user_vm') and (`vmlimit`.`tag` is null)))) left join `resource_count` `vmcount` on(((`account`.`id` = `vmcount`.`account_id`) and (`vmcount`.`type` = 'user_vm') and (`vmcount`.`tag` is null)))) left join `account_vmstats_view` `runningvm` on(((`account`.`id` = `runningvm`.`account_id`) and (`runningvm`.`state` = 'Running')))) left join `account_vmstats_view` `stoppedvm` on(((`account`.`id` = `stoppedvm`.`account_id`) and (`stoppedvm`.`state` = 'Stopped')))) left join `resource_limit` `iplimit` on(((`account`.`id` = `iplimit`.`account_id`) and (`iplimit`.`type` = 'public_ip')))) left join `resource_count` `ipcount` on(((`account`.`id` = `ipcount`.`account_id`) and (`ipcount`.`type` = 'public_ip')))) left join `resource_limit` `volumelimit` on(((`account`.`id` = `volumelimit`.`account_id`) and (`volumelimit`.`type` = 'volume') and (`volumelimit`.`tag` is null)))) left join `resource_count` `volumecount` on(((`account`.`id` = `volumecount`.`account_id`) and (`volumecount`.`type` = 'volume') and (`volumecount`.`tag` is null)))) left join `resource_limit` `snapshotlimit` on(((`account`.`id` = `snapshotlimit`.`account_id`) and (`snapshotlimit`.`type` = 'snapshot')))) left join `resource_count` `snapshotcount` on(((`account`.`id` = `snapshotcount`.`account_id`) and (`snapshotcount`.`type` = 'snapshot')))) left join `resource_limit` `templatelimit` on(((`account`.`id` = `templatelimit`.`account_id`) and (`templatelimit`.`type` = 'template')))) left join `resource_count` `templatecount` on(((`account`.`id` = `templatecount`.`account_id`) and (`templatecount`.`type` = 'template')))) left join `resource_limit` `vpclimit` on(((`account`.`id` = `vpclimit`.`account_id`) and (`vpclimit`.`type` = 'vpc')))) left join `resource_count` `vpccount` on(((`account`.`id` = `vpccount`.`account_id`) and (`vpccount`.`type` = 'vpc')))) left join `resource_limit` `projectlimit` on(((`account`.`id` = `projectlimit`.`account_id`) and (`projectlimit`.`type` = 'project')))) left join `resource_count` `projectcount` on(((`account`.`id` = `projectcount`.`account_id`) and (`projectcount`.`type` = 'project')))) left join `resource_limit` `networklimit` on(((`account`.`id` = `networklimit`.`account_id`) and (`networklimit`.`type` = 'network')))) left join `resource_count` `networkcount` on(((`account`.`id` = `networkcount`.`account_id`) and (`networkcount`.`type` = 'network')))) left join `resource_limit` `cpulimit` on(((`account`.`id` = `cpulimit`.`account_id`) and (`cpulimit`.`type` = 'cpu') and (`cpulimit`.`tag` is null)))) left join `resource_count` `cpucount` on(((`account`.`id` = `cpucount`.`account_id`) and (`cpucount`.`type` = 'cpu') and (`cpucount`.`tag` is null)))) left join `resource_limit` `memorylimit` on(((`account`.`id` = `memorylimit`.`account_id`) and (`memorylimit`.`type` = 'memory') and (`memorylimit`.`tag` is null)))) left join `resource_count` `memorycount` on(((`account`.`id` = `memorycount`.`account_id`) and (`memorycount`.`type` = 'memory') and (`memorycount`.`tag` is null)))) left join `resource_limit` `gpulimit` on(((`account`.`id` = `gpulimit`.`account_id`) and (`gpulimit`.`type` = 'gpu') and (`gpulimit`.`tag` is null)))) left join `resource_count` `gpucount` on(((`account`.`id` = `gpucount`.`account_id`) and (`gpucount`.`type` = 'gpu') and (`gpucount`.`tag` is null)))) left join `resource_limit` `primary_storage_limit` on(((`account`.`id` = `primary_storage_limit`.`account_id`) and (`primary_storage_limit`.`type` = 'primary_storage') and (`primary_storage_limit`.`tag` is null)))) left join `resource_count` `primary_storage_count` on(((`account`.`id` = `primary_storage_count`.`account_id`) and (`primary_storage_count`.`type` = 'primary_storage') and (`primary_storage_count`.`tag` is null)))) left join `resource_limit` `secondary_storage_limit` on(((`account`.`id` = `secondary_storage_limit`.`account_id`) and (`secondary_storage_limit`.`type` = 'secondary_storage')))) left join `resource_count` `secondary_storage_count` on(((`account`.`id` = `secondary_storage_count`.`account_id`) and (`secondary_storage_count`.`type` = 'secondary_storage')))) left join `resource_limit` `backup_limit` on(((`account`.`id` = `backup_limit`.`account_id`) and (`backup_limit`.`type` = 'backup')))) left join `resource_count` `backup_count` on(((`account`.`id` = `backup_count`.`account_id`) and (`backup_count`.`type` = 'backup')))) left join `resource_limit` `backup_storage_limit` on(((`account`.`id` = `backup_storage_limit`.`account_id`) and (`backup_storage_limit`.`type` = 'backup_storage')))) left join `resource_count` `backup_storage_count` on(((`account`.`id` = `backup_storage_count`.`account_id`) and (`backup_storage_count`.`type` = 'backup_storage')))) left join `resource_limit` `bucket_limit` on(((`account`.`id` = `bucket_limit`.`account_id`) and (`bucket_limit`.`type` = 'bucket')))) left join `resource_count` `bucket_count` on(((`account`.`id` = `bucket_count`.`account_id`) and (`bucket_count`.`type` = 'bucket')))) left join `resource_limit` `object_storage_limit` on(((`account`.`id` = `object_storage_limit`.`account_id`) and (`object_storage_limit`.`type` = 'object_storage')))) left join `resource_count` `object_storage_count` on(((`account`.`id` = `object_storage_count`.`account_id`) and (`object_storage_count`.`type` = 'object_storage')))) left join `async_job` on(((`async_job`.`instance_id` = `account`.`id`) and (`async_job`.`instance_type` = 'Account') and (`async_job`.`job_status` = 0)))))
character_set_client: utf8mb4
collation_connection: utf8mb4_0900_ai_ci
1 row in set (0.00 sec)

mysql>  SHOW CREATE VIEW cloud.account_netstats_view;
ERROR 1146 (42S02): Table 'cloud.account_netstats_view' doesn't exist

mysql> SELECT TABLE_NAME FROM information_schema.VIEWS WHERE TABLE_SCHEMA='cloud' AND VIEW_DEFINITION LIKE '%account_netstats_view%';
Empty set (0.00 sec)