Skip to content

Add image URL caching support for testcloud provisioner - #5146

Open
HabibRh26 wants to merge 9 commits into
mainfrom
issue-2124
Open

HabibRh26 wants to merge 9 commits into
mainfrom
issue-2124

Conversation

@HabibRh26

Copy link
Copy Markdown
Contributor

Pull Request Checklist

  • implement the feature
  • write the documentation
  • extend the test coverage
  • update the specification
  • adjust plugin docstring
  • modify the json schema
  • mention the version
  • include a release note

@LecrisUT

LecrisUT commented Sep 7, 2026

Copy link
Copy Markdown
Member

From the original issue I am not sure if this feature is still needed. testcloud does cache the images now even by default. Maybe tmt can offer some settings to control it.

@HabibRh26
HabibRh26 force-pushed the issue-2124 branch 3 times, most recently from 20d5717 to de4a303 Compare September 7, 2026 23:47
@LecrisUT LecrisUT added this to planning Sep 8, 2026
@github-project-automation github-project-automation Bot moved this to backlog in planning Sep 8, 2026
@happz happz moved this from backlog to implement in planning Sep 8, 2026
@therazix therazix moved this from implement to review in planning Sep 15, 2026
@therazix therazix linked an issue Sep 15, 2026 that may be closed by this pull request
Comment thread tmt/steps/provision/testcloud.py Outdated
default=7,
option='--image-cache-age',
metavar='DAYS',
help="Maximum age of cached image URLs in days, 7 by default.",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Click & co. will take care of displaying the default value, please, don't include it in the help text. It would be slightly different if the behavior was more complex, but simple values like this and image-cache do not need to be mentioned.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to limit ourselves tyo just days, as we have Pint available, which cn convert between units easily. See e.g. log_size_limit in tmt.steps.report.reportportal, it's using Size instead of int, and employes special conversions to convert from the input - 1 kB, 10 GB, ... - to a value in bytes. I think we should support something similar here, and allow input like 1d, 3600s, or 72h. It would require the change of this field type, and addition of custom callbacks for conversion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Click & co. will take care of displaying the default value, please, don't include it in the help text. It would be slightly different if the behavior was more complex, but simple values like this and image-cache do not need to be mentioned.

Addressed in 88109c0

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to limit ourselves tyo just days, as we have Pint available, which cn convert between units easily. See e.g. log_size_limit in tmt.steps.report.reportportal, it's using Size instead of int, and employes special conversions to convert from the input - 1 kB, 10 GB, ... - to a value in bytes. I think we should support something similar here, and allow input like 1d, 3600s, or 72h. It would require the change of this field type, and addition of custom callbacks for conversion.

Addressed in df3892a

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well also add show_default=True 🙂

Comment thread tmt/steps/provision/testcloud.py Outdated
help="List locally available images.",
)

image_cache: bool = field(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I would expect this to be a path to the "image cache". How about use-image-cache or similar?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in c15b6a6

Comment thread tmt/steps/provision/testcloud.py Outdated
def prepare_image() -> None:
self._image = testcloud.image.Image(self.image_url)
self.verbose('qcow', self._image.name, 'green')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 893f09e

@psss

psss commented Sep 15, 2026

Copy link
Copy Markdown
Member

@HabibRh26, please update the pull request title and description to match the actual changes.

@HabibRh26 HabibRh26 self-assigned this Sep 15, 2026
@HabibRh26 HabibRh26 changed the title add --use-local-image flag to skip image download in testcloud provis… Add image URL caching support for testcloud provisioner Sep 15, 2026
@HabibRh26

Copy link
Copy Markdown
Contributor Author

@HabibRh26, please update the pull request title and description to match the actual changes.

updated

Comment thread tmt/steps/provision/testcloud.py Outdated
Comment thread tmt/steps/provision/testcloud.py Outdated
are treated as days for backward compatibility.
"""

from pint import Quantity

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use global imports unless strictly needed by circular dependencies

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

followed the existing pattern from normalize_data_amount

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that may have been an artifact from when this was an optional dependency. It is a mandatory dependency now, so can move it (and the others) to global import

Comment thread tmt/steps/provision/testcloud.py Outdated
Comment thread tmt/steps/provision/testcloud.py Outdated
)

image_cache_age: 'Size' = field(
default=tmt.hardware.UNITS('7 day'),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can change this to string and normalize would still be applied. Then show_default and show it in a nicer format

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, '7 days' should work.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like our assumption is incorrect here 🤔 something broke with this change

Comment thread tmt/steps/provision/testcloud.py Outdated
@therazix therazix added plugin | testcloud The testcloud virtual provision plugin ci | full test Pull request is ready for the full test execution step | provision Stuff related to the provision step labels Sep 16, 2026
@HabibRh26
HabibRh26 requested review from LecrisUT and happz September 16, 2026 21:47
Comment thread tmt/utils/__init__.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci | full test Pull request is ready for the full test execution plugin | testcloud The testcloud virtual provision plugin step | provision Stuff related to the provision step

Projects

Status: review

Development

Successfully merging this pull request may close these issues.

Allow using already downloaded testcloud images

6 participants