Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -610,8 +610,20 @@ Behavior:
Purge already deleted projects (remove residual metadata, optionally ignoring backend errors) using a Manage API token.

```
php cli.php storage:deleted-projects-purge [--ignore-backend-errors] <manageToken> <stackUrl> [--ignore-backend-errors] <projectIds>
php cli.php storage:deleted-projects-purge [-f|--force] [--ignore-backend-errors] <url> <token> <projectIds>
```
Arguments:
- `url` (required): Stack URL, including `https://`.
- `token` (required): Manage API token.
- `projectIds` (required): Comma-separated list of project IDs to purge (e.g. `1,7,146`), or `ALL` to purge every deleted project that isn't purged yet.

Options:
- `--force` / `-f`: Actually purge the projects. Without this flag, the command only reports what would be purged (dry-run).
- `--ignore-backend-errors`: Ignore errors from the backend and just delete buckets and workspaces metadata.

Behavior:
- With `projectIds` set to `ALL`, lists all deleted projects via the Manage API (paginated) and purges each one not already purged.
- For each project ID, skips it if already purged or not found, otherwise purges it and waits (up to 10 minutes) for the purge to complete.

### Set data retention for multiple projects
Set data retention days for specific projects listed in a CSV piped via STDIN.
Expand Down
46 changes: 44 additions & 2 deletions src/Keboola/Console/Command/DeletedProjectsPurge.php
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ protected function configure(): void
->setDescription('Purge deleted projects.')
->addArgument('url', InputArgument::REQUIRED, 'URL of stack including https://')
->addArgument('token', InputArgument::REQUIRED, 'manage api token')
->addArgument('projectIds', InputArgument::REQUIRED, 'IDs of projects to purge (separate multiple IDs with a space)')
->addArgument('projectIds', InputArgument::REQUIRED, 'Comma-separated IDs of projects to purge, or "ALL" to purge every deleted project')
->addOption('ignore-backend-errors', null, InputOption::VALUE_NONE, "Ignore errors from backend and just delete buckets and workspaces metadata")
->addOption('force', null, InputOption::VALUE_NONE, 'Actually perform destructive operations (purge). Without this flag, the command will only simulate actions.');
}
Expand Down Expand Up @@ -48,7 +48,13 @@ protected function execute(InputInterface $input, OutputInterface $output): int
'url' => $url,
'token' => $token,
]);
$projectIds = array_filter(explode(',', $projectIds), 'is_numeric');

if (strtoupper(trim($projectIds)) === 'ALL') {
$projectIds = $this->fetchAllDeletedProjectIds($client, $output);
$output->writeln(sprintf('Found %d not yet purged deleted project(s).', count($projectIds)));
} else {
$projectIds = array_filter(explode(',', $projectIds), 'is_numeric');
}

foreach ($projectIds as $projectId) {
$this->purgeProject(
Expand All @@ -63,6 +69,42 @@ protected function execute(InputInterface $input, OutputInterface $output): int
return 0;
}

/**
* @return int[]
*/
private function fetchAllDeletedProjectIds(Client $client, OutputInterface $output): array
{
$projectIds = [];
$limit = 100;
$offset = 0;

do {
$output->writeln(sprintf('Fetching deleted projects (offset %d)...', $offset));
$deletedProjects = $client->listDeletedProjects([
'limit' => $limit,
'offset' => $offset,
]);

foreach ($deletedProjects as $deletedProject) {
assert(is_array($deletedProject));
if (($deletedProject['isPurged'] ?? false) === true) {
continue;
}
$projectIds[] = (int) $deletedProject['id'];
}

$output->writeln(sprintf(
' - got %d project(s), %d not yet purged so far',
count($deletedProjects),
count($projectIds),
));

$offset += $limit;
} while (count($deletedProjects) === $limit);

return $projectIds;
}

private function purgeProject(
Client $client,
OutputInterface $output,
Expand Down
Loading