Need help about keyring and config files!

[gilbertoferreira]

Hi there

I am trying to use thie PVE_Ceph-Disaster_Recovery script.
I have 3 nodes.
This nodes was used as 3 nodes ceph cluster.
As purpose, I reinstall all the nodes, keeping a backup of the following files/keyring:
Critical Keyrings
/var/lib/ceph/mgr/-/keyring
/var/lib/ceph/mon/-/keyring
/etc/pve/priv/ceph.client.admin.keyring
/etc/pve/priv/ceph.mon.keyring
/etc/pve/priv/ceph.client.bootstrap-osd.keyring
Configuration Files
/etc/ceph/ceph.conf (cluster config)
/etc/pve/ceph.conf (Proxmox cluster-synced config)
/var/lib/ceph (daemon state and store data)

But I don't what to do with this files!

Another doubt:
Do I need to create/install ceph on all nodes and the add monitors/manager, like, as I create a new ceph cluster?
Note: I have 3 OSD, 1 per node, intact.
I am using Proxmox VE 9 and CEPH SQUID.

Thanks

[SoFMeRight]

You will have to create and install ceph on all nodes. Make sure they have dns so you can hit them with their hostnames. DO NOT recreate the OSDS.. lvm command in the script should activate them. 🤞🏽 You will also need to make sure those backups of those files remain intact. Keep a copy and get a copy in place when ceph is reinstalled. You will need the cluster to believe it is still the same old ceph cluster essentially. :( I dont think anywhere suggests to reinstall it makes it a wee bit harder. But not impossible! I hope you get this working :<

[SoFMeRight]

I also hope to god you remove your disks if you ran the script titled ceph-purge-data_loss_if_osds_connected.sh

Note: I have 3 OSD, 1 per node, intact.

This ^ is good. Thats all you need if you were 3 nodes and were 2/3 replicated. You can copy the monmap to the new host. Dont use my script if you already have a good monmap

[SoFMeRight]

If you have working host and just basically need quorum you just need to hopefully havent done anything yeeeeet... You could copy the monmap to the new host. Let me know your actual state Im gonna read again Im super ADHD, and Idk if you are live so I just started replying immediately cause disaster recovery scenario lol

[SoFMeRight]

This may sound crazy but if you want to ask me questions about this directly you are more than welcome to join r/homelabs discord and @ me there. Or also message me on discord. I just cant guarantee I will see it so I mentioned a community discord that I could be pinged at. But I will do my best to reply when I see it! I made this page in github because I had mishap in November so I feel it to my core I may readjust some things to be more clear from my observations in you reading over it.. I should add more stuff about states that can be recoverable much easier without the script and also if you are reading this in a disaster recovery scenario have a first headline of DOs and DONTs. Im glad you reached out!

[gilbertoferreira]

Thank you so much for you messages.
For the moment, I am cool with all that and don't need any rush-rush with this.
In time I will contact you in discord. For now on I am involved in some many other projects that I can reach you right now...
Thanks.