Hi, thank you for making DNABERT-S publicly available!
I am trying to fine-tune DNABERT-S using data from the RiboGrove database, and I’ve run into a question about how the training dataset was constructed.
The paper that introduces DNADERT-S states that the dataset consists of nearly 2 million randomly selected sequence pairs from 30 thousand unique genomes. If I understand correctly, this implies that some genomes are represented by multiple sequence pairs in the final dataset — is that right?
If so, I’m not sure how the contrastive learning process handles this. As I understand it, within a batch, each i-th pair is treated as positive, while all other pairs in the batch are treated as negative. Suppose now that i-th pair originates from genome G. Then it seems highly likely that another pair (say, the j-th) in the same batch would also originate from G. Wouldn’t treating the j-th pair as negative be incorrect in that case?
Unfortunately, I failed to infer this detail from the paper or the repository.
I would really appreciate knowing whether this was considered or addressed during training. And it isn’t too much trouble, could you please share the details on how the 2 million pairs were sampled from the genomes?
Thanks very much for your time!
I also reached out by email (hanliu@northwestern.edu) a few days ago but wanted to raise it here as well in case that’s a better channel. I am sorry for the inconvenience.
--
One more separate consideration. The README references an example file data/debug_train.csv, but it there is no such file present in the repository. Would it be possible to upload it?
Hi, thank you for making DNABERT-S publicly available!
I am trying to fine-tune DNABERT-S using data from the RiboGrove database, and I’ve run into a question about how the training dataset was constructed.
The paper that introduces DNADERT-S states that the dataset consists of nearly 2 million randomly selected sequence pairs from 30 thousand unique genomes. If I understand correctly, this implies that some genomes are represented by multiple sequence pairs in the final dataset — is that right?
If so, I’m not sure how the contrastive learning process handles this. As I understand it, within a batch, each i-th pair is treated as positive, while all other pairs in the batch are treated as negative. Suppose now that i-th pair originates from genome G. Then it seems highly likely that another pair (say, the j-th) in the same batch would also originate from G. Wouldn’t treating the j-th pair as negative be incorrect in that case?
Unfortunately, I failed to infer this detail from the paper or the repository.
I would really appreciate knowing whether this was considered or addressed during training. And it isn’t too much trouble, could you please share the details on how the 2 million pairs were sampled from the genomes?
Thanks very much for your time!
I also reached out by email (hanliu@northwestern.edu) a few days ago but wanted to raise it here as well in case that’s a better channel. I am sorry for the inconvenience.
--
One more separate consideration. The README references an example file
data/debug_train.csv, but it there is no such file present in the repository. Would it be possible to upload it?