Different output on ROCm

[danielzgtg]

The output text is different on ROCm GPU. It should have been the same.

Code

def to_gpu(x):
    return x #.to("cuda:0") # Uncomment to test
tokenizer = AutoTokenizer.from_pretrained("microsoft/GODEL-v1_1-large-seq2seq")
model = to_gpu(AutoModelForSeq2SeqLM.from_pretrained("microsoft/GODEL-v1_1-large-seq2seq"))
def generate(instruction, knowledge, dialog):
    if knowledge != '':
        knowledge = '[KNOWLEDGE] ' + knowledge
    dialog = ' EOS '.join(dialog)
    query = f"{instruction} [CONTEXT] {dialog} {knowledge}"
    input_ids = to_gpu(tokenizer(f"{query}", return_tensors="pt")).input_ids
    outputs = model.generate(input_ids, max_length=128, min_length=8, top_p=0.9, do_sample=True)
    output = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return output
instruction = f'Instruction: given a dialog context, you need to response empathically.'
# Leave the knowldge empty
knowledge = ''
dialog = [
    'Does money buy happiness?',
    'It is a question. Money buys you a lot of things, but not enough to buy happiness.',
    'What is the best way to buy happiness ?'
]
# dialog = ["Hey my name is Thomas! How are you?"] # Uncomment to test
response = generate(instruction, knowledge, dialog)
print(response)

requirements.txt

absl-py==1.0.0
astunparse==1.6.3
cachetools==5.1.0
certifi==2022.12.7
charset-normalizer==3.0.1
click==8.1.3
contourpy==1.0.5
cycler==0.11.0
filelock==3.9.0
flatbuffers==1.12
fonttools==4.37.4
gast==0.4.0
google-auth==2.6.6
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.46.3
h5py==3.7.0
huggingface-hub==0.12.1
idna==3.4
joblib==1.2.0
keras==2.9.0
Keras-Preprocessing==1.1.2
keybert==0.7.0
kiwisolver==1.4.4
libclang==14.0.1
Markdown==3.3.7
markdown-it-py==2.2.0
matplotlib==3.6.1
mdurl==0.1.2
nltk==3.8.1
numpy==1.24.2
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
oauthlib==3.2.0
opt-einsum==3.3.0
packaging==23.0
pandas==1.4.2
Pillow==9.2.0
protobuf==3.19.4
psutil==5.9.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.14.0
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2022.1
PyYAML==6.0
regex==2022.10.31
requests==2.28.2
requests-oauthlib==1.3.1
rich==13.3.1
rsa==4.8
scikit-learn==1.2.1
scipy==1.10.1
sentence-transformers==2.2.2
sentencepiece==0.1.96
six==1.16.0
tensorboard==2.9.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.9.1
tensorflow-estimator==2.9.0
tensorflow-io-gcs-filesystem==0.26.0
termcolor==1.1.0
threadpoolctl==3.1.0
tokenizers==0.13.2
torch==1.13.1+rocm5.2
torchvision==0.14.1
tqdm==4.64.1
transformers==4.26.1
typing_extensions==4.5.0
urllib3==1.26.14
Werkzeug==2.1.2
wrapt==1.14.1

Output

CPU	GPU
Money doesn't buy happiness. It only gives you money for things you have. It doesn't buy happiness.	Be happy, not to be.
Hello Thomas, I’m fine. How are you?	Hi, I'm good. How are you?