Skip to content

[Bug]: Inaccuracy introduced when non-zero fractional bits #110

Description

@cameronpoe

Summary

In the code I am trying to run, I am working with numbers that are over 64 bit with no fractional bits. At some point, I need to bitshift down and store in a new Fxp variable that has fractional bits. However, it seems that assigning an Fxp variable with non-zero fractional bits any whole number that is larger than 64 bits causes inaccuracy. I assume this is because something under the hood is converting to np.float64 or np.int64. However, this is not expected behavior.

Minimal reproducible example

a = Fxp(None, signed=True, n_int=63, n_frac=0, round='around', saturate='overflow')
a.set_val(2**63 - 1)

b = Fxp(None, signed=True, n_int=65, n_frac=0, round='around', saturate='overflow')
b.set_val(a + a + a)
print(a)
print(b)
print(a.info(verbose=3))
print(b.info(verbose=3))

c = Fxp(None, signed=True, n_int=65, n_frac=2, round='around', saturate='overflow')
c.set_val(a + a + a)
print(c)
print(c.info(verbose=3))

Actual behavior

9223372036854775807
27670116110564327421
dtype = fxp-s64/0
Value = 9223372036854775807
inaccuracy = True
extended_prec = True

Signed		=	True
Word bits	=	64
Fract bits	=	0
Int bits	=	63
Val data type	=	<class 'int'>

Upper		=	9.223372036854776e+18
Lower		=	-9.223372036854776e+18
Precision	=	1.0
Overflow	=	saturate
Rounding	=	trunc
Shifting	=	expand

None
dtype = fxp-s66/0
Value = 27670116110564327421
inaccuracy = True
extended_prec = True

Signed		=	True
Word bits	=	66
Fract bits	=	0
Int bits	=	65
Val data type	=	<class 'int'>

Upper		=	3.6893488147419103e+19
Lower		=	-3.6893488147419103e+19
Precision	=	1.0
Overflow	=	saturate
Rounding	=	trunc
Shifting	=	expand

None
2.7670116110564327e+19
dtype = fxp-s68/2
Value = 2.7670116110564327e+19
inaccuracy = True
extended_prec = True

Signed		=	True
Word bits	=	68
Fract bits	=	2
Int bits	=	65
Val data type	=	<class 'float'>

Upper		=	3.6893488147419103e+19
Lower		=	-3.6893488147419103e+19
Precision	=	0.25
Overflow	=	saturate
Rounding	=	trunc
Shifting	=	expand

None

Expected behavior

I would expect the b and c to be the same exact value, except that c has two fractional bits that are 0.

fxpmath version

0.4.10

Python version

3.13.2

NumPy version

2.3.1

Operating system

Ubuntu 26.04

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions