HashXHistoryPrefixRow issue with endianness of TxNums written to DB #91

New issue

Open

opened 2022-09-01 19:53:17 +02:00 by moodyjon · 1 comment

moodyjon commented

2022-09-01 19:53:17 +02:00

(Migrated from github.com)

The endianness should be corrected to big-endian like other numbers in the database OR it should be locked to be little-endian specifically.

For now, I am correcting (herald.go) HashXHistoryValue implementation to read TxNums in little-endian form.

Python scribe when run on my machine (MacOS ARM64) produces little-endian txnums for hashXhistory prefix. (I just looked up M1 mac behavior, and it's little-endian by default) But there could be some more exotic platform that writes txnums big-endian fashion. The endianness should be corrected to big-endian like other numbers in the database OR it should be locked to be little-endian specifically. For now, I am correcting (herald.go) HashXHistoryValue implementation to read TxNums in little-endian form.

jackrobison commented

2022-09-14 03:10:40 +02:00

(Migrated from github.com)

This is a consequence of using an array.array('I'), which uses the native endianness. And I think it's actually worse - docs say It can be 16 bits or 32 bits depending on the platform. (https://docs.python.org/3/library/array.html).
Perhaps another library offers good enough performance and the ability to specify endianness, or we could make our own fast little endian uint32 array serializer/deserializer in cython.

This is a consequence of using an `array.array('I')`, which uses the native endianness. And I think it's actually worse - docs say `It can be 16 bits or 32 bits depending on the platform.` (https://docs.python.org/3/library/array.html). Perhaps another library offers good enough performance and the ability to specify endianness, or we could make our own fast little endian uint32 array serializer/deserializer in cython.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/hub#91

No description provided.