Claim abandon fails - 64: non-mandatory-script-verify-flag (Data push larger than necessary) #242

New issue

Closed

opened 2018-12-04 22:13:37 +01:00 by tzarebczan · 6 comments

tzarebczan commented

2018-12-04 22:13:37 +01:00

(Migrated from github.com)


64: non-mandatory-script-verify-flag (Data push larger than necessary)
[0100000001a7bfdb31abc9ca5eed68fc7da260476cd1e1d5216186d142ff8c150c1014fd36000000006a47304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366012103d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506ffffffff0174180f00000000001976a914322ab934ea45bde72d3f9c8bb0dc00eb0b031f0188ac00000000]

decoded tx:

{
  "txid": "f913c3b658ba4715000207fd2b925b8ea581e62c1ba093d74ba91f197d1d9c31",
  "size": 191,
  "version": 1,
  "locktime": 0,
  "vin": [
    {
      "txid": "36fd14100c158cff42d1866121d5e1d16c4760a27dfc68ed5ecac9ab31dbbfa7",
      "vout": 0,
      "scriptSig": {
        "asm": "304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366[ALL] 03d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506",
        "hex": "47304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366012103d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.00989300,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 322ab934ea45bde72d3f9c8bb0dc00eb0b031f01 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914322ab934ea45bde72d3f9c8bb0dc00eb0b031f0188ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "bHJXfJPMdafmSgyH2vcWLhPsfLJhwcb8uy"
        ]
      }
    }
  ]
}```

When abandoning certain claims, they are rejected by the mempool. Not sure if this is related to the dust issue we saw in the past (https://github.com/lbryio/lbrycrd/issues/186) as the value here is higher. ```2018-12-04 12:08:15,722 WARNING torba.client.basenetwork:28: Wallet server returned an error. Code: -1 Message: the transaction was rejected by network rules. 64: non-mandatory-script-verify-flag (Data push larger than necessary) [0100000001a7bfdb31abc9ca5eed68fc7da260476cd1e1d5216186d142ff8c150c1014fd36000000006a47304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366012103d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506ffffffff0174180f00000000001976a914322ab934ea45bde72d3f9c8bb0dc00eb0b031f0188ac00000000] ``` decoded tx: ```1d [lbry@victor-dev:~] [lbryumserver] 1 $ ./lbrycrd-cli decoderawtransaction 0100000001a7bfdb31abc9ca5eed68fc7da260476cd1e1d5216186d142ff8c150c1014fd36000000006a47304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366012103d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506ffffffff0174180f00000000001976a914322ab934ea45bde72d3f9c8bb0dc00eb0b031f0188ac00000000 { "txid": "f913c3b658ba4715000207fd2b925b8ea581e62c1ba093d74ba91f197d1d9c31", "size": 191, "version": 1, "locktime": 0, "vin": [ { "txid": "36fd14100c158cff42d1866121d5e1d16c4760a27dfc68ed5ecac9ab31dbbfa7", "vout": 0, "scriptSig": { "asm": "304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366[ALL] 03d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506", "hex": "47304402206df8764da78d06d2c721b788803e0114539408e171280655c783bc7e861b693a02204f563d71754ce7fe648f35a2baa951d2afb7e868880ac0d3b819b3dd10f27366012103d07fe8a0deb7423f552bd1c182251c8a926951f688801e3b2ba681148783a506" }, "sequence": 4294967295 } ], "vout": [ { "value": 0.00989300, "n": 0, "scriptPubKey": { "asm": "OP_DUP OP_HASH160 322ab934ea45bde72d3f9c8bb0dc00eb0b031f01 OP_EQUALVERIFY OP_CHECKSIG", "hex": "76a914322ab934ea45bde72d3f9c8bb0dc00eb0b031f0188ac", "reqSigs": 1, "type": "pubkeyhash", "addresses": [ "bHJXfJPMdafmSgyH2vcWLhPsfLJhwcb8uy" ] } } ] }```

kaykurokawa commented

2018-12-12 19:29:07 +01:00

(Migrated from github.com)

@tzarebczan on frequency of this:

It's hard to tell, but I ran into on spee.ch and another user did also a few weeks ago.
happens to 2 claims I tried on spee.ch (out of like 30 that I abaonded)

@tzarebczan on frequency of this: It's hard to tell, but I ran into on spee.ch and another user did also a few weeks ago. happens to 2 claims I tried on spee.ch (out of like 30 that I abaonded)

kaykurokawa commented

2019-01-07 21:55:49 +01:00

(Migrated from github.com)

The transaction with the claim that the above transaction posted by tom is abandoning

  "hex": "01000000012e4eb792214ac2d5088b9d0a3d117c5cae3f4085e5ef8e867586720fbb0360ee010000006a473044022060c2e6dd693cc863d70d20cbd6972923018f020b1a087e2bd7686f3be4707c0d02207fabb2d2296a24ebdc0b3f9b1457c50c4b1426f1875be616f0b6ac824f55a8aa01210346feb5f53dac025b49c8c97e5b46a6f810c5ae18e62d046c744825f924ae915bffffffff0240420f0000000000fd4b01b52c4861726c65794a6f686e73746f6e65566963746f72696142617272794672617564732d5468756d626e61696c4dff00080110011a9a0108011252080410011a364861726c6579204a6f686e73746f6e65202b20566963746f726961204261727279203d20467261756473202d205468756d626e61696c22002a07537065652e636832012038004a0052005a001a42080110011a30ea6818d431d23d9a7e8171d4088ac0d4f0912e3c2c687dba62d87b71ebaa2dc00563f4625cc0e66e8428ab60f5a40e2b220a696d6167652f6a7065672a5c080110031a40abfec8cd2831acd8011a813fe0ee1ea5f06caa0f0334455cd1b0d2e2d5c376f008419ab9adadb068459c7cc234d985c3007441b7bd642fcbd976bc3401dcfa0c22143b979b5151f88bb1cc96a2f180a218da73c91e066d7576a9147cfbcc0abbd12c538801840487defc02b90a309e88ac800f3a09000000001976a914fad4fc21d4c27cb3fee3b33d0cecab7785eb5c7788ac00000000",
  "txid": "36fd14100c158cff42d1866121d5e1d16c4760a27dfc68ed5ecac9ab31dbbfa7",
  "size": 533,
  "version": 1,
  "locktime": 0,
  "vin": [
    {
      "txid": "ee6003bb0f728675868eefe585403fae5c7c113d0a9d8b08d5c24a2192b74e2e",
      "vout": 1,
      "scriptSig": {
        "asm": "3044022060c2e6dd693cc863d70d20cbd6972923018f020b1a087e2bd7686f3be4707c0d02207fabb2d2296a24ebdc0b3f9b1457c50c4b1426f1875be616f0b6ac824f55a8aa[ALL] 0346feb5f53dac025b49c8c97e5b46a6f810c5ae18e62d046c744825f924ae915b",
        "hex": "473044022060c2e6dd693cc863d70d20cbd6972923018f020b1a087e2bd7686f3be4707c0d02207fabb2d2296a24ebdc0b3f9b1457c50c4b1426f1875be616f0b6ac824f55a8aa01210346feb5f53dac025b49c8c97e5b46a6f810c5ae18e62d046c744825f924ae915b"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.01000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_CLAIM_NAME 4861726c65794a6f686e73746f6e65566963746f72696142617272794672617564732d5468756d626e61696c 080110011a9a0108011252080410011a364861726c6579204a6f686e73746f6e65202b20566963746f726961204261727279203d20467261756473202d205468756d626e61696c22002a07537065652e636832012038004a0052005a001a42080110011a30ea6818d431d23d9a7e8171d4088ac0d4f0912e3c2c687dba62d87b71ebaa2dc00563f4625cc0e66e8428ab60f5a40e2b220a696d6167652f6a7065672a5c080110031a40abfec8cd2831acd8011a813fe0ee1ea5f06caa0f0334455cd1b0d2e2d5c376f008419ab9adadb068459c7cc234d985c3007441b7bd642fcbd976bc3401dcfa0c22143b979b5151f88bb1cc96a2f180a218da73c91e06 OP_2DROP OP_DROP OP_DUP OP_HASH160 7cfbcc0abbd12c538801840487defc02b90a309e OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "b52c4861726c65794a6f686e73746f6e65566963746f72696142617272794672617564732d5468756d626e61696c4dff00080110011a9a0108011252080410011a364861726c6579204a6f686e73746f6e65202b20566963746f726961204261727279203d20467261756473202d205468756d626e61696c22002a07537065652e636832012038004a0052005a001a42080110011a30ea6818d431d23d9a7e8171d4088ac0d4f0912e3c2c687dba62d87b71ebaa2dc00563f4625cc0e66e8428ab60f5a40e2b220a696d6167652f6a7065672a5c080110031a40abfec8cd2831acd8011a813fe0ee1ea5f06caa0f0334455cd1b0d2e2d5c376f008419ab9adadb068459c7cc234d985c3007441b7bd642fcbd976bc3401dcfa0c22143b979b5151f88bb1cc96a2f180a218da73c91e066d7576a9147cfbcc0abbd12c538801840487defc02b90a309e88ac",
        "type": "nonstandard"
      }
    }, 
    {
      "value": 1.54800000,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 fad4fc21d4c27cb3fee3b33d0cecab7785eb5c77 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914fad4fc21d4c27cb3fee3b33d0cecab7785eb5c7788ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "bbbYmdxaojfFijmkA5GWbqHRAXAt6mYCqi"
        ]
      }
    }
  ],
  "blockhash": "2d46fc8d70a7a7adb8f0975357f3cf09c5c0023ed5f87fca459b7cb07047d8f0",
  "confirmations": 114793,
  "time": 1528564810,
  "blocktime": 1528564810
}

The transaction with the claim that the above transaction posted by tom is abandoning ```{ "hex": "01000000012e4eb792214ac2d5088b9d0a3d117c5cae3f4085e5ef8e867586720fbb0360ee010000006a473044022060c2e6dd693cc863d70d20cbd6972923018f020b1a087e2bd7686f3be4707c0d02207fabb2d2296a24ebdc0b3f9b1457c50c4b1426f1875be616f0b6ac824f55a8aa01210346feb5f53dac025b49c8c97e5b46a6f810c5ae18e62d046c744825f924ae915bffffffff0240420f0000000000fd4b01b52c4861726c65794a6f686e73746f6e65566963746f72696142617272794672617564732d5468756d626e61696c4dff00080110011a9a0108011252080410011a364861726c6579204a6f686e73746f6e65202b20566963746f726961204261727279203d20467261756473202d205468756d626e61696c22002a07537065652e636832012038004a0052005a001a42080110011a30ea6818d431d23d9a7e8171d4088ac0d4f0912e3c2c687dba62d87b71ebaa2dc00563f4625cc0e66e8428ab60f5a40e2b220a696d6167652f6a7065672a5c080110031a40abfec8cd2831acd8011a813fe0ee1ea5f06caa0f0334455cd1b0d2e2d5c376f008419ab9adadb068459c7cc234d985c3007441b7bd642fcbd976bc3401dcfa0c22143b979b5151f88bb1cc96a2f180a218da73c91e066d7576a9147cfbcc0abbd12c538801840487defc02b90a309e88ac800f3a09000000001976a914fad4fc21d4c27cb3fee3b33d0cecab7785eb5c7788ac00000000", "txid": "36fd14100c158cff42d1866121d5e1d16c4760a27dfc68ed5ecac9ab31dbbfa7", "size": 533, "version": 1, "locktime": 0, "vin": [ { "txid": "ee6003bb0f728675868eefe585403fae5c7c113d0a9d8b08d5c24a2192b74e2e", "vout": 1, "scriptSig": { "asm": "3044022060c2e6dd693cc863d70d20cbd6972923018f020b1a087e2bd7686f3be4707c0d02207fabb2d2296a24ebdc0b3f9b1457c50c4b1426f1875be616f0b6ac824f55a8aa[ALL] 0346feb5f53dac025b49c8c97e5b46a6f810c5ae18e62d046c744825f924ae915b", "hex": "473044022060c2e6dd693cc863d70d20cbd6972923018f020b1a087e2bd7686f3be4707c0d02207fabb2d2296a24ebdc0b3f9b1457c50c4b1426f1875be616f0b6ac824f55a8aa01210346feb5f53dac025b49c8c97e5b46a6f810c5ae18e62d046c744825f924ae915b" }, "sequence": 4294967295 } ], "vout": [ { "value": 0.01000000, "n": 0, "scriptPubKey": { "asm": "OP_CLAIM_NAME 4861726c65794a6f686e73746f6e65566963746f72696142617272794672617564732d5468756d626e61696c 080110011a9a0108011252080410011a364861726c6579204a6f686e73746f6e65202b20566963746f726961204261727279203d20467261756473202d205468756d626e61696c22002a07537065652e636832012038004a0052005a001a42080110011a30ea6818d431d23d9a7e8171d4088ac0d4f0912e3c2c687dba62d87b71ebaa2dc00563f4625cc0e66e8428ab60f5a40e2b220a696d6167652f6a7065672a5c080110031a40abfec8cd2831acd8011a813fe0ee1ea5f06caa0f0334455cd1b0d2e2d5c376f008419ab9adadb068459c7cc234d985c3007441b7bd642fcbd976bc3401dcfa0c22143b979b5151f88bb1cc96a2f180a218da73c91e06 OP_2DROP OP_DROP OP_DUP OP_HASH160 7cfbcc0abbd12c538801840487defc02b90a309e OP_EQUALVERIFY OP_CHECKSIG", "hex": "b52c4861726c65794a6f686e73746f6e65566963746f72696142617272794672617564732d5468756d626e61696c4dff00080110011a9a0108011252080410011a364861726c6579204a6f686e73746f6e65202b20566963746f726961204261727279203d20467261756473202d205468756d626e61696c22002a07537065652e636832012038004a0052005a001a42080110011a30ea6818d431d23d9a7e8171d4088ac0d4f0912e3c2c687dba62d87b71ebaa2dc00563f4625cc0e66e8428ab60f5a40e2b220a696d6167652f6a7065672a5c080110031a40abfec8cd2831acd8011a813fe0ee1ea5f06caa0f0334455cd1b0d2e2d5c376f008419ab9adadb068459c7cc234d985c3007441b7bd642fcbd976bc3401dcfa0c22143b979b5151f88bb1cc96a2f180a218da73c91e066d7576a9147cfbcc0abbd12c538801840487defc02b90a309e88ac", "type": "nonstandard" } }, { "value": 1.54800000, "n": 1, "scriptPubKey": { "asm": "OP_DUP OP_HASH160 fad4fc21d4c27cb3fee3b33d0cecab7785eb5c77 OP_EQUALVERIFY OP_CHECKSIG", "hex": "76a914fad4fc21d4c27cb3fee3b33d0cecab7785eb5c7788ac", "reqSigs": 1, "type": "pubkeyhash", "addresses": [ "bbbYmdxaojfFijmkA5GWbqHRAXAt6mYCqi" ] } } ], "blockhash": "2d46fc8d70a7a7adb8f0975357f3cf09c5c0023ed5f87fca459b7cb07047d8f0", "confirmations": 114793, "time": 1528564810, "blocktime": 1528564810 } ```

tzarebczan commented

2019-01-08 19:35:22 +01:00

(Migrated from github.com)

I have a wallet file with the claim that has this issue if anyone needs it (sent to @kaykurokawa already)

kaykurokawa commented

2019-01-21 02:12:39 +01:00

(Migrated from github.com)

The problem originates from the claim that is being abandoned that I posted above, txid: "36fd14100c158cff42d1866121d5e1d16c4760a27dfc68ed5ecac9ab31dbbfa7"

If you check the value component of the name claim on vout:0, you can check it is exactly 255 bytes long. This means that it should use OP_PUSHDATA1 (0x4c) to push the data on to the stack, however OP_PUSHDATA2 (0x4d) is used, and this will trigger the CheckMinimalPush() to return false.

It looks like the current version of Torba handles this properly as seen here: https://github.com/lbryio/torba/blob/master/torba/client/basescript.py#L54 . The above transaction is from June 9th 2018, so I think it was created using the now deprecated lbryum code base.

There is some additional tricky things here as well related to how lbrycrd pareses valid claim transactions. https://github.com/lbryio/lbrycrd/blob/master/src/nameclaim.cpp#L59 . You will notice that claim names and values of size 1 character cannot be pushed onto the stack using, OP_1NEGATE, OP_1, OP_2, ... OP_16 , since these op codes are > OP_PUSHDATA4 . Doing so results in the claim being invalid. It looks like both lbrycrd ( https://github.com/lbryio/lbrycrd/blob/master/src/script/script.h#L438 ) and torba never uses OP_1NEGATE... OP_16 to push claim names and values on to the stack so it should not create invalid claims. However, these kinds of pushes will also get caught by CheckMinimalPush() and will get rejected by network policy.

As for the simple solution, I believe it would be fairly safe to disable this network policy implemented by CheckMinimalPush(), it was originally implemented here
698c6abb25 (diff-be2905e2f5218ecdbe4e55637dac75f3) for BIP62. BIP62 https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki however has been withdrawn and is replaced by segwit. I believe Bitcoin Core still retains this network policy, I presume just to encourage people to be more efficient in their data pushes.

A bit more involved solution would just be to disable CheckMinimalPush() for claim transactions so people can abandon their claims created in lbryum and avoid potential issues with the OP_1NEGATE.. I think this method could also be fine and maybe more preferable.

The error code that is being flagged here is: SCRIPT_ERR_MINIMALDATA , and it is checked here https://github.com/lbryio/lbrycrd/blob/master/src/script/interpreter.cpp#L210 by function CheckMinimalPush() . This function tries to verify that when pushing data to the stack, the method that is used is of minimal size (as there are multiple ways of pushing data on to the stack, https://en.bitcoin.it/wiki/Script). It is not consensus policy, it is network policy and is called when accepting transaction to the mempool. The problem originates from the claim that is being abandoned that I posted above, txid: "36fd14100c158cff42d1866121d5e1d16c4760a27dfc68ed5ecac9ab31dbbfa7" If you check the value component of the name claim on vout:0, you can check it is exactly 255 bytes long. This means that it should use OP_PUSHDATA1 (0x4c) to push the data on to the stack, however OP_PUSHDATA2 (0x4d) is used, and this will trigger the CheckMinimalPush() to return false. It looks like the current version of Torba handles this properly as seen here: https://github.com/lbryio/torba/blob/master/torba/client/basescript.py#L54 . The above transaction is from June 9th 2018, so I think it was created using the now deprecated lbryum code base. There is some additional tricky things here as well related to how lbrycrd pareses valid claim transactions. https://github.com/lbryio/lbrycrd/blob/master/src/nameclaim.cpp#L59 . You will notice that claim names and values of size 1 character cannot be pushed onto the stack using, OP_1NEGATE, OP_1, OP_2, ... OP_16 , since these op codes are > OP_PUSHDATA4 . Doing so results in the claim being invalid. It looks like both lbrycrd ( https://github.com/lbryio/lbrycrd/blob/master/src/script/script.h#L438 ) and torba never uses OP_1NEGATE... OP_16 to push claim names and values on to the stack so it should not create invalid claims. However, these kinds of pushes will also get caught by CheckMinimalPush() and will get rejected by network policy. As for the simple solution, I believe it would be fairly safe to disable this network policy implemented by CheckMinimalPush(), it was originally implemented here https://github.com/bitcoin/bitcoin/commit/698c6abb25c1fbbc7fa4ba46b60e9f17d97332ef#diff-be2905e2f5218ecdbe4e55637dac75f3 for BIP62. BIP62 https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki however has been withdrawn and is replaced by segwit. I believe Bitcoin Core still retains this network policy, I presume just to encourage people to be more efficient in their data pushes. A bit more involved solution would just be to disable CheckMinimalPush() for claim transactions so people can abandon their claims created in lbryum and avoid potential issues with the OP_1NEGATE.. I think this method could also be fine and maybe more preferable.

lbrynaut commented

2019-01-30 16:09:05 +01:00

(Migrated from github.com)

https://bitcoincore.org/en/2016/01/26/segwit-benefits/

Related: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2014-November/006878.html https://bitcoincore.org/en/2016/01/26/segwit-benefits/

BrannonKing commented

2019-09-17 04:40:17 +02:00

(Migrated from github.com)

I put @kaykurokawa 's suggested fix into master. It now disables the minimal push validation on claims.