The arisen::binary_extension type

Let's fully explain what the arisen::binary_extension type is, what it does, and why we need it for contract upgrades in certain situations.

You can find the implementation of arisen::binary_extension in the arisen.cdt repository in file: arisen.cdt/libraries/arisenlib/core/arisen/binary_extension.hpp.

Our primary concern when using this type is when we are adding a new field to a smart contract's data structure that is currently utilized in an arisen::multi_index type (AKA a table), or when adding a new parameter to an action declaration.

By wrapping the new field in an arisen::binary_extension, you are enabling your contract to be backwards compatible for future use. Note that this new field/parameter MUST be appended at the end of a data structure (this is due to implementation details in arisen::multi_index, which relies on the boost::multi_index type), or at the end of the parameter list in an action declaration.

If you don't wrap the new field in an arisen::binary_extension, the arisen::multi_index table will be reformatted in such a way that disallows reads to the former datum; or in an action's case, the function will be uncallable.

But let's see how the arisen::binary_extension type works with a good example.

Take a moment to study this smart contract and its corresponding .abi.

This contract not only serves as a good example to the arisen::binary_extension type, but can also be used as a gateway for developing smart contracts on the arisen protocol.

binary_extension_contract.hpp

#include <arisen/contract.hpp>         // arisen::contract
#include <arisen/binary_extension.hpp> // arisen::binary_extension
#include <arisen/datastream.hpp>       // arisen::datastream
#include <arisen/name.hpp>             // arisen::name
#include <arisen/multi_index.hpp>      // arisen::indexed_by, arisen::multi_index
#include <arisen/print.hpp>            // arisen::print_f

class [[arisen::contract]] binary_extension_contract : public arisen::contract {
public:
   using contract::contract;
   binary_extension_contract(arisen::name receiver, arisen::name code, arisen::datastream<const char*> ds)
      : contract{receiver, code, ds}, _table{receiver, receiver.value}
   { }

   [[arisen::action]] void regpkey (arisen::name primary_key);                ///< Register primary key.
   [[arisen::action]] void printbyp(arisen::name primary_key);                ///< Print by primary key.
   [[arisen::action]] void printbys(arisen::name secondary_key);              ///< Print by secondary key.
   [[arisen::action]] void modifyp (arisen::name primary_key, arisen::name n); ///< Modify primary key by primary key.
   [[arisen::action]] void modifys (arisen::name primary_key, arisen::name n); ///< Modify secondary key by primary key.

   struct [[arisen::table]] structure {
      arisen::name _primary_key;
      arisen::name _secondary_key;
         
      uint64_t primary_key()   const { return _primary_key.value;   }
      uint64_t secondary_key() const { return _secondary_key.value; }
   };

   using index1 = arisen::indexed_by<"index1"_n, arisen::const_mem_fun<structure, uint64_t, &structure::primary_key>>;
   using index2 = arisen::indexed_by<"index2"_n, arisen::const_mem_fun<structure, uint64_t, &structure::secondary_key>>;
   using table  = arisen::multi_index<"table"_n, structure, index1, index2>;

private:
   table _table;
};

binary_extension_contract.cpp

#include "binary_extension_contract.hpp"

using arisen::name;

[[arisen::action]] void binary_extension_contract::regpkey(name primary_key) {
   arisen::print_f("`regpkey` executing.\n");
   
   auto index{_table.get_index<"index1"_n>()}; ///< `index` represents `_table` organized by `index1`.
   auto iter {index.find(primary_key.value) }; ///< Note: the type returned by `index.find` is different than the type returned by `_table.find`.
   
   if (iter == _table.get_index<"index1"_n>().end()) {
      arisen::print_f("`_primary_key`: % not found; registering.\n", primary_key.to_string());
      _table.emplace(_self, [&](auto& row) {
         row._primary_key   = primary_key;
         row._secondary_key = "nothin"_n;
      });
   }
   else {
      arisen::print_f("`_primary_key`: % found; not registering.\n", primary_key.to_string());
   }

   arisen::print_f("`regpkey` finished executing.\n");
}

[[arisen::action]] void binary_extension_contract::printbyp(arisen::name primary_key) {
   arisen::print_f("`printbyp` executing.\n");
   
   auto index{_table.get_index<"index1"_n>()};
   auto iter {index.find(primary_key.value) };
   
   if (iter != _table.get_index<"index1"_n>().end()) {
      arisen::print_f("`_primary_key`: % found; printing.\n", primary_key.to_string());
      arisen::print_f("{%, %}\n", iter->_primary_key, iter->_secondary_key);
   }
   else {
      arisen::print_f("`_primary_key`: % not found; not printing.\n", primary_key.to_string());
   }

   arisen::print_f("`printbyp` finished executing.\n");
}

[[arisen::action]] void binary_extension_contract::printbys(arisen::name secondary_key) {
   arisen::print_f("`printbys` executing.\n");
   
   auto index{_table.get_index<"index2"_n>()};
   auto iter {index.find(secondary_key.value)};
   
   if (iter != _table.get_index<"index2"_n>().end()) {
      arisen::print_f("`_secondary_key`: % found; printing.\n", secondary_key.to_string());
      printbyp(iter->_primary_key);
   }
   else {
      arisen::print_f("`_secondary_key`: % not found; not printing.\n", secondary_key.to_string());
   }

   arisen::print_f("`printbys` finished executing.\n");
}

[[arisen::action]] void binary_extension_contract::modifyp(arisen::name primary_key, name n) {
   arisen::print_f("`modifyp` executing.\n");
   
   auto index{_table.get_index<"index1"_n>()};
   auto iter {index.find(primary_key.value)};
   
   if (iter != _table.get_index<"index1"_n>().end()) {
      arisen::print_f("`_primary_key`: % found; modifying `_primary_key`.\n", primary_key.to_string());
      index.modify(iter, _self, [&](auto& row) {
         row._primary_key = n;
      });
   }
   else {
      arisen::print_f("`_primary_key`: % not found; not modifying `_primary_key`.\n", primary_key.to_string());
   }

   arisen::print_f("`modifyp` finished executing.\n");
}

[[arisen::action]] void binary_extension_contract::modifys(arisen::name primary_key, name n) {
   arisen::print_f("`modifys` executing.\n");
   
   auto index{_table.get_index<"index1"_n>()};
   auto iter {index.find(primary_key.value)};
   
   if (iter != _table.get_index<"index1"_n>().end()) {
      arisen::print_f("`_primary_key`: % found; modifying `_secondary_key`.\n", primary_key.to_string());
      index.modify(iter, _self, [&](auto& row) {
         row._secondary_key = n;
      });
   }
   else {
      arisen::print_f("`_primary_key`: % not found; not modifying `_secondary_key`.\n", primary_key.to_string());
   }

   arisen::print_f("`modifys` finished executing.\n");
}

binary_extension_contract.abi

{
    "____comment": "This file was generated with arisen-abigen. DO NOT EDIT ",
    "version": "arisen::abi/1.1",
    "types": [],
    "structs": [
        {
            "name": "modifyp",
            "base": "",
            "fields": [
                {
                    "name": "primary_key",
                    "type": "name"
                },
                {
                    "name": "n",
                    "type": "name"
                }
            ]
        },
        {
            "name": "modifys",
            "base": "",
            "fields": [
                {
                    "name": "primary_key",
                    "type": "name"
                },
                {
                    "name": "n",
                    "type": "name"
                }
            ]
        },
        {
            "name": "printbyp",
            "base": "",
            "fields": [
                {
                    "name": "primary_key",
                    "type": "name"
                }
            ]
        },
        {
            "name": "printbys",
            "base": "",
            "fields": [
                {
                    "name": "secondary_key",
                    "type": "name"
                }
            ]
        },
        {
            "name": "regpkey",
            "base": "",
            "fields": [
                {
                    "name": "primary_key",
                    "type": "name"
                }
            ]
        },
        {
            "name": "structure",
            "base": "",
            "fields": [
                {
                    "name": "_primary_key",
                    "type": "name"
                },
                {
                    "name": "_secondary_key",
                    "type": "name"
                }
            ]
        }
    ],
    "actions": [
        {
            "name": "modifyp",
            "type": "modifyp",
            "ricardian_contract": ""
        },
        {
            "name": "modifys",
            "type": "modifys",
            "ricardian_contract": ""
        },
        {
            "name": "printbyp",
            "type": "printbyp",
            "ricardian_contract": ""
        },
        {
            "name": "printbys",
            "type": "printbys",
            "ricardian_contract": ""
        },
        {
            "name": "regpkey",
            "type": "regpkey",
            "ricardian_contract": ""
        }
    ],
    "tables": [
        {
            "name": "table",
            "type": "structure",
            "index_type": "i64",
            "key_names": [],
            "key_types": []
        }
    ],
    "ricardian_clauses": [],
    "variants": []

Take note of the action regpkey, and the struct structure in con.hpp and con.cpp; the parts of the contract we will be upgrading.

binary_extension_contract.hpp

[[arisen::action]] void regpkey (arisen::name primary_key);
struct [[arisen::table]] structure {
    arisen::name _primary_key;
    arisen::name _secondary_key;

    uint64_t primary_key()   const { return _primary_key.value;   }
    uint64_t secondary_key() const { return _secondary_key.value; }
};

binary_extension_contract.cpp

[[arisen::action]] void binary_extension_contract::regpkey(name primary_key) {
   arisen::print_f("`regpkey` executing.\n");
   
   auto index{_table.get_index<"index1"_n>()}; ///< `index` represents `_table` organized by `index1`.
   auto iter {index.find(primary_key.value) }; ///< Note: the type returned by `index.find` is different than the type returned by `_table.find`.
   
   if (iter == _table.get_index<"index1"_n>().end()) {
      arisen::print_f("`_primary_key`: % not found; registering.\n", primary_key.to_string());
      _table.emplace(_self, [&](auto& row) {
         row._primary_key   = primary_key;
         row._secondary_key = "nothin"_n;
      });
   }
   else {
      arisen::print_f("`_primary_key`: % found; not registering.\n", primary_key.to_string());
   }

   arisen::print_f("`regpkey` finished executing.\n");
}

And their corresponding sections in the .abi files:

binary_extension_contract.abi

{
    "name": "regpkey",
    "base": "",
    "fields": [
        {
            "name": "primary_key",
            "type": "name"
        }
    ]
}
{
    "name": "structure",
    "base": "",
    "fields": [
        {
            "name": "_primary_key",
            "type": "name"
        },
        {
            "name": "_secondary_key",
            "type": "name"
        }
    ]
}

Now, let's start up a blockchain instance, compile this smart contract, and test it out.

~/binary_extension_contract $ arisen-cpp binary_extension_contract.cpp -o binary_extension_contract.wasm
~/binary_extension_contract $ arisecli set contract arisen ./
Reading WASM from /Users/john.debord/binary_extension_contract/binary_extension_contract.wasm...
Publishing contract...
executed transaction: 6c5c7d869a5be67611869b5f300bc452bc57d258d11755f12ced99c7d7fe154c  4160 bytes  729 us
#         arisen <= arisen::setcode               "0000000000ea30550000d7600061736d01000000018f011760000060017f0060027f7f0060037f7f7f017f6000017e60067...
#         arisen <= arisen::setabi                "0000000000ea3055d1020e656f73696f3a3a6162692f312e310006076d6f646966797000020b7072696d6172795f6b65790...
warning: transaction executed locally, but may not be confirmed by the network yet

Next, let's push some data to our contract.

~/binary_extension_contract $ cleos push action arisen regpkey '{"primary_key":"arisen.name"}' -p arisen
executed transaction: 3c708f10dcbf4412801d901eb82687e82287c2249a29a2f4e746d0116d6795f0  104 bytes  248 us
#         arisen <= arisen::regpkey               {"primary_key":"arisen.name"}
[(arisen,regpkey)->arisen]: CONSOLE OUTPUT BEGIN =====================
`regpkey` executing.
`_primary_key`: arisen.name not found; registering.
`regpkey` finished executing.
[(arisen,regpkey)->arisen]: CONSOLE OUTPUT END   =====================
warning: transaction executed locally, but may not be confirmed by the network yet

Finally, let's read back the data we have just written.

~/binary_extension_contract $ cleos push action arisen printbyp '{"primary_key":"arisen.name"}' -p arisen
executed transaction: e9b77d3cfba322a7a3a93970c0c883cb8b67e2072a26d714d46eef9d79b2f55e  104 bytes  227 us
#         arisen <= arisen::printbyp              {"primary_key":"arisen.name"}
[(arisen,printbyp)->arisen]: CONSOLE OUTPUT BEGIN =====================
`printbyp` executing.
`_primary_key`: arisen.name found; printing.
{arisen.name, nothin}
`printbyp` finished executing.
[(arisen,printbyp)->arisen]: CONSOLE OUTPUT END   =====================
warning: transaction executed locally, but may not be confirmed by the network yet

Now, let's upgrade the smart contract by adding a new field to the table and a new parameter to an action while NOT wrapping the new field/parameter in an arisen::binary_extension type and see what happens:

binary_extension_contract.hpp

+[[arisen::action]] void regpkey (arisen::name primary_key, arisen::name secondary_key);
-[[arisen::action]] void regpkey (arisen::name primary_key);
struct [[arisen::table]] structure {
    arisen::name _primary_key;
    arisen::name _secondary_key;
+   arisen::name _non_binary_extension_key;

    uint64_t primary_key()   const { return _primary_key.value;   }
    uint64_t secondary_key() const { return _secondary_key.value; }
};

binary_extension_contract.cpp

+[[arisen::action]] void binary_extension_contract::regpkey(name primary_key, name secondary_key) {
-[[arisen::action]] void binary_extension_contract::regpkey(name primary_key) {
   arisen::print_f("`regpkey` executing.\n");
   
   auto index{_table.get_index<"index1"_n>()}; ///< `index` represents `_table` organized by `index1`.
   auto iter {index.find(primary_key.value) }; ///< Note: the type returned by `index.find` is different than the type returned by `_table.find`.
   
   if (iter == _table.get_index<"index1"_n>().end()) {
      arisen::print_f("`_primary_key`: % not found; registering.\n", primary_key.to_string());
      _table.emplace(_self, [&](auto& row) {
         row._primary_key   = primary_key;
+        if (secondary_key) {
+           row._secondary_key = secondary_key;
+         }
+         else {
            row._secondary_key = "nothin"_n;
+         }
      });
   }
   else {
      arisen::print_f("`_primary_key`: % found; not registering.\n", primary_key.to_string());
   }

   arisen::print_f("`regpkey` finished executing.\n");
}

binary_extension_contract.abi

{
    "name": "regpkey",
    "base": "",
    "fields": [
        {
            "name": "primary_key",
            "type": "name"
+       },
+       {
+           "name": "secondary_key",
+           "type": "name"
        }
    ]
}

{
    "name": "structure",
    "base": "",
    "fields": [
        {
            "name": "_primary_key",
            "type": "name"
        },
        {
            "name": "_secondary_key",
            "type": "name"
+       },
+   {
+           "name": "_non_binary_extension_key",
+           "type": "name"
        }
    ]
}

Next, let's upgrade the contract and try to read from our table and write to our table the original way:

binary_extension_contract.abi

{
    "name": "regpkey",
    "base": "",
    "fields": [
        {
            "name": "primary_key",
            "type": "name"
        },
        {
            "name": "secondary_key",
+           "type": "name$"
-           "type": "name"
        }
    ]
}
{
    "name": "structure",
    "base": "",
    "fields": [
        {
            "name": "_primary_key",
            "type": "name"
        },
        {
            "name": "_secondary_key",
            "type": "name"
        },
    {
+           "name": "_binary_extension_key",
+           "type": "name$"
-           "name": "_non_binary_extension_key",
-           "type": "name"
        }
    ]
}
{
    "name": "secondary_key",
+   "type": "name$"
-   "type": "name"
}

Note the $ after the types now; this indicates that this type is an arisen::binary_extension type field.

{
    "name": "_binary_extension_key",
+   "type": "name$"
-   "type": "name"
}

Now, let's upgrade the contract again and try to read/write from/to our table:

~/binary_extension_contract $ cleos set contract arisen ./
Reading WASM from /Users/john.debord/binary_extension_contract/binary_extension_contract.wasm...
Publishing contract...
executed transaction: 497584d4e43ec114dbef83c134570492893f49eacb555d0cd47d08ea4a3a72f7  4696 bytes  648 us
#         arisen <= arisen::setcode               "0000000000ea30550000cb6a0061736d01000000018f011760000060017f0060027f7f0060037f7f7f017f6000017e60017...
#         arisen <= arisen::setabi                "0000000000ea305581030e656f73696f3a3a6162692f312e310006076d6f646966797000020b7072696d6172795f6b65790...
warning: transaction executed locally, but may not be confirmed by the network yet
~/binary_extension_contract $ arisecli push action arisen printbyp '{"primary_key":"arisen.name"}' -p arisen
executed transaction: 6108f3206e1824fe3a1fdcbc2fe733f38dc07ae3d411a1ccf777ecef56ddec97  104 bytes  224 us
#         arisen <= arisen::printbyp              {"primary_key":"arisen.name"}
[(arisen,printbyp)->arisen]: CONSOLE OUTPUT BEGIN =====================
`printbyp` executing.
`_primary_key`: arisen.name found; printing.
{arisen.name, nothin}
`printbyp` finished executing.
[(arisen,printbyp)->arisen]: CONSOLE OUTPUT END   =====================
warning: transaction executed locally, but may not be confirmed by the network yet
~/binary_extension_contract $ arisecli push action arisen regpkey '{"primary_key":"arisen.name2"}' -p arisen
executed transaction: 75a135d1279a9c967078b0ebe337dc0cd58e1ccd07e370a899d9769391509afc  104 bytes  227 us
#         arisen <= arisen::regpkey               {"primary_key":"arisen.name2"}
[(arisen,regpkey)->arisen]: CONSOLE OUTPUT BEGIN =====================
`regpkey` executing.
`_primary_key`: arisen.name2 not found; registering.
`regpkey` finished executing.
[(arisen,regpkey)->arisen]: CONSOLE OUTPUT END   =====================
warning: transaction executed locally, but may not be confirmed by the network yet

Nice! The smart contract is now backwards compatible for the future use of its tables and/or actions.

  • Just keep these simple rules in mind when upgrading a smart contract. If you are adding a new field to a struct currently in use by a arisen::multi_index be SURE to:
  • add the field at the end of the struct.
  • wrap the type using an arisen::binary_extension type.